This page gives an overview of the Level 1 "single-frame" processing that results in calibrated exposures.
Baseline Documents
The primary documents are:
- LSE-163 (Data Products Definition Document)
- LDM-151 (Data Management Applications Design)
Also relevant are:
- LSE-180 (Level 2 Calibration Plan, as there may be some applicability even to Level 1)
Inputs (for a nominal science visit)
- Two crosstalk-corrected "snap" images from Camera, including four wavefront sensor images
- Calibration "master" frames and models (designated at start of night):
- Bias (from Calibration Products Production, CPP, as needed)
- Other amp/CCD info (gains, read noise, brighter-fatter coefficients, ...)
- Dark (if necessary, from CPP as needed)
- Non-linearity (from CPP as needed)
- Flat (synthesized by CPP from previous day's broadband data and month's narrowband data)
- RHL the details on flat generation are TBD
- Fringe (if necessary, from CPP as needed, modified by model fit?)
- RHL Probably multiple fringes (the OH results in more than one component)
- Defect and hot pixel list (from CPP as needed)
- Astrometric and photometric reference catalog
- Thresholds, default PSF, and other algorithm configuration parameters
- RHL Tricky. default PSF may need some history or iteration (e.g. if the seeing is steadily improving through the night)
Overall Process
- For each "snap" image in a visit, including wavefront sensors (TBD: any changes for wavefront sensors?):
- For each amplifier:
- Convert to floating point
- Detect and mask (but do not interpolate) saturation (TBD: not mentioned in LDM-151)
- Do overscan correction by averaging columns, fitting 1D function, and subtracting row by row
- Do bias correction by subtracting master bias frame
- Do dark correction (if necessary) by subtracting master dark frame scaled by exposure time (RHL: coefficient possibly a function of temperature?)
- Assemble amplifiers into a CCD including trimming prescan/overscan
- Correct for non-linearity, along with any temperature dependence
- Do flat correction by dividing by a normalized master flat, assuming a nominal flat spectrum for all sources
- RHL: the choice of spectrum is still TBD. More likely an average sky spectrum.
- Do fringe correction if necessary depending on filter by subtracting a best-fit modelled fringe pattern frame
- RHL Maybe more than one component. In theory it's not obvious that we should estimate the fringe coeffs per chip, but it's probably OK.
- Update the image variance (TBD: not mentioned in LDM-151)
- Mask and interpolate over defects (TBD: not mentioned in LDM-151)
- Unmask saturated hot pixels (mark them as only BAD, not SAT) (TBD: not mentioned in LDM-151)
- Interpolate over saturated pixels (TBD: not mentioned in LDM-151)
- Mask and interpolate over NaNs (TBD: not mentioned in LDM-151)
- RHL where do these NaNs come from?
- Combine two "snap" CCD images from a visit (not for wavefront sensors):
- Reject cosmic rays based on two images (TBD: simple subtraction, morphological analysis, more?)
- RHL we need a PSF before we can do morphological CR rejection. We'll probably do a morpho in the difference between the images, but that depends on the atmosphere and telescope.
- Add images; assume no warping or realignment is necessary
- RHL we won't know for sure until comCam or beyond. It's the same question as whether we can do a straight subtraction for CR rejection. If we do need to do some simple warp/match we'd do it before the CR step to allow us to subtract.
- Using a default PSF:
- Estimate the background and subtract it
- RHL At high Galactic latitude we can probably avoid a subtraction – a single number can be added to the threshold. Down in the plane it's going to be more fun.
- Detect and do initial measurement of sources on the image
- Use sources to determine a PSF
- Second-moment, catalog, and object size star selectors are options
- RHL Probably catalog in steady state
- Use PCA to generate spatially-varying PSF model (TBD: How accurate does the PSF need to be for Level 1 processing?)
- RHL PCA is a possible model of the individual PSFs. The spatial model is another question. One implementation of both aspects is the current pcaPsf
- Now repeat using the real PSF:
- Estimate the background and subtract it
- Uses large cells (256 or 512 pixels on a side) and clipped mean
- Ignores pixels that are part of sources
- Akima spline used to estimate background level in each pixel
- RHL I'm not sure of the algorithm: the cells, the clipped mean, and the spline are all TBD. But as we just need this for WCS/Photocal it seems reasonable for Level 1
- Detect and do initial measurement of sources on the image
- Use sources to do astrometric calibration to determine the WCS
- Do photometric zero-point determination by fitting the measured sources with a photometric catalog
- RHL there's no single zero-point when it's cloudy. We'll need a model TBD