| Name | Should be corrected by DM? | Comments | Requirements (eotest, LCA-10103) | Requirements (LSE-61) | Inputs needed for correction | Rubin DM Stack Code ISR: lsst/ip_isr CPP: lsst/cp_pipe
| eotest code (eotest/python /lsst/eotest/sensor/) |
|---|
| 1 | Bias frames: 2D image (master bias frame) | | - Bias frame stability with BOT data (Jim)
- There are correlations of the bias mean variations and the spikes in the bias stdev values following saturated or high flux exposures: persistence/deficient clear:
- Problems with first bias in a given acquisition (and here)
statistics of the first exposure are often different from subsequent exposures (Duncan) In 9-raft tests, this behavior can be seen in > ~10% of runs for any given sensor; up to 90% of runs for some sensors. (Duncan)
| - None. Other effects rely on the removal of the electronic bias offset per segment (LCA-10301).
| - DMS-REQ-0060: The DMS shall construct on an as-needed basis an image that corrects for any temporally stable bias structure that remains after overscan correction. The Bias Residual shall be constructed from multiple, zero-second exposures where the overscan correction has been applied.
| | - isrTask.py
- Call "biasCorrection", implemented in isrFunctions.py: subtracts a given biasMaskedImage from the image to correct.
| - Taken as part of "Dark Integrations" (2D bias frame is a zeroth-order term of darks)-→ Image Files Data Sets (LCA-10103)
|
| 2 | Bias frames: row by row (Overscan subtraction) | | - Using row-wise overscan subtraction (median), a lot of the temporal variation reposted with BOT data in the first point above (Bias frame stability with BOT data ) is reduced to the ~1ADU level, but there are still some CCDs that have amps with substantial signal variation in time up to ~10-20 ADUs, i.e., R22_S22 and R22_S10 (Jim Chang).
- In DM, we have added row-wise median overscan subtraction to the existing options (Jira: DM-23238).
- Section 8.1 (Bias Offset Correction) of LCA-10103-A indicates that they take the mean of the overscan row and then fit a cubic polynomial to them. In ip_isr/isrFunctions.py we have these options and several more.
| |
| | - isrTask.py
- Overscan: corrects DC offset ("pedestal") from fitting a function to columns of overscan region. Function: overscanCorrection() in isrFunctions.py.
| - eotest/image_utils.py: there's the function in there called "bias_image()" which calls the function "bias_func()". The latter computes the bias by fitting a polynomial (def=linear) to the mean of each row of the selected overscan region.
Bias offset is removed by computing a row by row correction using the data in the overscan columns. For each row in the segment, the mean value of the pixels in the serial overscan is computed. A cubic fit is then created for the function: overscan mean vs. row number. For each row of pixels in the image, a bias offset correction is calculated using the fit parameters. That value is then subtracted from that row in the segment (LCA-10301) |
| 3 | Dark current | | - Measured to be ~0.04 e/pix/sec: from Run 6806D (mode of histogram).
- Dark integrations are images acquired after the CCD has been cleared of the charge and then the dark current signal has been allowed to integrate as the signal for a specified time in the absence of any photon flux.
| Dark current (95th percentile) shall not exceed 0.2 electrons per pixel per second. Measurement to be made at a detector temperature of -95±1° C with dark current integration time of no less than 500 seconds. (LCA-128) | - DMS-REQ-0282: The DMS shall produce on an as-needed basis a dark current correction image, which is constructed from multiple, closed-shutter exposures of appropriate duration. The effectiveness of the Dark Correction shall be verified in production processing on science data
| | - isrTask.py
- Calls "darkCorrection", implemented in isrFunctions.py: subtracts a scaled version of a give dark exposure from he image to correct. The scaling factor is (exposure darktime) / (calib darktime).
| - darkCurrentTask.py (compute 95th percentile dark current in units of e-/sec/pixel)
- Dark current is measured by calculating the amount of charge collected in each pixel during a dark current integration of a given duration. In order to eliminate the effects of charge generated by cosmic ray impacts, the 5 images in each of the data set are medianed together to create clean dark current images. These images are then bias-offset corrected. Multiplication by the system gain constant (e-/DN), and division by the integration time (seconds) provides a measure of the dark current per pixel in electrons per second. These pixel values are then sorted and the 95th percentile value is calculated (LCA-10103).
- Discussion from LCE-61: The need for a dark current correction will have to be quantified during Commissioning. Collecting closed-dome dark exposures may be deemed necessary to monitor the health of the detectors, even if not used in calibration processing.
|
| 4 | Bad pixel mask. | | - eotest: Includes: bright pixels in dark frames, dark pixels in flat frames, death pixels, etc.
- ISR correction in DM: uses a defects list, which is produced by using a list of darks and flats (or a master flat and a master dark), making a histogram, and identifying as defects outliers beyond a certain threshold ("bright" and "dark" pixels).
- From DMTN-101: We expect the camera team to deliver a list of bad (i.e. ones that are unusable) and suspect (i.e. ones that should raise a flag during processing) pixels. Need to compare with the ones made by DM.
| - CCD-012: Active imaging area and Cosmetic Quality: the active imaging area shall contain no fewer than 16,129,000 pixels. The active imaging area excludes the 9 pixels closest to any edge of the device. The active imaging area definition also excludes any pixel within 5 rows of any midline anti- bloom stop.
Within this active imaging area, a maximum of 0.5% of the pixels can be designated as defective. Defective pixels are defined as the following: (a) any pixel with more than 5e-/s of dark current;
(b) any pixel with 500nm photoresponse less than 80% of the mean;
(c) any pixel in any column in which more than 20 contiguous pixels have more than 5e-/s of dark current;
(d) any pixel in any column in which more than 100 contiguous pixels have a 500nm photoresponse less than 80% of the mean;
(e) any pixel in any column in which a pixel traps more than 200 electrons; Any defective columns (according to defective pixel definitions d-e above) shall be separated by at least 10 columns. (LCA-128) | - DMS-REQ-0059: The DMS shall produce on an as-needed basis a map of detector pixels that are affected by one or more pathologies, such as non-responsive pixels, charge traps, and hot pixels. The particular pathologies shall be bit-encoded in, at least, 32-bit pixel values, so that additional pathologies may also be recorded in down-stream processing software.
| | - ip_ir/isrTask.py
- For SAT:
- saturationDetection() and saturationInterpolation() in isrTask.py
- For SUSPECT: suspectDetection() in isrTask.py
- maskPixelsFromDefectList(maskedImage, defectList, maskName='BAD') in isrFunctions.py, assuming a defect list.
- UNMASKEDNAN: I don’t know if we ever expect to use it, but there’s currently an option in isrTask.py to do so (doNanMasking). It could also be used when interpolating masked pixels. I think it’s good to have the option.
| Class to handle masks for a full sensor. You can add mask planes, they will be combined with an "||" ("OR"), and you can get bits to be used with an afwMath.makeStatistics object. Note: The current "main" currently uses the following files: mask_files = ('bright_pix_mask.fits', 'CCD250_DEFECTS_mask.fits') Bright pixel defects and bright column defects are detected using dark integration images. Find bright pixels and bright columns above a threshold specified in units of e- per second per pixel. The mask that is generated will be identified as mask_plane. - DarkPixels.py
Find dark pixels and dark columns above a threshold specified in units of e- per second per pixel.
The mask that is generated will be identified as mask_plane.
- trapTask.py
Task to find traps from pocket-pumped exposure
Calls "Traps.py": uses a pocket-pumpet image, lists traps (x,y,size) per amplifier. Has method to write results to a FITS binary table. "Traps.py", in turn, uses the class defined in "TrapFinder.py", uses method in Kotov+14
|
| 5 | Full Well | | | - CCD-008: Blooming Full well. Pixel charge capacity shall be no greater than 175,000 electrons. Note: Defined as the point at which the detector response (volts out divided by mean per pixel integrated photo-generated charge) saturates when illuminated by a flat field (LCA-128)
|
| - PTC (ramp of pairs of flats)
| - Not part of ISR (blooming full well delivered by Camera subsystem?)
| Compute full well. Fit linear and quadratic functions to data, with constraint at inferred variance at 1000 e-. Full well obtains where the maximum fractional deviation between the linear and quadratic curves is greater than some maximum value, which is 10% as specified in LCA-128.
- DetectorResponse.py also has a full well calculation in "full_well()" function.
|
| 6 | Bleeding (or blooming), midline | | - Midline blooming in ITL (Craig, 2018MAR19):
- Blooming crosses the midline:
- As the charge increases, the well initially does not fill during integration but is overfilled when the charge is compressed into half the volume during transfer. In this case, blooming stops at the midline.
- When the charge gets high enough that the well is filled during integration blooming spills across the midline.
- Consistent with the ITL device not having a “bloom stop” implant at the midline.
- Slope of blooming profiles appears to be driven by the lateral electric field between parallel and serial voltages.
- When the spot is near the edge, blooming reaches the edge before spreading laterally. When it is near the center, it begins to spread laterally before it reaches the edge. This behavior is not fully understood. Dependent on the number of parallel transfers?
- more severe in e2v than in ITL; about 25 pixels should be marked at the midline in e2v when calculating gains.
|
|
|
| | - See "bad pixel mask" row above
|
| 7 | (Non-)Linearity | | - Linearizer in DM: in the PTC task, fit a polynomial (standard basis) to the mean vs time curve. Fractional non-linearity residuals 0.2%-0.4% with LATISS data (DM-23045), using polynomials of degree 3 or 4.
- Eotest vs DM linearity (2020JUL06, Craig):
- Flux dependent multiplicative offset between the mean values. The ratio (Eotest/DM flux) falls from 1.0 to 0.9987 at the lowest flux of 290 ADU.
- Robert Lupton Do we know how stable the linearity is as the gain, bias level, ... change?
I don’t think we have studied this. As the gain or bias changes with, for example, temperature? Can you elaborate more on what you have in mind to test for this? Should we file a ticket? What would the goal of the ticket be? Functional form: The first tests with LATISS data show small (0.2-0.4%) fractional non-linearity residuals using a polynomial of degree 3 or 4 in the standard basis. I think we need to keep monitoring these residuals with more data and explore other functional forms if necessary.
| Detector response (volts out divided by median per pixel integrated photo-generated charge) to flat field illumination shall be linear to within ±2% over the range of 1000 to 90,000 electrons. The response of the CCD is the mean detected signal vs. incident flux. (LCA-128) |
| - PTC (ramp of pairs of flats)
- Range of levels is studied by the flat pairs:
| - isrTask.py (apply Function or LUT)
- "linearizer" is in isrTask.py. It is a linearizing functor; a subclass of lsst.ip.isrFunctions.LinearizeBase. The class is defined in ip_isr/python/lsst/ip/isr/linearize.py. Look-up-table, or function(e.g., quadratic).
| - linearityTask.py, uses DetectorResponse.py (general class to compute detector response characteristics such as full well and linearity from pairs of flats).
Compute detector response vs incident flux for pairs of flats.
|
| 8 | Gain: drift over time and temperature | | - Change of about 0.15% over a period of a few hours: run 6792D (Jim)
- Gain Stability in Run 6870D (Seth): relation between the relative gains and ASPIC temperatures during a sequence of flats
- 'Relative gain' for each amplifier is gain/mean(gain) where gain = (average counts in an amplifier)/flux, and the mean is evaluated over all 196 flats.
- variations of gain correlate well with the temperature of the ASPICs
- slow gain variations will remain even after the ASPIC temperatures are held more constant, although the scale is small (<0.1%). A very few sensors have obvious jumps in gain (at the 0.05% level).
|
| - 1% over 12 hours, 0.1% over 1 hour (Roodman+18)
| - Time series of pairs of flats
| - ptc.py in cp_pipe: fit var(diffIm) vs mean(difIm)to polynomial, Eq. 16 or Eq. 20 of Astier+19.
|
|
| 9 | Gain: variation (amplifiers, methods) | | - Gain comparison DM vs Eotest (ptc and Fe55) (Craig)
- DM gains vs Eotest Fe55: ~1% agreement
- Eotest gains vs Eotest Fe55: ~0.5-0.7% agreement (open question: linearity, etc? see here (Aaron))
- After masking the edge pixels, the agreement is now of ~0.01% (update on 2020-07-23)
- DM v. Eotest discrepancy:
Still under investigation. We’ve been looking at things such as differences in the means (agreement to 0.04% except for edge amplifiers; currently being investigated) and variances and the impact of crosstalk correction (negligible). - After masking the edge pixels, the agreement is now of ~0.01% (update on 7/23/2020)
|
|
| - Fit to PTC including quadratic BFE term.
| - ptc.py in cp_pipe: fit var(diffIm) vs mean(difIm)to polynomial, Eq. 16 or Eq. 20 of Astier+19.
- isrTask.py assumes that the information about the gain, per amplifier, will be in Detector of Exposure. Otherwise, it is an input parameter.
| - Fe55GainFitter.py (fit Fe55 signal distribution to obtain the system gain. Two Gaussian model of Mn K-alpha and K-beta lines for Fe55 tests. The ratio of peak locations is fixed at the line energy ratio, and the line widths are assumed to be the same/
|
| 10 | Serial CTE | | | >0.999995 |
| | - None: should there be a mask plane marking regions with bad CTE?
| - cteTask.py: calls EPERTask() in eperTask.py
CTI is calculated by EPER in overscan using 500 nm superflat (see LCA-10103 9.4.3 for formula). |
| 11 | Parallel CTE | | | >0.999997 |
| | - None: if there are pixels that presented bat CTI, and then they were corrected, should we flag them somehow (similar to SUSPECT)? If we are convinced that the residuals of the correction (for both pCTI and sCTI) are small enough, then I guess we don’t need to do this.
| - cteTask.py: calls EPERTask() in eperTask.py
CTI is calculated by EPER in overscan using 500 nm superflat (see LCA-10103 9.5.3 for formula). |
| 12 | CTE-like/trapping issues: deferred charge | | |
|
| - Flats to use pocket-pumping technique
|
|
|
| 13 | QE (ugrizY) | | - .QE or response spatial variations: structures seen in QE acquisitions for e2v in [320 nm, 1100 nm] (Yousuke Utsumi)
- Annealing (no shift along with wavelength; 1%, 30px; 5%, 360px)
- tree rings (< 1%)
- fringing (Fringing in red: moving around; 5%, variable wavelength): the fringing pattern itself moves in the column plot of the slides linked above. It is the consequence of the spatial wavelength changing as wavelength changes. The 5% was the number of the amplitude at a certain wavelength Yousuke found.
| Quantum efficiency in the various LSST filter pass-bands is defined in 6 separate specifications. They are as follows. See LCA-128 for precise definitions of pass-bands and measurement wavelengths: - CCD-021: u Band QE. u Band QE shall exceed 41%. u band defined as 321nm to 391nm FWHM
- CCD-022: g Band QE. g Band QE shall exceed 78%. g band defined as 402nm to 552nm FWHM
- CCD-023: r Band QE. r Band QE shall exceed 83%. r band defined as 552nm to 691nm FWHM
- CCD-024: i Band QE. i Band QE shall exceed 82%. i band defined as 691nm to 818nm FWHM
- CCD-025: z Band QE. z Band QE shall exceed 75%. z band defined as 818nm to 922nm FWHM
- CCD-026: y Band QE. y Band QE shall exceed 14.4%. y band defined as 930nm to 1070nm FWHM
|
| - Monochromatic Flats over a range of wavelengths:
- Flat comparison
- resolution for monochromator is 1.54 nm.
- LDM-51, 4.2.9: Sets of ‘monochromatic’ (c. 1nm bandwidth and spacing) flat-field screen images taken with no filter/glass in the beam.
- Collimated Beam Projector:
- provides complementary information to the monochromatic dome flats: page 3 of this paper
- LDM-51, 4.2.10: Type of datasets taken with CBP (for multiple goals)
| - isrTask.py
- flatCorrection() in isrTask.py calls flatCorrection in isrFunctions.py which "scaledDivides" the image given a flat.
- Fringe subtraction: ip_isr/firnge.py
| - qeTask.py: Compute QE curves from the wavelength scan dataset. Uses method "calculate_QE" in class "QE_data", defined in QE.py.
|
| 14 | Coffee stains (a.k.a "Dragon breath", "foam") | | - QE or response spatial variations (Yousuke)
- ITL devices, no shift along with wavelength, 5%, various sizes
- No sign of astrometric shifts correlated with these variations so far, indicating that they are indeed surface QE effects.
- They are wavelength-dependent too.
- Limitations to photometry (Pierre Antilogus, Pierre Astier):
- ITL-068 (ATS), peak to peak dispersion of the QE in UV ~>10%, non-uniformities in the coating applied to the CCD surface
- Fast (~5%) in U bad variation of the QE in some areas: can't do precise photometry.
- Surface features in UV-band (A. Bajat)
- No temperature dependencies,
- Wavelength dependency
- The pixels affected by the nonuniformity of the coating are easily recognizable by their wavelength-dependent fit.
- The spatial distribution shows the corners and peripheries of the sensors are more affected by the nonuniformities of the coating
- Future: e2v, full focal plane.
- Analysis so far with existing images taken with ts7-1/ts8. New analysis will be with BOT data.
|
|
|
|
|
|
| 15 | Comb-like pattern | | - QE or response spatial variations (Yousuke)
- ITL, a few percent in amplitude, only visible between 1000 nm and 1100 nm, seen in SLAC and BNL Raft level testings, but not obvious in a sensor level.
- Update at July 2020 DESC virtual meeting by Alex Broughton:
- You can see them on ITL sensors on the raft-level test, but not on sensor-level tests.
- They show up in low-IR wavelengths on the order of a few percent in magnitude (ranges from 3-7% above background, and is highly wavelength-dependent
|
|
|
|
|
|
| 16 | Voltage Testing And Optimization | | - Camera optimization plan
- Findings based on BOT data (Claire, July 2020)
- Understood, fixes proposed:
- Current/heat issues: ASPIC first stage in saturation outside image readout ⇒ fix sequencer.
- Long-range correlations: issues with 1st and 2nd stage reset in e2v CCD ⇒ Voltage update.
- Need some more work:
- Glitches in parallel clocks. Could it explain why “large” parallel swing is noisy? ⇒ Needs more tests.
- VBSS variation during readout ⇒ In situ investigation needed.
- CCD temperature glitches during readout ⇒ In situ investigation needed + extra analysis.
- Need more scan data.
- Running e2v in bipolar mode helps with PTC, tearing, CTI, etc.
|
|
|
| |
|
| 17 | Cross talk (correction matrix) | | - Single Raft
- e2v: Inter-sensor crosstalk consistent with Gaussian noise (single raft)
- ITL: Inter-sensor crosstalk histogram shows a large deviation from Gaussian noise, likely due to the "noisy" sensors (single raft)
- Between rafts
- Results extremely consistent with Gaussian noise.
- No detectable physical crosstalk above the threshold between ITL and E2V rafts.
- Crosstalk derivation on LATISS data with the DM stack
- Need to test the current algorithm with better data (on-sky LATISS data used above determined to not be good enough)
- Improvements to the current algorithm by Chris Waters (DM-24760):
- gen2->gen3
- Preliminary support for inter-sensor and inter-raft crosstalk (currently, it is only intra-sensor).
- Preliminary support to implement Non-linear crosstalk:
- Chris has not implemented this yet (July 2020). The measured ratios and input fluxes are available to the solver and can be used once there’s data to test or an expected model, he says. He also points out that it should not be hard to do once we have a functional form, and he envisions it as another table in the crosstalk calibration.
- Non-linear crosstalk
- Some BOT studies (A. Snyder)
- Seen as part of crosstalk impact of trails with streak simulator (A. Bradshaw)
- Robert Lupton Why do we need a streak simulator to study non-linear crosstalk?
| - Output-to-output crosstalk shall not exceed 0.19% with a goal of 0.02% (CCD-013, LCA-128)
| - DMS-REQ-0061: The DMS shall, on an as-needed basis, determine from appropriate calibration data what fraction of the signal detected in any given amplifier on each sensor in the focal plane appears in any other amplifier, and shall record that fraction in a correction matrix. The applicability of the correction matrix shall be verified in production processing on science data.
| - Crosstalk correction matrix
| - isrTask.py
- CrosstalkTask in crosstalk.py applies the crosstalk correction given cross (crosstalkSources→ is this same as crosstalk matrix). The calculation is done by subtractCrosstalk in crosstalk.py.
- Robert Lupton Has DM accepted and validated measurements from eotest's crosstalkTask.py? Can we run this code ourselves?
- Not yet. As discussed, we will work on this (getting the data and scripts run at NCSA).
| - crosstalkTask.py (matrix calculation from a set of spot image files), calls crosstalk.py, which contains "system_crosstalk" (aggressor amp has single illuminated column) and "detector_crosstalk" (detector crosstalk from a spot image in aggressor) functions.
- Intra-sensor crosstalk for now only
- Inter-sensor, inter-raft, and non-linear crosstalk correction planned
|
| 18 | PRNU: Pixel Response Non-uniformity (LCA-10301) or Photoresponse non-uniformity (LCA-128) | | - Flat fields give a map of variations in pixel response non-uniformity (PRNU), with contributions from changes in sensitivity(QE) and pixel size.
| - CCD-027: PRNU. The measured variation in photoresponse at wavelengths of 350 nm, 450 nm, 500 nm, 620 nm, 750 nm, 870 nm, and 1000 nm shall be less than 5% rms across the CCD. (LCA-128)
|
| | - QE→ flats (see above)
- Pixel area→ astrometric solution (WCS)
| - prnuTask.py , uses class defined in prnu.py
|
| 19 | Overscan Levels | | - Using row-wise overscan subtraction (median), a lot of the temporal variation reposted with BOT data in 1) (Bias frame stability with BOT data ) is reduced to the ~1ADU level, but there are still some CCDs that have amps with substantial signal variation in time up to ~10-20 ADUs, i.e., R22_S22 and R22_S10 (Jim) (referring to "mean(ADU) vs frame index" plots in here )
- Robert Lupton What does this mean? 10-20 ADUs after per-row correction?
- In DM, we have added row-wise median overscan subtraction to the existing options (Jira: DM-23238).
- CCD Assembly: "trims out nonphysical pixels". Measured to get biases.
- Robert Lupton Is the correlated noise coming from row-by-row overscan being allowed for (or is DESC willing to sign off on it not being important)?
I’m not sure about what you mean here. What does it mean for DESC to “sign-off” of something not being important? Representatives of Working groups such as SAWG or WLWG, based on their science roadmap, or something like that?
|
|
| | - See "Bias row-by-row" above
|
|
| 20 | Tearing→rabbit ears | | |
|
|
| - Plans for demonstrating that LSST camera data is free of tearing structures?
- One option would be to use Pierre's automated code to monitor the tearing and its time evolution.
From Pierre Antilogus: Even if we think we have it under control, we should keep an eye on it. Some sensors (with some specific defects) are more sensitive to it, so this could mean that they degraded in time and that the setup that controlled the tearing at some point does not work anymore. So, this should be monitored with time.
| - tearing_statistics.py:
Code to compute Pierre Antilogus' tearing detection statistics. See https://jira.slac.stanford.edu/browse/LSSTTD-1273Under certain operating conditions, E2V sensors exhibit undesirable "tearing" features.
Tearing does not impact raft EO specification and is currently only detectable by visual examination of flatfield images. Pierre Antilogus has developed code for automated tearing detection.
We request this feature to be incorporated in the standard EO test framework. (from SLAC JIRA link above)
|
| 21 | Tearing→divisadero | - Maybe (possible correction, or just masking)
| - Measurements and detection on flats:
- After consultation with A. Bradshaw and A. Roodman (July 2020)
- looks like holes stuck at the amp boundaries
- Seems closer to an additive effect rather than a multiplicative one.
- Study of its impact on objects is an important outstanding to-do
- Will occur during EO testing
- Determine whether the pixels are usable, a correction is possible, or they should be masked.
- Open questions (as of July 2020):
- Flux or Wavelength dependence?
- Photometric/Astrometric/Shape impact?
- Study with Flats and Artificial
Stars
- Voltage optimization
- Optimization of e2v clock voltages, keeping BSS leakage in a reasonable range and trying to mitigate Divisidero tearing
- So far (Pclk_Low, P_Swing)=(-6.0, 9.6) have shown the best result for eliminating divisadero tearing.
- We will explore the vicinity of this point by a 3x3 grid with a spacing of 0.3V or less.
- Modeling:
- Craig Lage: simulations seem to match measurements
- Andrew Rasmussen: electrostatics, with the right boundary conditions imposed, seem to explain with reasonable quantitative fidelity, the fixed pattern artifacts we've been looking at. They include divisadero tearing, the midline redistribution feature, edge roll-off as well as now-less-common tearing features
|
|
| - Flats, for characterization.
|
| - From Andre Bradshaw (SLAC, July 2020):
- below 1% error in "~sky-level" flat fields at 3 pix from the boundary
- No hardware solution yet
- plans to test various voltage configs in full focal plane testing.
- Poisson simulations of hole distributions being developed by C. Lage to have a better understanding and come up with a possible correction.
- Masking: best current idea
|
| 22 | Persistence | | - The cure for the tearing in e2v (bipolar mode + clear in “inversion”), and the way ITL is run (bipolar), does reduce a lot (remove?) the persistence (Pierre Antilogus, July 2020).
- Tearing is a slightly different phenomenon from usual "persistence" which is often charge trapping at the surface (silicon/silicon dioxide interface). Tearing seems to be related to holes (rather than the electrons) not being cleared (Adam Snyder, July 2020).
|
|
|
| | - persistenceTask.py: Compute the deferred charge as a function of time in darks taken after a saturated flat has been taken.
|
| 23 | Dipoles | | - ITL, seen in super flats, can change signs.
- Solved by running the ITL parallel clock gates correctly (A. Roodman, July 2020)
Pierre Antilogus (July, 2020): In any case, we should check that they are 100% gone due to the way we run the ITL sensor now
(they can be hard to see by eye when they are small …and it should be checked for each new sequencer used), but my guess/hope is that this problem is 100% gone.
|
|
|
| | |
| 24 | Diffusion contribution to PSF | | | - CCD-028: Point Spread Function. The PSF due to charge diffusion shall not exceed 5 μm rms (0.2300 FWHM). Note: The PSF measurement is via analysis of Fe55 x-ray generated charge packets. Algorithm for analysis shall be provided by LSST
- The PSF due to charge diffusion is measured using the Fe55 data set
|
|
| - None. Not ISR. Part of PSF model.
| - fe55_psf.py: Fit 2D gaussian to Fe55 footprints to determine the Gaussian width of the charge dispersed signals. For each footprint, also compute the probability of the chi-square fit.
Distribution of dispersion parameters is fit with model in Kotov+12.
|
| 25 | Amplifier Glow | - Yes (part of dark subtraction)
| - Pierre Astier (July 2020):
- There are two types of glow:
- "Serial Glow": continuous serial flush done during the exposure to avoid that large saturation overflow the full CCD, which generates glow at the location of defects on the serial register. The locations of those defects are expected to be stable, as well as the size of the glow, and proportional to the exposure length.
- Very small and stable (at least in e2v; we have less knowledge on ITL)
- However, the method to deal with blooming might change: we could also set the serial register in a specific configuration which avoids the overflow to go back in the imaging area (this is Pierre's preferred method).
- "Accident glow": due to a diode failure in the bulk (this is specific to deep depleted CCD, and the high voltage applied to the bulk). These are due to a break down:
- There's an example of the appearance of such a defect due to a configuration above what a given CCD could stand, which went away once we went back to a normal configuration.
- However, the fear is that such defects appear due to some “aging” / incident (e.g, a power-up, which is the largest stress for a CCD) and then don’t go away. Such glow can be extreme, and even send light to other CCDs (we had such an example)
- In this case, we should detect them as soon as possible. If they are extreme, they should be detected at the slow control level ( increase in current deliver to the CCD), but we should have specific monitoring of this in images
- The minimum will be to implement a detection/monitoring in DARK image taken regularly (once a day and each time you power up the focal plane)
- It would be nice to have something running on sky images too, but Pierre is not 100% convinced that that’s needed: the effect appears at the largest stress = power-on, so monitoring on DARK should do the job. However, this should be checked, as we don't have much information on this type of effect yet.
- This type of glow can appear anywhere; the preferred location is on the 4 CCD sides, but not necessarily on the amplifier/serial register: we had one which was ~a few 100 lines above the serial register: plot
|
|
|
|
|
|
| 26 | Read Noise (incl. correlations) | | | - CCD-007: Read noise. Read noise shall not exceed 8 electrons rms (with a goal of 5 electrons rms) at a pixel rate consistent with CCD-006, single CDS, system noise subtracted. Measurement to be made at a detector temperature of -95±1° C (LCA-128 )
For each amplifier on the CCD, the read noise is calculated using data that are generated at a pixel rate that produces a 2 second frame readout time. A system gain measurement is made using the Fe-55 x-ray gain measurement method described in the previous section (LCA-10103) |
|
| - isrTask.py:
- updateVariance(): read noise is calculated from the ``overscanImage`` if the ``doEmpiricalReadNoise`` option is set in the configuration; otherwise the value from the amplifier data is used.
| - ReadNoiseTask.py: Compute read noise distributions for a sample of images. Bias and system readout noise exposures, the latter for determining the noise contribution from the electronics, must be provided. Uses
noise_dists and NoiseDistributions from read_noise.py.- read_noise.py: noise_dists, NoiseDistributions
|
| 27 | Fringing | | - Discussion from LCE-61: Fringing is likely to affect only the reddest filters, where the CCD substrate becomes semi-transparent to incident light.
- Measured in flat fields and QE acquisitions:
- here (Yousuke)
- Fringing in red (i. z, Y):
- moving around
- 5%, variable wavelength
- and here (Andy)
- Measurements by Pierre et al:
- There are two sources of fringes, one "high-frequency" associated to the CCD thickness that will probably not be visible on the sky, and one "low-frequency" associated to a few micro thicknesses (glue/space at the CCD bottom?) that are easily seen everywhere (sky, lab ...).
- Robert Lupton: Doesn't the BF comment belong somewhere else?
|
| - DMS-REQ-0283: The DMS shall produce on an as-needed basis an image that corrects for detector fringing. The effectiveness of the Fringe Correction shall be verified in production processing on science data.
| | - isrTask.py
- fringe.py: FringeTask: We measure fringe amplitudes at random positions on the science exposure and at the same positions on the (potentially multiple) fringe frames and solve for the scales simultaneously.
this method samples the image footprint at random, then uses those positions in both the master fringe (normalized to amplitude 1) and the science image to measure the appropriate multiplicative factor to apply the correction: corrected = science - factor * fringe. With multiple fringes, it's a least-squares solution to the different signals on the different frames.
| |
| 28 | Tree rings | | - ITL device measurements at BNL: ~ 0.9% amplitude, weak wavelength dependence (HyeYun)
- ITL and e2V: dependence of amplitude on Vbb (bias voltage), radius from center.
- Should be part of the astrometric model per filter (calibrate.py). Profiles can be determined as in Plazas+14; applied as in Bernstein+17. NOTE: Plazas+14 determine radial profiles from flats, and then integrate them to get astrometric profiles.
| - Driven by required RSM astrometric and photometric residuals requirements.
|
| - Flats (photometric profiles), the response of detector to focused light (CBP, c.f., "star-flats" in DES; astrometric profiles)
| - If their scale is large compared to objects of interest, then they can be treated as part of WCS like in DES
- Park+17 measure the tree rings in an LSSTC candidate CCD (ITL-3800C-107); they show periods of ~50 pix.
- HyeYun Park reports tree-rings amplitudes (maximum peak - minimum peak, divided by average flux) of ~0.04% (ITL at BSS=50V) and ~0.03% (e2V at BSS=70V), at a distance of about 5500 pixels from the center of the rings. Closer to the edge, the amplitude was close to 0.1%, although it changed among devices.
- If their scale is not large compared to the objects of interest, then we could think about other corrections such as interpolation, as Robert suggests.
| |
| 29 | Brighter-fatter | | - Current corrections (Antilogus+14, Coulton+17) remove ~90% of the effect. Will likely have to improve on this for LSST.
- Measurements (ITL) and Poisson simulations by Craig Lage show that the BF effect is sufficiently linear that the Coulton+17 correction algorithm (implemented in the DM stack) may be adequate for BF effect correction. The method corrects for >90% of the effect on spot images. Using the back-on-the envelope estimation of multiplicative biases (c.f, weak lensing) caused by the BFE performed in Mandelbaum+14, it is estimated that to reach m=0.001-0.003 (estimated for LSST) the correction needs to improve by a factor of 2-5.
- Improvements:
- CTI impact on BFE (see 9).
- Fringing impact of BFE (see 29).
- Thickness difference in the CCDs may induce BF nonuniformity (through a local field/BSS changes).
| - Requirement driven by residuals in PSF modeling (e.g., rho statistics)
|
| - PTC (flats), correlations/kernel from flats
| - isrTask.py; cp_pipe,
- isrTask calls brighterFatterCorrection() if requested.
- makeBrighterFatterKernel.py from cp_pipe is supposed to provide the kernel to apply Coulton correction.
| |
| 30 | Edge distortions | | - 10-12 pixels around edges: centroid shifts, ellipticity distortions:
- lateral field effect (static). Should be part of the astrometric model per filter (calibrate.py). DECam masks 30 pixels from the edge of each CCD, then measures the profiles directly with star flats. We can use starflats or perhaps CBP if it has an appropriate spatial resolution. For LSST, we probably will not be able to mask a lot of these.
- In addition to distortion, need to check if the edge distortion also depends on the number of counts in a flat (it does for DECam chips at <0.005 level at <20 pix and <0.001 level for >25 pix). There might be some physical effect at the edge that does not fit the simplest model of a static electric field changing the pixel footprints on the sky.
| - Driven by required RSM astrometric and photometric residuals requirements.
- Also, by how much total detector area can be tolerated to be masked.
|
| - Flats (photometric profiles),response of detector to focused light (CBP, c.f., "star-flats" in DES; astrometric response)
| - See the comments above for tree rings.
- To measure the intensity variation of the distortions, if present, we can use flats as in DES: Gary Bernstein calculated this in DECam by averaging 300 rows at the edge of one CCD (S9) from the average of two flats of the PTC at some time T in the r band. After normalizing this row profile by the median of a section away from the ramp, he divided it by the corresponding profile at t=3s. If the edge behavior is independent of the background level and the nonlinearity corrections are accurate, then the ratio should be 1. He found a deviation at the <0.005 level for >20 pixels from the edge, and of <0.001 for >=25 pix from edge
| |
| 31 | Spider Legs | | - Old references to spider legs
- The effect has not been seen/studied in a while (> 4 years, as of July 2020).
- It does not necessarily mean that it has gone away; we should keep an eye for it.
|
|
|
|
|
|
| 32 | Bamboo | | - Images
- Slide by P. O'Connor in 2014: "Understood to be an artifact of the mask-writing process. Future devices would use an earlier mask-writing method that does not show this effect ."
- Decreases with higher back-bias voltage.
- Seen in some of the prototype sensors, but not an issue in the focal plane (Aaron Roodman, Andrew Bradshaw, July 2020). However, Pierre Antilogus warns that we should still keep an eye on them as they will not just go away with flat-fielding and we should assess their impact on astrometry.
|
|
|
|
|
|
| 33 | Flat fields | |
|
| - DMS-REQ-0063 (monochromatic):
The DMS shall produce on an as-needed basis an image that corrects for the color-dependent, pixel-to-pixel non-uniformity in the detector response. The images in the cube shall be constructed from exposures at multiple wavelengths of a uniformly illuminated source. The effectiveness of the flat-field shall be verified in production processing on science data
| - Broadband and monochromatic Flats
| - isrTask.py
- Calls "flatCorrection()" implemented in isrFunctions.py: scaled division or multiplication of an afw.image.Masked image to correct given a flat masked image (provided by CPP?).
- Robert Lupton Need more detail here! Distinctions between surface brightness and flux flats. Colours. etc. How far in the blue will things work well? Where is the colour of the sky coming from?
The reference flat, as defined in eq. 6 of Bernstein+17 (https://arxiv.org/pdf/1706.09928.pdf), is the quantity needed to homogenize the photometry of the survey. It is usually estimated with a source with near uniform surface brightness, which is referred to as “flat field”. For DEcam, the dome flat deviates from the reference flat by about +-5%. The dome flat confuses pixel-area variation and scattered light with the desired measure of response to focused light. To correct the errors in the dome flat, the star flat function is introduced. The time dependence of the reference flat within an epoch is captured by the global relative calibration function: eq. 31 of Bernstein+17 above. - Discussion in https://dmtn-101.lsst.io/DMTN-101.pdf on flat fields, section 10.
| - Flat fields (type of data set):
- A ramp of pairs to do PTC to get pixel full well and linearity curves.
- Superflat: median of 25 flats of equal exp. time to identify:
- Discussion from LCE-61: Monochromatic flat-fields are expected to be produced no more frequently than monthly, owing to the time required to obtain the exposures
|
| 34 | Filter Transmission | |
|
|
|
| - isrTask.py
Provided by camera, could also be measured with CBP, as explained in section 12 of DMTN-101. That section also mentions that at 1nm resolution, it would take several hundreds of exposures to get the transmission curves, but that under certain assumptions (which assumptions?) this number could be reduced.
|
|
| 35 | Ghost and ghouls | | - Ghost: things you can ray trace (non-sequential)
- light hits focal plane bounces back up to some other surface, and then back down onto the focal plane again. Should be able to model and correct these (Chris Z. W.)
- Ghouls:
- non-standard, nickname for “scattered light” off of surfaces not part of the optical system, and therefore cannot be predicted with ray tracers.
- You can't raytrace: screwheads, shuttler, baffles
- CBP can trace ghosts: section 10.4 (Flats for Photometry) of DMTN-101
- Robert Lupton How are you thinking of correcting the ghosts in the ISR? Also, some of the effects are in the flats where we take them out
- DMTN-080 “Coaddition Artifact Rejection and CompareWarp” lists ghosts as transient artifacts that get rejected during coaddition, so perhaps it is not strictly ISR.
|
|
|
|
|
|
| 36 | Bias offset/shift | | - Offset/shift of ~3-25 ADU in bias level.
- Solved by reducing ASPIC gain (by ~1e/ADU) video signal < 195k
- See CAP-213 for an adjust-in-gain fix in AuxTel.
- In general:
- The current setup should be OK to keep the effect for e2v devices.
- The problem is not closed yet, as we have not demonstrated that all ITL devices are ok with the current setup. There will be work on ITL devices in the Northern Fall of 2020 in Paris (pandemic permitting).
- Reference of talk that presents the cure: here
- Camera page on the topic: here
- For e2v, this seems to be under control with the current setup, but the problem, in general, is still on as it has not been demonstrated yet that the ITL devices are OK under the current setup.
From Pierre Antilogus: Not sure yet what the exact cause in the video channel is (Claire and Poierre will investigate in the REB), but how it appears is clear: if a signal on the video channel is above a given value , then you have a bias shift . The way to prevent this is to reduce the gain at the entry of the video channel , using the programmable gain of the second amplification stage of the ASPIC. By reducing the signal size the problem goes away. For e2v, Pierre thinks it’s easy to have this under control (the current setup should be ok, and all e2v devices are approximately the same ). For ITL devices. it’s more difficult: some of them have long saturation tails (some can produce incredibly large signals , well above full well), and Pierre doesn't think that the current setup has been demonstrated to be Ok for all of them. Still, at some point it should be the case. For the moment the only document that presents the cure is (SAWG 5th Feb 2019) The camera subsystem has a page on this topic and has some updates since the initial talk above: here The problem is not closed yet, as we have not demonstrated that all ITL devices are ok with the current setup. If this is not the case, we will have to update the setup, but we still think that such setup exists, and that effect will be under control. In Paris they should go back to work again in the lab during the Northern Fall of 2020 (?) on ITL saturation, with an ITL device presenting an extra-long saturation tail, so their work will include the bias shift issue for this type of ITL with long-tail saturation.
|
|
|
|
|
|
| 37 | ADU favoritism |
| - Acquisition-related issues (ARI), SAWG
- Related to Binary Offset Effect:
Pierre Antilogus: the effect was found some years ago but has not been studied much. the effect is there and it is unexplained, and no cure found so far. the size and/or impact of the effect is not really studied (it’s highly variable from an ADC to another one). so this is in my (and the camera’s) to-do list, including to work at the level of the acquisition micro-code to look for a fix there, but for the moment we should consider that the problem is there and that we should work on it (quantification/impact of it at least ), and live with it.
|
|
|
|
|
|
| 38 | Pixel-to-pixel memory effect |
| - Acquisition-related issues (ARI)
- Pierre Antilogus (July 2020):
It’s like serial CTE but comes from the electronics, so it has different physical properties (can be a longer range, a kind of offset of the baseline). We haven’t done much work on this; it is quite smaller than CTE, and we will probably include some of it, if any, in the serial CTE correction.
|
|
|
|
|
|
| 39 | Jitter and jitter cleaner | | - Acquisition-related issues (ARI), SAWG
- Low and high-frequency time jitters
- Low: only impact integration:
- gain and pedestal variation between pixels.
- High: between reset and signal integration
- In addition to above, badly subtracted reset level
- Can be controlled with jitter cleaner. However, the plan for the moment is not to run with the jitter cleaner (it costs about 1-.5 Watts per REB, and we probably want to save that). For the moment no jitter study has been done on the full focal plane so its impacts are unknown (need to talk with Aaron so this is done).
- Consequences, if left uncorrected:
- The signal we send to the ADC is the result of a subtraction between the integration of a “signal” and a “zero” level. If there is a difference in the integration time due to jitter in the FPGA clock, this operation adds a noise. Also, the jitter on the “signal” integration, generate noise on the gain value from one pixel to another.
- By construction, the size of the expected FPGA jitter is small and should not impact us … but we have seen that noise on the grounding, can disturb the FPGA clock and increase in a sizeable way its jitter…. and noise on a grounding seems to be present in RAFTs.
- Even if we have to live with this extra-noise/gain fluctuation we may need to suppress them for some measurements, in particular for the measurement of the Brighter-Fatter parameters, for which a periodic gain change over an image would prevent a correct measurement of the correlation. In this case, we may for these measurements activate the jitter cleaner, at least in some REBs.
- Remark: if we observe such “gain noise”, its size, >~ 10^-4, would be completely negligible for photometry, but this would impact the BF measurement if it is associated to a periodic noise.
|
|
|
|
|
|
| 40 | Misplaced pixels/ empty pixels | | - Acquisition-related issues (ARI), SAWG
- In some large flux flats, pixels in the image area in a few recurrent locations and for more than one amplifier per image have flux at bias level.
- Seen up to October 2017 in BNL raft data, and not seen since: to be investigated.
- We also had pixels with non-physical values, it was definitely a DAQ issue that got fixed (Claire, July 2020)
|
|
|
|
|
|
| 41 | Analog overshoot in the signal chain | - Yes (with CTI and deferred charge correction?)
| - One component of the deferred charge studied by Adam
|
|
|
|
|
|
| 42 | Missing codes in A/D converter output= ADC distortion |
| - Also known as differential non-linearity. No missing codes, just a bias towards 0 or 1 for certain bits. So the plot of ADC code vs. Analog signal, which should look like a regular staircase, instead must be an irregular staircase - ie. differential non-linearity (Aaron, July 2020)
- Measurements by Adrian:
- In a worst-case scenario, the 1 and 2 bit will be biased to ~0.60 or ~0.40 in some range of ADU. This translates to a 10% increased chance of reading 3 ADU above/below the actual value, therefore reading 3 e- more/less in E2V sensors (gain of 1.095 e-/adu) and 4 e- more/less in ITL (1.277 e-/adu). These values are small compared to the noise. It is not obvious how one would model and correct for this effect due to the wide range of behaviors and statistical nature.
|
|
|
|
|
|
| 43 | Lateral field effects- micro-WCS distortions | | - generic description of pixel-size effects on small scales that can impact WCS, through lateral field distortion: edges (30), tree rings (32), bamboo (34), things from step-and-repeat masking, PNRU (17)
- pixel size variations contribution to PRNU
- For an LSST sensor prototype with low PRNU (0.4%), the impact on astrometry and photometry is negligible but scales linearly with the PRNU. Therefore, for sensors with PRNU values closer to the specification of 5%, additional work to separate and model the effects of local QE variation and pixel grid distortions may be required.(Baumer+15, Baumer+18)
|
|
|
|
|
|
| 44 | "Misc. video-chain badness" |
| - Things that might be bad in the video chain that either aren't covered directly by the above list but impact them (hidden variable type thing) or haven't been discovered yet but live in there (Merlin), e.g.:
|
|
|
|
|
|
3 Comments
Steve Ritz
This is great. A few suggestions and items to update (then you can delete the comment):
Christopher Waters
Is there a listing somewhere with images of the various non-standard effects ("rabbit ears", "spider legs", etc)? It'd be good for everyone (including me, asking because I don't) to know what each of these look like so we don't report known issues.
Andrés Alejandro Plazas Malagón
Examples of images of sensor effects will live in Images of sensor anomalies