DRP Metrics Monitoring 2020-07-10

12pm - 12:50pm EDT

Yusra's Zoom

Attendees:

Agenda:

• Dashboard:  Chronograf HSC RC2 Dashboard
• What is HSC RC2 /datasets/hsc/repo/rerun/RC   (chosen by Lauren MacArthur)( DM-11345 - Getting issue details... STATUS )
• What is the "DC2" precursor CI dataset /datasets/DC2/repoRun2.2i/rerun/   (chosen by Hsin-Fang Chiang )( DM-22954 - Getting issue details... STATUS ) 
• Start with 5 metrics:
  • How were these computed?
  • What happened for them the last 6 months?  Annotate changes live (and intro to nightly data from the validation_data_hsc dashboard if need specific date estimate)
    • Annotations added: 
      • w_2020_26: Use FGCM in RC2 (tag: DRP)
      • May 8- June 3 window: DM-23825: PsfEx compiling prob w/ move to conda-forge Fixed DM-25210 (tag: DRP)
• w_2020_26 was the most recent rerun.
  • What was noteworthy in this rerun?  
    •  Started using FGCM for photometric calibration. 
  • We don’t have to debug live, if we spend more than 3 minutes on a question we’ll assign someone or group to look into it for next time
    
    • Eli Rykoff To investigate source of photometric offsets introduced by FGCM in w26 07 Aug 2020 
    • Lauren MacArthur To add tract number to stellar_locus metrics in colorAnalysis 07 Aug 2020 (Done on   DM-25863 - Getting issue details... STATUS )
• What do we expect from w_2020_30? (to be run week of July 27)
  • No major features expected
• What new metric can we expect next time?
  • Rho statistics computed by coaddAnalysis. 
    • Arun Kannawadi Add a couple rho statistics to the dashboard. 
• What else? (Actions from last week)

Minutes:

Part 1: Introduction to precursor datasets

RC2: How was it chosen?

• Selected from public HSC PDR1 data. Limited in terms of size and processing time. Should be processed every 2 to 4 weeks. Monitor performance.
• 3 tracts (1.8 square degrees). Intentionally selected to be not representative.
  • COSMOS field (ultra? deep): small dithering. Tract 9813
  • 2 tracts in wide field.
  • Known issues: e.g., too good seeing, large galaxies
  • All bands (grizy), but none of the new HSC filters (r2 and i2). One narrow band in COSMOS.

DC2: Two tracts. Recommended by the DESC. Standard test tracts. Run 2.2i. Year 1 and partial year 2. WFD survey. Not as deep as deep drilling fields.

DC2 will probably be the basis of Data Preview 0 (DP0).

RC3 likely to be a superset of RC2. Larger contiguous area might be valuable. Perhaps we should create a space for people to discuss design of RC3.

Currently 5 metrics are regularly computed by validate_drp. Let's talk about these. Looking at COSMOS tract of HSC RC2

Astrometry. AM1 is repeatability of angular separation between pairs of stars.

• Jump due to change meas mosaic to joint cal
• Conda environment update, which caused problem for PSFEx. Compiler error fixed. Brought AM1 back to baseline for v20 release.
• KT fixed the PSFEx issue on June 3, 2020. (DM-25210), The backported fixes were all applied to w_2020_22 on June 4. (DM-25221)
• There are annotations. We had annotated jointcal algorithm turned on for example. We didn't annotate the PSFEx problem. We'll try to annotate the PSFEx change live. (Simon Krughoff made a suggestion about using tags to search for annotations that I didn't fully catch)

Aside: Another dashboard to know is "DRP metrics from CI". This corresponds to nightly jenkins runs on HSC. Smaller dataset. Don't run global calibration task, e.g., jointcal.

PSF ellipticity residuals. TE1. There was a bug in validate_drp. Splitting matches into bins, had left the matches in a state where there was a tight bin in magnitude. Those changes were annotated. SRD TE1 say tracking a median. validate_drp seems to be using an average. This would be worth ticketing. In the new framework, the statistic for aggregation is set in configuration.

Photometric repeatability. PA1. Change in most recent RC2 run. Change was doing photometry with FGCM instead of jointcal. Remember that COSMOS is small dithering. FGCM needs more information from overlapping wide fields. There isn't enough information from wide field in RC2, and exacerbated by limited dithering. Let's check other tracts to see. Use "variables" button at the top. GAMA field does not show increase in photometric repeatability spread. VVDS field also got worse.

Stellar locus. Computed by pipe analysis. Galactic extinction by object rather than per field. There is option to use average Galactic extinction per tract. In w_2020_26, there was a big merge. Empty metric file. Lauren just submitted ticket for review by Yusra. Those will show up in a month.

Color analysis produces plots and metrics. Stellar locus. Limit to selection of stars. Based on extendedness. Has to be classified as star in all bands. These are the gray points. wperp is g-r color. Plan fitted to locus. Red dashed line for absolute cut. There is some iteration and clipping to get final set of points for fitting. There are multiple statistics computed, some of which account for photometric error. Lines are the Gaussians with fitted standard deviation. There is some intrinsic width. Ideally, this w_perp is small, indicating a tight spread. The intrinsic width depends on a number of factors, e.g., magnitude range, field selection, in part because of metallicity dependence of stellar colors and sampling different stellar pops. FGCM solution is chromatic, so not surprising that there would be a shift. Jointcal is non-chromatic. We don't know if shift is quantitatively exactly what we would expect.

There is a tool for compare coadd. Can check

Color analysis is done on coadd measurements. Intrinsic width is not just related to photometric calibration, many other aspects enter.

compare visit circular aperture. We can see changes for a specific visit between FGCM and jointcal.

Part 2: What's new? Looking ahead to w_2020_30

rho statistics. Including PSF ellipticity residuals. Generalized to include correlations between ellipticity, ellipticity residuals, and size. Add to chronograph. No formal requirements from SRD. Make own set of requirements based on guidance from WL. Arun is working on this.

Changes to pipelines in w_2020_30? Nothing expected.

Suggest to meet August 7th in same timeslot.

Will we make a regular practice of converting RC2 to Gen-3. Plan to create new repo each weekly. Schema is changing.

We are tracking scalars. Should we be storing other quantities?

Action Items