Versions Compared

Old Version 4

changes.mady.by.user Keith Bechtol

Saved on

compared with

New Version Current

changes.mady.by.user Lauren MacArthur

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Meeting recorder - Keith (last 6 meetings were:  Fred, Nate, Jeff, Clare, Arun, Leanne) 
  • Announcements
  • Review Action items from last month
  • Processing Status
  • Review w_2022_44 rerun: 
    • Jira
      serverJIRA
      serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2
      keyDM-36763
      • Smooth processing, no major issues
      • We expect problematic patch to pass in w48
    • Recall history:
      • w12: psfex
      • w16: piff  (bad size residuals)
      • w20: finalizeCharacterize (bad apcorr configs → bad stellar locus)
      • w22: lanczos11 + apcorr configs (better stellar locus and size residuals approx equivalent to psfEx!
      • w24: PIFF kernelSize to 25. new scarlet lite storage.
      • w28: attempt at fixing measure failures:  DM-35722
      • w32: First RC2 at SLAC. subtractImages compatibility mode on
      • w36: subtractImages Compatibility mode off
      • w40: 
      • w44:
    • Chronograf, plot-navigator
      • Chronograf is down, so cannot look at metrics easily right now
      • w44 is similar to w40 in content because already tagged by the time w40 was finished
      • Currently 3 plots from analysis tools. Sophie has added more plots on stellar locus that should be coming in w48.
      • PSF size fractional residuals plots
        • Suggest to add horizontal line at zero to better see if there is a trend with respect to magnitude
      • New in w44 is astrometry refcat plots
        • sky plot over the focal plane
        • scatter plots that show residuals with respect to magnitude
        • Lauren is working on plot showing residuals as a function of focal plane position and RA/Dec for a given collection at visit level. Average per detector. Clare offers to help with review. 
          Jira
          serverJIRA
          columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
          columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
          serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2
          keyDM-34625
        • Josh is interested in intradetector effects as well
        • Josh is also interested in residuals with respect to high and low zenith angle
    • Open tickets for the errors:
      • DM-36930 Robustify PIFF so that we're back to 0 characterizeImage and calibrate failures. 
        • Looking at example icExp that shows large regions of interpolation and "wonky behavior" for Piff PSF modeling. If we did config change, this would pass threshold cuts. Even when Piff does evaluate properly, there is factor ~3 in size (determinant radius) variation over the image. Probably we don't want such a detector to be included in coadds.
        • Want to make PSF models not wonky.
        • We want metrics to identify detectors that have large regions without PSF model stars and detectors that exhibit large amount of variation. Check number of PSF stars and distribution across the detector. Some of this should go inside the PSF determiner, and some would be afterburner.
        • This problem was caught for the wrong reasons and we were lucky to have caught the problem.
        • PSFEx was able to handle this example image in w12, but less of the image was masked, so not directly comparable. We have not yet run the experiment to see whether PSFEx can successfully handle this image with the larger amount of masking.
        • Hypothesize that PSF modeling is failing because not enough PSF anchors distributed across the image.
        • Summary:
          • More robust PSF modeling
          • Add metrics to identify PSF model failures
            • Check for large variation across the CCD
            • Check if very different from neighboring CCDs
            • Check that every quadrant / octant / other spatial region of image has a good PSF star (note that we should exclude the vignetted regions from such an analysis)
    • What pipeline changes to we expect from w48?
      • DM-36265 (fewer failures from subtractImages)
      • DM-36998 Templates take <  half space on disk.
      • 9697 patch 7 detection should succeed.
      • Clear up of deblender log files to be more informative with less distracting information
    • New plots and metrics to expect from w48?
      • Multiple stellar locus color-color plots and metrics
        • Compare those metric values against faro output
        • Details of signal-to-noise cuts and selection of bands to apply criteria will impact metric values, so be careful to be consistent
      • Peter has been developing visualizations of photometric residuals as a function of focal plane position with the intent to add these to analysis tools. Initial testing LATISS images already shows that photometric residuals are reduced with the new filters with respect to the previous filters
      • Sophie has compiled a list of plots from analysis_drp that still need to be ported to analysis_tools: https://photos.app.goo.gl/Y1QPtcRv5jT8C9WZ6
        • Organized into core and extended lists
        • Add color-color stellar locus plots to the list
  • Update on PSF analysis from the 100 sq deg run (Arun Kannawadi )
    • Summary report from HSC Gen3 processing on 100 deg^2
    • Changes
      • SDSS → Scarlet (deblender choice)
      • Consistent star selection (same stars used across all visits)
      • PSFEx → Piff (PSF modeling choice)
    • Measured stellar and PSF
      • Ellipticity residuals appear centered on zero
      • Size residuals show bias (T/T_psf - 1 is negative). Replacing PSF modeling code alone not expected to solve this issue. Contributes to multiplicative bias to be calibrated with simulations. Additive bias can be calculated with rho statistics.
        • Disappointed that commutator change did not improve
        • Need to be careful about fixing all the cases; need to give careful thought
      • Rho statistics
        • Gray shaded region is requirement. Darker gray region shows transition to linear scale.
        • rho_1: On scales less than 1.5 deg, within requirement, but exceed requirements on larger scales.
          • Considering the reserved and isolated stars. If bouncing around positive and negative, likely consistent with zero. Error bars likely underestimated.
        • rho_5 near constant offset for PSF stars is a direct result of the PSF size residuals discussed above
        • rho_2, rho_3, rho_4 all within requirements
      • Debiased PSF moments
        • rho_1: almost no difference
      • Conclusion
        • Expect to be able to meet requirements in PDR4, even though factor sqrt(2) tighter. We can relax rho_1 requirement and tighten rho_3 requirement.
      • Discussion of isolated star
        • Extendedness cuts applied in coadd
  • Time to doodle for a new meeting time? 
  • AOB:
    • USDF is doing maintenance for ~4 days after thanksgiving so the next w48 should be a bit delayed. 

...