Mondays 12PT (3 - 3:50pm ET)

Yusra's Zoom: https://princeton.zoom.us/my/yusra

Attendees:

Regrets:

Agenda:

  • Meeting recorder -  Orion (last 6 meetings were:  Jim, Clare, Dan, Erfan, Lee, Robert, Arun) 
  • Announcements
  • Action items from last time:
    • Scaled-size scatter cuts  (LM says not merged)
    • Reference-catalog matching plots need to get closed out (not on main, yet, but unblocked)
    • Manifest checker: counts of quanta that failed and succeeded (merged but not run)
    • scarlet lite as a standalone package  (on 39)
    • FGCM logic change not lookup table (on 39)
    • Some mysteries from last week
  • Any other changes past or expected?
    • DM-39970 (Dan) -- changes the default max to peak for SDSS centroid
  • Processing Status
    • Review the w_2023_41 DC2 (Orion): 
      • wPerp from faro_tract ran out of memory multiple times. There was some talk of retiring this version of wPerp, since it also exists in analysis_tools? 
    • Processing of w_2023_43 RC2 (Jen/Orion)
      • About to proceed to Step 5
      • skyObjectMean has errors 
        ValueError: Column unknown_ap09Flux specified in parameters not available in parquet file.
        
        but it looks like the data id is 
        dataId={band: 'unknown', skymap: 'hsc_rings_v1', tract: 9813}
        so recommend re-run or rescue with proper band constraints.
      • skyObjectStd failed with error 

        ValueError: Column ~_pixelFlags_edge specified in parameters not available in parquet file. 

        it looks like a data id is 

        dataId={band: 'white~nd', skymap: 'hsc_rings_v1', tract: 9615}

        This doesn't look normal to me (Orion). Could this also be because of the missing band constraints ?

  • Review the w_2023_41 rerun:
    • https://usdf-rsp.slac.stanford.edu/plot-navigator/dashboard_gen3
    • https://usdf-rsp-dev.slac.stanford.edu/chronograf/
    • Lauren: big jumps in some metrics is expected!
      • DM-41554 - Getting issue details... STATUS  
      • Stamp sizes for piff vs. psfex reveals extremely high psf sizes
      • Huge spike ~0
      • Something is going seriously wrong with piff and maybe it has to do with the stamp size
      • Luckily the thresholding metrics caught on
      • Size scatter ~0 but we have NaN for psfTraceRadiusDelta – NaN's are skipped so then other values should not  be close to zero
      • Many plots available on the ticket
      • Lauren re-ran these and found no issue on HSC-RC2-- as opposed to the DC2.
        • We are probably running this very wrong on our DC2's!
        • tract constraints need to be gone so the CM team needs to re-do their submission files.
        • Selecting on tract means we don't get full visits, so there's not much for finalizeCharacterization to work with.
        • We should be comparing the initial and final PSF measurements if possible.
    • Photometric repeatability plots:
      • very weird holes!
      • shape doesn't make sense!
      • jagged edges in the healsparse maps
    • Yusra: Tract constraints on step 1 and step 2 were actually on purpose because the tracts are a lot smaller and to our knowledge we don't run any full focal plane tasks
    • Eli: some tasks maybe do? Given the holes we see, none of the piff psf's should work
    • "Strange things are afoot"
      • Lauren: The tract constraints are having an affect (if they shouldn't make a difference, what are we seeing?)
    • Jim: cannot have the tract constraints; Yusra: everything is WAY faster if we have them
    • Jim: isolated star catalogs expect to tile the sky
      • If you have a detector that is not one of your target tracts, isolated star creation will break if there is a tract constraint
      • unless we expand the list of tracts to include adjacent tracts
    • Eli: isolated stars in Lauren's run actually look fine
    • Lauren to Eli: it seems like she got consolidatedMaps but without the tract constraints they were not consolidated. Does that make sense?
      • Eli: ?? no.
      • TBD
    • Big mystery #2, unclear if connected: Jump in all coadd psf metrics for the u-band. 
      • possible connection to scarlet_lite standalone package?
      • Yusra: "10^-2 is objectively bad"
      • Jim: does this metric only look at isolated stars? It probably should. (although Fred confirms that this should have no effect due to weighting of blended stars)
        • Possible solution: remove blended stars from the metric.
  • Review the w_2023_43 rerun:
    • Recall what we expected:
    • Recall 2023 history:
      • w03: Issues with using finalize characterize PSFs downstream. 
      • w07: ip_diffim. Cannot find plugin (fixed on DM-38209) run with ticket branch. Issues with finalizeCharacterize downstream gone.
      • w11: GBDES on.  detectAndMeasure segfaults (fixed with hours to spare for w15) 
      • w15: quasi-random skyObject placement (DM-23781), source selectors use isPrimary (DM-39141)
      • w19: Parallax and PM on (DM-37943), astrometric match improved (DM-38808), cleared mask plan of the template. (DM-38901)
      • w23: RA/Dec columns renamed
      • w27: No major pipeline changes
      • w32: Use GAIA DR3 refcat in GBDES
      • w35: 
      • w39: scarlet_lite as a standalone package. 
  • What pipeline changes do we expect for w_2023_47?
    • "Rashly predict" that AB and AA will get into metrics – Clare
    • Sophie: new analysis tools potentially
    • Eli: finish in putting defaults in from LATISS for scaled size thing
    • No errors in step 4!!!!! !!!!! !!!!
  • Show and tell:
  • AOB:
    • DM-41554 - Getting issue details... STATUS needs discussing
    • Tanaka-san plot!
      • AA1: Clare: since writing, after discussion with Robert, decided to implement differently
        • look at the median of the absolute value of the difference between the RA and Dec positions of the reference catalog
        • we might actually want to not take an absolute value; take the actual difference
        • Clare thinks that with more visits, you can get a better fit of the astrometry and get a better fit between the science measurements and the reference catalog
        • Way below science requirements so that's "always a comfort" (smile) 




Action Items

DescriptionDue dateAssigneeTask appears on
  • Add a plot with fakes stats to the dashboard. Sophie Reed 
04 Sep 2020Sophie ReedDRP Metrics Monitoring 2020-08-07
  • Clare: add analyzeMatchedVisitsCore to drp_pipe step8
DRP Metrics Monitoring 2023-06-26
  • Sophie: make a new list for outstanding analysis_drp plots that require moving, send to Jim
DRP Metrics Monitoring 2023-06-26
  • turn catchFailures on in calibrate. Add flag to indicate that deblender failed because PSF is bad. 
DRP Metrics Monitoring 2022-10-31
  • Yusra AlSayyad Eric's account was deleted; we need to make sure he has all his logs. 
Yusra AlSayyadDRP Metrics Monitoring 2021-06-14
  • Arun Kannawadi Modify rho stats in pipe_analysis  to use debiased moments (see  DM-30751 - Getting issue details... STATUS ). 
Arun KannawadiDRP Metrics Monitoring 2021-04-19
Arun KannawadiDRP Metrics Monitoring 2021-03-01
  • Yusra AlSayyad Do a rerun with w50 PS1 refcat and one with shrunk refcat errors. 
Yusra AlSayyadDRP Metrics Monitoring 2021-01-04
  • Jeffrey Carlin Add an absolute astrometry match-to-refcat metric to dashboard  DM-34153 - Getting issue details... STATUS
Jeffrey CarlinDRP Metrics Monitoring 2021-01-04