NBReport - Simon needs a more concrete description of need. Current use case is too broad. RHL: don't want to invent requirements up front. Need to start doing verification first. RHL:
Robert Lupton Scope: What is the role of notebooks in certifying calibration products RHL to come up with a proposal.
Little progress - noo full parity btw G2/3 ppls, Y would like metrics calculated on the G2 oprocessed data before OK-ing a release. Eli is writing a script to compare results Slack conversation here: https://lsstc.slack.com/archives/C4JQP6FRS/p1623111517088600,
Access to schema of tract object table is possible: See this thread.
Can access for the tract object and source tables.
Need to do this because on the RSP, could not load the full tract into memory, so a workaround was needed. Suggestion from Eli to get the parquet schema without loading anything.
Simon's new dataset. Minimum of checking. 8 visits from 5 bands in cosmos, just with 6 central chips from HSC. Run that dataset through DRP Step 1 (SFP, JC, FGCM)/2 (warp/coadd)/3(meas FP object creation). Only 4 patches with full coverage. All this with w24 ran to completion with no error in quanta. Eli said FGCM outputs were not terrible. Will be (intention is to) better than validation_hsc - more visits/bands. Still in the ci / testing./reg. issues phase. repo contains only raw/calib and not outputs. Need a shared location at NCSA
Leanne Guy Need to test and refine the RC2 dataset and run faro on tit.
Keith: For Fall 2021 epic, wearing commissioning hat, I would personally like to prioritize implementation of a collection of metrics to provide basic quality evaluation of individual detectors / visits, in other words, metrics that would be run immediately upon taking on-sky images, and which could be used as part of an on-the-fly analysis system providing feedback to observers.
Propose to develop these metrics using HSC RC2
Propose to use sourceTable as main catalog input (work in progress
Jira
server
JIRA
serverId
9da94fb6-5771-303d-a785-1b6c5ab0f2d2
key
DM-26214
)
Needs reference catalogs as input (work in progress
Jira
server
JIRA
serverId
9da94fb6-5771-303d-a785-1b6c5ab0f2d2
key
DM-30732
)
Example metrics: number of detected objects (flagged objects), throughout, image quality, sky brightness, matches to reference catalogs, effective depth, anomaly detection
Question: Could we apply these metrics to AuxTel imaging to attempt to perform a closed loop test?
Part of the thinking here is that I want to start getting more specific about the types of information we need to address the use cases that have been developed, having some example metrics to use to think more concretely about visualizations and generally moving information where it needs to go.
RHL: Auxtel is one way to do it, can also run on HSC single frame data. Reply DC2 data throught the camera data could also be used.