|Notes and Action Items
|News, announcements and task review
- sandbox notebook - there have been other requests for a sandbox. Simon will come back with a proposal next week
- How to handle failure modes
- Need CI dataset that is fit for purpose (highest priority)
- There is distinction between values recorded lsst.verify.Measurement, what gets saved in json, and what gets recorded in chronograph
- Idea of status codes as extras in lsst.verify.Measurement
- Create a backlog Jira ticket for thinking about metric status codes?
- faro dashboard for RC2
- InfluxDB is distinct from chronograph visualization. Can create dashboard separate from persisting values in database.
- Leanne to follow up with Yusra
Bugs, issues, topics, etc of the week
Reprocessing status and metrics review
Spreadsheet with plans/progress tracking.
Dashboards can be found here: https://chronograf-demo.lsst.codes/sources/2/dashboards
- Need to replace validation_data_hsc_gen3 (3 bands, 2-3 visits/band).
- New subset has 5 bands and 5-8 visits / band . Includes parquet tables.
- Plan to update the nightly processing:
- Jeff to run faro on Simon's gen3_rc2_subset dataset to test and review outputs
- Review outputs
- Iterate on dataset until happy
- When happy replace CI dataset.
- TE metrics & stellar locus metrics measure on coadds
- Need to fix stellar locus metric
- TE metrics use sources not deblended by scarlett. We can not run deblending by removing from the pipeline.
- CI specific pipeline definition to retarget deblending.
- Could also change the SNR to 3sigma (from 5sigma)
- Fall 2021 epic
DM-30748Getting issue details...
- Backlog epic:
DM-29525Getting issue details...
DM-26214Getting issue details...
- New metrics should be added using the new parquet file approach
- Need to migrate existing metrics and deprecate the FITS file classes with high priority.
- Make one ticket per context. Need to create an equivalent base class following the template and create the yml files to point to the base class.
- Add DM license preamble to all python files
DM-31135Getting issue details...
- Started this for all the files touched so far
- Create / improved doc strings
- Homogenize run method signatures across all the measurement tasks. Suggest in general to inherit run method from CatalogMeasurementBaseTask base class, which can accept and pass keyword arguments. We should avoid passing dataIds to measurement tasks were possible. For example, we can use FilterLabel or list of FilterLabels instead of passing the dataIds if we just need to know which bands are being used.
DM-31061Getting issue details...
- Added most of the utility for loading external reference catalogs into the CatalogMeasurementBaseTask base class
- Add unit tests for task code in faro
- Use consistent DM class naming convention for base classes
DM-31136Getting issue details...
- Started this
- Development of reference catalog loading capability in faro with proper configuration options led to discovery and resolution of a bug in LoadReferenceCatalogTask. Thanks to Eli.
DM-31188Getting issue details...