All
CI status, bugs, issues of the week
- Spring 2022 epic
- Backlog epic:
- Progress on conversion to parquet
- Remaining infrastructure in faro to support adding many more metrics
- Relationship with visualization
- Visualization capability within faro
- Connection to analysis_drp – understanding the relationship between these two software packages
- Infrastructure for visualizing ensembles of metric values / statistics / correlations
- Diagnostic capability
- Partly addressed with shelf
- Other tooling for understanding an unexpected metric value – related to visualization above
- Metric naming convention and metric package specifications
- Review of documentation
- Review of unit tests / coverage
- List of metrics to be implemented / review of metric specifications
- Any additional analysis contexts needed?
- Persisting metrics as lsst.verify.Measurement objects
- Performance question when scaling up to thousands of metric values
- Matching routine for using external catalogs.
- Relationship with visualization
- MatchedCatalogTable base class merged last week
- Rough example of PA1 using MatchedCatalogTableMeasurement (Git branch tickets/DM-32613)
- Just using aperature photometry 12 as the measure since the PsfFlux column in not available in current matched table.
- DM-33959 has been bumped up the priority list for pipelines. We're expecting a fix to land in w15. DM-31855 is essentially a duplicate so this has been marked as 'won't fix. This would apply astrometric and and photometric calibrations to source parquet tables.
- Would a second matching step need to run to make table of matched sources?
- Join once and persist or join every time we run a metric.
- There is a value is
- We could profile the joining now using the isolatedStarAssociationTask keys as an index
Data processing campaigns
- RC2/DC2 reprocessing epic :
- w_2021_40 RC2:
- w_2021_44 DC2 :
- w_2021_40 RC2:
- AuxTel - Erik to give update
- AP/DiffIm
- Prior issues with AuxTel photometric calibration issues now understood, a bug to the calibrated fluxes written to parquet tables was discovered and a fix is under development (see
). For now, we can still calibrate fluxes using AFW tables and photoCalib object. Calibration is good but shows some systematics in scatter.
- I need to generate/register a skymap for the different fields in order perform co-addition, is there a way to do this outside of repo/main to allow for experimentation?
- butler register-skymap --config-file /path/to/file /path/to/repo (could experiment on rc2subset, example config in obs_lsst )
- Discrete skymaps will become more work than worth at some point.
- Option would be to make a small repository and do a test with discrete skymap. Then make skymap in repo/main
- Are there any single-visit/detector level metrics are implemented now in Faro that are worth running on the afw tables we have already generated?
- Keith and Erik to schedule a meeting to talk about next steps for computing metrics to support AuxTel observing
- What quantities are most useful to AuxTel during the night
- Merlin is probably the person to talk to about current AuxTel data processing and RubinTV visualization of metric values
- Short term
- analysis_drp, producing many QA plots. Person is Sophie Reed. Potentially generate a webpage of useful plots. Another source of
Potential co-working session ideas here
Next meeting
