| Item | Who | Pre-meeting notes | Minutes |
|---|
| News, announcements and task review | |
| - Move to zoom from next week. We will use the existing zoom channel.
- Co-work - look to merge with pipelines
- Thought about feature requests → will assess influx 2 first and then look at what features we would like but that are not present. Alerts are in influx 2.0 which will be useful.
|
CI status, bugs, issues of the week | All | - verify_drp_metrics CI jobs
| - Much variation in metrics in the last 7 days in AM/PA/TE metrics
- Unlikely that there is a ticket commit every daya that led to this
- Check input catalogs to faro first to check that the input columns are stable - CI intermediate data products are persisted
- Checkout daily and see if we even get the same numbers.
|
| Faro Development status | | - Fall 2021 epic
| Jira |
|---|
| server | JIRA |
|---|
| columnIds | issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution |
|---|
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-30748 |
|---|
|
- Backlog epic:
| Jira |
|---|
| server | JIRA |
|---|
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-29525 |
|---|
|
- Progress on conversion to parquet
| - Parquet conversion - tickets assigned to convert all the metrics that need matched catalogs as input. Base classes that used the forced photometry table will not be able to handle AM repeatability metrics due to fixed centroid, so cannot compare the astrometric solution. Need a new catalog matcher that takes the per-visit source catalogs. Keith planning to do this in the next week
- The need to match the source catalogs for astrometric repeatability raises the N-way matcher topic again. One solution would be to use the object table as the reference and do matches of source catalogs against the object table, effectively associating sources with an object id.
- TEx conversions - Erik & Jeff have it running for all analysis contexts.
- Multiples found in the output table of a multimatch selection of individual visits. Erik to look into it.
- TEx algorithm convert to accept parquet as input. Verified with FITS.
- Ready for a PRPF: Comparison to truth table reference catalogs
- Comparison with refcats that are not part of the regualr processing (e.g Gaia, Pan-STARRS), look to implement a simple metric to understand the process of working with any refcat in the butler context. presentation link DM-31985
|
Data processing campaigns | | - RC2/DC2 reprocessing epic :
| Jira |
|---|
| server | JIRA |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-26911 |
|---|
|
- w_2021_40 RC2:
| Jira |
|---|
| server | JIRA |
|---|
| columnIds | issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution |
|---|
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-32248 |
|---|
|
- w_2021_44 DC2 :
| Jira |
|---|
| server | JIRA |
|---|
| columnIds | issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution |
|---|
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-32391 |
|---|
|
- AuxTel
- AP/DiffIm
| - Erik: Merlin has finished a first run through of AuxTel data to handle SF processing. Handed over instructions. Keith ran to completion (see discussion here). Will start processing AuxTel data.
- Data is at NCSA, latiss data repo
- Can we move to Antu for processing. Erik will ping RHL on this.
- Goal is to have near real-time processing at summit
|
| DP0.2 | | | - Still working on SF processing sign-off
|
| AOB | | Potential co-working session ideas here | Last meeting of 2021 on 14 December. Meet again in 18 January 2022 |