| Time | Who | Pre-meeting Notes | Notes and Action Items |
|---|
| News, announcements and task review | | |
|
| CMR |
| Current draft of DMTR-311 | - Jeff made a ticket to use update columns and has BPS job running now that should finish soon (~30 min). The change was made in the default rather than the yaml configuration.
- Should we be using "slot" designations in general? Depends. One does have to be careful where the slots are pointing.
- Where does documentation exist for the columns? Not an immediate answer. We expect several of the names/columns to change.
- The ellipticity residuals were the big change in reported metric values.
- Need to specify high memory (40 G) when running the BPS jobs to compute metric on full tract.
- Even with largest container in RSP, cannot load full tract catalog from object table into memory.
- Getting the schema is a good ticket for middleware. Should be able to retrieve the schema.
- Check to see if there is composite dataset type for the schema
- faro should be transitioning to use object table and loading a subset of columns
- Need to figure out how to specify the subset of columns to load in the connections class.
- Might have to make a runQuantum function to do this, though it would be nice if could specify as argument in the input in connections class
- Considering to use w18 instead of w14 for v22 pipelines release
- One issue that would need to backported, but might change conda dependencies. Release management reasons.
- There is a w18 RC2. Should we run faro on w18, Monika included. Jeff would need to re-run TE1 and TE2. Then would have w18 results in hand.
- w22 processing in underway
|
| Status of reprocessing |
| - w_2021_14 (RC2)
- w_2021_16 (DC2 gen3) (thanks Dan)
| - Now running faro with BPS, need documentation
|
| Bugs, issues, etc of the week |
| - Apparent TEx performance degradation in bands other than i
| Jira |
|---|
| server | JIRA |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-30359 |
|---|
|
|
|
| Development status |
| Review progress on the current development epic | Jira |
|---|
| server | JIRA |
|---|
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-26993 |
|---|
|
Backlog epic: | Jira |
|---|
| server | JIRA |
|---|
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-29525 |
|---|
|
| - Spring 2021 epic was intended to be finished 31 May 2021
- Next week could be an opportunity to do a comprehensive review of current epic and make priorities for the next several months
- One high-priority item would be using object table and source table (parquet format)
|
| AOB |
| - Reference catalogs interface development:
| Jira |
|---|
| server | JIRA |
|---|
| serverId | 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 |
|---|
| key | DM-30054 |
|---|
|
- What feedback would we like to receive from science pipeline developers from faro presentation on Tuesday 2 June?
- What metrics do we expect pipelines to be producing in situ?
- Visualization
- Dedicated data structure to hold metric values in butler repo?
- What inputs should be used for metric calculation (e.g., parquet tables vs. direct outputs of pipelines)?
- External reference catalogs
- Adding reference catalogs
- External calibrations
- Spatial scales for computing metrics (e.g., patch/tract versus other spatial partitioning scheme)
- Communication between faro development and science pipeline development
| - Simon is updating validation_data_hsc CI dataset. Need a Gen3 dataset to use for getting started. Central 6 sensors from something like 10 visits. Calibrations and reference catalogs.
- For science pipelines meeting,
- Not necessarily trying to necessarily propose solutions, or get into design discussions, but try to find out who to connect with in DM to make progress
- "What can faro do for you?"
|