Date

Location

Browser

Room System

Phone Dial-in

https://bluejeans.com/690803707

or 

https://ls.st/rsv


  1. Dial: 199.48.152.152 or bjn.vc
  2. Enter Meeting ID: 690803707 -or- use the pairing code

Dial-in numbers:

  • +1 408 740 7256
  • +1 888 240 2560 (US Toll Free)
  • +1 408 317 9253 (Alternate Number)

Meeting ID: 690803707

Attendees

Regrets

Metric Tracking Dashboard

URL: 

Discussion items

ItemWhoPre-meeting notesNotes and  Action Items 
News, announcements and task review

DM Development Workflow.

Slides on faro package presented at DM Science Pipelines meeting last week

  • Science Pipelines would like faro metrics to be routinely shipped to SQuaSH and appearing on chronograf dashboard
  • Question of best way to handle situations when the metric definition (algorithm for calculating) changes. Annotate in SQuaSH dashboard.

Development workflow: 

  • Reviewer should not approve until successful
  • Make sure to rebase with respect to matter
  • Start policy to ensure successful Jenkins run prior to merge of faro PRs.   
  • Need to routinely ship faro metrics to SQuaSH and display on chronograf dashboard
    • Proposed approach to teach Yusra to run
    • How do we make this automated? Need a script to automate. We can help Yusra to provide information about what needs to go into the script.
    • For the metrics of interest, the definitions exist.
  • Simon Krughoff to schedule a time to meet with Yusra or designate from DRP team to see how to to ship faro metrics to squash.   
CMRCurrent draft of DMTR-311
  • Jeff spoke to Yusra and they decided to use w14 for v22.0.0 and backport the scarlet fix. Yusra wants to see a direct comparison of faro metrics on direct Gen3 processing and Gen2 to Gen 3 conversion version.
    • However, faro won't run on the converted repo with changes made for external calibration support. Eli is working on a fix.
    • Yusra is interested to confirm that Gen2 to Gen3 is equal to or slightly better than the native Gen3 version (Gen3 version does not yet have all the features of Gen2 to Gen3 conversion version)
Bugs, issues, etc of the weekAll 

Review of any issues that have come up during the week with faro and metric computation 

 : do we do this in faro?

What is the status in faro of these old validate_drp tickets: 

: this error appears to have been propagated to faro.

  •  to fix DM-16555  in faro . Can we get this fixed before issuing the CMR
  • Jeffrey Carlin to investigate and decide if relevant for faro  
  •  - do the filtering before assembling the matched catalog 
Status of reprocessing 
  • What versions are we reprocessing now? 
  • What will be the weekly version for R22.0.0?
  • Complete CMR for R22.0.0 – see notes above for w14
  • w18. One of the tracts failed in the coadd step. Monika offered to re-run. 2 of 3 tracts were fine. Third made single-frame but not coaddition step.
  • w22 processing is already running. Note that since Eric Morganson left the project, Morgan Schmitz is trying to get up to speed with RC2 processing.
  • Simon is getting close to have small 6 chips in 9 visits for 5 bands. Successfully processed all through coadd except g band. Picking better visits should fix. All using butler import/export tools. Native Gen3. This would replace validate_drp_hsc. Size is 14 GB including reference catalogs, calibrations. Repo contains raws. Simon has done the fix to get reference catalogs from only the relevant portion of the sky.
Development status 
  • Spring 2021 epic was intended to be finished 31 May 2021
  • Next week could be an opportunity to do a comprehensive review of current epic and make priorities for the next several months
    • One high-priority item would be using object table and source table (parquet format)

Review progress on the current development epic

Backlog epic: 

  • Keith Bechtol Check to see whether we can access the schema of tract object tables (who?)  
AOBNext meeting 2021-06-15.


List of tasks (Confluence)