Date

Location

Browser

Room System

Phone Dial-in

https://bluejeans.com/690803707

or 

https://ls.st/rsv


  1. Dial: 199.48.152.152 or bjn.vc
  2. Enter Meeting ID: 690803707 -or- use the pairing code

Dial-in numbers:

  • +1 408 740 7256
  • +1 888 240 2560 (US Toll Free)
  • +1 408 317 9253 (Alternate Number)

Meeting ID: 690803707

Attendees

Regrets

Metric Tracking Dashboard

URL:

Discussion items

ItemWhoPre-meeting notesNotes and  Action Items 
News, announcements and task reviewWelcome to Peter Ferguson
  • Peter will work with SITCOM. Milky way science background, DES. Delve science validation experience. 

Bugs, issues, topics, etc of the week

All
  • NaN values json issue
    • Change values that are output by faro in lsst.verify.Measurement objects? We likely still want to have NaNs internally for faro so this isn't an attractive option.
    • Sanitize values at time of creating of json files?
    • Pending updates to lsst.verify dispatching code?
  • Robert Lupton We have just started being able to take LATISS/ComCam calibrations and process them on the mountain including (v. soon!) checks for quality of the calibration products. Robert would like to produce some longer-term summary of how this goes (e.g. did the bias change slowly over the last month)
    • Opportunity to start executing this as a ‘campaign’ and use it to build processes for campaign management
  • faro soon to be added to ci_hsc_gen3 and ci_imsim. This ticket also includes many updates that help to homogenize and simplify arguments that are passed to measurement task run methods. Big thanks to Dan for these updates and to Hsin-Fang for suggestions on adding faro to CI testing. DM-31382 - Getting issue details... STATUS
  • Longer term remaining task to add faro to DRP.yaml? See this thread and notes from last week.
    • Would ensure that faro is run automatically with DRP processing
    • Jeff is implementing a version of this in  DM-31326 - Getting issue details... STATUS

faro could potentially run some analysis contexts at intermediate stages of DRP processing (e.g., metrics that only require single-frame processing). This would be closer to how we plan to actually run at scale.

  • NaNs issue
    • Likely requires a fix to lsst.verify. Need to create a ticket. Krystof may be the person to do the implementation in lsst.verify.
    • Also, related but distinct from the above, we have many NaNs right now because we need better validation dataset.
  • Leanne Guy  to follow up with Ian about who looks after lsst.verify  Update: Ian says AP can still cover lsst.verify and to add him as watcher 
  • Jeffrey Carlin  to create a ticket for Nan/None issue. Add Ian Sullivan as watcher. Ticket is here:   DM-31519 - Getting issue details... STATUS
  •  Calibrations
    • Getting close to being able to take daily afternoon calibrations with LATISS and ComCam. How are these changing over time? Aaron Roodman, Tony Johnson, Eric Charles, Jim Chiang, interested in this topic from camera side. Would like some representatives from SV side at this meeting. This is about montoring the performance of master calibrations over time (e.g., trending on months timescale). FAFF is more of a point analysis. ComCam flat field lamps are always available, but maybe would need to be done after hours so as not to interrupt daytime activities.
      • What diagnostics do we want out of camera?
      • How do we do the trending?
    • Lots of discussions going on right now around data flow through to visualization. This is one large data flow problem. 
    • Data flow at present:
      • Alysha runs script for biases and darks
      • Raw images are ingested in Gen3 repo on mountain and at NCSA
      • Processed with OCPS, Science Pipelines calibration products pipeline. Compute running on the mountain. Haven't yet figured how to transfer calibrations from NCSA back to mountain. We could attempt to do processing at both locations and compare. Are there logs?
      • cp_verify (just about working)
    • Need observing/logging system. Could do registry query by date. (aware of the problem)
    • Colin and Robert will represent this group at Roberts meeting with Camera
  • Erik Dennihy  to work out with mountain people if LATISS or comcam is better to use.  RHL wants a large enough dataset to start looking at calibrations   Update: For illuminated images, ComCam is a bit easier to use since AuxTel requires active summit support for clearance on slewing the telescope and turning on the lamp. ComCam is always available unless otherwise under testing/maintenance and the lamp is left on. For darks/biases, both ComCam and LATISS are regularly available. 
  • Faro in CI 
    • Will allow more comprehensive integration tests and be run with the main DRP yaml. 
    • More frequent running will give better insight ot the performance. 
    • Tested by Jeff - ticket will be merged shortly 
  • Faro in DRP.yaml and integrate it at different stages. 
    • Need different pipeline yaml files that correspond to the different stages. 
    • Will faro be a step 1b) or integrated into step 1) . CS thinks that HFC wants to integrate everything into a single step. KSK points out that it is computationally more efficient. 
    • DRP.yaml part works, Jeff needs to add scripts to run it in jenkins
    • Naming : need to provide a name
  • Jeffrey Carlin Ask HFC and execution what is the best way to integrate faro. Update: See this Slack post and subsequent conversation. Jeff will confirm that things are working and then pass info along to Hsin-Fang/Yusra.   

Reprocessing status and metrics review  

  • Status of gen3_rc2_subset . 8 visits per band, 5 bands = 40 visits. Central 6 sensors per visit. 3 patches that have some coverage from all 40 visits. Smallest dataset that has complete coverage in all 5 bands. Currently called gen3_rc2_subset.
    • Also being used to see tutorials on lsst pipelines io.
  • rc2_small?   15GB dataset.  100s when reduced
  • Jeff tried running already with this dataset - executes but not looked at the resutls 
  • KSK needs to update the jenkins scripts to run automatically and use a native gen3 repo. 
  • Leanne Guy Create tickets for KSK to update DMTN-91 to include details of this new dataset  and updating the jenkins scripts and configurations to enable running on the new dataset. 
Development status 
  • Fall 2021 epic  DM-30748 - Getting issue details... STATUS
  • Backlog epic:  DM-29525 - Getting issue details... STATUS
Monitoring calibrationsWe're just starting to be able to take afternoon calibrations on the summit, and to run cp_verify to see if they are good.  I'm just starting to think about how to report the results, especially the trends with time.  Is this a rôle for this group?
AOB

Next meeting  



List of tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Leanne Guy to talk to Science Pipelines (Yusra) about when do this transfer  
19 Oct 2021Leanne Guy2021-09-28 Science Metric Development Agenda and Meeting notes
  • Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
26 Oct 2021Leanne Guy2021-08-31 Science Metric Development Agenda and Meeting notes
  • Colin to ask about capturing ideas for improvement to the stellar locus algorithm   
30 Nov 2021 2021-11-09 Science Metric Development Agenda and Meeting notes
  • Jeffrey Carlin to review metric specification package organization and the relationship to formal requirements documents
Jeffrey Carlin2022-04-19 Science Metric Development Agenda and Meeting notes
  • Colin Slater to make a preliminary draft agenda for a workshop to clarify visualization use cases for science verification and validation
Colin Slater2022-04-19 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol Schedule a time to have focused discussion on verification package, potentially next status meeting
Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes
  • Keith Bechtol to make a ticket to better understand mapping of these camera and calibration products characterization efforts to verification documents and the focus of these efforts. Discuss with the SCLT
Keith Bechtol2021-09-14 Science Metric Development Agenda and Meeting notes