Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


ItemWhoPre-meeting notesNotes and  Action Items 
News, announcements and task review
  • Keith has run timing tests on loading metrics into memory via the RSP 
  • Question from first-look analysis discussions: we put information in butler, but then how do we get this information to the observer? Is there an all-done signal returned from OCPS? Do we envision routine monitoring / querying?
  • Commissioning test planning, based on DM experience
    •  write tests "just-in-time". 
    • mapping of system-level test cases to test plans and tying these to milestones
    • have test plans for data taking on the summit
  • Erik: LATISS vs ComCam dataset. For illuminated images, ComCam is a bit easier to use since AuxTel requires active summit support for clearance on slewing the telescope and turning on the lamp. ComCam is always available unless otherwise under testing/maintenance and the lamp is left on. For darks/biases, both ComCam and LATISS are regularly available. 

Timing tests 

  • 30 mins to just load metrics into memory in the RSP. 
  • With w_2021_33 on RSP, takes 10 seconds to queryDatasets for 48K measurements.
  • Takes 1m52s to load 10K Measurements into memory.
  • What about using parquet files? We don't have to use lsst.verify as the im-memory storage. Could use astropy tables. Simon thinks yaml parsing is the bottleneck.  Metrics computed on small scales will inevitable result in  irreducible I/O. 
  • Discussion to date has been informal. This is really a request with a commissioning hat on. 

sandbox dashboard

  • manage with tags in the interim 
  • sasquatch product from Angelo  will unify squash/efd. 

define faro failure status task:

  • marked as done – Jira ticket 
  •  Keith Bechtol Leanne Guy to start to plan verification activities for commissioning.   
  •  Keith Bechtol writes a ticket to DM architecture to describe need for loading metric values in memory efficiently. Start by looking at the use cases.  

Bugs, issues, topics, etc of the week

  • mypy might be tricky as we have to add type hints. Not mandated but new packages are using them. 

Reprocessing status and metrics review  

  • Rename jenkins job to "verify_drp_metrics". 
  • Verification dataset to be renamed "rc2_subset". 
  • Simon can now set up jenkins jobs and data repository sorted ASAP – expect to have metrics from CI & new dataset for next meeting. 

PDR3 processing 

  • Planned for in the coming months 
  • Focus on a more complete version of faro for this processing 
  • PDR3 paper ... link
  •  Leanne Guy arrange to disucss at a future meeting if there are metrics from PDR3 & this paper that we might want to include in faro.   
Development status 
  • Fall 2021 epic 
  • Backlog epic: 

PR for faro documentation, 

both conceptual overview and how to use

  • Is there a list of HSC RC2 reprocessed repos to point to?
  • What should go in the package README file? Probably this should point to the package documentation on
  • Should we include documentation on dispatching metrics?
  • Getting high-level documentation to appear in regular documentation build Currently a stub for high-level documentation appears in daily documentation build
  • Sufficient work to start a review now

Co-work for development: 

  •  Keith Bechtolto set up a doodle poll for a co-work session for faro development  

 No meeting next week due to JDR. Next meeting:  

Agenda page for JDR: