Date

 

Location

Zoom alias : https://ls.st/rsv

Zoom:  https://noirlab-edu.zoom.us/j/96120781994?pwd=bCtpT3Q3b2RsU1ZBRUFaSnUrZXo3Zz09#success


Attendees

Regrets


Useful links

Metric tracking dashboards

Rubin science pipelines weekly change log and  Github repo: https://github.com/lsst-dm/lsst_git_changelog

Status of jenkins jobs and slack channle for notifications:  #dmj-s_verify_drp_metrics slack channel 

Discussion items

ItemWhoPre-meeting notesMinutes
News, announcements and task review

All

Welcome to the last meeting of 2021 

Faro paper - our proposed paper has been accepted. Abstracts due to the project by 2021-12-20. SPIE abstract submission 2022-01-05

Peter and Erik to be added to Project author list database.

SPIE conference in Montreal. Requested a talk and paper.

Slack channel will be open for business, but no meetings.

CI status, bugs, issues of the week

All
  • verify_drp_metrics CI jobs
  • Check variation in metrics seen last week 


Jeff  did some more exploration as to why the metrics are changing nightly. Did 2 back-to-back rc2-subset. Differences in coadds. Subtracted the images from each other → pixel level differences (and obviously diff number of sources in the catalogs) FGCM zeropoints. Model written out by FGCM each time is different. 

Are the visit-level images are identical? Need to check this

Seems to be some non-deterministic in the calibration. 

Not sure whether it is random aspect of the algorithm, or related to the order of inputs.

Astrometry does not change but was see nightly fluctuations in photometric measured. 

TE1/2/Stellar locus are measured on coadds - observed nightly variation is symptomatic of non-determinism in the zeropoints since those are related to weights when building the coadd.

Jeff will reach out to Lauren to coordinate. 

Jeff will check the config files that were saved in butler to see if the FGCM random seed is set (and to what)

Faro Development status 
  • Fall 2021 epic
  • Backlog epic: 
  • Progress on conversion to parquet
  • TeX conversions

PF: Match-probabilistic folded into some of the parquet-based metrics. Refcat loader returns a NP array. Need to convert format.  

Faro is using the new RefCatLoader so we have some freedom to adapt it directly.

Peter and Keith will look into this. 

Need to review selecters to filter the catalogues - e.g SNR and how we apply this → cowork 


Data processing campaigns  

  • RC2/DC2 reprocessing epic :
    • w_2021_40 RC2:  
    • w_2021_44 DC2 :
  • AuxTel
  • AP/DiffIm
  • AuxTel Imaging Processing
    • we have advanced to using BPS to do batch processing of data, performed some zoology on pipetask failures. Quick summary: for 266 images from the night of Nov 2nd sqr degree run (includes some intermediate wavefront sensing images), 108/266 passed characterizeImage task, 70/108 passed calibrate task (includes rough photometric and astrometric calculations). Common failures identified, next step is to explore configuration tweaks and add table generation (writeSourceTable, transformSourceTable) to BPS job. 
    • Got notes from Lee Kelvin on how to create discrete skymaps needed for co-addition further down pipeline
    • LATISS is currently warm following a series of power outages on the summit, December run was cancelled and January run is at risk but no decision has been made. Hope is that in the next available run we will install r and i filters to get multi-band coverage on the same 1sqr degree field. 
    • There is an outstanding task to add ATLAS REFCAT2 as reference catalog in order to include photometric calibration in single-frame processing:
DP0.2

Faro for DP0.2


AOB

Potential co-working session ideas here


Next meeting 18 January 2022


List of tasks (Confluence)