DP0 general information:
- SLAC channel: #ops-data-previews
- Meetings: Very optional biweekly meetings 8am Tuesday
- Deadline: Pipeline for DP0.2 freeze June 30th 2021. (Patches for correcting performance of already extant components will be accepted for a couple months after)
Accounting update for Summer:
- Last call for story points on March/April/May: PREOPS-450.
PREOPS-450Getting issue details...
- Scheduled AP folks: Ken: 32, Ian: 7.5, JohnP: 15, Chris: 15 :thankyou:
- Epic for June/July/August: f21a-a&p-preops
PREOPS-477Getting issue details...
- Scheduled: Assume same as PREOPS-450
Only Variable Sources in WFD are SNe: https://arxiv.org/pdf/2010.05926.pdf
Good for SNe, and validating that everything else is flat
https://community.lsst.org/t/shared-gen3-data-repositories-ready-for-some-use/4845 This includes all of the DESC DC2 DR6 WFD raws and processing that will be used in DP0.1 (it’s a clone of the IDF repo - or rather the converse). That includes the original calibs used for the DESC processing. The dataset processed approximately monthly by DM is a subset of this, and there are two TAGGED collections that contain the raws for these important subsets:
2.2i/raw/DP0: raws for DP0
2.2i/raw/test-med-1 (DM monthly processing subset: tract 3828/3829
The full DESC-run DR6 WFD processing can be found in the 2.2i/runs/DP0.1 collection.
Pending reruns as of:
w_2021_14 HSC rerun will have best seeing templates
w_2021_16 DC2 rerun will contain difference images and diffim lightcurves
- New for June 8:
What's left as of April 26 2020:
- Are false positive rates on DC2 reprocessing as expected?
- Rerun a tract using DR6. What’s a suitable template? Check with Bruno on what they did
- Weird diasource centroids off footprint and off chip (DRP will look at this in pair coding again)
- Ability to handle science images with seeing better than the templates. Decide whether to
- deconvolve anyway
- swap science and template
- Associate DiaSources into DiaObjects (At the very minimum do something (DONE) like: https://github.com/LSSTDESC/dia_pipe/blob/master/python/lsst/dia/pipe/multimatch_association.py)
- and then rename Objects to PreliminaryObjects and associate DiaObjects + PreliminaryObjects = Objects? (DONE)
- DOES NOT include variability stats for each of the objects (just centroids to seed forced photometry)
- CBM: One comment, centroiding a DiaObject is, as I referred on the call, a DiaObject summary statistic so we will be expecting to compute at least a few summary stats on the DiaSource history according to the above statement.
- Some summary stats.
- Use CoaddPsfs from all warps in getTemplate task
- Get data from multiple-tracts for complete templates.
- Compare the calexp lightcurves vs. the diffim lightcurves. Does this behave as expected? Is dependence on seeing as expected?
- Ensure that whatever we do for association eventually gives us all diaObjects forced phot on coadds too.
Standard Data Model:
- ForcedSource SDM specifications (including psFlux and psDiffFlux)
- Visit Table
- CcdVisit Table
- New as of June 8:
- Instructions for how to build a pipeline and run it on bps. (Meredith)
- Datasets to run:
- 1.5 year depth is run by Monika 3828, 3829 DRP pipeline
- 6 year depth is up to us to run 4431 DRP pipeline
- Implementation of RFC-775 (all)
- Look into slot_shapeFlag (Ken)
- QA on w24 DC2 rerun (Eric)
- QA on w24 (adapt Meredith's notebooks to use parquet tables)
Minimum Viable Product version would be Bob Armstrong’s pipelines: https://github.com/LSSTDESC/dia_pipe