Date & Time

Date  , NCSA-3000, 9:00 - 17:00 PT

This meeting precedes the DMLT F2F meeting


BrowserRoom SystemPhone Dial-in

  1. Dial: or
  2. Enter Meeting ID: 103664856 -or- use the pairing code

Dial-in numbers:

  • +1 408 740 7256
  • +1 888 240 2560 (US Toll Free)
  • +1 408 317 9253 (Alternate Number)

Meeting ID: 103664856



Pre-meeting planning

Please add ideas to the table of suggested topics

Questions to be addressed in each topic: 

  • Where are we now and what do we need to do to get where we want to be?
  • How are we going to verify and validate the algorithmic performance (e.g requirements and anything missing for requirements)?
  • What are the "success criteria" for these algorithms?
  • What, if any studies do we need to do to (e.g SST study, engage science collaborations, interact with other outside groups)?

Agenda:  room = NCSA 3000

TimeTopicChairNotes & Actions
9:00 - 9:05WelcomeLeanne Guy
  • Review agenda and goals of the meeting
9:05 - 9:15Jira for DM Science
  • Review of new Jira based tracking of SST work 
    • The processes as described at DM System Science Team Work Methodology will be implemented in full starting S19. 
    • DM Science team work can be tracked on the DM Science Kanban Board
    • All stories in  DM-19345 - Getting issue details... STATUS  should be closed. For liaison stories - please add an estimate of how much time you spent interacting with the SC and describe any activities
    • Storied in  DM-18608 - Getting issue details... STATUS  should be closed or moved to  DM-19987 - Getting issue details... STATUS
  • All DMSST to move their incomplete and ongoing tasks from f19 epics to s19  epics Leanne Guy  
9:15 - 10:30Difference Imaging 
  • A&L algorithms vs ZOGY?
  • Both algorithms are implemented. What do we need to do for validation?
  • DCR correction?
10:30 - 10:50Break
10:50 - 12:30Deblending and crowded fields
  • How do we know that we have succeeded in deblending (definition of success).
    • What do we need for the ORR and what is needed for the longer term survey
    • Machine learning for source measurement and/or deblending.
  • Crowded fields
    • Update from PST meeting discussions and discussions with SMWLV group (Leanne)
      • As described in  PST-77 - Getting issue details... STATUS  following consultation with the SMWLV SC and at the PST meeting, DM leadership has been mandated to try to identify resources to improve the performance of LSST stack pipelines in crowded fields. 
      • Zeljko Ivezic suggested we present the DM plan at the 2019 PCW
      • TVS SC has set up a crowded fields task force
    • How could we show that we are making progress and not deferring this indefinitely
12:30 - 13:30Lunch
13:30 - 15:00Deep object measurement 
  • What is the status of the current thinking on Cell based coadds vs  Multifit vs ?

  • Does this impact the coadds we would build?
15:00 - 15:20Break
15:20 - 15:50Background fitting
  • Different science cases have very different needs for background modeling and subtraction
  • How do we support this wide variety of (possibly conflicting) requirements?
15:50 - 17:00Verification needs
  • What do we need in terms of verification, testing, QA to get the pipelines to a well functioning integrated system
  • How can we involve external stakeholders  (e.g science collaborations). 
    • Small trials with science collaborations to put in place processes 

Session Notes

(Please correct if I have misrepresented your statements)

Image differencing  & DCR

  • paraphrase: Main thing is to evaluate whether we hit reqs based on AL, if not we can go onto Zogy.
    • KSK: Preconvolution isn't yet working, needed to make deconvolution work
    • Would be nice to do a bakeoff, but we should focus on getting AL successful as higher priority.
    • Eric: "Main focus is to get in a position to hit requirements as soon as possible, with just one algorithm"
  • LPG: Could we do some sort of monthly-roll up, i.e. what Meredith is generating manually? Could load the monthly run? 
    • KSK: There is a templated notebook system, meant for periodic reports. Would that be useful for AP?
  • Eric shows 3 Decam fields of detections, show false positive features. Patch-sized problems have been fixed recently.
    • ZI: Are these false positives a strong function of SNR? EB: Good question, don't know
    • EB: Edge flag needs to be used to throw away those sources, but just hasn't been added to the query yet.
    • DMTN-21 getting ~50 det/ccd, but current hits seems to be ~on average ~hundreds per ccd.
  • RHL: Brighter Fatter? Not being corrected right now. Means the kernel is being built from bright stars and then applied to the faint stars.


  • RHL: What happens if you run Ian's code on a telescope with an ADC [as a null test]? Good question, haven't tried. 
  • EB: Current opsims often producing pessimal image sets with the same airmasses, but opposite sign of the parallactic angle. Gives no leverage for fitting.
  • Discussion: Current DCR code doesn't handle PSFs that vary between input images into the coadd. Because it's pixel-based, the pixels don't know about their objects or their PSF.
  • Haven't gotten to RB yet. "What we need to train a RB model is the same as what we need to show we're meeting our requirements, i.e. a training set". Will need a larger dataset.
  • RHL: have to understand the full system before we go on sky, can't pick off bits and pieces. Need to understand non-dcr data before we understand dcr data. 
  • MWV: Focus on good templates with ~12 images?
    • RHL: That's the best case, we can do that before doing the harder part of small N. Learn different things from the two cases.
  • LPG: DESC is interested in joining the image differencing effort, how can we help this?
    • MWV: DESC definitely has energy on this, but are they going to inundate the AP team with questions/problems.
    • JB: "We're already under a lot of known-issues, just finding more issues doesn't necessarily help".
    • JB: DESC has a lot of decam experts, could they help getting obs_decam and calibrations in better shape, or is that too indirect for DESC to care. MWV: Software dev people are more from the French/CFHT side. 
    • Robert G: Still working on Decam issues from the DES side. Could reprocess the HITS fields with the DESDM, see if the same issues remain/is it something with the CP calibration side.
  • MWV: Could produce a coherent report on the DC2 image differencing


  • SDSS deblender: Spatial symmetry constraint.
  • Scarlet: Assumes each component in a source has the same morphology across bands, but different fluxes.  Includes spatial symmetry and radial flux monotonicity constraints.
  • KSK: What about objects with multiple physical components, bulge+disk.
    • JB: Need a way to tell it that that is ok, but theoretically it could.
  • Without constraints, failure mode is the big objects eat the flux from all their neighbors. Some concerns about what monotonicity means, implementation-wise. Still working on that.
  • Currently identifying peaks and then deblending with fixed set of peaks, but doesn't necessarily have to be that way (but other ways are probably harder).
  • Tried implementing some algorithms that don't work on the deblended pixels, only the direct pixels in the very central region of the peak. Good for colors for photo-z, but no hope at getting the total flux right.
  • LPG: slide says not the best for all cases, examples?
    • JB: Works best for weakly blended galaxies, wrong one for stellar fields and galaxy clusters.
  • Biggest cost probably comes in the per-step convolution process. Need to reduce the number of steps by having better initial guesses.
  • JB: Would like to have the computational performance optimization first, too hard to do scientific optimization now when the runtime is bad.
  • KSK: Mentioned earlier that if we don't need multifit then we can give that back to deblending
    • JB: And scarlet will happily eat all of that compute.
  • LPG: Need to get metrics to know whether we actually fit in the compute budget, especially w.r.t. multifit.
  • LPG: Has there been work on GPU?s
    • JB: Not so far, some of the other projects have been using frameworks like tensorflow to enable easy switching to GPUs. Would be super interesting for someone to try a GPU implementation.
  • KSK: Could we make this pluggable, or is the interface not definable enough?
    • JB: Could be, but worried that there might still be incompatibilities.

List of SST tasks (Confluence)

DescriptionDue dateAssigneeTask appears on
  • Robert Lupton Clarify the meaning of time in the object table. 1 sentence description in sdm_schemas, can link to a short DMTN.  Update 2022-02-09: Meeting to resolve this on 2022-02-21  
28 Feb 2022Robert Lupton2018-11-05 DM SST F2F Agenda and Meeting notes
  • Gregory Dubois-Felsmann check if SDM standardization is adequately represented in project documents, and whether DMTN-067 should be required.
31 Mar 2022Gregory Dubois-Felsmann2022-02-14 DM-SST Virtual F2F Agenda and Meeting notes
28 Feb 2023Leanne Guy2023-01-23 DM-SST Agenda and Meeting Notes
  • Leanne Guy talk to Steve R about presenting plans for the ShearObject table to PST and SciCollab chairs   
20 Mar 2023Leanne Guy2023-02-27 DM-SST Agenda and Meeting Notes
31 Mar 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  • Leanne Guy  talk to Gregory Dubois-Felsmann to review the original intent of the AFS-related Portal requirements before deciding on a course of action  
29 May 2023Leanne Guy2023-05-01 DM-SST Focus Meeting - Brokers in Commissioning
  • Leanne Guy Prepare to consult the PST on the question of providing compressed PVIs for AP outputs, to cover the period before the data become available in a DR.  
02 Jun 2023Leanne Guy2023-03-27 DM-SST Agenda and Meeting Notes
  • Jim Bosch Incorporate 30-60 day period for raws on disk into the strawman proposal and present to KT  
26 Jun 2023Jim Bosch2023-05-08 DM-SST Agenda and Meeting Notes
  • Parker Fagrelius Patrick Ingraham  how long will it take to do a scan as described? No need to scan the whole WL range but will require additional points outside nominal lambda range.  
30 Jun 2023Parker Fagrelius2023-03-27 DM-SST Agenda and Meeting Notes
31 Jul 2023Colin Slater2023-07-10 DM-SST Agenda and Meeting Notes
  • Eli Rykoff , Leanne Guy  Develop a proposal for what calibration processing, hardware, data we actually need and what will be needed for DR1. This has implications for the ORR and for prioritisation of work in commissioning  
31 Jul 2023Eli Rykoff2023-01-30 DM-SST Agenda and Meeting Notes
  • Yusra AlSayyad will look to see if there is any effort to help on option 1  
28 Aug 2023Yusra AlSayyad2023-08-14 DM-SST Agenda and Meeting Notes
  • Jim Bosch  Provide a physical example of that a  up on cell table would look like fo the Colin Slater / DAX team to review  
31 Aug 2023Jim Bosch2023-02-27 DM-SST Agenda and Meeting Notes
  •  "What is the pathway to defining the data products that are required to meet DMS-REQ-0266" Jeffrey Carlin   
30 Nov 2023Jeffrey Carlin2023-10-23 DM-SST vF2F Agenda and Meeting Notes
30 Nov 2023Gregory Dubois-Felsmann2023-10-23 DM-SST vF2F Agenda and Meeting Notes
30 Nov 2023Leanne Guy2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  • Jeffrey Carlin follow up with KT on DMS-REQ-0176 and DMS-REQ-0315 to update/disaggregate this for latest base/summit infrastructure split.  
30 Nov 2023Jeffrey Carlin2023-10-23 DM-SST vF2F Agenda and Meeting Notes
  • Jim Bosch Follow up on the possibility of investigating further the ability to process 2 collections in parallel.   
31 Jan 2024Jim Bosch2023-12-04 DM-SST Agenda and Meeting Notes
31 Jan 2024Jeffrey Carlin2023-12-04 DM-SST Agenda and Meeting Notes
Gregory Dubois-Felsmann2023-10-23 DM-SST vF2F Agenda and Meeting Notes

Overdue or Undated DM Science Team tickets

Key Summary T Created Updated Due Assignee Reporter P Status Resolution

LIT tickets of interest to DM Science

T Key Summary Assignee Reporter P Status Resolution Created Updated Due