The Science Pipelines team meets every Wednesday at 1pm Pacific Time/4pm Eastern Time to discuss various topics of interest. This page aggregates the presentation material from the past meetings.

https://princeton.zoom.us/my/yusra

Team Meeting Hosts

DatePresenterTopicLink

 

A deep dive on Detection and Measurement (continued)

 

NO MEETING

No meeting week

 

Everyone! Keith Bechtol can organize

Ops Rehearsal 3 Debrief

KB: suggested topic

Operations Rehearsal for Commissioning #3 Realtime Lessons Learned

 

Plotting feedback

 

A deep dive on Detection and Measurement

detection and measurement FAQs: https://docs.google.com/document/d/1RiQ9QfMGnKOSpDBvPxXoGIGCHzrgSI2EJhCtZXbIRkE/edit

Recording

 

Pierre-François Leget Formal introduction and summary of his previous work

 

Trys McCann Discussion about synthetic source injection for regular RC2 reprocessinghttps://docs.google.com/presentation/d/1HDH40Sc8wnFT2_srgzazTSFzyiEd-wN_IsU_g-yJDss/edit?usp=sharing

 

ComCamSim Pipeline Sprint Demohttps://docs.google.com/presentation/d/1IkCc9MKpp4aEpzqzq6VsRIlgay62305BqhEm1JT6gW8/edit?usp=sharing

 

Dan Taranu Update on MultiProFit and galaxy photometry/sizes

 

Arun Kannawadi Discussion about moments-based star/galaxy classifier

 

Analysis tools sprint demoCalibration/analysis tools sprint slide deck

 

CANCELLED for JTM

 

Alex Broughton BF mitigation strategy improvements

 


analysis tools sprintingKick off is during paircoding, and there's no talk planned, so we can just keep on co-working. Slides.

 

Pre-mortem II (continued)

Reviewing proposed mitigations:

https://docs.google.com/spreadsheets/d/1EvQL3lexsiVm5ZT4t7R4_nbTnFCcnewH48OfN2Es3CU/edit#gid=0

 

Pre-mortem II (continued)

https://app.sli.do/event/471uR19pwi6xt7GDJf44QE

Onward with reviewing December's brainstorming and turning the items into risks:https://docs.google.com/spreadsheets/d/1EvQL3lexsiVm5ZT4t7R4_nbTnFCcnewH48OfN2Es3CU/edit#gid=0

 

 

NO MEETINGHOLIDAYS

 

Yusra AlSayyad Pre-mortem IIhttps://docs.google.com/presentation/d/1PcF4yJvg3MTtPiP9aU3rHkQJQkyAIlH93B6AfoTFiZ0/edit?usp=sharing

 

Aaron Watkins Modifying SkyCorrectionTask to be low-surface-brightness compliant;  Developing skyCorr metrics using source injection. 

 

Eli Rykoff Calibration Strategy incl New ISR, a.k.a. Calibpalooza Plan. 

 




 

NO MEETINGTHANKSGIVING

 

Yusra AlSayyad Rubin's Hybrid Cloud strategy. https://docs.google.com/presentation/d/1qcOjZ5JaSiA5YuX_fZyB-tiVjk3qeReeB8FkTIOketw/edit?usp=sharing

 

All + Sierra and OrionCampaign Management and BPS gripes, tips, and show-and-tell. 

 

AllLATISS Config update demohttps://docs.google.com/presentation/d/1pJdewTyQEGHhivbBpbulAzXCD6Kqnvlgii4_ajce9kQ/edit?usp=sharing

 

NO MEETINGLATISS Config Sprint Cowork

 

Tour of LATISS data products

Anyone who has tips and info on the LATISS processing runs are welcome to show and tell

IntroToAuxTelLatissProcessing_18Oct2023.pdf

LatissPromptProcessing202310.pdf 

Prompt Processing with AuxTel Imaging Survey Data 2023

Ian Sullivan  & AP Team

metrics sprint closeoutCloseout slides

Amir Ebadati-Bazkiaei (In-Kind contributor from Macquarie University, Australia)

Bright Star Subtraction in the Science Pipelines

Link TBD

(I think this overlaps with the Butler/Client Fest - If this slot is required for that, we're happy to move to next week instead)

 

NO MEETING WEEK

 

Viraj Karambelkar (Visiting Princeton from Caltech) Data-processing pipeline for a new infrared time-domain surveyor - WINTER

 

Riley Clarke (visiting Princeton from U of Delaware)

 

Andrew Saydjari (visiting Princeton from Harvard)Photometry and background-related uncertainties in DECaPShttps://ui.adsabs.harvard.edu/abs/2022ApJ...933..155S/abstract

 

John Parejkoobs package config overrides and DM-39796

 

No meeting

 

allpost-PCW discussion

 

No meetingPCW

 

Yusra + everyonePractice run for Updates from Science Pipelines for PCWhttps://docs.google.com/presentation/d/1hQvlRA5_bNz3wgtjD1kf8FRErH4y-vVSnLQm7TDE-Og/edit?usp=sharing

 

Yusra office hours for PCW talk

 

Yusra (+ Jim)Major components left to write for the DRP pipeline

 

Jim Boschthe QuantumGraph generation algorithm, why it's slow, and how to speed it uphttps://docs.google.com/presentation/d/1HO7RHMgf4kUqD5bn-Hu2gZcJcraecwgN6Ts2yljFDGs/edit?usp=sharing

 

John Parejko, allMaking the next release not be preceded by "drop everything and get your deprecation removals in"

 

John ParejkoNew CalibrateImageTask

 

No meetingMeeting free week

 

Arun KannawadiSingle-visit star-galaxy classificationslides

 

No meetingMany people are celebrating Lupton or striking.

 

@fredChi**2 coadds

 

No meetingEveryone is bootcamping.

 

Keith Bechtol Commissioning update

slides

Commissioning Science Validation Bootcamp welcome slides

 

analysis_tools sprint demo

 

John Parejko Data model changes from RFC-906

 

Never allocate nodes again

 

LSSTCam updateslides

 


MEETING FREE WEEK

 

Bruno Sanchez Transient survey science: from pixels to cosmology: Part IIslides


Transient survey science: from pixels to cosmologyslides

everyonePost-Chile discussionSlides




 

Yusra AlSayyad Prep for JTM

Slides

 

Bruno Sanchez 
Bruno couldn't make it, so we talked about the upcoming Chile JTM.

 

Agnès Ferté Weak lensing and clustering cosmology with DES

 

Ari Heinze Methods for fast and accurate classification of difference image sources without machine learning


 

Iterative Single Frame Measurement and Global Calibration

 

Trys McCann (she/her)Sophie Reed Lee Kelvin 

Update on synthetic source injection Announcements

 

Sorting items needed for pipeline commissioning. Followup from Dec 14. 

Announcements

Commissioning Slides

 

Yusra AlSayyad News sharing

 

No meeting.Enjoy the break.

 

Brainstorm required data products to complete commissioning (follow up Nov. 30)Snapshot from Dec 2022

 

Aaron WatkinsLow surface brightness galaxies and sky subtraction

 

Keith Bechtol Planned schedule / activities for on-sky observationsslides

 

José Sánchez-GallegoSDSS V

 

Keith Bechtol Kick-off meeting for SIT-Com Science Unitsslides

 




 

Princeton visit debriefing

parsl-visualize demo (time permitting)

 

Nima Sedaghat Machine Learning topics

 

Brianna Smart The Diffuse Ionized Gas of the Magellanic Clouds

 

Erfan Nourbakhsh Unrecognized blending and cosmological inference

 

Erik Dennihy Batch Processing AuxTel datasets

 


NO MEETING

 

John Parejko A discussion on papers that should be written before/at the same time as DR1.

A short report on PFS wavelength solutions (which share some similarities with HSC/DECam astrometric solutions, hence the possible Rubin interest.  Plus it's cute).

Brianna Smart The Diffuse Ionized Gas of the Magellanic CloudsTO BE RE-SCHEDULED

Red galaxy errors

PCW decompressionJust discussion. Imagine it's Thursday night after the PCW. Your talks are done, you just had dinner, and are in the pool floating around with the rest of the pipelines folks. "What's up?"


PCW. NO MEETING

Practice PCW talk: Update from Science Pipelineshttps://docs.google.com/presentation/d/1aSD44xND8WMPbhaQ0gDXSXn17PvtQg0NvjJIjTI2Ygc/edit

Compensated Filters in Practice

 

Eli Rykoff Brief obs_decam Gen2 Deprecation Update

 

1) New output data products from the multi-band deblender
2) Refactoring drp_pipe

 

Source Injection with the Science PipelinesSource Injection with the Science Pipelines ticket.

 


DM Pipelines Bootcamp

 

Nate Lust New Analysis Tools Design Presentation

https://github.com/lsst/analysis_tools

Confluence to collect feedback: Analysis Tools Prototype Design

 


NO MEETING WEEK

 


NO MEETING. DMLT WEEK

 

John Parejko AP Runtime Characterization Sprint Demo

 


Emergent Issues

 

Lee Kelvin Unit tests for drp_pipe

 

History Of Background Estimation in the DRP Pipeline

 

Jim Bosch A Guided Tour of QuantumGraph Generation
27 Apr 2022

Leanne Guy

Interactive plotting in GaiaPostponed (date: TBD)
20 Apr 2022

Chris Waters 

Verification (cp_verify)cp_verify overview-DM.20220420.pdf
13 Apr 2022

John Parejko 

Krzysztof Findeisen 

 "The Prompt Processing Prototype" 

Eric's summary slides
06 Apr 2022

Yusra Alsayyad

Kian-Tat Lim

"Update from the third USDF workshop"

Workshop agenda

KT's talk

 

Sophie ReedAnalysis_drp Plotting Sprint DemoSprint Demo Slides

 

Ian SullivanDifference Imaging refactorImage Differencing refactor March 2022.pdf

 

Yusra AlSayyadPremortem Follow-upTrello Premortem Link

 

Meredith RawlsSatellites: IAU CPDQSSCI (lol), ZTF, and the Sun

 

Clare SaundersThe Present and Future of DM Streak Masking

 

Jim BoschConfiguration lessons learnedConfiguration is Hard Slides

 

Dan TaranuObject-to-Truth matches 

 

Jim BoschPlanning for Gen2 RemovalGen2 Middleware Removal Planning

 

Much ado about noise

DM-33320

 

Lee Kelvin's vistor Aaron WatkinsLow-surface-brightness ScienceSky subtraction Optimized for LSB Observations Slides

 

Yusra AlsayyadPremortem

 

Alex Amon (visitor)Weak lensing measurements & calibration

 

Lauren MacArthurDP0 processing plots

 

Robert LuptonNoise in Detectors Supporting non-destructive Reads (CMOS; HgCdTe)

 

Yusra AlsayyadProject updates

 

Spencer NelsonAP DC2 profiling

 

Dan TaranuDRP DC2 plots

 

Arun KannawadiShape measurement uncertainties

 

Meredith RawlsSatellite update

 

Yusra AlsayyadFinal finalizing of the DP0.2 pipeline

 

Yusra AlsayyadDC2/obs_lsst pipeline changes/finalization

 

Yusra AlsayyadReview Rows/Columns in the DP0.2 output Parquet TablesParquet DRP Table & Column Reconciliation Notebook

 

Nate LustMemory Management in Python

 

Josh MeyersPSF estimation update

 

Dan TaranuScarlet's treatment of bright stars

 


PCW recap

 

Yusra AlsayyadPCW prep

 

Nate LustBPS interface updates




 

Arun KannawadiMind the GAaP




 

Robert LuptonMultiband detectionMaximum Likelihood Detection (ChiSq) Slides



Final finalizing of the DP0.2 pipeline (Final)
Sep 15:  Yusra on : https://github.com/yalsayyad/dm_notebooks/blob/master/object-table/Parquet_DRP_Table_and_Column_Reconciliation.ipynb
Sep 1: Nate on memory management in Python
Aug 25: Psf-estimation update (Josh on Piff; Dan on Scarlets’s treatment of bright stars
Aug 18:  PCW recap
Aug 11: Cancelled for PCW
Aug 4: Prep for PCW (Yusra)
July 28: DESC meeting review and BPS interface updates (Nate)
July 21: DC2 QA Sprint Demo Round 2 (Yusra)
July 14: 
July 7:  Cancelled for holiday week
June 30: Yusra: Results of the Replan/Review
June 23: DC2 QA sprint demo.
June 16: DC2 QA sprint kickoff
June 9: Eric Charles: NWayMatch
June 2: Keith Bechtol & friends: faro
May 26: QA comparison party (whoot whoot)
May 19: Possible follow-up on discussion from last week. Also, Arun will be showing more results from his GAaP plugin.
May 12th: Lee and Sophie - setting up a more permanent set of synthetics (aka fakes) to be continually processed. This will be ~5-10 minutes of presentation followed by 10-15 minutes of discussion.
May 5th: Mario Juric - Trailed source fitting.
April 28th: Clare - Astrometry
April 21st: Robert
April 14th: Sophie and Nate - New actions and plotting tools
April 7th: Fred - Scarlet update
March 31st: Discussion of RC3 dataset

March 24th: Yusra - DMLT updates
Announcements:
https://www.youtube.com/watch?v=41_VUIPwWAk
https://www.flickr.com/photos/slaclab/sets/72157718685044792/
Ranpal started ordering 325 tshirts and patches
Replan update
Last call if you want to learn how to release the science pipelines for v22. target for release candidate is w_2020_14. There will  be a 23.x for the stack release that we use for DP0.2. 
https://community.lsst.org/t/shared-gen3-data-repositories-ready-for-some-use/4845
GAAP continued…. (Arun)
DMLT updates (Yusra) https://confluence.lsstcorp.org/pages/viewpage.action?pageId=148111379

March 17th: Arun Kannawadi - Formalism & preliminary results from GAAP

March 10th: Lee Kelvin - Fake Source Injections Update
Announcements:


March 3rd:
Announcements:
Please sign up for stack club on https://confluence.lsstcorp.org/pages/viewpage.action?pageId=100179225
See Visual Identity guide: https://docushare.lsst.org/docushare/dsweb/Get/Document-37294/20210212%20Visual%20Identity%20Manual%20%e2%80%94V7.pdf
Fill out form to get a T-shirt: Friday, March 5, 2021.   
Telescope parts are being assembled. 
Major thank you to Nate, Eli, John, Lauren and Fred for getting fixes in for weekly w10 Gen3 rerun. 
Robert Lupton: Multi band detection https://lsstc.slack.com/files/U2JPLS84X/F025R7S40CA/chisq.pdf

February 24th: Cancelled 

February 17th: ctrl_bps Tutorial

February 10th:

February 4:
DRP: where we’re going https://lsst.ncsa.illinois.edu/~yusra/download/drp-activities-as-of-2021-01.png

January 27
Robert Lupton and Eli Rykoff to lead discussion of aperture corrections

January 20: Meeting Canceled

January 13
Robert Lupton: New background estimation. luptonCompensatedApertureFluxes20210113.pdf

January 6
What to do about the w02 RC2  (Yusra, <10 min, https://github.com/yalsayyad/dm_notebooks/blob/master/calibration/Jointcal%20with%20Gaia%20RefCat.ipynb
Jim Bosch: DMTN-172