Date: Fri, 29 Mar 2024 14:33:37 +0000 (UTC) Message-ID: <1046870244.21902.1711722822318@confluence> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_21901_2136504728.1711722817307" ------=_Part_21901_2136504728.1711722817307 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The Science Pipelines team meets every Wednesday at 1pm Pacific = Time/4pm Eastern Time to discuss various topics of interest. This page aggr= egates the presentation material from the past meetings.
https://princeton.zoom.us/my/yusra
Date | Presenter<= /strong> | Topic | Link |
&n= bsp; |
|
||
&n= bsp; |
NO MEETING |
No meeting week | |
&n= bsp; |
|
||
= |
Plotting feedback | ||
&nbs= p; |
A deep dive on Detection and Measurement | detection and measurement FAQs: https://docs.google.com/docu= ment/d/1RiQ9QfMGnKOSpDBvPxXoGIGCHzrgSI2EJhCtZXbIRkE/edit | |
&nbs= p; |
Pierre-Fran=C3= =A7ois Leget | Formal introduction and summary of his previous = work | |
&nbs= p; |
Trys M= cCann | Discussion about synthetic source injection for = regular RC2 reprocessing | https://docs.google.com/presentation/d/1HD= H40Sc8wnFT2_srgzazTSFzyiEd-wN_IsU_g-yJDss/edit?usp=3Dsharing |
&nbs= p; |
ComCamSim Pipeline Sprint Demo | https://docs.google.com/presentation/d/1Ik= Cc9MKpp4aEpzqzq6VsRIlgay62305BqhEm1JT6gW8/edit?usp=3Dsharing | |
&nbs= p; |
Dan Taranu<= /a> | Update on MultiProFit and galaxy photometry/size= s | |
&nbs= p; |
Arun Kanna= wadi | Discussion about moments-based star/galaxy class= ifier | |
&nbs= p; |
Analysis tools sprint demo | Calibration/analysis tools s= print slide deck | |
&nbs= p; |
CANCELLED for JTM | ||
&nbs= p; |
Alex Broug= hton | BF mitigation strategy improvements |
BFE_ An ISR Roadmap.pdf<= /p> |
&nbs= p; |
analysis tools sprinting | Kick off is during paircoding, and there's no ta= lk planned, so we can just keep on co-working. Slides. | |
&nbs= p; |
Pre-mortem II (continued) | Reviewing proposed mitigations: https:/= /docs.google.com/spreadsheets/d/1EvQL3lexsiVm5ZT4t7R4_nbTnFCcnewH48OfN2Es3C= U/edit#gid=3D0 |
|
&nbs= p; |
Pre-mortem II (continued) | https://app.sli.do/= event/471uR19pwi6xt7GDJf44QE Onward with reviewing December's bra= instorming and turning the items into risks:https://docs.google.com/spreads= heets/d/1EvQL3lexsiVm5ZT4t7R4_nbTnFCcnewH48OfN2Es3CU/edit#gid=3D0 = td> | |
&nbs= p; &nbs= p; |
NO MEETING | HOLIDAYS | |
&nbs= p; |
Yusra AlSayyad&= nbsp; | Pre-mortem II | https://docs.google.com/presentation/d/1Pc= F4yJvg3MTtPiP9aU3rHkQJQkyAIlH93B6AfoTFiZ0/edit?usp=3Dsharing |
&nbs= p; |
Aaron Watkins | Modifying SkyCorrectionTask to be low-surface-br= ightness compliant; Developing skyCorr metrics using source injection= . | |
&nbs= p; |
Eli Rykoff<= /a> | Calibration Strategy incl New ISR, a.k.a. Calibp= alooza Plan. | |
&nbs= p; |
|||
&nbs= p; |
NO MEETING | THANKSGIVING | |
&nbs= p; |
Yusra AlSayyad&= nbsp; | Rubin's Hybrid Cloud strategy. | https://docs.google.com/presentation/d/1qc= OjZ5JaSiA5YuX_fZyB-tiVjk3qeReeB8FkTIOketw/edit?usp=3Dsharing |
&nbs= p; |
All + Sierra and Orion | Campaign Management and BPS gripes, tips, and sh= ow-and-tell. | |
&nbs= p; |
All | LATISS Config update demo | https://docs.google.com/presentation/d/1pJ= dewTyQEGHhivbBpbulAzXCD6Kqnvlgii4_ajce9kQ/edit?usp=3Dsharing |
&nbs= p; |
NO MEETING | LATISS Config Sprint Cowork | |
&nbs= p; |
Tour of LATISS data products |
Anyone who has tips and info on the LATISS processing runs are welcome t= o show and tell |
|
|
Ian Sullivan & AP Tea= m |
metrics sprint closeout | Closeout slides |
|
Amir Ebadati-Baz= kiaei (In-Kind contributor from Macquarie University, Australia) |
Bright Star Subtraction in the Science Pipelines= | Link TBD (I think this overlaps with th= e Butler/Client Fest - If this slot is required for that, we're happy to mo= ve to next week instead) |
&nbs= p; |
NO MEETING WEEK | ||
&nbs= p; |
Viraj Karambelkar (Visiting Princeton from Caltech) | Data-processing pipeline for a new infrared time-domain surveyor = - WINTER | |
&nbs= p; |
Riley Clarke (visiting Princeton from U of Delaw= are) | ||
&nbs= p; |
Andrew Saydjari (visiting Princeton from Harvard= ) | Photometry and background-related uncertainties = in DECaPS | https:/= /ui.adsabs.harvard.edu/abs/2022ApJ...933..155S/abstract |
&nbs= p; |
John Parejko | obs package config overrides and DM-39796 | |
&nbs= p; |
No meeting | ||
&nbs= p; |
all | post-PCW discussion | |
&nbs= p; |
No meeting | PCW | |
&nbs= p; |
Yusra + everyone | Practice run for Updates from Science Pipelines = for PCW | https://docs.google.com/presentation/d/1hQ= vlRA5_bNz3wgtjD1kf8FRErH4y-vVSnLQm7TDE-Og/edit?usp=3Dsharing |
&nbs= p; |
Yusra office hours for PCW talk | ||
&nbs= p; |
Yusra (+ Jim) | Major components left to write for the DRP pipel= ine |
|
&nbs= p; |
Jim Bosch | the QuantumGraph generation algorithm, why it's = slow, and how to speed it up | https://docs.google.com/presentation/d/1HO= 7RHMgf4kUqD5bn-Hu2gZcJcraecwgN6Ts2yljFDGs/edit?usp=3Dsharing |
&nbs= p; |
John Parejko, all | Making the next release not be preceded by "drop= everything and get your deprecation removals in" | |
&nbs= p; |
John Parejko | New CalibrateImageTask | |
&nbs= p; |
No meeting | Meeting free week | |
&nbs= p; |
Arun Kannawadi | Single-visit star-galaxy classification | slides |
&nbs= p; |
No meeting | Many people are celebrating Lupton or striking.<= /td> | |
&nbs= p; |
@fred | Chi**2 coadds | |
&nbs= p; |
No meeting | Everyone is bootcamping. | |
&nbs= p; |
Keith Becht= ol | Commissioning update | Commissioning Science = Validation Bootcamp welcome slides |
&nbs= p; |
analysis_tools sprint demo | ||
&nbs= p; |
John Parejk= o | Data model changes from RFC-906 | |
&nbs= p; |
Never allocate nodes again |
|
|
&nbs= p; |
LSSTCam update | slid= es | |
&nbs= p; |
|
MEETING FREE WEEK | |
&nbs= p; |
Bruno Sanc= hez | Transient = survey science: from pixels to cosmology: Part II | slides
|
|
Transient = survey science: from pixels to cosmology | slides | |
|
everyone | Post-Chile discussion | Slides |
&nbs= p; |
Yusra AlSayyad&= nbsp; | Prep for JTM | |
&nbs= p; |
Bruno Sanc= hez | Bruno couldn't make it, so we talked about the u= pcoming Chile JTM. | |
|
Agn=C3=A8s Fert=C3=A9 | Weak lensing and clustering cosmology with DES |
|
&nbs= p; |
Ari Heinze<= /a> | Methods for f= ast and accurate classification of difference image sources without machine= learning |
|
&nbs= p; |
Iterative Single Frame Measurement and Global Ca= libration | ||
&nbs= p; |
Trys McCann (she/her)Sophie Reed Lee Kelvin |
Update on synthetic source injection | Announcements |
&nbs= p; |
Sorting items needed for pipeline commissioning.= Followup from Dec 14. | ||
&nbs= p; |
Yusra AlSayyad&= nbsp; | News sharing | |
&nbs= p; |
No meeting. | Enjoy the break. | |
&nbs= p; |
Brainstorm required data products to complete co= mmissioning (follow up Nov. 30) | Snapshot from Dec 2022 | |
&nbs= p; |
Aaron Watkins | Low surface brightness galaxies and sky subtract= ion | |
&nbs= p; |
Keith Becht= ol | Planned schedule / activities for on-sky observa= tions | slides |
&nbs= p; |
Jos=C3=A9 S=C3=A1nchez-Gallego | SDSS V | |
&nbs= p; |
Keith Becht= ol | Kick-off meeting for SIT-Com Science Units | slides |
&nbs= p; |
|||
&nbs= p; |
Princeton visit debriefing parsl-visual= ize demo (time permitting) |
||
&nbs= p; |
Nima Sedaghat&= nbsp; | Machine Learning topics | |
&nbs= p; |
Brianna Smart<= /a> | The Diffuse Ionized Gas of the Magellanic Clouds= | |
&nbs= p; |
Erfan Nourbakhsh= | Unrecognized blending and cosmological inference= | |
&nbs= p; |
Erik Denni= hy | Batch Proc= essing AuxTel datasets | |
&nbs= p; |
|
NO MEETING<= /span> | |
&nbs= p; |
John Parejk= o | A discussion on papers that should be written be= fore/at the same time as DR1. | |
|
A short report on PFS wavelength solutions (w= hich share some similarities with HSC/DECam astrometric solutions, hence th= e possible Rubin interest. Plus it's cute). |
|
|
|
Brianna Smart<= /a> | The Diffuse I= onized Gas of the Magellanic Clouds | TO BE RE-SC= HEDULED |
|
Red galaxy errors | ||
|
Yusra AlSayyad et al. |
PCW decomp= ression | Just discussion. Imagine it's Thursday night aft= er the PCW. Your talks are done, you just had dinner, and are in the pool f= loating around with the rest of the pipelines folks. "What's up?" |
|
|
PCW. NO MEETING | |
|
Yusra AlSayyad et al. |
Practice PCW talk: Update from Science Pipelines= | https://docs.google.com/presentation/d/1aSD44xND8WMPbhaQ= 0gDXSXn17PvtQg0NvjJIjTI2Ygc/edit |
|
Compensated Filters in Practice | ||
&nbs= p; |
Eli Rykoff<= /a> | Brief obs_decam Gen2 Deprecation Update | |
&nbs= p; |
1) Fred Moolekamp |
1) New output data products from the multi-band =
deblender 2) Refactoring drp_pipe |
|
&nbs= p; |
Source Injection with the Science Pipeline= s | Source Injection with the Sc= ience Pipelines ticket. | |
&nbs= p; |
DM Pipelines Bootcamp | ||
&nbs= p; |
Nate Lust&nbs= p; | New Analysis Tools Design Presentation | https://github.com/lsst/anal= ysis_tools Confluence to collect feedback: Analysis Tools Prototype Design = td> |
&nbs= p; |
NO MEETING WEEK | ||
&nbs= p; |
NO MEETING. DMLT WEEK | ||
&nbs= p; |
John Parejk= o | AP Runtime Characterization Sprint Demo | |
&nbs= p; |
Emergent Issues | ||
&nbs= p; |
Lee Kelvin= | Unit tests for drp_pipe | |
&nbs= p; |
History Of Background Estimation in the DRP Pipe= line | ||
&nbs= p; |
Jim Bosch&nbs= p; | A Guided Tour of QuantumGraph Generation | |
27 Apr 2022 |
|
Postponed (date: TBD) | |
20 Apr 2022 | Chris W= aters |
Verification (cp_verify) | cp_verify overview-DM.20220420.pdf |
13 Apr 2022 | John Pa= rejko |
"= The Prompt Processing Prototype" |
Eric's summary slides |
06 Apr 2022 | Yusra Alsayyad Kian-Tat Lim |
"Update fr= om the third USDF workshop" | |
&nbs= p; |
Sophie Reed | Analysis_drp Plotting Sprint Demo | Sprint Demo Slides= a> |
&nbs= p; |
Ian Sullivan | Difference Imaging refactor | Image Differencing refactor March 2022.= pdf |
&nbs= p; |
Yusra AlSayyad | Premortem Follow-up | Trello Premortem Link |
&nbs= p; |
Meredith Rawls | Satellites: IAU CPDQSSCI (lol), ZTF, and the Sun= | |
&nbs= p; |
Clare Saunders | The Present and Future of DM Streak Masking | |
&nbs= p; |
Jim Bosch | Configuration lessons learned | Configuration is Hard Slides |
&nbs= p; |
Dan Taranu | Object-to-Truth matches | |
&nbs= p; |
Jim Bosch | Planning for Gen2 Removal | Gen2 Middleware Removal Planning |
&nbs= p; |
Much ado about noise | ||
&nbs= p; |
Lee Kelvin's vistor Aaron Watkins | Low-surface-brightness Science | Sky subtraction Optimized for LSB Observations Slides |
&nbs= p; |
Yusra Alsayyad | Premortem | |
&nbs= p; |
Alex Amon (visitor) | Weak lensing measurements & calibration | |
&nbs= p; |
Lauren MacArthur | DP0 processing plots | |
&nbs= p; |
Robert Lupton | Noise in Detectors Supporting non-destructive Re= ads (CMOS; HgCdTe) |
|
&nbs= p; |
Yusra Alsayyad | Project updates | |
&nbs= p; |
Spencer Nelson | AP DC2 profiling | |
&nbs= p; |
Dan Taranu | DRP DC2 plots | |
&nbs= p; |
Arun Kannawadi | Shape measurement uncertainties | |
&nbs= p; |
Meredith Rawls | Satellite update | |
&nbs= p; |
Yusra Alsayyad | Final finalizing of the DP0.2 pipeline | |
&nbs= p; |
Yusra Alsayyad | DC2/obs_lsst pipeline changes/finalization | |
&nbs= p; |
Yusra Alsayyad | Review Rows/Columns in the DP0.2 output Parquet = Tables | Parquet DRP Table & Colu= mn Reconciliation Notebook |
&nbs= p; |
Nate Lust | Memory Management in Python | |
&nbs= p; |
Josh Meyers | PSF estimation update | |
&nbs= p; |
Dan Taranu | Scarlet's treatment of bright stars | |
&nbs= p; |
PCW recap | ||
&nbs= p; |
Yusra Alsayyad | PCW prep | |
&nbs= p; |
Nate Lust | BPS interface updates | |
&nbs= p; |
Arun Kannawadi | Mind the GAaP | |
&nbs= p; |
Robert Lupton | Multiband detection | Maximum = Likelihood Detection (ChiSq) Slides |
Final finalizing of the DP0.2 pipeline (Final)
Sep 15: Yusra on : https://github.com/yalsayyad/d=
m_notebooks/blob/master/object-table/Parquet_DRP_Table_and_Column_Reconcili=
ation.ipynb
Sep 1: Nate on memory management in Python
Aug 25: Psf-estimation update (Josh on Piff; Dan on Scarlets=E2=80=99s trea=
tment of bright stars
Aug 18: PCW recap
Aug 11: Cancelled for PCW
Aug 4: Prep for PCW (Yusra)
July 28: DESC meeting review and BPS interface updates (Nate)
July 21: DC2 QA Sprint Demo Round 2 (Yusra)
July 14:
July 7: Cancelled for holiday week
June 30: Yusra: Results of the Replan/Review
June 23: DC2 QA sprint demo.
June 16: DC2 QA sprint kickoff
June 9: Eric Charles: NWayMatch
June 2: Keith Bechtol & friends: faro
May 26: QA comparison party (whoot whoot)
May 19: Possible follow-up on discussion from last week. Also, Arun will be=
showing more results from his GAaP plugin.
May 12th: Lee and Sophie - setting up a more permanent set of synthetics (a=
ka fakes) to be continually processed. This will be ~5-10 minutes of presen=
tation followed by 10-15 minutes of discussion.
May 5th: Mario Juric - Trailed source fitting.
April 28th: Clare - Astrometry
April 21st: Robert
April 14th: Sophie and Nate - New actions and plotting tools
April 7th: Fred - Scarlet update
March 31st: Discussion of RC3 dataset
March 24th: Yusra - DMLT updates
Announcements:
https://www.youtube.com/watch?v=
=3D41_VUIPwWAk
https://www.f=
lickr.com/photos/slaclab/sets/72157718685044792/
Ranpal started ordering 325 tshirts and patches
Replan update
Last call if you want to learn how to release the science pipelines for v22=
. target for release candidate is w_2020_14. There will be a 23.x for=
the stack release that we use for DP0.2.
https://community.lsst.org/t/shared-ge=
n3-data-repositories-ready-for-some-use/4845
GAAP continued=E2=80=A6. (Arun)
DMLT updates (Yusra) https://confluence.lsstcorp=
.org/pages/viewpage.action?pageId=3D148111379
March 17th: Arun Kannawadi - Formalism & preliminary results from GA= AP
March 10th: Lee Kelvin - Fake Source Injections Update
Announcements:
March 3rd:
Announcements:
Please sign up for stack club on https://conflue=
nce.lsstcorp.org/pages/viewpage.action?pageId=3D100179225
See Visual Identity guide: https://docushare.lsst.org/=
docushare/dsweb/Get/Document-37294/20210212%20Visual%20Identity%20Manual%20=
%e2%80%94V7.pdf
Fill out form to get a T-shirt: Friday, March 5, 2021.
Telescope parts are being assembled.
Major thank you to Nate, Eli, John, Lauren and Fred for getting fixes in fo=
r weekly w10 Gen3 rerun.
Robert Lupton: Multi band detection https://lsstc.slack.com/files/U2JPLS84X/F025R7S40CA/chisq.pdf
February 24th: Cancelled
February 17th: ctrl_bps Tutorial
February 10th:
February 4:
DRP: where we=E2=80=99re going https://lsst.ncsa.illinois.edu/~yusra/download/drp-activities=
-as-of-2021-01.png
January 27
Robert Lupton and Eli Rykoff to lead discussion of aperture corrections
January 20: Meeting Canceled
January 13
Robert Lupton: New background estimation. luptonCompensatedAperture=
Fluxes20210113.pdf
January 6
What to do about the w02 RC2 (Yusra, <10 min, https://githu=
b.com/yalsayyad/dm_notebooks/blob/master/calibration/Jointcal%20with%20Gaia=
%20RefCat.ipynb)
Jim Bosch: DMTN-172