2pm Pacific Time
Zoom: https://washington.zoom.us/j/98846655020
attending:
- Eric Bellm
- Ian Sullivan
- Meredith Rawls
- Mario Juric
- Ari Heinze
- Brianna Smart
- Kenneth Herner
- Hsin-Fang Chiang
- John Parejko
- Bruno Sanchez
- Harshit Rai
regrets
Topics for discussion:
- Project updates (Eric Bellm , Ian Sullivan ):
- startracker movie: https://lsstc.slack.com/archives/C041MQGD8HM/p1709958475193229
- PCW (week of July 22): attendence, hotel, registration comments
- Auxtel processing status (all):
- slower processing: seems to be driven by forced photometry on junk
- diaPipe went from 10sec in January, now up to almost a minute
- 90% of time is associated with loading diaForcedSources from APDB, computing new ones. 45 seconds is APDB access, 10 seconds is image processing
- Ian: many sources are only detected once, we could restrict ourselves to only sources that we think are good. Eric: agreed, but some single detections are exactly the science sources we want to get forced measurements
- John: there are several things we call forced photometry–which is the one that's meant here? KF: whatever the diaForcedSource subtask is doing
- Eric: we can move some of this outside of the AP loop. Krzysztof: we'll blow up the compute if it takes ten minutes+ with a year of history
- KF: 250k DIASources, half as many DIAObjects, 144 fields. 1700 per field. John: ten seconds for the image bits seems slow
- Ian: we'll see this scaling pretty quickly with the Ops Rehearsal. Do we try to get something in ahead of the freeze on the 27th?
- Krzysztof: AuxTel is going to to hit this in a serious way within the month
- PP was running weekly 5 for most of Feb., weekly 8 for late Feb/early March–Ian will check if centerAll filtering was in for most of that
- slower processing: seems to be driven by forced photometry on junk
- Ops Rehearsal (all):
- Pipelines for Ops Rehearsal #3
- Operations Rehearsal for Commissioning #3 Orientation Meeting Wednesday 8am
- soft freeze for AP pipeline this Wednesday, hard freeze next Wednesday. Currently we're not planning to cut a release branch; planning to use the weekly, figure out how to do hotfixes if needed. Krzysztof: we have to use the science pipelines docker containers as input to Prompt Processing release. Could use dailys, but that will bring in non-bugfix code changes as well
- Hsin-Fang: when can we expect next visit messages from the summit, sample file products? Ian: no news, we should bug people
- Priorities from now-ComCam (Eric Bellm ):
- Pipelines Infrastructure (John Parejko, Krzysztof Findeisen ):
- Krzysztof: just ops rehearsal prep
- John: trying to finalize the deblending in diffim. thinks he has a way forward after chatting with Jim
- Precursor processing & Campaign Management ( Kenneth Herner , Erin Howard):
- no updates from Ken & Erin
- Bruno: managed to process the simulated variable ComCamSim data. Runs are completed, getting ready to compare to truth simulation for the various flavors of diffim
- Satellites (Meredith Rawls ):
- nothing to report
- Image differencing algorithms, DCR ( Bruno Sanchez , Ian Sullivan ):
- see above
- _TEMPLATE mask planes have landed
- analysis tools afterburner is now in ap_verify CI, which will highlight any breakages going forward
- alert distribution (Brianna Smart , Eric Bellm ):
- Bri: did a BPS run that produced alerts: success. With Krzysztof, put credentials into Prompt Processing. Alerts made it into alert distribution with little latency from Prompt Processing! There were some possible slowdowns, so investigating further
- ML (Nima Sedaghat Harshit Rai , Eric Bellm ):
- Harshit: graduated!
- Solar-system processing (Ari Heinze, Kurlander Jake , Mario Juric ):
- Mario:
- github.com/mjuric/mpsky: service to query for SSOs in the footprint. Currently running on epyc, returns in 3msec
- worked with Ari, now have a make_tracklets pipelineTask running in USDF. now working on heliolincc and orbit determination
- Mario:
- Review CI (https://usdf-rsp-dev.slac.stanford.edu/chronograf/):
- no changes of note
- Review outstanding action items
- QA meeting (May):
- Ops rehearsal debrief
- AOB
- square services bootcamp
Action Items
Description | Due date | Assignee | Task appears on |
---|---|---|---|
| 17 Jul 2023 | Eric Bellm | AP Pipeline Meeting, 2023-06-26 |
| 11 Sep 2023 | Eric Bellm | AP Pipeline Meeting, 2023-08-21 |
| 25 Sep 2023 | Eric Bellm | AP Pipeline Meeting, 2023-08-14 |