BlueJeans: https://bluejeans.com/426716450

attending: Eric Bellm Ian Sullivan Krzysztof Findeisen Meredith RawlsJohn SwinbankUnknown User (gkovacs)

Topics for discussion

  • pre-meeting discussion:
    • DM-19184 is going to need attention soon.
  • LSST 2019 session proposals?  due next Monday
    • Pipelines status report?  AP & DRP?  maybe this falls under the "time domain day of focus"?  that would be a public-facing thing
    • What about inter-DM discussions?  Krzysztof Findeisen wonders if it's time to check in again with Princeton about metric processing
    • discussion of QAWG outcomes/recommendations?  John points us to: DMLT Discussion of QAWG recommendations  potential for status and demos of qa tooling:  metrics, qa_explorer, etc.
    • with Eric M?  integrating alert distribution at the data facility?   maybe also prompt processing/middleware at the data center?  John thinks these are different audiences and hence perhaps different sessions
  • HiTS reprocessing (Meredith Rawls):
    • cleaning & closing tickets
    • pair coded some tests for ap_pipe with Krzysztof Findeisen .  Hard to test in general because it's linking lots of other package, most missing coverage should go in sub-tasks.  But high-level coverage is probably okay.  Some things they thought of but didn't do.
  • DES processing (Unknown User (emorganson)):
    • report via Slack:
      1) Coordinated the Kafka Engineering alert system between Tucson and our DB people. It turns out that everything has been built. It just needs to be put together. 2) May have mentioned that I got Maria's alert_stream fully working on our Kubernetes machine. 3) Doing OPS_REHEARSAL this week will report on anything interesting next week 4) We got faster machines that will be more like what processing will occur on. Gonna test the current pipeline on them to get more accurate timing results.
  • ap_association (Unknown User (cmorrison)):
    • Chris is out this week and next
  • DCR (Ian Sullivan ):
    • wrote code to compute PSF on DCR coadds; in review with John P.  Definitely improves the shape of the PSF, although percentage changes are not large.  Does not resolve the catastrophic failures from the variable PSF tests last month.  That was tracked down in a bug in the test–some of the PSFs stored were wonky.  Haven't re-run yet.
  • image differencing (Unknown User (gkovacs) ):
    • showing fits to kernel basis functions per kernel candidate source spatially over the image (8x8). (Not identical to the A&L cells in which the local solution is derived).  Default is 27 coefficients; Gabor is putting more description in the docstrings
    • also a plot of the global solution evaluated in each spatial region
    • good solution has fewer  sources being fit, and coefficients are typically smaller in value (few, as opposed to hundreds).  Krzysztof Findeisen raises the overfitting concern given the large number of parameters; 30x30 pixels, so
    • good solution has about all of the power in the 0th (Gaussian) component; bad has lots of high-order power as the various high-order terms beat against each other.  Should we be regularizing?  (or simply tuning minimum sigma, or cutting the number of allowed parameters)
    • Showing a very nice flowchart of the logic
    • John suggests Gabor ask Robert for advice on how to proceed to improve these solutions
    • may be some fit initialization caused by starting from the FWHM only?  general speculation about least squares being sensitive/finicky; lots of configurability in afw.math... (in doxygen, we find Cholesky LDLT, Cholesky_LLT, LU, EIGENVECTOR)
  • Review CI (https://chronograf-demo.lsst.codes/):
  • AOB
    • Ian requests a DCR reprocessing when the monthly HiTS is in.