AP Pipeline Meeting, 2020-11-30

2pm Pacific Time

BlueJeans: https://bluejeans.com/426716450

attending:

• Eric Bellm
• Unknown User (cmorrison)
• Ian Sullivan
• Kenneth Herner
• Krzysztof Findeisen
• Unknown User (gkovacs)
• Meredith Rawls

regrets:

Topics for discussion:

• Gen 3 conversion (all):
  • Krzysztof Findeisen :
    • nothing on Gen 3; spent last week on afw.filter, done (almost) soon

• Precursor processing (Meredith Rawls ):
  • merged stack-built calibs!
  • also started doing the same work in Gen 3 at the Gen3 party programming session!  So far everything is mostly working?
    • gen 3 decam still doesn't have linearity correction or crosstalk–Chis W says he doesn't expect it until Feb.?  so this is a blocker
  • HSC COSMOS
• report from NCSA (Unknown User (emorganson)  + Unknown User (manninga) et al): 
  • they are out.  We are talking 
  • We are going to lean into Postgres for our interim APDB
    
    • what do we need?
      • user accounts for AP
      • ability to write
      • ability to read others' schemas
    • Meredith: each user by default has a different place they can read and write
      • ideally we'd like everyone to be able to read–this seems like it's straightforward to grant
      • default schema is your username
        • psql and python can easily
        • dax apdb currently doesn't let us specify a schema to write to–so it only always writes to the user's default schema (their username)
        • so to do another run, has to go rename the default schema in the psql command line, make a new empty default. can't run two independent runs at once
        • so maybe we need a ticket to modify dax_apdb
          • Meredith points out we don't necessarily want to be able to blow away other's runs
          • we should talk to Colin/Fritz here
            • Chris points out that this isn't really an operations mode of apdb, so he suspects we'll have to work around it–either a config, or add something to the URL.  Worrying about us getting tied up in postgres specifics
        • Meredith: would be nice to be able to share data in Postgres with others
          • Eric: not going to be able to point them to a db on lsst-devel.  sql dump could work
• ap_association/SDM/fakes (Unknown User (cmorrison)):
  • just merged the pipelineTask that matches injected fakes to the DIASources
  • has successfully run a metric that uses that output, will futz some with numbers to make sure it's working as expected
• Crowded field photometry (Ian Sullivan ):
  • two different runs of ap pipe (2013-2015, and vice-versa)-seeing in 2015 was vastly better, so lots of nasty deconvolution artifacts in 2013-2015
• Image differencing (Unknown User (gkovacs) ):
  • PSF subtraction notebook merged.
  • Working on draft technote
• Preops project (Kenneth Herner  and all):
  • put a ticket in review with Krzysztof
• Review CI (https://chronograf-demo.lsst.codes/):
  • nothing of note
• Review outstanding action items

• AOB

Action Items