Unit tests for ap_verify require HITS2015 data, but we are not allowed to add a package to the stack if it depends on a git-lfs repository. We now skip this, so the dataset is not a hard dependency.

Outstanding questions DM-12853 - Getting issue details... STATUS :

  • What do we do with the --rerun argument to ap_verify?
  • ap_verify return status

We agreed that we have reviewed these questions and will mark the ticket resolved, even if we haven't answered them.

We need a proper design of ap_verify before we can actually answer those questions.

Krzysztof Findeisen is blocked on the lack of a coherent plan of what AP verify should be capable. We could get Krzysztof Findeisen's set of proto-requirements onto Confluence as bootstrapping for formal requirements/design documentation.

  • John Swinbank: set up Confluence pages for requirements and design for ap_pipe & ap_verify  
  • Krzysztof Findeisen: get proto-requirements for ap_verify listed on the Confluence page above  

Crosstalk for DECam is nearly done, thanks to Meredith Rawls; expecting that to be in review within a day or two.

Plans for the next month

What would it take to throw more data at AP pipe?

Make it a command line task!

  • Factor out ingestion
  • Make 13163 to simply be “create a command line task”
  • Add another ticket to create a simple ingestion script; that need not be a prerequisite for the command line task.
  • But self-ingestion is necessary for CI.
  • We are agreed that ingestion does not belong in AP pipe, since it should mirror the operational system as closely as possible.
  • Meredith will try to come up with an estimate to complete for that by next week.

Association:

  • Is a task (thumbs up)
  • Is too slow (thumbs down)
  • Needs some design decisions to be revisited.
  • Currently we're storing things in a SQLite database.
  • We need to think about the L1 data model; should talk to Fritz about how we're moving forward with this.
  • But Chris has clear ideas about where he wants to go in the relatively short term. We'll set that as an immediate goal. He'll walk us through that on the whiteboard at the edition of this meeting in 2 weeks.

ingestion:

  • Krzystof will work on ths.
  • Bringing ingestion into ap_verify.
  • Fix outstanding ingestion bugs, e.g. DM-12662.
  • Understand why this is a hard problem.
  • Ultimately, the AP team will find itself on the hook for obs_decam maintenance. This is a good opportunity to understand why that's hard!