Science Pipelines and SQuaRE
Eliminate boilerplate in source files
Jonathan is working on this. Will be done after the HSC fork is merged. Will be an automatic transition managed by SQuaRE.
Switching to NumPy docstring format
Waiting for the new documentation build system, "LSST the Docs". Should be in the next week or two. Will be discussed in the "decision making" session tomorrow.
Still work to be done in determining the best way to document tasks and how tutorials and examples should be published
Continuous Integration
What does it mean to have "enough" CI to perform major changes? Suggestion that we need all the supported cameras processed through to measurement on coadds every week. This should include regression tests as well as tests for absolute values and how they relate to the science requirements.
What is a supported camera? SQuaRE emphasize that we should not guarantee to support in perpetuity every obs_ package that is submitted. Agreed that they should have designated "sponsors" who will maintain them. However, if a change to the stack causes a test to break for a specific camera, our first assumption should be that it's an algorithmic error rather than a bug in obs_foo.
Plans for nightly QA runs that will run large scale (~hours) tests. Regular CI will run shorter (minutes) tests.
Limited developer support in Spring 16 and lsst-dev shared stack
SQuaRE will provide only limited developer support during the short Summer 16 issue. Science Pipelines identify one issue deserving of immediate support as the shared stack on lsst-dev. JDS will act as point of contact for SQuaRE on this.
Process Control and Data Access
SUI/T and Architecture
Science Pipelines and Architecture
Previous coding standard decisions
Python 3, idiomatic Python, etc.
Almost all of this work has been blocking on the HSC merge. However, the onus is then on Science Pipelines to decide if and when to schedule the work: Architecture will not make this decision. Should assess:
- Is work done as a big bang or incrementally?
- If work is incremental, how does it impact ongoing tickets?
Rough estimate that the effort to make our Python code "idiomatic" would be two weeks of work for three people. Important to have enough test coverage in place to make sure this doesn't break anything; running an HSC data release candidate should be sufficient.
The plan is to carry out the Python 3 work in a "big bang" hack session at the August All Hands.
AstroPy
There will me a meeting with the AstroPy developers at UW in ~1 month. Where we draw the line in terms of stack integration has a huge impact on the amount of effort required. Aim to have a rough idea of this before the meeting, but expect to refine it based on input from the AstroPy folks. An ideal outcome might be to produce AstroPy affiliated packages which provide C++ APIs that we can use in the stack.
Process Control and SQuaRE
SUI/T and Data Access
We all would like to exercise the SUI to Data Access system in a basic way regularly, deploy nightly out of NCSA. The idea is to get this going with a very small repository. Then, when we receive the PanSTARRS data, SUI can start to use that data to write a robust set of regression tests.
Jacek would like to get the small set of available around July-Augus 2016. Before then SUI will deliver example queries so we can verify we’re prepared to run the tests.
Once the pan stars data arrives it should take Data Access about a month to get the pan stars data ready. Note, we don’t know what format it’s coming in; we will need to think about how to load it.
Remote Butler - SUI really wants a Java client to the remote Butler, for the Firefly service. We need to discuss it with the architecture group, but perhaps we should consider *not* doing the python butler client for now, and doing the Java one instead.