This is a design sprint to kick off the LSST Science Pipelines documentation reboot. Our goal is to a create a tangible vision of what the Science Pipelines documentation will be. Questions we want to answer are:
Who are the users of Science Pipelines documentation? What does each group want to get out of the Science Pipelines and its documentation? Do those needs conflict? Do we need to prioritize one user group in the initial implementation?
What are the boundaries of the Science Pipelines documentation (the site at https://pipelines.lsst.io)? What are adjacent documentation projects that the Science Pipelines documentation might link against?
What's the curriculum for learning the Science Pipelines? What are the concepts that the Science Pipelines documentation site needs to cover? How are these concepts organized (hierarchically or as a bottom-up information network). Do different types of users need specific entry points into the documentation and Science Pipelines itself?
What kinds of content are we going to be producing? What do the templates of these topic types look likes?
How are concepts unique to Science Pipelines, like tasks and command line tasks, documented in both a code and information architecture sense?
Intended Sprint Products
These are suggested products and outcomes from the sprint:
A map of the science pipelines site. This map should resolve into individual HTML/reStructuredText documents (topics in Every Page is Page One terminology). Each topic should be annotated with:
Topic name.
Content purpose and scope.
Topic type (i.e., template).
Adjacent topics (topics that link into this page; topics this page will link out to).
Topic types and templates. Each template shapes how different types of topics are written. Examples can be: API reference, task, command line task, tutorial project, conceptual overview, recipe. See Every Page is Page One Chapter 9: EPPO Topics Conform to a Type.
Tuesday December 6: campfire chat at Bentley's or elsewhere.
Wednesday December 7. 9:00 am to 5:00 pm. LSST Workroom.
Thursday December 8. 9:00 am to 5:00 pm. LSST Workroom.
Friday December 9. 9:00 am to 5:00 pm (or as participants depart). LSST Workroom.
Discussion items
Time
Item
Who
Notes
What is the scope of the "Science Pipelines" documentation site?
Note that important obs packages are outside lsst_distrib; omitting the obs packages would reduce the current usability of the Pipelines documentation.
Technical constraint: tightly coupled packages should be documented together since docs will be versioned tightly with the codebase (docs embedded in Git; also known as 'docs as code'). This is an LSST the Docs feature: https://sqr-006.lsst.io.
We agreed that a lot of middleware (things beyond Princeton and UW) should be included in pipelines.lsst.io because of the tight API integration, including:
task/supertask framework
butler
logging
display packages
Example: document the Butler API in pipelines.lsst.io, but document the DAX service elsewhere.
Example: document the Firefly display package in pipelines.lsst.io, but document Firefly itself elsewhere.
There is a list of obs packages that will be supported. These will be included in pipelines.lsst.io.
lsst.validate packages will be in pipelines.lsst.io.
Think of pipelines.lsst.io as documentation for the open source project that might be used in other contexts besides LSST AP and DRP pipelines (other observatories, building L3 data products). Data release documentation will specify exactly how the Science Pipelines were used to build a data release.
Boundary between Pipelines docs and the Developer Guide
Should the pipelines documentation cover developer and build-oriented topics currently in the DM Developer Guide? Do pipelines users need to be able to create Stack packages to make Level 3 data products?
lsstsw and lsst-build
Structure of Stack packages (including sconsUtils and EUPS details)
etc?
developer.lsst.io is intended to define policies and practices specific to DM staff. We can't use it as documentation to end users.
If the build and packaging system are described in pipelines.lsst.io, it could be awkward for other software projects, like Qserv and Sims, that also depend on EUPS/sconsUtils/lsst-build/lsstsw, etc..
However, putting build/packaging documentation in pipelines.lsst.io probably makes the most sense for astronomers extending the stack. pipelines.lsst.io is already where astronomers will look to learn how to write new packages against the Pipelines API. Overall, we can just learn that pipelines.lsst.io is where build and packaging is fundamentally documented.
Who are our users?
What user group should be prioritized?
What are common activities that this group wants to achieve? What documentation will assist with that?
DM developers in construction
Need API references most.
Currently learn APIs by introspection or reading the source and code that uses an API. Doxygen isn't useful.
It's natural to organize documentation (to some extent) according to units of EUPS packages, given that doc content should live with code. Should every EUPS package have a topic page and be linked from the homepage (like the astropy docs do for sub-packages)? Are there exceptions where documentation that may live in an EUPS package should actually be organized altogether independently of EUPS package structure?
To what extent should documentation refer to EUPS packages (e.g., afw) versus Python namespaces (lsst.afw)?
What is the structure of the documentation homepage?
The homepage is important for orienting users. The structure of the homepage should present a coherent vision for what the Science Pipelines are and how they're used.