Date
Attendees
Goals
This is a design sprint to kick off the LSST Science Pipelines documentation reboot. Our goal is to a create a tangible vision of what the Science Pipelines documentation will be. Questions we want to answer are:
- Who are the users of Science Pipelines documentation? What does each group want to get out of the Science Pipelines and its documentation? Do those needs conflict? Do we need to prioritize one user group in the initial implementation?
- What are the boundaries of the Science Pipelines documentation (the site at https://pipelines.lsst.io)? What are adjacent documentation projects that the Science Pipelines documentation might link against?
- What's the curriculum for learning the Science Pipelines? What are the concepts that the Science Pipelines documentation site needs to cover? How are these concepts organized (hierarchically or as a bottom-up information network). Do different types of users need specific entry points into the documentation and Science Pipelines itself?
- What kinds of content are we going to be producing? What do the templates of these topic types look likes?
- How are concepts unique to Science Pipelines, like tasks and command line tasks, documented in both a code and information architecture sense?
Intended Sprint Products
These are suggested products and outcomes from the sprint:
- A map of the science pipelines site. This map should resolve into individual HTML/reStructuredText documents (topics in Every Page is Page One terminology). Each topic should be annotated with:
- Topic name.
- Content purpose and scope.
- Topic type (i.e., template).
- Adjacent topics (topics that link into this page; topics this page will link out to).
- Topic types and templates. Each template shapes how different types of topics are written. Examples can be: API reference, task, command line task, tutorial project, conceptual overview, recipe. See Every Page is Page One Chapter 9: EPPO Topics Conform to a Type.
- Timelines. Timeline for content and for documentation infrastructure.
Prep Work/Background Reading Material
- Read Every Page is Page One.
- Jonathan Sick's "Pipelines Documentation Site Organization Sketch" is on clo.
- LDM-493: Data Management Documentation Architecture.
- Potentially relevant design docs, which may be cross-referenced with or otherwise relate to Science Pipelines docs:
- validate-base documentation
- Astropy documentation
Meeting Logistics
- Tuesday December 6: campfire chat at Bentley's or elsewhere.
- Wednesday December 7. 9:00 am to 5:00 pm. LSST Workroom.
- Thursday December 8. 9:00 am to 5:00 pm. LSST Workroom.
- Friday December 9. 9:00 am to 5:00 pm (or as participants depart). LSST Workroom.
Discussion items
Time | Item | Who | Notes |
---|---|---|---|
What is the scope of the "Science Pipelines" documentation site?
|
| ||
Boundary between Pipelines docs and the Developer Guide Should the pipelines documentation cover developer and build-oriented topics currently in the DM Developer Guide? Do pipelines users need to be able to create Stack packages to make Level 3 data products?
|
| ||
Science Pipelines docs and LDM-151 |
| ||
Who are our users?
|
Summary
| ||
EUPS Packages as units of organization
|
|
| |
What is the structure of the documentation homepage?
| Frameworks.
Twinkles workflow.
Homepage structure. | ||
Where should concepts of science interest (such as algorithm details) be documented?
|
| ||
How should examples and tutorials be produced?
| |||
How should C++/Python API reference documentation be produced? | |||
Listing topic types and templates
| Preliminary listing. | ||
Community.lsst.org and the docs |
| ||
Tagging command line tasks | We'll have lots of lists of command line tasks in two places: module topic pages and in processing context sections of the home page.
|
LSST Science Pipelines
- Installation and setting up
- Processing data: a tutorial
- Release Notes
- Community, and getting help
- How to report issues
- How to contribute
Processing Data
The Processing Data section is oriented around command line entrypoints (command line tasks or supertasks) and documents processing patterns and algorithmic considerations.
The sections are patterned around typical user pipelines and processing/measurement contexts (single frames, coadds, difference imaging, and multi-epoch datasets). Contexts are slightly different from LDM-151 Section 5 headers. For example, we treat coaddition and difference imaging as different contexts.
In each section, there will be:
- Overview pages that provide a narrative to command line processing and algorithms.
- Tutorials that illustrative command line tasks with realistic datasets.
- Lists of command line tasks, linking to their reference pages. Command line task reference pages are hosted inside package documentatation. Command line task reference pages also link to task reference pages. Organize command line tasks between:
- processing data
- measuring data
Data ingest
Single Frame
- Overview — what do we do in a single frame context. Then link to processCcd.
- Processing entrypoints list
- processCcd
Coaddition
- Overview topics
- Processing entrypoints
- makeCoaddTempExp
- assembleCoadd
- Measurement entrypoints
Difference imaging
- Overview topics
- Processing entrypoints
- Measurement entrypoints
Multi-epoch object characterization
- Processing
- Measurement
Postprocessing
- ...
Frameworks
The Frameworks section is API module oriented. lsst modules (in a Python sense) are organized into framework groups. Each framework group section contains:
- Overview topics that orient a reader to the APIs in the framework.
- List of modules, linking to module topic pages (shown below).
Data structures
- Overview
- lsst.afw.image - Images
- lsst.afw.table - Tables
- ...
Geometry framework
- Overview
- lsst.afw.geom - Geometry primitives
- ...
Measurement framework
- Overview
- ...
Modelling framework
- Overview
- ...
Task framework
- Overview
- ...
Display framework
- Overview
- ...
Data access framework
- Overview
- lsst.daf.base
- lsst.daf.persist
Observatory interfaces
- Overview
- Building observatory interfaces
- CFHT (obs_cfht)
- HSC (obs_hsc)
- ...
Validation framework
- Overview
- lsst.validate.base
- lsst.validate.drp
Logging
- lsst.log
Debug
- lsst.debug
Build and continuous integration
- Packaging
- Dependency packaging
- CI datasets
- utils
- sconsUtils
- lsstsw
lsst.module.name — Readable name
Context establishment paragraph.
Links to related modules, pages, and disambiguation.
Design/High Level Overview
If necessary?
Command line tasks (or supertasks)
- Overview (if necessary)
- Listing of command line task pages (autogenerated; alphabetical)
Tasks
- Overview (if necessary)
- Listing of tasks (autogenerated; alphabetical)
Using the lsst.module.name API
- Links to API concept pages
- If it has a C++/Python API
Python API reference
- list of API object reference pages
C++ API reference
- list of API object references pages
Packaging
- Link to EUPS package/GitHub repository
- Dependencies: auto-generated graph/list of EUPS dependencies
Related documentation
- Linked design documents
- Linked technotes
- Linked papers
- Linked Community conversations