"Like Winter2015 Package Reorganization Planning, only not impossible!"

Bubbles of Like-Minded Packages

 old packages (not complete)purpose
basicbase, daf_base, utils, pex_exceptionslow-level packages used by nearly all higher-level code (selectively used by qserv)
butlerdaf_butlerUtilsbutler and data access middleware
scimeas_*, ip_*, coadd_*, pipe_taskshigh-level science pipelines
taskpipe_base, pex_configpipeline definition base classes and configuration
afwafwlow-level science data structures and algorithms
ctrlctrl_*execution middleware
qservqserv_*distributed database
qatesting_pipeQA, testing_displayQAquality analysis and visualization tools
ui science user interface
production  
legacypex_policy, pex_logging, daf_persistencecode we have retired or want to retire soon

Motivations for Splitting and Merging Packages

When We Split Packages

  • Low-level components are stable, but high-level components are not.
  • Some components are needed by a higher-level package, but other components are not.
  • Components are not related to each other conceptually.

When We Merge Packages

  • Users of any of the components typically want all of the components.
  • Interfaces are tightly coupled and cannot be modified separately.
  • Components have circular dependencies.
  • Components are conceptually similar.

What Reorganization Won't Fix

  • Build times: we need to refactor how we handle intrapackage Swig to make a difference here.
  • Organization-as-documentation vs. organization-for-dependencies: we can't simultaneously optimize both of these metrics, and sometimes they conflict.
  • Names: some of them will always be terrible.  Let's not get distracted talking about them now (save that for the individual RFCs!)

Specific Proposals

  • base, utils, pex_exceptions, daf_base: merge all of these?
  • meas_algorithms: move Jarvis/shapelet code to separate, legacy package.
  • afw: move Wcs from afw::image to afw::coord
  • afw: move Kernel and warping from afw::math, Psf from afw::detection to (e.g.) afw::convolution
  • meas_*, coadd_*, pipe_tasks (all of these end up with sci_ prefixes?)
    • one package focused on single-frame/forced processing interfaces and utilities:
      • subtask interfaces (for CalibrateTask + subtasks, measurement, deblending detection)
      • mid-level driver code - not CmdLineTasks (currently split across pipe_tasks and meas_algorithms)
      • miscellaneous utilities (CRs, backgrounds, repair - mostly in meas_algorithms)
    • one package for standard PSF implementations (currently a subset of meas_algorithms)
    • one package for standard deblender implementations (current meas_deblender)
    • one package for standard astrometry implementations (current meas_astrom + AstrometryTask from pipe_tasks)
    • one package for standard measurement plugins (current meas_base, probably meas_extensions_photometryKron too)
    • one package for coaddition; includes command-line drivers (from coadd_* and pipe_tasks)
    • one package for single-frame/forced processing command-line drivers
    • one package for image differencing
    • one package for source model fitting (meas_modelfit) - separate from other measurement plugins due to dependencies, build time
    • one package for HSM shape measurements (meas_extensions_shapeHSM) - separate from other measurements due to dependencies, external origin
  • shapelet, skymap: just add a sci_ (or maybe afw_?) prefix; there will be more libraries like this in the future: astronomy-specific utility (not plugin or algorithm) code that isn't as broadly useful as what's in the current afw

To Legacy

  • pex_policy
  • pex_logging
  • daf_persistence
  • afw::formatters
  • ap
  • datarel

 

 

 

 

  • No labels