Page History

Winter2015 Package Reorganization Planning

• BaBar had many packages (3.5MLOC, 40% in core library, rest application-specific)
  • Had 1000 packages in production builds; easy to create new ones; made it easy to manage dependencies
  • 5 sub-pipelines doing completely different things, sharing only limited common code (10%)
  • Evolved that way; could possibly have worked with a different approach
  • Did not work with one package per area of responsibility; needed to use them for dependency control
  • Could work within system touching only a few of the packages
  • Working within shared environment; global build provided most of code; developers only changed a little
  • Testing depended on stack of libraries, but those were already built
  • Also had tool to test effects of lower changes on upper packages
  • Renamings weren't particularly difficult
  • Abstraction/interface + implementation(s) led to separate packages
  • No one person knew whole picture, though
  • Hard to take a slice of the code and deliver it to people
    • Could have used a coarser-grained organization on top of fine-grained
• Frossie: want limited number to improve adoptability by outsiders
  • Agree with interface/implementation split
  • Want to be able to test packages in isolation; not install lots of dependencies to build/test a single one
  • Test control flow of tasks without science by stubbing/mocking it out
    • GPDF: But can't stub out all science and still be useful
• But afw in particular is a library with a wide interface that is hard to mock out
• Linux kernel model:
  • Build via a hierarchy of drivers
  • Stack should be thought of as a Linux/Anaconda distribution: install whole thing or parts
  • Or are we more like PyPI/CPAN/Ruby gems?
• Repos and packages don't have to be aligned
• GPDF: Class-rich data objects lead to complex dependencies
  • As opposed to something like byte streams
• GPDF: Algorithms built as layers of functionality also lead to more dependencies
• Issue is more distribution of packages and making them easily used for development (and not forcing rebuilding of the entire stack)

• No discussion or action

• Want fewer timezones overall, particularly between system logs and application logs
• Unclear whether better to shift the application logs to GMT or system logs to local time
• Tentative conclusion: probably best to try to get all logs into GMT
• Also crontabs and other things?
• Still a potential issue with TAI and UTC, which are a lot easier to confuse than different timezones

How legalistic should the SAT be?

• No discussion or action

• People were not prepared to follow up
• Mario promises to look over this by Thursday

Data Release Production Overview

• No discussion
• Mario promises to look over this by Thursday