SAT Meeting 2014-09-02

• (traveling)

• (sick)

• (very interested party)

• (interested party)

Winter2015 Package Reorganization Planning

• More than a couple of packages may always need a tool
• Build times affected by SWIG which depends on namespaces and possibly .i file structure
  
  • AndyS also has a scanner fix to avoid unnecessary rebuilds
• May need complex layering of geometry to separate Eigen dependency from Qserv
• JBecla: Qserv has separate build system because it has many modules/products within single repo and package
  • New organization would have multiple Python modules but still have eups package/product and git repo aligned
• MJ: Why not 1000 packages (and repos)?
  • FE: Disorienting, hard to keep in your head
  • RHL: BaBar had many (GPDF not available)
  • FE: Could do if have conceptual grouping (but tools don't really support)
  • JBosch: Need to build and support our own tooling
    • MJ: Android has something like this
    • JBosch: But may be difficult to have that interact with eups
    • MJ: And still need tooling anyway for more than a couple of packages
• FE: Doesn't have to be uniform; can split out some small bits for sharing
• FE: How many repos do I need to touch to do a small task?  Should be small (ideally one)
• FE: Short build times needed to get fast test turnaround
  • scons should take care of that with multiple modules per package
• RHL: Don't want people to have to build from source
• DP: Controlling change, especially deploying to production, difficult with bigger packages; hard to get just changes you want and not ones you didn't
• RHL: Things that are changing should be minimal in size
  • Others: Not clear that size matters; this has to occur with small or big packages
• RHL: Working on shared machines is different than independent machines; shared stack helps but requires smaller packages
  
  • RHL: Want most things to be stable
• RHL's shared development model:
  • Stack that everyone shares, kept up to date
  • Only packages needed to be built from source are the ones you are working on
  • Essentially treat many packages like internal third-party dependencies
  • KTL: Could use containers to deliver guaranteed environment
  • JBosch: Valuable when working on large volumes of real data; less so for small changes
• FE: Always need to transition from rapid change to stable deployment; requires release engineer
• MJ: Worried about pulling in unnecessary things (especially third-party dependencies)
  
  • Have some of both right now: afw is merged; pex_* and ctrl_* and meas_* are broken up.
  • JBosch: Almost lowest-level code has almost all third-party dependencies
• JBosch: Could split primitives into stable and unstable
• Containers: not for everyone, but does solve some problems for people needing pushbutton environments
• FE: Need a minimum size for reuse to develop a community
• Splitting out third-party dependencies: cfitsio (would require a lot of refactoring work), minuit, gsl might be possible
  • JBosch: Likely lose more than we gain trying to split these out
• MJ: We should be like a Linux distribution with a core and lots of optional additions rather than like Fermi/GLAST with everything included
  • FE: This is a spectrum