Date

Jul 16, 2014

Attendees

Goals

  • Get together, resolve all bottlenecks, make progress faster, exchange ideas

Discussion Items

master inadvertently broken (merge of DM-622)

  • probably incorrect merge
  • fixed in u/fjammes/dm-967 during the meeting, reviewed and merged


How to install system dependencies?

  • Can't do through eups
  • Maybe special package that checks and reports missing dependencies?
  • Check with Kian-Tat Lim?
    • (KTL) We don't have a solution for this in the stack yet.  My preference is currently for a "stub" package that checks for system dependencies.  I don't want to have system dependency installation in newinstall.sh, which should only be for installing eups.

Running on multi-node

  • runServer, people not happy at large scale
  • pmg used by LHC/Atlas
  • ideally, just be flexible and use whatever is recommended for a given cluster


contrib/eupspkg/* packages that Fabrice Jammes created

  • Should there be a better place for these things?
  • Ask Kian-Tat Lim?
    • (KTL) LSST/external/* once they have been reviewed.

xrootd integration tests

  • Get the latest custom xrootd package, see DM-944
  • Get qserv from u/daniel/kirkwood
  • Jacek will set up test environment
  • identified first issue: context not passed during the meeting

25 node cluster for Qserv test

  • Likely available in Sept, maybe even in Aug, Fabrice will double check

Refmatch

  • About week away from testing
  • Includes name resolution
  • Can't support outer joins that are spatial
    • So queries like "find objects without matches" would not work
    • Chat with Mario Juric about it, maybe in Phoenix?
    • That is because now we concatenate results from two joins: one against subchunk data, one from overlap
    • Possible fix: maintain in memory subchunk, and subchunk+overlap (now we have subchunk and overlap). But This doubles memory requirement. Need to run through spreadsheet and check cost increase
    • Or... load to memory subchunk+overlap, filter out overlap through a non-materialized view


boost dependency checker script (by Andy Salnikov)

  • code non-trivial, naming conventions in boost crazy, likely to change in the future, code needed in more than one place
  • needed for partitioner (see DM-666)
  • want to put the boost dependency checker into sconsUtil
  • but sconsUtil depend on eups
  • partitioning package is simple, does not want all these dependencies
  • sentiment: maybe better to duplicate code for checking boost versions than use sconsUtil?
  • get advice from Kian-Tat Lim
    • (KTL) I suggested to Daniel that maybe we should punt on all this and just require the user to specify where Boost is (equivalent of "--with-boost=").  We can of course look in BOOST_DIR if eups has set that.