Date

Jul 16, 2014

Attendees

• Jacek Becla

• Andy Salnikov

• Unknown User (danielw)

• Serge Monkewitz

• Fabrice Jammes

Goals

• Get together, resolve all bottlenecks, make progress faster, exchange ideas

Discussion Items

master inadvertently broken (merge of DM-622)

• probably incorrect merge
• fixed in u/fjammes/dm-967 during the meeting, reviewed and merged

How to install system dependencies?

• Can't do through eups
• Maybe special package that checks and reports missing dependencies?
• Check with Kian-Tat Lim?
  • (KTL) We don't have a solution for this in the stack yet.  My preference is currently for a "stub" package that checks for system dependencies.  I don't want to have system dependency installation in newinstall.sh, which should only be for installing eups.

Running on multi-node

• runServer, people not happy at large scale
• pmg used by LHC/Atlas
• ideally, just be flexible and use whatever is recommended for a given cluster

contrib/eupspkg/* packages that Fabrice Jammes created

• Should there be a better place for these things?
• Ask Kian-Tat Lim?
  • (KTL) LSST/external/* once they have been reviewed.

xrootd integration tests

• Get the latest custom xrootd package, see DM-944
• Get qserv from u/daniel/kirkwood
• Jacek will set up test environment
• identified first issue: context not passed during the meeting

25 node cluster for Qserv test

• Likely available in Sept, maybe even in Aug, Fabrice will double check

Refmatch

• About week away from testing
• Includes name resolution
• Can't support outer joins that are spatial
  • So queries like "find objects without matches" would not work
  • Chat with Mario Juric about it, maybe in Phoenix?
  • That is because now we concatenate results from two joins: one against subchunk data, one from overlap
  • Possible fix: maintain in memory subchunk, and subchunk+overlap (now we have subchunk and overlap). But This doubles memory requirement. Need to run through spreadsheet and check cost increase
  • Or... load to memory subchunk+overlap, filter out overlap through a non-materialized view

boost dependency checker script (by Andy Salnikov)

• code non-trivial, naming conventions in boost crazy, likely to change in the future, code needed in more than one place
• needed for partitioner (see DM-666)
• want to put the boost dependency checker into sconsUtil
• but sconsUtil depend on eups
• partitioning package is simple, does not want all these dependencies
• sentiment: maybe better to duplicate code for checking boost versions than use sconsUtil?
• get advice from Kian-Tat Lim
  • (KTL) I suggested to Daniel that maybe we should punt on all this and just require the user to specify where Boost is (equivalent of "--with-boost=").  We can of course look in BOOST_DIR if eups has set that.