Date

23 Sep 2015

Attendees

• Vaikunth Thukral, John Gates, Brian Van Klaveren, Andy Salnikov, Fritz Mueller, Unknown User (npease), Jacek Becla

• Kian-Tat Lim for part of the meeting

Discussion items

SQLAlchemy

• new db module pushed to master (and Qserv and dax_* migrated)
• Brian working on unit tests based on sqlite
  • complication: python sqlite driver does not return column types
• looks like we are still including mysqldb in qserv, need to revisit

Docker

• Fabrice has docker-based multi-node integration test
• next step: integrate with CI, discuss next week with Frossie

• current hack: docker VM name = host name, so in practice we can run only one docker vm per machine. This is related to xrootd depending on host name. We could put each VM on network rather than bridging network to get around it. Need to work with Andy Hanushevsky on breaking dependency on hostname.

Query processing

• OBJECTID restrictor is like the IN restrictor. Remove it and switch to using IN if no performance penalty (do it as a new story)

• passing restrictor name as string, lots of string comparisons, maybe const int would be faster (do as a new story)

Data provenance

• planning a kick-off discussion next Thursday 1-3pm

Query cancellation

• new code much cleaner, seeing problems at scale (see DM-3943 and DM-3936)

CSS design

• need to remember consistent number of parameters
• catch 22, plugins need css info to understand query, so can't take snapshot of only dbs/tables involved in a query before plugins are called

RFC-96 (logging)

• maybe Tim Jenness could implement the log part?
• we will have to deal with migrating qserv (lots of lines to change, can't automate)

xswap

• running into some timeouts if run for too long, need to restart services
  • understand better
• use kpi-testing like setup in the long term to do measurements

Next week

• Frossie Economou at SLAC next Wed, Qserv mtg next Wed all about DB-SQUARE related topics (docker, documentation, CI, scons/eups troubles etc)

Action items

• Jacek Becla, revisit usage of mysqldb in qserv, create stories to capture switching to new db module. Done, see DM-3947 and DM-2949
• Jacek Becla, announce new db module to DM team (once we vet it through sqllite tests)