Date

May 14, 2014

Attendees

• Jacek Becla

• Unknown User (danielw)

• Serge Monkewitz
• Unknown User (bchick)
• Fabrice Jammes

Goals

• Discuss blockers, short term plan and design

Discussion Items

DM-510

• remove objIdIndex
• rename drivingTable and keyColName to dirTable, dirColName (dir == director)

Unassociated sources

• special case, complicates design
• we need an estimate of the count
• options: 
  • dummy object for each unassociated source (problem: Object table grows)
  • one dummy object per chunk for all unassociated sources in a chunk (problem: complicates design elsewhere)
  • maintain a separate table for all unassociated sources (preferred solution for now)
• Open a new issue (Jacek)

Doing subchunking with overlap

• for background, see Daniel's email sent to qserv-l 5/3/2014
• Write short description why we need subchunking (Daniel)
• Decide in ~Sept, maybe we can outsource this problem to mysql developers, after all, subchunking = optimization needed because optimizer is dumb 
• and ask Monty how hard

DM-666

• Serge wrote a patch
• Fabrice tested, made one tiny fix, packaging now

DM-220

• Daniel found memory leak
• Add comments what we want AndyH to implement/fix (Daniel)

DM-195 (logging)

• finished another iteration
• better docs on trac
• few more features suggested by K-T, biggest item is: want both boost and printf, question: do we want
• to maintain both
• printf is much faster than boost (based on tests)
• playing with speeding up boost version
• one super wrapper? No
• For packaging:
  • need to change namespaces
  • look at the partition.git
  • push to contrib (Bill), then Serge will take a look
  • package name: "logging"? Double check with Mario Juric and Kian-Tat Lim
  • expecting to have it packaged by next meeting

DM-70

• Trying to decide how much to rewrite
• Daniel wants to hear what was ugly about existing code eg AsyncQueryMgr

DM-365

• would like to rename names, but eups package must have the same name as git repo
• wants: qservdata --> qserv_testdata
• find out if it is ok to rename report, and how easy (Jacek)
• qserv should not depend on qservdata

DM-648

• It is related to DM-481
• Discussed last wee and assigned to Andy Salnikov, Fabrice Jammes wants to have a look and will try to implement

DM-516

• see comments, just submitted (no locking guarantee, consider using kazoo locking, removing KVInterface)

DM-198

• progress, but not as far as planned

DM-626

• "distinct" was never handled
• "order by" - old parser passed everything

DM-207

• divide into: 
  • migrate Qserv into the new logging system, remove old logging (2 weeks)
  • polishing what messages are printed, debug levels etc

New issues opened since last meeting:

• DM-661 Parser has inverted order for "limit" and "order by"
• DM-659 running individual query in the automated test suite
• DM-658 automated tests should use their own dedicated database, not "LSST"
• DM-657 qserv initialization broken when previously registered database does not exist
• DM-656 selecting from non-partitioned table is broken
  • was table unpartitioned?Jacek will check
• DM-655 unknown column derails Qserv
  • squashing will be rewritten in DM-70
• DM-664 "out of range value" message when running qserv-testdata (loader.py)
• DM-666 DM-366 partition package should use sconsUtils to detect eups-related boost

Action Items

• Jacek Becla update DM-510
• Jacek Becla open issue about unassociated sources
• Unknown User (danielw) Write short description why we need subchunking
• Unknown User (danielw) Add comment about valgrind messages in DM-220
• Jacek Becla Determine name for logging package
• Jacek Becla,  Add comments to DM-70: what was ugly about existing code eg AsyncQueryMgr
• Serge Monkewitz Add comments to DM-70: what was ugly about existing code eg AsyncQueryMgr
• Unknown User (bchick) Add comments to DM-70: what was ugly about existing code eg AsyncQueryMgr
• Jacek Becla find out if ok to rename git module
• Jacek Becla break DM-207 into two
• Jacek Becla check on DM-656
• Jacek Becla update DM-655