Date
Attendees
Local: Igor Gaponenko, Vaikunth Thukral, John Gates, Brian Van Klaveren, Andy Hanushevsky
Remote: Fabrice Jammes, Fritz Mueller
Discussion items
Fritz:
- At DMLT in Tucson - very productive, 5 year re-plan and resource loading needs to be finished 80% by end-of-week
- Management folks present to answer all questions
- Preparation for reviews is underway
- Bringing up DAX services on PDAC should be our point of focus for the upcoming week
Fabrice:
- PDAC - Qserv is installed but still some issues with container management
- Basic installed image/version and integration tests are working
- Loading SDSS data in local instance at Clermont. Handing over that config to Igor just in case
- Stripe82 bulk loading prep is finished, loader is installed on all nodes and data is on GPFS
- Can use integration test scripts to figure out (reverse engineer) loading procedure like Jason Alt did when he visited SLAC (not recommended)
- Group decided to name the database at NCSA "stripe82"
- For backup at PDAC (since no solution exists yet) - can manually copy data dir after loading is finished
No update on Swarm yet, just minor xrootd config improvements
Consider Rocket instead of Docker for containers?
John:
- Finished Qserv related issues from last week, on to PDAC
- Reviewing Fabrice's ticket DM-7104
- Having trouble pulling Qserv images from docker hub, Fabrice will look into it
- Need another node for developer specific tasks/monitoring
Another procurement of machines coming up next month, can ask for one dedicated for that
- Alternate option - make a VM in OpenStack and figure out networking options needed
Brian:
- Working on Docker containers for services at PDAC
- Mostly busy with Slack migration
SLAC HR told Brian to talk to University HR to create new Big Data positions, for systemwide reclassification definition
Igor:
Working on QA between data at in2p3 vs PDAC
coadd, images and forced sources along overlap covers small fraction of the sky, hole in the patch
Consider having 2 DBs in Qserv - "stripe82" and "stripe82_v1"
Gregory requested having coadd images accessible on GPFS, which is now done and all coadds appear to be there
Finishing migration from in2p3
Trying a loader config in a distributed fashion rather than how it was done earlier
Chunk to worker mapping - consider preloading mapping before loading data with qserv-admin
Fritz's DHT prototype could help Igor in figuring out how to parallelize the loader - service model
AndyH:
- Recently back from CERN
- Reading up on sizing models
Vaikunth:
Continue to debug spurious recorded values. Narrowed down - observed different numbers for same query!
Diff file shows the different values when query submitted at different times on exact same data (same number of lines in result too)
Fabien ran his queries directly on the backend REST interface, Vaikunth uses the browser Sense plugin, not sure if that is the cause of the bug/problem
Vaikunth and Fritz @ LBNL on Nov 1 for XSWAP renewal strategy meeting
Other/Monitoring:
Nagios @ PDAC is default, we should ask for Ganglia and not sure whether we should use ELK
Lots of options available, more discussion and follow up between Fritz and Fabrice/Fabien
- Consider moving entire ELK stack config from in2p3 and go all out on monitoring at PDAC? Could also help Vaikunth for XSWAP work