Date
Attendees
- @AndyS @gapon @vaikunth @natepease @jgates @FabriceJammes @MichaelKelsey @AndyHanushevsky
Status Reports
@FabriceJammes: Monitoring work (DM-5885). Will sit with Fabien to provide monitoring reports from large scale tests. Fixed docker git script (DM-6130). Command line complex, will provide docs. Will work on Docker swarm (DM-5967). With Walid (student at IN2P3) working to set up scripts – can he get NCSA account?
@natepease: Butler stuff. RFC-184 closed as adopted; working on implementation. Storage factorization.
@vaikunth: Progress cleaning up Kibana dashboard, now gets only nodes we care about. Can adopt and do analysis on data; some binning effects due to pre-aggregation?
@AndyS: Finishing tech note about L1 database schema, revision, etc. Need reviewer before publishing (soon!). Next step to start prototyping implementation. Biggest unknown: new L1 for each release, or accumulate forever? Will look at MySQL vs. PostgreSQL. Must be partitioned, based on time -- newest is active; close when full and open new one. Avoid searches across multiple partitioned.
@gapon: First week; getting up to speed. Still need HipChat account, NCSA, IN2P3 (request access to QServ cluster). Want to build/test software before leaving. Get VMware installed on Mac.
@MichaelKelsey: Adding objectId to internal containers in lsst/partition, in order to dump secondary index.
@jgates: Threads throttled much more than expected on cluster (DM-5910). Possible solution: use single pool of threads to allocate among requests; enforces maximum total usage. Bigger problem: proxy gets too much data back, then dies -- eats system resources rapidly then stops, taking czar with it. Very narrow window of "safe" queries which both exercise new code and doesn't kill proxy with results.
@AndyHanushevsky: Will discuss face-to-face w/John some changes to simplify his proxy stuff. Logging in XRootd essentially done except for interface to QServ logger (DM-4473). Sequencing releases to coordinate with QServ in process. Working on memory management stuff as well.