Skip to end of metadata
Go to start of metadata

Date

Attendees

  • @AndyS @gapon @vaikunth @natepease @jgates @FabriceJammes @MichaelKelsey @AndyHanushevsky

Status Reports

Fabrice: Monitoring work ( DM-5885 - Getting issue details... STATUS ).  Will sit with Fabien to provide monitoring reports from large scale tests.  Fixed docker git script ( DM-6130 - Getting issue details... STATUS ).  Command line complex, will provide docs.  Will work on Docker swarm ( DM-5967 - Getting issue details... STATUS ).  With Walid (student at IN2P3) working to set up scripts – can he get NCSA account?

Nate: Butler stuff.  RFC-184 closed as adopted; working on implementation.  Storage factorization.

Vaikunth:  Progress cleaning up Kibana dashboard, now gets only nodes we care about.  Can adopt and do analysis on data; some binning effects due to pre-aggregation?

Andy S: Finishing tech note about L1 database schema, revision, etc.  Need reviewer before publishing (soon!).  Next step to start prototyping implementation.  Biggest unknown: new L1 for each release, or accumulate forever?  Will look at MySQL vs. PostgreSQL. Must be partitioned, based on time -- newest is active; close when full and open new one.  Avoid searches across multiple partitioned.

Igor: First week; getting up to speed.  Still need HipChat account, NCSA, IN2P3 (request access to QServ cluster).  Want to build/test software before leaving.  Get VMware installed on Mac.

Michael: Adding objectId to internal containers in lsst/partition, in order to dump secondary index.

John: Threads throttled much more than expected on cluster ( DM-5910 - Getting issue details... STATUS ). Possible solution: use single pool of threads to allocate among requests; enforces maximum total usage.  Bigger problem: proxy gets too much data back, then dies -- eats system resources rapidly then stops, taking czar with it.  Very narrow window of "safe" queries which both exercise new code and doesn't kill proxy with results.

Andy H: Will discuss face-to-face w/John some changes to simplify his proxy stuff.  Logging in XRootd essentially done except for interface to QServ logger ( DM-4473 - Getting issue details... STATUS ).  Sequencing releases to coordinate with QServ in process.  Working on memory management stuff as well.