Skip to end of metadata
Go to start of metadata

Date

Attendees

Discussion items

Qserv software on in2p3 cluster

  • unify how we install/manage qserv @in2p3, see newly opened DM-3199

Large scale tests

  • debugging,  latest problems look like a bug in log4cxx

Ingest

  • almost done, dealing with complexities of aliasing in afw table code

mysql-based kv interface

  • flat key/value structure
  • children - implement through a separate table that keeps info about relationships
  • ok to keep key length under 512 bytes

managing credential files for integration tests

  • config files in home dir? AndyS not convinced. Need to find "better" solution

Integration tests and  CI

  • already have dockr file that runs single-node, reuse that and develop a script that could be used by CI system
  • later support multi-node tests (harder to do, but useful to have)

Planning discussion

  • distributed loader
    • Can't test without infrastructure (fast shared fs). Discuss w/NCSA at Bremerton
    • do exploration and plausible design, postpone implementation until we have hardware to test it on
    • do we need epic for loading images? Discuss w/K-T
  • non-partitioned tables
    • maybe duplicate most frequently columns, use shared fs for the rest
    • simplest would be to duplicate all non-partitioned tables
    • revisit if we could duplicate all non-partitioned tables on all nodes
    • need infrastructure (scalable shared fs). Discuss w/NCSA at Bremerton
    • BTW, we still have to support L3 shared fs
  • dbserv
    • POST related story is for async queries, move to S16
  • near neighbor optimizations
    • neighbors-of-neighbors idea won't work (e.g. won't find neighbor if it is far away and there are no objects in between that could be used for chaining)
    • worth exploring how to optimize, not high priority
    • explore through XSWAP