Notes from the previous meeting

Discussion items

(tick)Project news

Fritz Mueller :

  • camera in Chile
  • annual budget scrub is happening
(tick) IDF

Fritz Mueller :

  • Qserv -int  and -dev have been eliminated to save cost
  • Qserv -prod  is still up awaiting SLAC IT to open Qserv ports at USDF for IDF
  • Igor Gaponenko 's VM was left
  • DP02 bucket is left
    • Igor Gaponenko:
      • we may need these data for ingesting into the Kubernetes-base Qserv at USDF
      • there is tons of useful info on the ingest workflow and data transformations attempted during the ingest effort 2 years ago
      • If I'm not mistaken, Kian-Tat Lim might  mentioned that these buckets are being moved to SLAC(?)
        • Fritz Mueller ":
          • we need to double-check that
          • if that's not the case then we should move the bucket into the cold store which would be a significant reduction in cost
          • we also need to talk to the IN2P3 colleagues to see what they have

Notes from the USDF meeting:

    • no changes for Qserv
    • there was a subsequent discussion on opening ports
    • Fritz Mueller: waiting for SLAC IT / Security to open ports

Andy Salnikov :

  • worked on schema migration for APDB
  • need to extend the 
  • AP folks tried Cassandra's instance and discovered differences (mostly in assumptions and interfaces) between the new database and SQL (PostgreSQL). I'm working on resolving the issues.
  • Fritz Mueller people should be aware of the schema evolution limitations in Cassandra. It's not the same as for RDBMS.
  • Andy Salnikov we should have another person working on APDB as it's a time and effort-consuming project
(tick)Qserv releases

The current production release at USDF is:

The current production release at USDF is:

The next release to support user-generated data products has been built as well:

(tick)Qserv operator

Igor Gaponenko :

  • extended the operator to make it compatible with the existing Qserv release (the latest 2 releases)
  • it's supposed to be published as the release 2024.5.1-rc1 
  • ran into problems with building the release due to possible incompatibility between the operator, Golang, or Google Operator
  • still investigating

Igor Gaponenko :

  • Discussed an option to reimplement the operator in Python using the "Kopf" framework:
(tick)New Qserv

John Gates :

  • working on Czar-workers communications w/o using XROOTD
  • using "uber-jobs" to construct messages at Czar and reconstruct those at workers
  • it's a bit difficult to get both XROOTD/SSI and the new protocol working at the same time
  • a goal is not to miss (or break) anything important

Fritz Mueller :

  • ACTON ITEM for John Gates:
    • it would be great to get periodic technical presentations on the ongoing work
    • drafting an intermediate PR would be another option
      • John Gates:
        • I will try to do it by the end of the week
        • the code cleanup would be needed for that

Action items