You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Date

Attendees

Goals

  • Add topics to the table below

Discussion items

TimeItemWhoNotes

Project newsFritz Mueller 

News from DLMT

News from the ongoing 3rd Data Facilities Planning Workshop - 2022-04-05/07:

  • meeting recordings and presentations are available (see the link above)
  • service transition from NCSA is the biggest concern
  • Fritz Mueller will report on moving HSC reprocessing to USDF

B50 offices: more pressure is expected later this year


Progress on topics discussed at the last meeting Database Meeting 2022-03-30


Plans for building a new Qserv release to be deployed at IDF

Fritz Mueller 

Igor Gaponenko: the latest version of the Replication/Ingest system introduced the worker registration service which requires follow up changes in the qserv-operator. Details in: DM-33376 - Getting issue details... STATUS


Readiness for ingesting the DP0.2 products into Qserv

Discussed improvements to the architecture of the catalog ingest system (in a broad sense, including the "inner" Replication/Ingest system and the "outer" ingest workflows):

  • we are expected to take care of translating Parquet files and Felis schema files into intermediate products to be ingested into Qserv
  • should we consider moving the partitioning stage into the "inner" Ingest system rather than doing this at a level of the ingest workflow?
  • we need to keep thinking about the API to the "outer" ingest workflows for users (to reduce impedance for ingesting user-generated data products)

Problems with creating local table indexes at workers during catalog ingest

Fabrice Jammes 

Context: exceptions are thrown by the improved version of the qserv-ingest tools (the Python code) when attempting to create indexes. The problem is still being investigated.


Action items

  •  
  • No labels