Page History

Fritz Mueller:

• generator power at Summit due to power line maintenance from the "mainland" to Summit
• no major news from TCAM; everyone is getting ready for joined operations review (SLAC, end of February); A chance of Qserv demo (via notebooks)
• DP03 is gaining more attention on DMLT (it's now on the weekly agenda) 
• DP03 won't be ingested into Qserv. This is the Solar System catalog.

Colin Slater :

• no formal requirements document yet for DP03 from GPDF

Fritz Mueller on the baseline ("aspirational") schema changes:

• the current version doesn't correspond to the pipeline reality
• many view the schema from the pipeline perspective and find some aspects of the schema confusing
• it's useful for us (the DAX team) to have some reference documents even though they may not be fully compatible with the ongoing data model of the pipeline
• no schema browser exists yet for the "aspirational") schema

Colin Slater (to continue on the subject)

• the situation has changed significantly over the last 2 years
• before DP02 no tables were generated, and other tables looked differently; now the DP02 covers ~90% of the baseline schema; there will be differences in the number of columns, column names, and data types 

Fritz Mueller: reasons, solution, lessons?

• an attempt to build Qserv failed at the CI test
• fundamental reason:
  • we depend on the latest version of the Python packages
  • (though) MariaDB and mysql-proxy versions are fixed after some earlier (last year) incident
  • change in the Oracle MySQL connector
  • change in MyPy
  • suspected a problem in the multi-statement SQL execution (triggered by smig).
  • a bug in the latest connector is suspected
  • or it could be a problem in our own code that might not behave correctly
  • documentation build was broken for some packages
• solution:
  • unrolled back a few packages
• lessons:
  
  • pros and cons for freezing packages vs following and using the latest and greatest
  • Andy Salnikov: it's the same problem in ATLAS, where people rely on the latest versions of the software packages
  • Igor Gaponenko: set up another (parallel) CI thread to be run "on a timer" by pulling the latest and greatest packages

John Gates: Any further progress? 

Fritz Mueller: Are we going to deploy and test the new Qserv release 2023.1.1-rc2  at IDF this week? 

Yes, it will be deployed and tested first at qserv-int. This may happen tomorrow.

The latest version of both has been built. Fabrice's QA test has passed.

Fritz Mueller and Fabrice Jammes will work on merging both packages into Qserv as the next project