Database Meeting 2023-04-26

Fritz Mueller :

• summit shut down in late May or early June planned for an electric upgrade
• leaving this Tue, be back by mid-May.
• need to set up the planning exercise for Qserv after getting back from Chile: remaining bugs, features that were planned to be done and which havn'e, new features, and what else are we going to do next. It's also a subject for the budgetary situation. Will begin looking at JIRA tickets to make a list of what's left. GPFD, Frossie, Colin and the DAX team to be a part of the discussion. Stuff assignments are going to be a topic. We might have a chance to make a case to get another Qserv developer abroad. Not guaranteed though. First, we need to set up a plan for the "end game".
• Colin Slater and Yusra are at Fermilab 

Igor Gaponenko:

• qserv6prod  expanded from 6 to 15 nodes in the same way it was done earlier for qserv6dev. The instance is back to production.
• Fritz Mueller there was an issue with TAP upstream (missing schema for DP02). It's been resolved. Verified that Notebooks work with new Qserv in both sync and async query submission modes. So, functionally it works
• a surprising latency has been observed in query turnaround compared with IDF
• we should also have the MOnkey running against that instance. Need to check.

Fritz Mueller, Fabrice Jammes: any updates on the status of the Kubernetes-based deployment mode?

• Fritz Mueller will talk to Yee informally to see if any formal steps are needed to move it forward.

Done. This is in the main  branch of Qserv. Anything else we need to discuss here?

Fritz Mueller will move to the next release branch after 

Fritz Mueller: there was a plan to modify Qserv's RelationGraph to generate faster queries where the RefMatch tables are involved. The idea is not to materialize sub-chunks and to put a restrictor on the sub-chunks for the "mother" tables. In this case, MySQL would benefit from indexes on the latter.

• no progress on this yet
• will work on it later

Igor Gaponenko :

• the EPIC branch has been cleaned up, commits rearranged and rebased against the latest state of the main  branch.
• DM-38069 - Getting issue details... STATUS
• Qserv locks were detected when testing the branch using a full-sky NN query against the 150k chunks catalog GAIA_DR2 at USDF. Further tests indicate the same problem exists in the current state of the main branch. The official release  2023.4.1-rc1 looks fine though. It's possible the problem was introduced in one of the following branches:
  • DM-38730 - Getting issue details... STATUS
  • DM-38257 - Getting issue details... STATUS
• this needs to be further investigated

Fritz Mueller proposed a few ideas on what to investigate next:

• test Qserv at the intermediate state in between the above-mentioned merges
• increase the large result limit to allow queries to complete
• run the COUNT_STAR version of the queries to see if we would have the same problem

John Gates :

• add more log messages

Andy Hanushevsky :

• enable SSI tracing to see if "Finish" get called

Fabrice Jammes:

• still having trouble with the UNIX account/password at SLAC

Fabrice Jammes:

• Greg's cluster at UkDF is working now

Context:

• PREOPS-3374 - Getting issue details... STATUS