Database Meeting 2023-06-28

Date

28 Jun 2023

Time Item Notes

Ingesting user-generated data products

Colin Slater: this is a high-priority project for the team. The main focus should be on the temporary tables

requirements are supposed to be outlined in TAP: Interface to future Qserv temporary-table upload capability. The document is presently nearly empty.

"New" Qserv

Igor Gaponenko the PR is in code review:

John Gates thinks that once this file-based protocol will be deployed, the worker lock-up problems would disappear.

John Gates proposed enforcing large result limits at workers which would be possible with the file-based protocol.

On the multi-Czar option:

a couple of years ago John Gatestried an implementation based on running a few clones of Czar. Though, we didn't do a lot of testing of that version beyond verifying that it basically works.
Igor Gaponenko mentioned that this option may not be presently usable from the TAP perspective which expects a single "port of entry" (host:port) for Qserv. No load-balancing mechanism exists in TAP yet to allow forwarding queries to multiple ports. Besides, if such a mechanism existed, there would still be a problem with queries submitted via the ASYNC protocol.
Colin Slater insisted that the stability & functionality of Qserv has presently significantly higher priority than the multi-czar option.