the new version of the R-I system adds the worker registry that needs to be integrated into the operator
the latest bugfix release of Qserv should be used by the operator: qserv/lite-qserv:2022.4.1-rc1
support for the liveliness probe is about to be merged
DM-34404
-
Getting issue details...STATUS
Status of DP02
team
DP02 is supposed to be ready in June 2022
Support for ingesting the image metadata tables Visit and CcdVisit into Qserv has been added. Both tables have been test-ingested into the
Questions/problems that are still remaining:
technical aspects of using Felis as a "source of truth" for preparing table schemas as needed by the Qserv ingest protocol (JSON is expected, while Felis s still producing the "traditional" MySQL CREATE TABLE statements.
publishing/unpublishing/extending catalogs to add more tables and adding rows to existing tables. What protocol/API do we need here?
manually ingest tables Visit and CcdVisit into Qserv into the small first and into IDF after that
work on improving the Ingest system to add support for adding tables in the long-run
then work with Colin Slater on ingesting the final version of the Object table into IDF (dev or int?). No need to ingest it into the small cluster at NCSA. Then work on improving the Ingest system system
get in touch with Hsin-Fang Chiang on how to use Felis for generating table schema in the JSON representation, and using the ingest workflow
concerned about 8,000 MySQL partitions limit per table in the long run
the first option would be to let users do it using SciSql (the current implementation is not suitable for that). A possible solution would be to have the right UDF function in SciSql for this transformation.
another option would be to set up views on the object tables
Igor Gaponenko: the Ingest system could be improved to support creating views
Fritz Mueller: at the meantime, we could create views manually in IDF
Discussion on using/supporting views in Qserv:
Igor Gaponenko: we need to see if Qserv correctly supports views. Views need to be described in Qserv's CSS.
Andy Salnikov: indexing won't work on views. Perhaps it's better to implement views at czar.
Unknown User (npease): Query support has been implemented for unconstrained COUNT(*).
Igor Gaponenko: we need to discuss how to construct the row counter tables in Qserv, if this needs to be done automatically at each ingest or consider this as an optional optimization to be done after the fact (publishing catalogs) as we do it for the table local indexes at workers.