another "hot" topic is the DM all-hands meeting in Chile (preliminarily March 13-17, 2023):
Yusra sent around the questionary for those who may be interested in participating (online versus in-person, arriving the weekend before or staying the weekend after)
It's formally for DM "construction" (not for "operation") which might be an issue for some of us.
a collection of tables (5 databases) within Qserv source tree within itest_src
a purpose of many queries is not well understood (or documented) with documentation links pointing to TRAC
Fritz Mueller we may still have the original TRAC pages migrated to Confluence
Action item: need to document each query
some queries are meant to test the non-existing functionality of Qserv (some sort of the "wish list" for future improvements?)
Fritz Mueller those might be based on the initial survey of what functionality was expected from Qserv
some may exist for testing the home-grown SQL parser (before migrating to ANTLR4
about 50% of the test queries are presently disabled (marked as FIXME, etc.)
some of those (disabled tests) are needed to cover the current functionality of Qserv
some might be disabled when migrating the tests to the new lite container or because the required functionality wasn't present in the Replication/Ingest system at the time of the migration
there is quite a bit of duplication between the tests (and catalogs)
some data ( CSV ) files are compressed, while others aren't. It's not clear why and what it's meant to test.
only 3 (out f 5) catalogs are presently tested
Conclusions:
it's a bit of a mess in there
Qserv coverage is not complete (or excessive) in some areas
the BIGGEST problem (for myself) was with using very specific table names that imply certain semantics in the context of LSST ("Object", "Source", etc.). Although the initial motivation behind that decision is clear, this naming convention is presenting a big obstacle in understanding what is actually being tested in Qserv. The semantics of some LSST tables has been changed since the original Data Model.
The proposal to be discussed:
revisit the test cases
eliminate duplicates and obsolete tests
add in the missing tests
come up with the Qserv-specific naming convention for the tables to reflect their role within Qserv ("director", "child", "ref-match", "fully-replicated", etc.)
refine the table schemas (and data) to leave only the essential columns (required for Qserv and for the referential integrity of the schemas), and add a few of the "payload" columns as needed where tests require row selection based on those values (shared scans, testing the WHERE clause etc.)
