- Brief discussion of the current 1 arcmin "overlap distance" in the Qserv partitioning scheme. This is documented in LDM-135, section 3.3.6:
To avoid [the need for data exchange among shard servers], each partition can be stored with a pre-computed amount of overlapping data. This overlapping data does not strictly belong to the partition but is within a preset spatial distance from the partition's borders. Using this data, spatial joins can be computed correctly within the preset distance without needing data from other partitions that may be on other nodes.
Overlap is needed only for the Object Catalog, as all spatial correlations will be run on that catalog only. Guided by the experience from other projects including SDSS, we expect to preset the overlap to ~1 arcmin, which results in duplicating approximately 30% of the Object Catalog.
- Gregory Dubois-Felsmann expressed concern about whether the specific value of this limitation is well understood in the LSST science community, and suggested that we should do outreach no later than this year to ensure that it is and to collect any feedback that might affect our design. Private outreach to various collaborations followed by a presentation at the August All-Hands?
- Brief discussion of alternatives to true two-point queries, such as binning (by GROUP_BY). Any alternative strategies like this that emerge should themselves be added to our reference queries.
- Zeljko Ivezic suggested that in advance of the AHM we work to identify specific science use cases that require spatial joins, and understand those queries in more detail, before making any public presentation. He asked Gregory Dubois-Felsmann to file a ticket to record the need for this work.
- Fritz Mueller noted that, although the 1 arcmin value is documented (and under DM-level change control - gpdf) in LDM-135, it does not appear to be visible in any higher-level documents.