Data Access Hangout 2014-11-24

Date

24 Nov 2014

Example urls for dataCat?

Testing qserv for SUI

jdbc issues fixed, still need to install latest qserv at ipac and verify
want more rapid cycle
ultimately will have machine(s) at NCSA, it is ~2 months away, purchasing now
short term: install qserv at slac for IPAC
- open mysql for selected ipac machine(s), ipac team can use ssh tunnel to connect

Performance optimizations

SUI wants high performance, near direct access speed, reconstructing fits headers requires making a copy of file
don't design something that rules out direct file access (direct file access might be needed for SUI, we won't expose it for end-users)

html-type access

not urgent
SUI can access qserv directly
but for qserv-proprietary interfaces, it should be web-wrapped from the beginning

User upload tables

tension between having one simple interface for users, vs multiple for performance reasons
example: 3-row upload table vs 3-million-row, opportunities to optimize for small tables
- these optimizations belong to SUI level
Level 3 in LDM-240 planned for 2019-2020 time frame
- SUI needs user upload tables much earlier
- doable, unless we want uploads concurrently with user queries
also, cross-matching will be essential during commissioning
- not clear if it would be done via database though

SUI requirements

sui team going through all sui requirements, exposing hidden requirements (like need for user upload tables in Qserv)

Next meeting