Leanne Guy

News from stack-club : SST roster

Summary of EuroDAC meeting

• Europeans are putting in a joint funding request to the EU to set up an independent DAC. There is much interest around how User generated data products will work

  • The DES model is that all users have own user spaces, direct queries and joins against DES tables are possible. There are quotas at group level. DES has working groups that are somewhat equivalent to LSST SCs. If a table is considered to be of widespread use it is moved up to a release data space and is not counted against a user space. The process to decide what gets "moved up" is very informal. This is manageable with only 400 users.
  • This model is analogous to that of SDSS / Gaia
  • DESC DC2 plans are to use flat files for User Generated products and provide an interface. At this stage they do not plan to use a DB for UG data products.

• Leanne Guy plan to write a TN for UG use cases with input from Michael Wood-Vasey for DESC 30 Aug 2019 DM-23297 - Getting issue details... STATUS   

Gregory Dubois-Felsmann

Colin Slater

Summary of points in order of priority

• Authorisation and Authentication processes were worked on at the level of token flow diagrams. NCSA working on federated identity management
• Demos of detection footprint visualization integrated with table browsing. Probably the only LSST specific visualization that will be delivered given the descope
• Demo of firefly with Jupyterlab integration mechanism. The user no longer needs to know Firefly host details. 
• DASK visualization with Gaia data
• TAP service demo that meets requirements for cross platform integrations based on asynchronous TAP queries, working with TOPCAT and supporting VOTable.
• CADC cloudbursting to amazon was also discussed which could. IVOA service implementation discussed,Fritz wants to prioritize delivering a working TAP service for the Portal first (within the constraints of the descope)
• Spark/DASK relative merits discussed. Parquet as file format and backing store has been converged upon. But the allocation and management of the computational resources is still an open topic. Still building up a N2DB system piecewise. A lot of progress has been made.
• Note RFC-465 that implementd the baseline of Parquet as a formal storage for tabular data and the input format for QServ loading
• Review of supported VO standards. Adopting CADC TAP service
  
  

• Gregory Dubois-Felsmann will create an RFC for DMTN-090 and supported VO standards 25 Jul 2019 

Simon Krughoff

Based on the list of tasks defined on the bootcamp preparations page, can we decide which middleware version to use and what (if any) functionality needs to be worked on to be ready for the bootcamp

• Simon was absent this week

Melissa Graham

• Wide fields Astronomy in Canada
• Swift-LSST Synergies

Answers to questions around computational resources of the Science Platform still leave people with the impression that what LSST will provision in the US DAC will not be enough

Colin Slater added that we need to understand how our DAC and system interacts with reousrces beyond our control (e.g independent DACs) and how they will drive our design

Bulk download is currently specified as being DAC to DAC. There is no provision for DAC → individual. We need to resolve this before the LSP review. It will be a big problem if data rights holders cannot download the LSST catalogs.

• Leanne Guy to trigger second look at DAC → individual model for bulk download 25 Jan 2019   DM-18877 - Getting issue details... STATUS

See list below

• Given other priorities the demo of the proposed technology for glossaries and acronyms is delayed until end 2018
• LSP design review draft charge is late - the draft a list of presentations date is hence shifted to end November
  

•  Existing SUIT use cases (LDM-244) have been resurrected  Gregory Dubois-Felsmann and Leanne Guy  to review offline existing SUIT use cases (LDM-244) 25 Jul 2019  - rendered irrelevant by DM-10 Portal "freeze", superseded by  DM-23046 - Getting issue details... STATUS