Skip to end of metadata
Go to start of metadata

Date

Attendees

Goals

  • Advance the re-integration of the LSP components in the merged Kubernetes environment

Discussion items

TimeItemWhoNotes
Status of getting kubectl access to Kubernetes development environment for devs
  • See IHS-1637 - Getting issue details... STATUS
  • Michelle Butler says that nothing should have changed here and that SUIT developers should have kubectl access to the development environment.  She has asked for the list of such developers to be explicitly added to that ticket.

Progress in re-establishing the PDAC DAX services and Portal on lsst-lsp-int
  • DAX restart completed, see DM-17059 - Getting issue details... STATUS
  • Starting to work on reconnecting the Portal, partially blocked by IHS-1637, see DM-17072 - Getting issue details... STATUS , now assigned to Loi Ly , who should coordinate with Adam Thornton if any updates are needed at the cluster level.
  • It would be useful to have a discussion about how to actually implement the relocatability of the cross-Aspect connections (so that, by default, e.g., lsst-lsp-stable's Portal would refer to lsst-lsp-stable DAX services, and lsst-lsp-int's Portal would refer to lsst-lsp-int DAX services), but allowing for overrides. Adam Thornton has some ideas for this emerging from last week's successful effort to add a configuration override for the Notebook-to-Firefly connection.
    • Notebook need to talk to Firefly; currently handled with override environment variable
    • Notebook may eventually need to talk to DAX
    • Portal needs to talk to DAX
    • DAX shouldn't need to talk to anyone else
    • Should this be handled with service names? Possibly not if we want users to be able to change it.  Also effectively bypasses ingress including authentication; better for internal service connections.

Establishing a basic Firefly service on lsst-lsp-stable


  • What is a good way for the IPAC group to communicate to the LSP "ops team" (currently some combination of NCSA & SQuaRE) what container images are to be used for each instance? (Discuss, e.g., DockerHub image tagging conventions.)
  • For now, we're trying to have Adam Thornton coordinate, so communication should go through him.  As this is likely to prove unscalable, we'll need to come up with official procedures for release and deploy of new versions both in development and in production.



Authentication/authorization progress

It is really urgent to get this functional.

  • Status of rights-checking micro-service
  • Status of authentication proxy
  • Both are blocked on getting the ID token through the proxy to services behind it.  Most existing services apparently use access tokens instead.  Likely need to customize our own fork to pass the ID token or else use the access token to access the userinfo endpoint.  ETA still end of this week.
  • Client Aspects should use JWT authenticators (one exists for JupyterLab) rather than contacting the userinfo endpoint.
  • Brian Van Klaveren needs new IAM client ids for the OAuth2 proxy; these requests go to CILogon help, not IHS.
(if time permits)Progress on IVOA-ish DAX services
  • Status of SODA service work
    • We would be first to implement SODA and underlying VO services in Python on the server side; have to implement the entire "stack".
    • Not clear whether calling out from Java to Python is more difficult (wasn't really evaluated; seems to be possible due to Hadoop/Python example, but VO services may need more astronomical concepts in the framework code rather than in external "job" code).
    • First milestone is DALI; second is images; ETA for completion of both is this cycle (end of S19).
    • Using existing PDAC datasets as tests.
  • Status of TAP-QServ connectivity
    • Christine making progress after being blocked on syntax issues with qserv_* pseudo-functions. Expecting other ADQL issues afterwards.  Month to 6 weeks before running.  Simple queries can run now, but geometry will take a while.  Will come up with a list of increasingly-complex queries so that support can be "checked off".
    • Current portal has multi-cone queries that may be difficult for Qserv.  Should be supportable at Qserv layer, although possibly not with good performance (although performance is partly the point).  Not a particularly high priority.
    • Portal can retrieve records for a particular Object or Source, but needs an areaspec?  Object should not, as the "secondary index" provides an Object ID to chunk mapping.  Source is too big for a "secondary index", but part of its ID could potentially be decoded and used as a spatial hint.  We need to prepare for indexing columns within chunks (e.g. magnitudes), once they're identified.
(if time permits)Progress on WebDAV workspace access
  • WebDAV working with SciTokens but not yet ID tokens; should be simple once proxy is fixed.

Action items

  •