| Time | Item | Who | Notes |
|---|
| Log Diet |
| Andy, Igor, John, and Nate will do their bit to downgrade or remove a log statement, each on their own ticket.
Log rotating is not turned on for PDAC. BUT, we figure it's better to wait for k8s:
1. right now we log to standard out, and that gets redireceted to a file.
2. k8s has it's own logging capture from stdout
3. so we don't want to change logging to a file
4. which means we don't want to use the log4cxx log to file + rotation in favor of waiting for k8s
We don't want to turn off xrd logging, even though it's very noisy. |
| Auth | Andy S, Christine, Colin | Per Fritz: db auth is coming over the horizon. @chb expressed some technical concerns to me at PCW wrt. the
current “plan” (pushing auth down to the czar level), and @andy-s expressed concerns as well wrt. the amount
of work involved in supporting the newer auth-plugin-capable mysql client protocol at the proxy/czar level.
Can/should we enforce auth at the TAP layer instead? (Discuss; use both sides of paper if necessary.)
Fritz says the current plan is NOT to allow users to directly connect to qserv though it would be nice to not
throw this away entirely. Discussion
----------
ANDY: There is not the right people here to discuss auth.
CHB: what can we show in the top layer?
COLIN: nothing from other users.
CHB: so I need to be super careful at the top layer.
CHB:
Use a service account for qserv? Or not?
Will we allow qserv connection directly - if so do we need auth at a lower level?
ANDY:
Managing authentication is not in the scope of qserv, we need someone else to provide us credentials.
Authorization for access to the tables will have to be qserv's responsibility:
- no one else wants to do it
- there is no easy way to export the role to another entity
- we want to rely on mysql authorization to do this.
CHB: Who is reponsible for granting auth to users? What about user data?
ANDY: I hope it's an external tool to manage it, and qserv can synchronize it's auths with that tool.
CHB:
TAP has to be aware of all the tables in the database that it's going to use - it has to be aware of the user
tables & their schema & whatnot. And it will apply authorization to use of those tables. This support exists
for temporary data upload (stored only for that query) and existing permanent data, but not yet for long-term
user data. CADC might be working on their auth-aware parts of the TAP service for this, and we might be able
leverage that.
There was some discussion of user groups; Colin says group-level auth sounds quite valuable.
COLIN: Users will only have access (or not) to specific data releases. |
| Test-ingest of Science Pipeline products into Qserv using the new Ingest System | Igor, Joanne, Colin and Hsin-Fang | A plan is that Igor will work with Hsin-Fang to process and ingest the first batch (tract) of the HSC data using a simple “pusher” script. The goal here is to work out various aspects of configuring (the future) workflow. (edited) |
| Discussion about current issues with query dispatch in Qserv. | John, Igor, Andy S, Nate | It seems there are too many queries getting serially queued & sent to workers. It is not known exactly what the bottleneck is. There is some discussion that we need to lower the number of separate queries sent, but also some of us think we need to understand exactly what the problem is so we don't run into a similar issue again later. |