Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Who/WhenFeedback from meeting

Anchor
SC2018
SC2018

(Please provide some brief notes from your attendance at the meeting, particularly with an eye toward continuity with the next SST representative who attends.)



  • Phil Marshall discussed a list of high-level projects that Stack Club participants could undertake. See
    https://github.com/LSSTScienceCollaborations/StackClub/issues?q=is%3Aopen+label%3Aproject+label%3Aopportunity 
  • Alex Drlica-Wagner expressed concerns about difficulties for Stack Club participants attempting to use the /project/shared/data workspace. Post-meeting discussions with Simon Krughoff resulted in a proposed recommendation to the Stack Club to use the /project/stack_club directory, which had already been created, for shared work within the Club. See the related issue
    Jira
    serverJIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2
    keyIHS-1319
    .









2019 

Anchor
SC2019
SC2019




Free space!  No meeting this week.




discussed v17 changes
Discussed audience for Stack Club.  Focus on processing steps good for a dedicated interested community.  But won't really be able to engage larger Science Collaborations community until there are data products following the DPDD accessible through the LSP.




Unable to attend due to meetings. No replacement was found

(Phase has shifted with June 7 → June 14.  Schedule is now June 14 + bi-weekly.)

Discussed plans for PCW Stack Club session.  Will have 6 junior members presenting Notebooks covering a range of topics; partial travel funding for these junior members provided by LSSTC.

Desire to make it on Tuesday afternoon to seed work and exposure for people to continue working on projects throughout the week.

Detailed planning for PCW workshop.  Some confusion about the new federated login system to the LSP (I advised most to stick to NCSA for now), and some problems with /jhome/ not being present which were reported to #dm-lsp.
Ran through 4 demo notebooks that will be presented at the Stack Club session at the PCW on Wednesday next week.
Quiet meeting - everyone one just worked in silence, at least until I had to leave for the SST meeting.
A few reviews of notebooks in PRs (in the background; I wasn't involved).  Quiet meeting, mostly people working independently.

Leanne GuyJeffrey CarlinMerlin Fisher-Levine attended. Quiet meeting - Leanne offered some commentary on notebooks she went through, and Alex+Douglas tested a Gen3 example notebook. 

We tried going through a test workflow of doing a PR with reviewNB.com.  It worked ok, but they didn't seem super excited to pick it up without it being integrated with GitHub.
Worked with Alex on exploring Gen3. Discovered Registry.getAllCollections(), and after some questions in #dm-middleware found that Registry.queryDatasets() is what we were looking for to discover what data was available in a repository.

Alex: want to morph stack club into a 6 week course for SC members: but probably not until ~March (so continued DM participation in the standard mode is useful until then)
DM help useful in that phase but less vital: “instructors” will be stack club members
    but DM participation would be welcome (either as help, or as participants) subject to overall caps

working on gen3 butler notebooks, some camera data

Alex had some questions about setting up weeklies on lsst-dev (vs. the LSP), and which weeklies are marked as current/default in each case.

Lee found that https://pipelines.lsst.io/getting-started/data-setup.html is broken now that ci_hsc is a metapackage–Eric encouraged him to file a ticket.

Greg Madejski asked me to comment on plans for introducing the science user environment for LSST to the community, and for documenting it.  This led to my walking him and Jeffrey Carlin through the current state of the LSP and talking about where it is going.  I emphasized that the Science Pipelines stack and the LSP are "peers" - we need to provide documentation, tutorials, etc. for both.  The stack is readily usable  in the Notebook Aspect of the LSP, of course, but each side of this pair is useful in its own right.  (The stack can be installed on a user's own computer; the LSP can be used without the Stack, obviously in the case of the Portal Aspect, but also in the Notebook Aspect by using pyvo, Astropy, PyViz, etc.)

Greg expressed considerable enthusiasm for even the present state of the Portal, especially a) the ability to bring a query from the Portal to the Notebook and b) the ability to bring in data from TAP queries to other archives, to use in concert with LSST data.

  •  Gregory Dubois-Felsmann will provide a documented "canned" example of Portal-to-Notebook (i.e., "a)" from the previous sentence) in notebook-demo  

I discussed the plans for the Data Preview releases (DP0 with precursor data, DP1 with ComCam data, and DP2 with LSSTCam SVV data).  We spoke briefly about the possible role of DESC DC2 data in DP0.  Greg expressed interest in something I've already been looking into: bringing the community-generated CatWISE catalog into the LSP (and adding it to DP0).

Given that I'm scheduled to be the contact for the next Stack Club session as well, Greg asked me if I could prepare for repeating this discussion with some presentation materials (as there was no advance notice of today's topic) at that meeting (on ).  I agreed to do this, and I think it's a great idea.  We can talk about this at the SST meeting on  .

Conversation revolved primarily around the planned "Stack Club School" in April of the coming year.  High level notes are as follows:
  • ~6x2 hour sessions held over either 6 or 12 weeks starting in April 2020
  • Expecting 10 to 20 students with at least one per science collaboration
  • Instructors will be pulled from previous stack club attendees
  • The hope is to have one DM rep at each of the sessions
  • Curriculum will cover material necessary to go from zero to stack club
  • Will need to be Gen3 aware

2020 

Anchor
SC2020
SC2020


  •  Colin Slater Stack Club   (this is the same week as AAS) 
Helped debug a possible problem with accessing read-only Gen3 repos for that tutorial (looks like it was just a version incompatibility), and discussed it a bit in general.

Sparse attendance.  Mostly people worked on their own.  Melissa Graham  is coming up to speed with the getting started docs and is looking to homogenize things like branch workflow etc.  Keith Bechtol was looking at the deblending notebooks and discovered that both examples were bit rotted.  I fixed one of them and Fred Moolekamphas agreed to look at the other one in coming weeks.

KSK: I didn't take into account that this was an off day in Tucson and no one was able to take Leanne's place on short notice.
  • Alex and Greg presented a proposed outline for the Stack Club School with presenters and potential DM individuals for support
  • Melissa helped several individuals getting logged in to the LSP and getting started with the intro notebooks

I actually was quite distracted for this meeting.  One of the users was having problems with her account and couldn't log in, so I spent most of the time firefighting that.

As far as I could tell, most people were just going through notebooks.

There were a couple of new people there, which was good to see.

Login issues; one user wasn't receiving duo pushes, so walked them through the manual code process. That did work, but it was hard to walk them through without a good written description.

User was wondering about the difference between x.getDimensions() vs. calling size(x); explained this as there being both pythonic and C++ish API paths.

Heard essentially no live discussion during the meeting, after an initial discussion about logistics and datasets for the upcoming course.  Spent most of my time there helping Merlin and Keith with mostly-unrelated-to-stack-club things, since we were all there anyway.
  • Minor annoyance that a user couldn't see what stack version they were using from the notebook, without knowing some magic commands.
  • One user got on the wrong track after selecting "avl-vpn-default" vpn group
  • Alex found that the twinkles repo was no longer readable.
  • Discussed stack-club addressing ticket 
    Jira
    serverJIRA
    serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2
    keyDM-4626
    . There are some existing examples brought over from DESC/NERSC that do not run with the current releases. First steps will be to update the existing examples. 
  • Discussion of  long-term data sets for notebooks (Simon and Alex)
  • Otherwise people mostly worked away on their projects
Missed much of the meeting due to sheer forgetfulness; joined near the end and answered a few questions.  Most work seemed to be prep for notebooks that may be useful at PCW sessions.

Simon Krughoff substituted for Eric Bellm , so Eric Bellm owes him one

This meeting was prep for the PCW.  I was in the main room while participants were largely in breakouts.




Nothing to report

  •  Stack Club  


2021 

Anchor
SC2021
SC2021







Every other week and 12th was an off week. 

  •  Stack Club  

  •   Stack Club  

  •   Stack Club  

  •   Stack Club  

...