Date
– (DMLT Days)
(T/CAM Day)
Remote access: (new ls.st link each day)
Attendees
- Jim Bosch
- John Swinbank
- Jeff Kantor
- Unknown User (xiuqin)
- Gregory Dubois-Felsmann
- Unknown User (ciardi)
- Robert Lupton
- Frossie Economou
- Kian-Tat Lim
- Mario Juric
- Jacek Becla remotely on Monday only
- Tim Jenness remotely
Goals
- Quarterly face to face meeting of DM Leadership Team (DMLT).
- Refer to LDM-294 for charter of the DMLT.
Carpooling
Arrivals
JFB: arriving ORD at 10:15pm on Sunday (could drive late that night, or Monday morning after staying one night in Chicago).
Jeff Kantor: arriving at MDW 12:05pm Sunday, renting car driving to Champaign
Robert Lupton Could arrive Monday morning or Sunday night. Would like to beg a ride
John Swinbank: likely arriving ORD at 05:02 on Monday morning. (Hoping to beg a lift from somebody driving from Chicago Monday morning.)
Unknown User (xiuqin): plan to arrive ORD on Sunday at 4:16pm by United. (may stay at Chicago and like to beg a ride to NCSA Monday morning.)
Frossie Economou: (plan to) Arrive Midway Sunday night, pick up car, drive to ORD for hotel, drive down early Monday morning with ADD YOUR NAME HERE IF YOU WANT A RIDE AROUND 7am MONDAY
Departures
JFB: departing ORD at 7:00pm on Wednesday.
Robert Lupton Could leave with Jim Bosch. I'll contact Laurie...
Jeff Kantor: departing NCSA ~3:00pm (depends on T/CAM day agenda), driving back to Chicago, turning in car (overnight near MDW, depart MDW Friday 8:30am)
John Swinbank: Departing ORD 8:16pm Thursday evening. (Hoping to beg a lift from somebody driving to Chicago Thursday afternoon).
Gregory Dubois-Felsmann: Departing ORD 8:20pm Thursday evening. Looking for a ride.
Unknown User (xiuqin): Plan to leave NCSA Thursday afternoon, fly out ORD Friday Morning at 10:17am by United.
Frossie Economou: plan to drive to ORD with ADD YOUR NAME HERE IF YOU NEED A RIDE on Thursday and fly out Friday morning
*** Note the outbound flight times from CMI are 2:45 pm and 6:05 pm
Logistics
The meetings will be held at the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana-Champaign.
NCSA is located at 1205 W. Clark Street, Urbana, IL, 61801
Local hospitality information:
- Hampton Inn (across the street from NCSA).
- Several hotels are available along I-74 between Neil Street and Prospect Avenue (Co Rd 1100 E). Note that they are not within walking distance of NCSA.
NOTE: Laurie Thomas is trying to book a block at the Hampton, contact her for hotel
Champaign Area: http://visitchampaigncounty.org/things-do/dining
Closest cafeteria : Beckman Center, west of the NCSA 1205 West Clark Street building
Agenda
Monday 18 May 2015 | ||||
| 10:00 | 10:30 | Meeting Prep: Set up and confirm audio/visual/telecom | NCSA any remote attendees | Refreshments |
| 10:30 | 11:00 | Meeting Start: Introductions and Objectives LSST Project Manager Remarks | ||
| 11:00 | 11:30 | DM Organization | ||
| 11:30 | 12:30 | Lunch | All | Self-pay, per diem, cafeteria or local place |
| 12:30 | 2:00 | DM Goals | Mario Juric | Develop a shared understanding of DM high-level goals, including:
|
| 2:00 | 2:15 | Break | Refreshments during break | |
| 2:15 | 4:15 | DM Architecture | Kian-Tat Lim | Develop a shared understanding of the high-level DM architecture, including:
(Provides a basis for later discussions of alert system data flow, VO, SUI, and TOWG) |
| 4:15 | 4:30 | Top worries and priorities | Initial thoughts on top areas that need to be attacked | |
| 4:30 | 5:30 | International networks and Brazil, IN2P3 MOAs | Discuss International Networks Status, Brazil, IN2P3 MOAs for Ops Support | |
Tuesday 19 May 2015 | ||||
| 8:15 | 8:30 | Meeting Preparation: Set up and confirm audio/visual/telecom | NCSA any remote attendees | Refreshments |
| 8:30 | 8:45 | Meeting Start: Daily Agenda | ||
| 8:45 | 9:30 | Development Process | Kian-Tat Lim | Improve our shared understanding of how we will be planning and executing software development |
| 9:30 | 10:00 | Clarifications and Discussions: Alert System Data Flow | with Simon Krughoff and Unknown User (xiuqin) | Develop a shared understanding of the alert system data flow from alert generation, storage in database, subscription, to alert distribution to users |
| 10:00 | 10:15 | Break | ||
| 10:15 | 11:00 | Clarifications and Discussions: VO and SUI | Define what needs to be done to coordinate with the VO and assign responsibilities Develop a shared understanding of the SUI visualization role in all aspects of the system, including debugging the pipelines, pipeline production, SDQA, the camera, and other teams | |
| 11:00 | 12:30 | Deliverables, status, and dependencies Part 1 | 06 Jacek Becla (presented by Kian-Tat Lim) 08 Unknown User (rlambert) | Develop a shared understanding of each group's near-term deliverables and dependencies on other groups |
| 12:30 | 1:30 | Lunch | All | Self-pay. per diem, cafeteria or local place |
| 1:30 | 3:30 | Deliverables, status, and dependencies Part 2 | 01 Frossie Economou | Develop a shared understanding of each group's near-term deliverables and dependencies on other groups |
| 3:30 | 4:00 | LDM-240 Prioritization | Kian-Tat Lim | Develop an initial draft of a tweaked LDM-240 attacking high priority areas first (KTL must leave at ~4 PM) |
| 4:00 | 5:30 | Key performance metrics and HSC/precursor data | Mario Juric | Develop a detailed plan for how key performance metrics will be measured, including amount of data, tests to be performed, cadence of tests, hardware resources required (note: no additional hardware is expected to be purchased or installed), software required, tentative assignments of who does what for remainder of S15 |
| 6:00 | 8:00 | Optional social dinner | All who want to attend | Self-pay, per diem, local place |
Wednesday 20 May 2015 | ||||
| 8:30 | 9:00 | Meeting Preparation: Set up and confirm audio/visual/telecom | NCSA any remote attendees | Refreshments |
| 9:15 | 9:45 | Comet VOEvent demo | Simon Krughoff | Quick prototype to show Comet-based event broker |
| 9:45 | 10:00 | Break | ||
| 10:00 | 11:00 | Python/C++ boundary
| Tim Jenness | Presentation of investigation into alternatives, discussion of costs and benefits Develop a list of further information required to reach a consensus and/or develop a transition plan |
| 11:00 | 12:00 | Technical Operations Working Group debrief | Convey an understanding of the TOWG charter, activities, and progress Start technical leads thinking about what will be needed to operate their systems and how to minimize operations costs | |
| 12:00 | 1:00 | Lunch | All | Self-pay. per diem, cafeteria or local place |
| 1:00 | 2:00 | General activities at NCSA of interest to LSST
| There are a few things / activity areas at NCSA that seem to be of interest to LSST. I'm off to investigate a bit more. 1) We have an emerging capability in another project that provides a frameworks for (apparently) both continuous integration, and also I think ad-hoc processing (think of capabilities for SUI and L3) in the same framework. 2) a concurent astro simulation / analysis meeting at NCSA 3) I may find more. | |
| 2:00 | 3:00 | Wrap up and plans for next meeting(s) | Jeff Kantor | Summarize actions and decisions from this meeting, dates and logistics for next meeting(s) |
| 3:00 | Adjourn | |||
Thursday 21 May 2015(T/CAM day) | ||||
| 8:30 | 9:00 | Meeting Preparation: Set up and confirm audio/visual/telecom | NCSA any remote attendees | Refreshments |
| 9:00 | 10:30 | Summer 2015 replan, including resource adjustments, staffing/hiring considerations | It will be close to 3 months into Summer 2015, so replan will be about due. | |
| 10:30 | 11:00 | Break | All | |
| 11:00 | 11:30 | FY15 Spend status, FY16 budgeting assumptions | Assess where we are with respect to budget and spending YTD, as well as planning for FY16. | |
| 11:30 | 12:00 | WBS/Control Account changes | Discuss proposed changes to WBS and/or Cost Accounts. | |
| 12:00 | 1:00 | SQuaRE/NCSA division of responsibility
| There are interfaces regarding development environment and tools, which are split across NCSA and SQuaRE. Need to clarify these in more detail and ensure WBS is consistent with this. | |
| 1:00 | Adjourn T/CAM Retreat | T/CAMs were looking for some quality time attached to a T/CAM training meeting or a DMLT meeting. (informal, may be en route to Chicago or at ORD) | ||
Suggested Agenda items
| Time | Item | Who | Notes |
|---|---|---|---|
| 1 hour | Virtual Observatory coordination | DMLT
| We need to rationalize VO interface responsibilities, continuing discussion from last session. Now that Brian Glendenning is US representative to IVOA, we have another avenue for impacting VO standards/protocols to address LSST requirements |
| 1 hour | SQuaRE/NCSA division of responsibility | There are interfaces regarding development environment and tools, which are split across NCSA and SQuaRE. Need to clarify these in more detail and ensure WBS is consistent with this. | |
| 2 hours | Summer 2015 replan | T/CAMs | It will be close to 3 months into Summer 2015, so replan will be about due. On Thursday T/CAM day. |
| 1 hour | FY15 spend status | T/CAMs | Assess where we are with respect to budget and spending YTD. On Thursday T/CAM day. |
| 4 hours | T/CAM retreat | T/CAMs | T/CAMs were looking for some quality time attached to a T/CAM training meeting or a DMLT meeting. On Thursday T/CAM day. |
| 1 hour | International networks and Brazil | Side session (not agenda item) to connect with Luiz da Costa to discuss Brazil MOA for network support. | |
| 2 hours | Python/C++ Boundary | DMLT (mostly DRP, AP, SUI, SQuaRE?) | Discussions on how to get to a more Pythonic (and AstroPy-friendly) interface, where the Python/C++ boundary should ultimately be in our code, and alternatives to Swig. This is driven mostly by a need to improve library interfaces that will be used by science users (hence the "mostly" teams in the Who column), but of course tool choices and library conventions may affect other teams as well. |
| 1 hour | Integration testing on HSC | SQuaRE, AP, DRP teams | DRP team will want to start running larger-scale tests on HSC data regularly in S15, and can contribute code to getting that working. Need to negotiate with SQuaRE (and AP) teams who should do what, and what the balance should be between fast-and-easy vs. future-proof in how this fits with other plans to improve CI. |
| 1 - 2 hours | Alert system data flow | UW, SUI, NCSA, SLAC, John Swinbank | To fully understand the alert system data flow, from alert generation, storage in database, subscription, to alert distribution to users. |
| 1-2 hours | General activities at NCSA of interest to LSST | Don +? | There are a few things / activity areas at NCSA that seem to be of interest to LSST. I'm off to investigate a bit more. 1) We have an emerging capability in another project that provides a frameworks for (apparently) both continuous integration, and also I think ad-hoc processing (think of capabilities for SUI and L3) in the same framework.
2) a concurent astro simulation / analysis meeting at NCSA
3) I many find more. |
| 1-2 hours | SUI visualization role in DM and LSST | all? | support for camera, pipeline, SDQA, and possibly other LSST teams |
| 1 hour | Technical Operations Working Group debrief | The TOWG has been working on use cases and resource estimates for data processing operations. The DM members of the TOWG would provide an overview of the TOWG assumptions, use cases, and estimates to the DM LT. |
Thursday Tony Johnson Visit — Agenda Items
| Time | Location | Topic | Who | Notes |
|---|---|---|---|---|
| 10:00-12:00 CDT | NCSA-2004 | Camera visualization | Tatiana Goldina Unknown User (jjt) Unknown User (xiuqin) Gregory Dubois-Felsmann Trey Roby | Interested parties are DM SUI group, Thaler's group. |
| TBD | Understanding glast/Fermi Authn | Location NCSA. Interested NCSA Parties are DM group, LSST ISO, NCSA security person | ||
| TBD | Understanding glast/Fermi file management | Location NCSA Interested NCSA parties are DM group. |
Reference documents
- LSST overview paper, arxiv:0805.2366, http://ls.st/op
- Science Requirements Document, LPM-17, http://ls.st/srd
- Data Products Definition Document (DPDD), LSE-163, http://ls.st/dpdd
- DM Technical Baseline, Collection-2511 - of which see especially:
Action items
From core DMLT meeting
- Kian-Tat Lim Define standing time slots for design discussions
- Gregory Dubois-Felsmann Create memo about whether a change request is needed to reflect the assignment of the alert mini-broker and interface responsibilities. : This overall issue went unresolved for a long time; however, there is now a consensus, as of the October 2016 T/CAM meeting, regarding the division of responsibilities among WBS elements. It will be reflected in the DM replan documentation.
- Jeff Kantor Discuss with Victor Krabbendam, Steven Kahn how can DM scientists provide input into discussion (e.g. at TOWG) regarding science operations, software evolution, archive/dp operations. What is the plan, roadmap for how TOWG and higher level governance team work leads to ops proposal, i.e the schedule, reviews, etc. Robert Lupton and others believe ops steady state is not until 3 - 5 years in, need at least 40% of construction staff level until then. Tim Jenness thinks that the next TOWG meeting is on Tucson operations. Unknown User (ciardi) says we should not pre-suppose locations of science operations, we should state what is needed.
NOTE: This message was passed on to Victor Krabbendam, he and Steven Kahn are aware and the process will be explained in the months ahead.
From T/CAM day
- Kevin Long will not require teams to be set for epics to be imported into Primavera (WBS and cycle will be required).
- Kevin Long: stories with 0 SPs will be imported into PMCS with a weight of 0. Stories with Null SPs will not be imported into PMCS.
- Kevin Long: add a "target completed story points to date" column to Excel summary sheet (that is: (estimated story points) * (elapsed months) / cycle length).
- Jeff Kantor: add "improvements" as equivalent to bugs & stories in workflow diagrams.
- Jeff Kantor: change "epic tests KPM" to "epic relates to KPM" in workflow diagrams.
- Jeff Kantor: set "DLP version" as equivalent to "DM cycle" in workflow diagrams.
- Jeff Kantor: remove "bloks" link between "DM epic" and "DLP milestone" in workflow diagrams.
- Jeff Kantor: discuss with Kian-Tat Lim the schedule for migrating LDM-240 from Excel to JIRA, including both a start date and a completion deadline.
- Jeff Kantor & Kevin Long: discuss the import of DLP information into PMCS.
1 Comment
Unknown User (xiuqin)
We could have T/CAM retreat Wednesday afternoon instead of Thursday afternoon. Everyone could catch a flight going home Thursday after lunch.