Skip to end of metadata
Go to start of metadata

Date

 –   (DMLT Days)

  (T/CAM Day)

Remote access: (new ls.st link each day)

Attendees

Goals

  • Quarterly face to face meeting of DM Leadership Team (DMLT).
  • Refer to LDM-294 for charter of the DMLT.

Carpooling

Arrivals

JFB: arriving ORD at 10:15pm on Sunday (could drive late that night, or Monday morning after staying one night in Chicago).

Jeff Kantor: arriving at MDW 12:05pm Sunday, renting car driving to Champaign

Robert Lupton Could arrive Monday morning or Sunday night.  Would like to beg a ride 

John Swinbank: likely arriving ORD at 05:02 on Monday morning. (Hoping to beg a lift from somebody driving from Chicago Monday morning.)

Unknown User (xiuqin): plan to arrive ORD on Sunday at 4:16pm by United. (may stay at Chicago and like to beg a ride to NCSA Monday morning.)

Frossie Economou: (plan to) Arrive Midway Sunday night, pick up car, drive to ORD for hotel, drive down early Monday morning with ADD YOUR NAME HERE IF YOU WANT A RIDE AROUND 7am MONDAY

Departures

JFB: departing ORD at 7:00pm on Wednesday.

Robert Lupton Could leave with Jim Bosch.  I'll contact Laurie...

Jeff Kantor: departing NCSA ~3:00pm (depends on T/CAM day agenda), driving back to Chicago, turning in car (overnight near MDW, depart MDW Friday 8:30am)

John Swinbank: Departing ORD 8:16pm Thursday evening. (Hoping to beg a lift from somebody driving to Chicago Thursday afternoon).

Gregory Dubois-Felsmann: Departing ORD 8:20pm Thursday evening. Looking for a ride. 

Unknown User (xiuqin): Plan to leave NCSA Thursday afternoon, fly out ORD Friday Morning at 10:17am by United.

Frossie Economou: plan to drive to ORD with ADD YOUR NAME HERE IF YOU NEED A RIDE on Thursday and fly out Friday morning

*** Note the outbound flight times from CMI are 2:45 pm and 6:05 pm

Logistics

The meetings will be held at the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana-Champaign. 

NCSA is located at 1205 W. Clark Street, Urbana, IL, 61801

Local hospitality information:

  • Hampton Inn (across the street from NCSA).
  • Several hotels are available along I-74 between Neil Street and Prospect Avenue (Co Rd 1100 E). Note that they are not within walking distance of NCSA.

NOTE: Laurie Thomas is trying to book a block at the Hampton, contact her for hotel

Champaign Area: http://visitchampaigncounty.org/things-do/dining 

Closest cafeteria : Beckman Center, west of the NCSA 1205 West Clark Street building 

Agenda

Monday 18 May 2015

10:0010:30Meeting Prep: Set up and confirm audio/visual/telecom

NCSA

any remote attendees

Refreshments
10:3011:00

Meeting Start:

Introductions and Objectives

LSST Project Manager Remarks

Jeff Kantor

Victor Krabbendam

 
11:0011:30DM Organization

Jeff Kantor

 
11:3012:30LunchAllSelf-pay, per diem, cafeteria or local place
12:302:00DM GoalsMario Juric

Develop a shared understanding of DM high-level goals, including:

  • Science goals
  • SRD and DPDD
  • Key performance metrics
  • LSST users
  • Role of precursor surveys/data
2:002:15Break Refreshments during break
2:154:15DM ArchitectureKian-Tat Lim

Develop a shared understanding of the high-level DM architecture, including:

  • Toolkit, services, productions
  • Science components: afw
  • Science algorithms: measurement, coaddition, MOPS, etc.
  • Data access middleware: Butler, Qserv, other databases, Web services
  • VO interfaces: alert publisher, alert broker, VO access protocols
  • SUI
  • SDQA
  • Control middleware: Config, Task, CmdLineTask, IPC, workflow/orchestration
  • DMCS, productions, and operations

(Provides a basis for later discussions of alert system data flow, VO, SUI, and TOWG)

4:154:30Top worries and priorities

Mario Juric

Kian-Tat Lim

Initial thoughts on top areas that need to be attacked
4:305:30International networks and Brazil, IN2P3 MOAs

Jeff Kantor

Donald Petravick

Discuss International Networks Status, Brazil, IN2P3 MOAs for Ops Support

 

 

Tuesday 19 May 2015

8:158:30Meeting Preparation: Set up and confirm audio/visual/telecom

NCSA

any remote attendees

Refreshments
8:308:45

Meeting Start:

Daily Agenda

Jeff Kantor

 
8:459:30Development ProcessKian-Tat LimImprove our shared understanding of how we will be planning and executing software development
9:3010:00Clarifications and Discussions: Alert System Data Flow

Kian-Tat Lim

with Simon Krughoff and Unknown User (xiuqin)

Develop a shared understanding of the alert system data flow from alert generation, storage in database, subscription, to alert distribution to users
10:0010:15Break  
10:1511:00

Clarifications and Discussions: VO and SUI

Gregory Dubois-Felsmann

with Unknown User (xiuqin)

Define what needs to be done to coordinate with the VO and assign responsibilities

Develop a shared understanding of the SUI visualization role in all aspects of the system, including debugging the pipelines, pipeline production, SDQA, the camera, and other teams

11:0012:30Deliverables, status, and dependencies Part 1

06 Jacek Becla (presented by Kian-Tat Lim)

07 Margaret Gelman

08 Ron Lambert
Develop a shared understanding of each group's near-term deliverables and dependencies on other groups
12:301:30LunchAllSelf-pay. per diem, cafeteria or local place
1:303:30Deliverables, status, and dependencies Part 2

03 Simon Krughoff

04 John Swinbank

05 Unknown User (xiuqin)

01 Frossie Economou
Develop a shared understanding of each group's near-term deliverables and dependencies on other groups
3:304:00LDM-240 PrioritizationKian-Tat LimDevelop an initial draft of a tweaked LDM-240 attacking high priority areas first (KTL must leave at ~4 PM)
4:005:30Key performance metrics and HSC/precursor dataMario JuricDevelop a detailed plan for how key performance metrics will be measured, including amount of data, tests to be performed, cadence of tests, hardware resources required (note: no additional hardware is expected to be purchased or installed), software required, tentative assignments of who does what for remainder of S15
6:008:00Optional social dinnerAll who want to attendSelf-pay, per diem, local place

 

 

Wednesday 20 May 2015

8:309:00Meeting Preparation: Set up and confirm audio/visual/telecom

NCSA

any remote attendees

Refreshments
9:159:45Comet VOEvent demoSimon KrughoffQuick prototype to show Comet-based event broker
9:4510:00Break  
10:0011:00

Python/C++ boundary

 

Tim Jenness

Presentation of investigation into alternatives, discussion of costs and benefits

Develop a list of further information required to reach a consensus and/or develop a transition plan

11:0012:00

Technical Operations Working Group debrief

Gregory Dubois-Felsmann

Donald Petravick

Robert Gruendl

Convey an understanding of the TOWG charter, activities, and progress

Start technical leads thinking about what will be needed to operate their systems and how to minimize operations costs

12:001:00LunchAllSelf-pay. per diem, cafeteria or local place
1:002:00General activities at NCSA of interest to LSST

 

Donald Petravick

There are a few things / activity areas at NCSA that seem to be of interest to LSST. I'm off to investigate a bit more.

1) We have an emerging capability in another project that provides a frameworks for (apparently) both continuous integration, and also I think ad-hoc processing (think of capabilities for SUI and L3) in the same framework.

2) a concurent astro simulation / analysis meeting at NCSA

3) I may find more.

2:003:00Wrap up and plans for next meeting(s)Jeff KantorSummarize actions and decisions from this meeting, dates and logistics for next meeting(s)
3:00 Adjourn  

 


Thursday 21 May 2015

(T/CAM day)

8:309:00Meeting Preparation: Set up and confirm audio/visual/telecom

NCSA

any remote attendees

Refreshments
9:0010:30

Summer 2015 replan,

including resource adjustments, staffing/hiring considerations

Jeff Kantor

Kevin Long

It will be close to 3 months into Summer 2015, so replan will be about due.
10:3011:00BreakAll 
11:0011:30FY15 Spend status, FY16 budgeting assumptions

Jeff Kantor

Kevin Long

Assess where we are with respect to budget and spending YTD, as well as planning for FY16.
11:3012:00WBS/Control Account changes

Jeff Kantor

Kevin Long

Discuss proposed changes to WBS and/or Cost Accounts.
12:001:00SQuaRE/NCSA division of responsibility

 

Donald Petravick

Frossie Economou

Jeff Kantor

There are interfaces regarding development environment and tools, which are split across NCSA and SQuaRE. Need to clarify these in more detail and ensure WBS is consistent with this.
1:00 

Adjourn

T/CAM Retreat

 

T/CAMs were looking for some quality time attached to a T/CAM training meeting or a DMLT meeting. (informal, may be en route to Chicago or at ORD)

Suggested Agenda items

TimeItemWhoNotes
1 hourVirtual Observatory coordinationDMLT

 

We need to rationalize VO interface responsibilities, continuing discussion from last session. Now that Brian Glendenning is US representative to IVOA,

we have another avenue for impacting VO standards/protocols to address LSST requirements

1 hourSQuaRE/NCSA division of responsibility

Donald Petravick

Frossie Economou

Kian-Tat Lim

Jeff Kantor

Gregory Dubois-Felsmann

Mario Juric

There are interfaces regarding development environment and tools, which are split across NCSA and SQuaRE. Need to clarify these in more detail and ensure WBS is consistent with this.
2 hoursSummer 2015 replanT/CAMsIt will be close to 3 months into Summer 2015, so replan will be about due. On Thursday T/CAM day.
1 hourFY15 spend statusT/CAMsAssess where we are with respect to budget and spending YTD. On Thursday T/CAM day.
4 hoursT/CAM retreatT/CAMsT/CAMs were looking for some quality time attached to a T/CAM training meeting or a DMLT meeting. On Thursday T/CAM day.
1 hourInternational networks and Brazil

Jeff Kantor

Donald Petravick

Side session (not agenda item) to connect with Luiz da Costa to discuss Brazil MOA for network support.
2 hoursPython/C++ BoundaryDMLT (mostly DRP, AP, SUI, SQuaRE?)Discussions on how to get to a more Pythonic (and AstroPy-friendly) interface, where the Python/C++ boundary should ultimately be in our code, and alternatives to Swig. This is driven mostly by a need to improve library interfaces that will be used by science users (hence the "mostly" teams in the Who column), but of course tool choices and library conventions may affect other teams as well.
1 hourIntegration testing on HSCSQuaRE, AP, DRP teamsDRP team will want to start running larger-scale tests on HSC data regularly in S15, and can contribute code to getting that working. Need to negotiate with SQuaRE (and AP) teams who should do what, and what the balance should be between fast-and-easy vs. future-proof in how this fits with other plans to improve CI.
1 - 2 hoursAlert system data flowUW, SUI, NCSA, SLAC, John SwinbankTo fully understand the alert system data flow, from alert generation, storage in database, subscription, to alert distribution to users.
1-2 hoursGeneral activities at NCSA of interest to LSSTDon +?

There are a few things / activity areas at NCSA that seem to be of interest to LSST. I'm off to investigate a bit more.

1) We have an emerging capability in another project that provides a frameworks for (apparently) both continuous integration, and also I think ad-hoc processing (think of capabilities for SUI and L3) in the same framework.

 

2) a concurent astro simulation / analysis meeting at NCSA

 

3) I many find more.

1-2 hoursSUI visualization role in DM and LSSTall?support for camera, pipeline, SDQA, and possibly other LSST teams
1 hourTechnical Operations Working Group debrief

Kian-Tat Lim

Donald Petravick

The TOWG has been working on use cases and resource estimates for data processing operations. The DM members of the TOWG would provide an overview of the TOWG assumptions, use cases, and estimates to the DM LT.

 

Thursday Tony Johnson Visit — Agenda Items

TimeLocationTopicWhoNotes
10:00-12:00 CDTNCSA-2004Camera visualizationTatiana Goldina Unknown User (jjt) Unknown User (xiuqin) Gregory Dubois-Felsmann Trey RobyInterested parties are DM SUI group, Thaler's group.
TBD Understanding glast/Fermi Authn Location NCSA. Interested NCSA Parties are DM group, LSST ISO, NCSA security person
TBD Understanding glast/Fermi file management Location NCSA Interested NCSA parties are DM group.

Reference documents

Action items

From core DMLT meeting

  • Kian-Tat Lim Define standing time slots for design discussions
  • Gregory Dubois-Felsmann Create memo about whether a change request is needed to reflect the assignment of the alert mini-broker and interface responsibilities. : This overall issue went unresolved for a long time; however, there is now a consensus, as of the October 2016 T/CAM meeting, regarding the division of responsibilities among WBS elements.  It will be reflected in the DM replan documentation.
  • Jeff Kantor Discuss with Victor KrabbendamSteven Kahn how can DM scientists provide input into discussion (e.g. at TOWG) regarding science operations, software evolution, archive/dp operations.  What is the plan, roadmap for how TOWG and higher level governance team work leads to ops proposal, i.e the schedule, reviews, etc. Robert Lupton and others believe ops steady state is not until 3 - 5 years in, need at least 40% of construction staff level until then.  Tim Jenness thinks that the next TOWG meeting is on Tucson operations.  Unknown User (ciardi) says we should not pre-suppose locations of science operations, we should state what is needed.  
    NOTE: This message was passed on to Victor Krabbendam, he and Steven Kahn are aware and the process will be explained in the months ahead.

From T/CAM day

  • Kevin Long will not require teams to be set for epics to be imported into Primavera (WBS and cycle will be required).
  • Kevin Long: stories with 0 SPs will be imported into PMCS with a weight of 0. Stories with Null SPs will not be imported into PMCS.
  • Kevin Long: add a "target completed story points to date" column to Excel summary sheet (that is: (estimated story points) * (elapsed months) / cycle length).
  • Jeff Kantor: add "improvements" as equivalent to bugs & stories in workflow diagrams.
  • Jeff Kantor: change "epic tests KPM" to "epic relates to KPM" in workflow diagrams.
  • Jeff Kantor: set "DLP version" as equivalent to "DM cycle" in workflow diagrams.
  • Jeff Kantor: remove "bloks" link between "DM epic" and "DLP milestone" in workflow diagrams.
  • Jeff Kantor: discuss with Kian-Tat Lim the schedule for migrating LDM-240 from Excel to JIRA, including both a start date and a completion deadline.
  • Jeff KantorKevin Long: discuss the import of DLP information into PMCS.

1 Comment

  1. Unknown User (xiuqin)

    We could have T/CAM retreat Wednesday afternoon instead of Thursday afternoon. Everyone could catch a flight going home Thursday after lunch.