You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 49 Next »

Logistics

COVID-19

Due to the ongoing COVID-19 situation, this meeting will be virtual. Please do not attempt to travel to Seattle — or to anywhere else — to participate.

Date

  – 

Location

Browser

Phone Dial-in

https://washington.zoom.us/j/95408203481

Please see #dm-camelot and/or e-mail for the password.

Meeting ID: 954 0820 3481

Dial by your location:

  • +1 253 215 8782 US (Tacoma)
  • +1 669 900 6833 US (San Jose)
  • +1 720 928 9299 US (Denver)
  • +1 971 247 1195 US (Portland)
  • +1 213 338 8477 US (Los Angeles)
  • +1 346 248 7799 US (Houston)
  • +1 602 753 0140 US (Phoenix)
  • +1 669 219 2599 US (San Jose)
  • +1 301 715 8592 US (Germantown)
  • +1 312 626 6799 US (Chicago)
  • +1 470 250 9358 US (Atlanta)
  • +1 470 381 2552 US (Atlanta)
  • +1 646 518 9805 US (New York)
  • +1 646 876 9923 US (New York)
  • +1 651 372 8299 US (St. Paul)
  • +1 786 635 1003 US (Miami)
  • +1 267 831 0333 US (Philadelphia)

Attendees

Agenda

Day 1: 2020-05-12

Time (Project)TopicCoordinatorPre-meeting notesRunning notes

Moderator: Leanne Guy


09:00WelcomeWil O'Mullane
  • Introductory remarks
  • Review agenda and code of conduct

09:10Project news
  • All ME20-03 variance narratives are now complete.
09:30Middleware status update
  • Timeline for deprecating Gen 2.
  • Current development activities.
  • Staffing plans moving forward.
  • Command line utility:
    • In the short term, will provide high-level repository management.
    • Ultimately, will provide limited query capability on the repository.
    • There is a document describing this.
  • July date for registry stability is of general interest.
  • Expect things to get busier in ~September, as Gen3 moves into general use.
  • Key transition point for developers is reprocessed HSC RC2 data being
    available in Gen3.
  • Quantum graph generation is slow, but reprocessing one tract or patch is
    fast enough.
  • Jim believes that scaling quantum graph generation to PDR2 in ~6 months is plausible.
  • What is the next tall pole after quantum graph generation?
    • There's nothing obvious.
    • Ingest may be a bit slower than Gen2.
  • Shared users are no longer an unsolved problem, but will be handled by
    naming conventions rather than technical mechanisms.
  • There was much discussion of the definition of the visit; it's not clear
    that this is really conclusive.
  • Not clear that adding new resources will really help with deadlines over the
    next couple of months, because the ramp up time is too long.
  • However, they would be useful for later in the year.
10:30Break

Moderator: Wil O'Mullane


11:00Calibration products
  • Brief review of the DMTN-148 proposals.
  • Is this document acceptable to the DMLT? What are the remaining open questions? How will we resolve them?
  • DMTN-148 is almost there suggest 2 weeks review by DMLT.
  • John to setup feedback system with Chris.

This should be baselined (change controlled)

Robert asks when we will start "acting on this" - e..g when could it be used for LATISS on the mountain. On going work from Andres and Merlin - where is the ingest and validate..

KT last stage getting from production system via oods to summit to be used for ISR on summit. Certified and transferred to where its needed.

Jim - Good to separate operations concerns (how its used on the mountain) from about the code and how we implement.  DMTN-111 could have the summit details.  Tim - no agreement on every curated calibration had class somewhere, one end - other is the certification

11:15Plans for IVOA and Python interfaces to time series data.
  • Follow-up on discussion within the DM-SST.
  • Headline: The DMLT agrees that the story we tell the community is that our data model is effectively two tables, and users will need to join them themselves.
  • General agreement about using PyVO and Pandas.
  • Are DIAForcedSources included?
    • The same considerations apply mapping DIAObject to DIAForcedSource.
    • Our feature computation may be based on DIASources or DIAForcedSources; a recommendation from Eric will be forthcoming.
  • Adding support for e.g. non-detection upper limits in feature computation is a possible, and may make the inputs to feature computation more complex. However, this should not be unmanageable.
  • How tightly coupled is the AP pipeline with the database? Is this a technical risk?
    • Reconstructing data structures from the AP pipelines based on VO interfaces would be challenging.
    • The details of feature computation are well abstracted and testable; they are not tightly coupled.
  • Plugins are implemented for feature computation below the task level; the master task takes a Pandas data frame as input.
  • None of these proposals are changes to previous promises made to the community.
  • In terms of announcements to the community, we suggest that this should be rolled into discussion of capabilities available for DP0.
    • Some discussion of a PST-SciCollab talk if necessary.
  • Eric Bellm — update time-series technote to contain a discussion of the way in which data will be presented to users.  
  • Gregory Dubois-Felsmann  — update the Science Platform design documentation to reflect that data access services should be tested with PyVO.  
12:30Break

Moderator: Wil O'Mullane


13:00Rebaselining & project scheduleWil O'Mullane
  • What's our current understanding of rebaselining?
  • How will DM respond to slips in the overall project schedule?
  • Calabrese coordinating mail pickup in Tucson.
  • Services which were used in commissioning/integration are easy to define as “done”.
  • Would be good to get a statement of thanks from project leadership to DM staff.
  • Aim to make “blurring” between construction and commissioning a positive opportunity.
  • Also look for opportunities in the deliveries to ops (but be careful that this is not blurring).
  • The details of financing, ramps, etc through FY22 & FY23 will have to be addressed on a case-by-case basis, depending on guidance from construction project management and the agencies.
  • Comments from Victor:
    • Covid 19 costs are not an appropriate use of current funding (baseline or contingency).
    • We will therefore adopt a new baseline; the so-called “over target baseline”.
    • The earlier we do this, the riskier it will be and the less accuracy it will have.
    • Currently seems like NSF will accept late replanning (October/November).
    • This is not an opportunity for us to reinstate previously-accepted descopes.
  • This information can be shared with the rest of the project.
  • Some concern expressed that operational priorities are different from construction priorities; we should be clear that staff transitioning do so in the project's interest, rather than just because it is financially expedient.
  • Victor is petitioning the agencies for a minimal status review this year.
  • The drawback of waiting longer for a rebaselining is that we have to live with bad metrics until it kicks in; might be an issue for reviews.
  • Not clear what the rebaselining process will be: could imagine an FDR-like process, but it's not clear that will be practical.
  • Concern raised that DM may be able to reach completion on close to original timescale.
  • Expectation is a 12 month delay with a cost of $3.5M per month. Do not believe there is a serious risk of this not being approved at the moment. Also do not believe there is a serious risk of being forced to accept technical compromises.
14:30 (at latest)Close

Day 2: 2020-05-13

Moderator: Gregory Dubois-Felsmann
09:00Plans for an interim data facility
  • What's the current status of the Google POC?
  • What's the timeline for an IDF decision (if one hasn't been made by the time of this meeting)?
  • What do we need to do to prepare for the IDF? How does it impact the other tasks that we are working on?

10:00Status of the APDB
  • Action item from the previous DMLT vF2F to report on the status of the APDB.
  • The DMLT acknowledges that this work may have been delayed by the focus on middleware development.

10:30Break
Moderator
11:00Status of ingestFritz Mueller
  • See Fritz Mueller's Slack message.
  • Should include:
    • Status update on HSC RC2 ingest to Qserv.
    • Discussion of longer term planning for ingest. What are the key questions? How do we get them answered?

12:30Break

Moderator: Frossie Economou


13:00Prompt processing
  • What exactly are “prompt services” (in terms of the product tree, system architecture, etc)?
  • What are the desires and use cases that have been advanced for an expanded scope “prompt processing” system?
  • How practical is it for the DM construction team to meet those desires?
  • If it is practical, what is the timeline and plan for doing so?

14:30 (at latest)Close

Day 3: 2020-05-14

Moderator
09:00Team statusJohn Swinbank

10:30Wrap-up
  • Actions and next meetings.
  • Virtual, 2020-11-16/19
    • This meeting will be virtual.
  • Tucson, 2021-02-22/25.
    • MCR booked - does not seem to clash with anything

11:00 (at latest)Close


Attached Documents

No files shared here yet.

Action Item Summary

DescriptionDue dateAssigneeTask appears on
  • Frossie Economou Will recommend additional Level 3 milestones for implementation beyond just the DAX-9 Butler provenance milestone.   
15 Mar 2022Frossie EconomouDM Leadership Team Virtual Face-to-Face Meeting, 2022-02-15 to 17
  • Kian-Tat Lim Convene a meeting with Colin, Tim, Robert, Yusra to resolve graph generation with per-dataset quantities (likely based on Consolidated DB work).  
18 Mar 2022Kian-Tat LimDM Leadership Team Virtual Face-to-Face Meeting, 2022-02-15 to 17
  • Frossie Economou Write an initial draft in the Dev Guide for what "best effort" support means  
17 Nov 2023Frossie EconomouDM Leadership Team Virtual Face-to-Face Meeting - 2023-Oct-24
  • Convene a group to redo the T-12 month DRP diagram and define scope expectations Yusra AlSayyad 
30 Nov 2023Yusra AlSayyadDM Leadership Team Virtual Face-to-Face Meeting - 2023-Oct-24
11 Dec 2023Gregory Dubois-FelsmannDM Leadership Team Virtual Face-to-Face Meeting - 2023-Oct-24
02 May 2024Frossie EconomouDMLT Meeting - 2024-04-22
22 May 2024 DMLT Meeting - 2024-04-22
  • Richard Dubois USDF part in data facilities for PSTN-017 and distrib processing ? 
22 May 2024Richard DuboisDMLT Meeting - 2024-04-22
22 May 2024Fabio HernandezDMLT Meeting - 2024-04-22
  • Tim Jenness - section on middleware for PSTN-017  
22 May 2024Tim JennessDMLT Meeting - 2024-04-22
  • Cristián Silva - section on summit/data acquisition  for PSTN-017  
22 May 2024Cristián SilvaDMLT Meeting - 2024-04-22

Pre-Meeting Planning

TopicRequested byTime required (estimate)Notes
Status of the APDB30 mins
  • Action from previous DMLT F2F, delayed due to DAX focus on middleware.
Plans for IVOA and Python interfaces to query time series data in Prompt and DR data products.30 mins (maybe 1hr, probably not) 
Status on RC2 ingest to Qserv30 mins
  • As requested at February DMLT 
Management of calibration products30  mins
  • Following the February DMLT meeting, Christopher Waters has drafted DMTN-148. Is the DMLT ready to sign off on that as our plan moving forwards?
  • Leanne Guy requests that we also talk about product ownership.
Plans for an interim Data Facility1 hour
  • What's happening with the Google POC?
  • What do we need to do to prepare for an iDF?
    • When might it happen?
    • Who needs to be involved?
Rebaselining1 hour
  • Everybody's talking about it, but what does it mean? Who will have to do what when?
  • Can we use this opportunity to get ahead of whatever Victor/Kevin/etc will ask for, and make sure DM comes out of the rebaselining process in good shape?
Prompt processing2 hours
  • What are the desires, use cases, requirements, plans, schedule, for an expanded scope “prompt processing” system, as requested by Robert Lupton?
  • What are “prompt services”, and what is the status of their product ownership?


  • No labels