Data Management
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

(back to the list of all DMLT meeting minutes)

Location

Browser

Room System

Phone Dial-in

https://gemini.zoom.us/j/93625401560?pwd=cjdFb1ZWeGx1eVJGSVBUVmpMRUg5UT09


Meeting ID: 936 2540 1560 

password: 161803

Dial closest IP: 162.255.36.11 (east coast) and 162.255.37.11 (west coast) Then use the Zoom meeting ID 936 2540 1560 as the dialing extension. For example: 936 2540 1560@162.255.37.11 or: 162.255.37.11##936 2540 1560 Password: 161803

Dial-in numbers:

+1 346 248 7799 (US Toll)

+1 669 900 6833 (US Toll) Meeting ID: 936 2540 1560 International numbers available: https://gemini.zoom.us/u/adcUNrbXzS

Time

10:15am PT

Attendees


Regrets

Discussion Items

ItemWhoNotes
Notes
last time
Project Updates

Wil O'Mullane

  • NSF visit good
  • office space La Serena - not enough  and not efficiently used
  • FY23 starts - amendments not yet approved but could be done this week
  • Ops contracts language for software not same as construction. Double check the IP language in your contracts and make sure it matches construction phasing, WOM/FE chasing up Contracts
  • Flights still expensive - overestimate TRs (its better than under estimate)
DEI


See  #inclusion and https://www.lsst.org/about/dei

Community link: https://community.lsst.org/c/eji/45

VF2F

 DM Leadership Team Virtual Face-to-Face Meeting - 2022-10-18,19

  • in 2 weeks
  • add topics - so far not enough for two full days
Review of Project Milestones


https://www.lsst.org/about/project-status

OPS epics
  • Wil does not see any labeled FY23 (crack whip)
  • Discussion about Tim's team and which epic they charge tickets to ("it's complicated")

 

T Key Summary Assignee Reporter P Status Resolution Created Updated Due
Refresh

Level 2 Milestones


See also LSST Verification & Validation Documentation and LDM-692 (VCD).

Test Plans due in the next 45 days

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

Milestones due in the next 45 days

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

Risk Review



Risk items needing review

Key Summary T Created Updated Due Assignee Status days since review
Refresh

Overdue risks (obligation date passed)

T Key Summary Assignee Reporter P Status Resolution Created Updated Due
Refresh

Risk mitigations due at the end of this month

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

Overdue risk mitigations

 

T Key Summary Assignee Reporter P Status Resolution Created Updated Due
Refresh

DMLT Travel & Availability this week



Any Other Business

Robert Lupton


Commissioning the Commissioning Cluster on Cerro Pachón 

  • Cristián Silva at the f2f we agreed to expand Yagan, we need to know by how much
  • Robert Lupton we never set up antu as the commissioning cluster - antu now gone
  • Gregory Dubois-Felsmann to clarify yagan is where rapid processing will run
  • Currently Parsl over SLURM at USDF .. may be more flexible for Yagan than PanDA
  • Colin Slater this will create a difference between summit and USDF
  • Rapid processing is same system as prompt processing which is what is running at the IDF which is neither PanDA nor Parsl/SLURM
    • Question to Paul Price and James Chiang as to whether Parsl has the same capabilities as HTCondor in BPS (at present and in the future) — need to enumerate missing capabilities.
  • Current USDF HTCondor is "personal" only — Expertise at NCSA available to help stand up HTCondor as a service at USDF
    • Likely can run batch system on Kubernetes-managed nodes, not necessarily bare metal, need to confirm; Robert Lupton and Gregory Dubois-Felsmann do not need to verify performance on Kubernetes at this point, only functionality; Steve Pietrowicz will confirm that HTCondor can run on Kubernetes; Richard Dubois will determine if Parsl/SLURM can run on Kubernetes
  • Current number of cores in yagan (now that antu has been moved) should be quite sufficient for near-term Rapid Processing

Tim Jenness 

ADASS registration (Tim will experiment on how to pay)
TCAM Stamdup

Frossie Economou

as needed
Action review



DMLT Action Items (Confluence quick tasks)

Confluence action items on DMLT meeting minutes pages are meant to be mainly low-level actions that can be completed quickly.  More substantial tasks should appear in JIRA.  The label "DMLT" can be used (with any component) to mark a task as resulting from a DMLT action item.

DescriptionDue dateAssigneeTask appears onLabel(s)
  • Frossie Economou Will recommend additional Level 3 milestones for implementation beyond just the DAX-9 Butler provenance milestone.   
15 Mar 2022Frossie EconomouDM Leadership Team Virtual Face-to-Face Meeting, 2022-02-15 to 17meeting-notesdmltdmlt-f2f
  • Kian-Tat Lim Convene a meeting with Colin, Tim, Robert, Yusra to resolve graph generation with per-dataset quantities (likely based on Consolidated DB work).  
18 Mar 2022Kian-Tat LimDM Leadership Team Virtual Face-to-Face Meeting, 2022-02-15 to 17meeting-notesdmltdmlt-f2f
15 Nov 2022Gregory Dubois-FelsmannDMLT meeting-2022-10-24dmltmeeting
  •  Kian-Tat Lim  Convene a meeting following up vF2F discussion about what to do with Jenkins  
24 Nov 2022Kian-Tat LimDM Leadership Team Virtual Face-to-Face Meeting - 2022-10-18dmltmeeting-notesdmlt-f2f
13 Feb 2023Gregory Dubois-FelsmannDMLT meeting-2023-01-30dmltmeeting
24 Apr 2023Gregory Dubois-FelsmannDMLT meeting-2023-04-17dmltmeeting

DMLT Action Items (JIRA)

JIRA tasks with label "DMLT" but not TESTPLANS:

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

1 Comment

  1. Paul Price

    For ctrl_bps_parsl:

    • bps cancel  is not really applicable: to cancel a run, you Ctrl-C  out of the bps submit  process that you're running.
    • bps report  is not yet implemented.
    • I have recently answered questions about whether it respects memory requests. The answer is that it depends on how you have it configured. A Slurm  site does not respect the memory requests because all the resources are identical. A TripleSlurm  site does respect the memory requests, and sends the job to the appropriate slurm block. I believe the WorkQueueExecutor is capable of respecting memory requests, but I don't think the WorkQueue site we have configured passes along the information.
    • What other capabilities are you interested in that I've omitted?