Data Management
Space shortcuts
Page tree

Versions Compared

Old Version 13

changes.mady.by.user Wil O'Mullane

Saved on

compared with

New Version Current

changes.mady.by.user Wil O'Mullane

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

ItemWhoNotes
Notes
last time
Project Updates

Wil O'Mullane

  • NSF visit good
  • office space La Serena - not enough  and not efficiently used
  • FY23 starts - amendments not yet approved but could be done this week
  • Ops contracts language for software not same as construction. Double check the IP language in your contracts and make sure it matches construction phasing, WOM/FE chasing up Contracts
  • Flights still expensive - overestimate TRs (its better than under estimate)
DEI


See  #inclusion and https://www.lsst.org/about/dei

Community link: https://community.lsst.org/c/eji/45

VF2F

 DM Leadership Team Virtual Face-to-Face Meeting - 2022-10-18,19

  • in 2 weeks
  • add topics - so far not enough for two full days
Review of Project Milestones


https://www.lsst.org/about/project-status

OPS epics
  • Wil does not see any labeled FY23 (crack whip)
  • Discussion about Tim's team and which epic they charge tickets to ("it's complicated")

 

Jira
serverJIRA
jqlQueryproject = PREOPS AND component = "Data Production" and labels = fy23
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

Level 2 Milestones


See also LSST Verification & Validation Documentation and LDM-692 (VCD).

Test Plans due in the next 45 days

Jira
serverJIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQuerylabels = TESTPLAN and resolution = Unresolved and due <= 45d order by id asc
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

Milestones due in the next 45 days

Jira
serverJIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQueryproject = DM and labels = L2Milestone and team not in ("Telescope and Site") and resolution = Unresolved and due <= 45d order by id asc
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

Risk Review



Risk items needing review

Jira
serverJIRA
columnskey,summary,type,created,updated,due,assignee,status,days since review
maximumIssues20
jqlQueryproject = "Risk Management" AND issuetype in (RM-Risk, RM-Opportunity) AND status = "Active Risk/Opportunity" AND component = "Data Management" AND "Days Since Review" >= 60 ORDER BY cf[14809] DESC, status DESC, cf[13108] ASC
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

Overdue risks (obligation date passed)

Jira
serverJIRA
jqlQueryproject = "Risk Management" AND issuetype in (RM-Risk, RM-Opportunity) AND status = "Active Risk/Opportunity" AND status = "Active Risk/Opportunity" AND component = "Data Management" AND "Obligation Date" < now() ORDER BY cf[14809] DESC, status DESC, cf[13108] ASC
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

Risk mitigations due at the end of this month

Jira
serverJIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQueryproject = "Risk Management" AND issuetype in (RM-Risk, RM-Opportunity) AND status = "Active Risk/Opportunity" AND status = "Active Risk/Opportunity" AND component = "Data Management" AND "Obligation Date" < now() ORDER BY cf[14809] DESC, status DESC, cf[13108] ASC
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

Overdue risk mitigations

 

Jira
serverJIRA
jqlQueryproject = rm AND issuetype = rm-handling AND status = Planned AND component = "Data Management" AND "Anticipated completion date" <= endOfMonth() AND "Anticipated completion date" <= now() ORDER BY status DESC, component ASC, assignee ASC, reporter ASC
serverId9da94fb6-5771-303d-a785-1b6c5ab0f2d2

DMLT Travel & Availability this week



Any Other Business

Robert Lupton


Commissioning the Commissioning Cluster on Cerro Pachón 

  • Cristián Silva at the f2f we agreed to expand Yagan, we need to know by how much
  • Robert Lupton we never set up antu as the commissioning cluster - antu now gone
  • Gregory Dubois-Felsmann to clarify yagan is where rapid processing will run
  • Currently Parsl over SLURM at USDF .. may be more flexible for Yagan than PanDA
  • Colin Slater this will create a difference between summit and USDF
  • Rapid processing is same system as prompt processing which is what is running at the IDF which is neither PanDA nor Parsl/SLURM
    •  Question to Paul Price and James Chiang as to whether Parsl has the same capabilities as HTCondor in BPS (at present and in the future) — need to enumerate missing capabilities.
  • Current USDF HTCondor is "personal" only — Expertise at NCSA available to help stand up HTCondor as a service at USDF
    •  Likely can run batch system on Kubernetes-managed nodes, not necessarily bare metal, need to confirm; Robert Lupton and Gregory Dubois-Felsmann do not need to verify performance on Kubernetes at this point, only functionality; Steve Pietrowicz will confirm that HTCondor can run on Kubernetes; Richard Dubois will determine if Parsl/SLURM can run on Kubernetes
  • Current number of cores in yagan (now that antu has been moved) should be quite sufficient for near-term Rapid Processing

Tim Jenness 

ADASS registration (Tim will experiment on how to pay)
TCAM Stamdup

Frossie Economou

as needed
Action review



...