Date:
Attendees: Unknown User (pdomagala), Margaret Gelman, Donald Petravick, Unknown User (kaylynr)
See the LSST Service Status Page
Note that the December maintenance window is moved from 12/21 to 12/14.
0 created, 0 resolved.
Unexpected reboot of lsst-qserv-db16 (IHS-606).
Spontaneous reboots of PDAC nodes has been an ongoing issue since Nov. 14. The last event was on Nov. 23. Datacenter infrastructure has been ruled out as a cause. The problem is determined to be with the servers themselves - likely on the system board.
Despite this issue, Igor Gaponenko was able to complete his data ingests and meet his Nov. 30 milestone.
On Mon. of this week, our engineering team and vendor tech support believe they have identified the likely cause and we've initiated an emergency change request. The fix entails firmware updates, which are currently being installed. Nodes will be unavailable as they are upgraded. This could be a lengthy process.
None created or resolved
This process primarily targets requests that can be handled with current level of effort (LOE) resources. This process is also designed to detect and redirect items to the EVMS process if they exceed LOE resources.
1 | Business Case & T/CAM Concurrence | Check that the submitter has stated a plausible business case and the relevant T/CAM agrees |
2 | Feasibility | Is the change well-formulated, address a project need and |
3 | Planning | A detailed implementation plan is created which takes into account impacts, resource needs, testing and verification. |
4 | Insertion | The plan is executed to implement the change. |
5 | Assessment | Verification of successful change, issues analysis, documentation and close-out. |
Key | Summary | Process Stage† | Reporter | P | Created | Resource track | Status |
---|---|---|---|---|---|---|---|
IHS-580 | DM developers need a build/test environment that supports docker containers | Feasibility | Joshua Hoblitt | 02/Nov/17 |
| ||
IHS-576 | Planning | Tim Morton | 02/Nov/17 |
| |||
Implement debug and normal queues for developers on the verification cluster | Planning | Yusra AlSayyad | 16/Nov/17 | LOE |
| ||
IHS-595 | Closed (inserted) | John Parejko | 08/Nov/17 | LOE |
| ||
IHS-613 | Closed (will not implement) | John Parejko | 16/Nov/17 | - |
|
Report format under development
None
Next meeting
Next PDAC meeting is tomorrow, .
Suspended until after the first of the year since Jeff is in Chile. However, I’m linked in to Chile IT.
Next meeting
(None)
New
From last week