Date
Attendees
Fritz Mueller, Andy Hanushevsky, Vaikunth Thukral, Unknown User (npease), Kian-Tat Lim, Andy Salnikov, Fabrice Jammes, John Gates
Discussion items
L1/AP Database
- Andy S is working to spec out the L1/AP simulated data source. This work will probably continue through to the end of July.
- Still a question of where to run this – need a lot of locally attached storage (100s of G) for an accurate simulation. Suggestion to peel off one of the IN2P3 nodes for this when necessary.
Shared-scans
- John wrapping up some bug fixes; these will carry over to next sprint.
- Still puzzled over some curious behaviors of the shared scans when running concurrent joins
- For example: scanned join of Source + Object takes 3hrs. scanned join of ForcedSource + Object takes 25m. But concurrent takes 4hrs!
- mlock oddness: doing something definitely, without 4hr scans take 8hrs. But, must wait for all mlocks to finish and only issue one at a time.
- Andy H is working on further memman rework.
Butler
- Nate has reworked DM-6459 (Butler repo refactor) after some design discussions with K-T, and is now working on associated unit tests.
- DM-5548 (Butler persistence improvements) is wrapped up in this work as well. Both expected to complete next week.
Docker
- Oualid's new OpenStack scripts have been merged.
- Fabrice has rebased his Swarm work on this.
- Now running into intermittent container failure problem. Investigating...
X-SWAP
- Vaikunth has revamped/updated the second half of the IN2P3 cluster
- Container running check in script always fails. Fabrice reports fixed some time pack. Vaikunth to check it he has fix.
- Nothing in the scripts right now removes dangling docker images. We should add.
- Vaikunth has run into some disk quota issues for logs; working with IN2P3 admins
Misc
- K-T asks re. Butler: "who should be responsible for maintaining/extending the obs packages?" Recommendation is for database team to own the framework, and a small number of well-documented, substantive reference implementations for e.g. mappers. Then sci-pi team can maintain the more scientific code built on top of these. Means we will need much better documentation.