Andy S is working to spec out the L1/AP simulated data source. This work will probably continue through to the end of July.
Still a question of where to run this – need a lot of locally attached storage (100s of G) for an accurate simulation. Suggestion to peel off one of the IN2P3 nodes for this when necessary.
Shared-scans
John wrapping up some bug fixes; these will carry over to next sprint.
Still puzzled over some curious behaviors of the shared scans when running concurrent joins
For example: scanned join of Source + Object takes 3hrs. scanned join of ForcedSource + Object takes 25m. But concurrent takes 4hrs!
mlock oddness: doing something definitely, without 4hr scans take 8hrs. But, must wait for all mlocks to finish and only issue one at a time.
Andy H is working on further memman rework.
Butler
Nate has reworked DM-6459 (Butler repo refactor) after some design discussions with K-T, and is now working on associated unit tests.
DM-5548 (Butler persistence improvements) is wrapped up in this work as well. Both expected to complete next week.
Docker
Oualid's new OpenStack scripts have been merged.
Fabrice has rebased his Swarm work on this.
Now running into intermittent container failure problem. Investigating...
X-SWAP
Vaikunth has revamped/updated the second half of the IN2P3 cluster
Container running check in script always fails. Fabrice reports fixed some time pack. Vaikunth to check it he has fix.
Nothing in the scripts right now removes dangling docker images. We should add.
Vaikunth has run into some disk quota issues for logs; working with IN2P3 admins
Misc
K-T asks re. Butler: "who should be responsible for maintaining/extending the obs packages?" Recommendation is for database team to own the framework, and a small number of well-documented, substantive reference implementations for e.g. mappers. Then sci-pi team can maintain the more scientific code built on top of these. Means we will need much better documentation.