Versions Compared

Old Version 2

changes.mady.by.user Fritz Mueller

Saved on

compared with

New Version Current

changes.mady.by.user Fritz Mueller

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Fritz Mueller, Andy Hanushevsky, Vaikunth Thukral, Unknown User (npease), Kian-Tat Lim, Andy Salnikov, Fabrice Jammes, John Gates

Discussion items

 L1/AP Database

  • Andy S is working to spec out the L1/AP simulated data source.  This work will probably continue through to the end of July.
  • Still a question of where to run this – need a lot of locally attached storage (100s of G) for an accurate simulation.  Suggestion to peel off one of the IN2P3 nodes for this when necessary.

Shared-scans

  • John wrapping up some bug fixes; these will carry over to next sprint.
  • Still puzzled over some curious behaviors of the shared scans when running concurrent joins
    • For example: scanned join of Source + Object takes 3hrs.  scanned join of ForcedSource + Object takes 25m.  But concurrent takes 4hrs!
    • mlock oddness: doing something definitely, without 4hr scans take 8hrs.  But, must wait for all mlocks to finish and only issue one at a time.
  • Andy H is working on further memman rework.

Butler

  • Nate has reworked DM-6459 (Butler repo refactor) after some design discussions with K-T, and is now working on associated unit tests.
  • DM-5548 (Butler persistence improvements) is wrapped up in this work as well.  Both expected to complete next week.

Docker

  • Oualid's new OpenStack scripts have been merged.
  • Fabrice has rebased his Swarm work on this.
  • Now running into intermittent container failure problem.  Investigating...

X-SWAP

  • Vaikunth has revamped/updated the second half of the IN2P3 cluster
  • Container running check in script always fails.  Fabrice reports fixed some time pack.  Vaikunth to check it he has fix.
  • Nothing in the scripts right now removes dangling docker images.  We should add.
  • Vaikunth has run into some disk quota issues for logs; working with IN2P3 admins

Misc

  • K-T asks re. Butler: "who should be responsible for maintaining/extending the obs packages?"  Recommendation is for database team to own the framework, and a small number of well-documented, substantive reference implementations for e.g. mappers.  Then sci-pi team can maintain the more scientific code built on top of these.  Means we will need much better documentation.

 

  (...work in progress...)