Time, Date & Place

12:00 Pacific 2019-08-02 on Amazon Chime https://chime.aws/1930107527

Attendees

Chris, Sanjay, Dino, Lorena, Greg Daues Hsin-Fang Chiang Unknown User (mbutler) 


regrets: Wil, K-T, Aaron

Discussion Items

  • Schedule of this meeting: Cancel 8/16 (LSST Project Community Workshop at Tucson), Continue on 8/23. Hsin-Fang will send out the new calendar invite. 
  • Hsin-Fang will give a short demo in the LSST Project Community Workshop DM session on Aug 12.  Plan to show ci_hsc workflow + condor annex + spot fleet + S3 datastore + sqlite registry on EFS; live demo a pipeline task. 
    • Sanjay and Dino:  good to mention the scalability & shareability of the data. For example S3 across countries e.g. S3 Chile, S3 France, S3 US)
  • Dino has been working on the RDS registry
    • Tried both copying from sqlite3 & ingest from scratch. Debugging..
    • Can try with POSIX datastore to simplify
  • Unknown User (mbutler)  NCSA has an object store too and can offer it for tests 
  • For scaling up, a larger dataset (HSC-RC2) is almost ready. A full tract of HSC-RC2 was processed successfully on NCSA with Gen3 software.
  • Defer discussions with Condor team: 
    • HTCondor S3 plug-in for data transfer. Next step?
    • annex on multiple aws regions? 


  • No labels