Data Management
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

Date

Attendees

Discussion items

DM-1618 (creation of distributed dbs)

  • for now, create from the data loader, refactoring that code and basing it on CSS now

DM-1619 (creation of distributed tables)

  • easy once we have DM-1618

DM-998 (multi node integration)

  • biggest problem: bring up the cluster
  • loading using the new loader is simple. Need DM-1618 and DM-1619

DM-211

MaxDB

  • Serge will attend
  • Create story, attach notes from mtg there

DM-681

  • probably 6 SP, not 4, close to finishing

DM-322

  • almost done

DM-219

  • a lot of cleanup already done
  • if can't finish this month, will close this ticket, open new one for later

DM-854

  • data set Fabrice chose to use uncovered issue with empty overlap tables
  • switching to other data set allows him to reproduce the problem

Empty overlap tables

  • query execution expects all chunks and all overlaps to exist, even if they are empty
    • this needs to be documented! (Daniel). Not sure where the best place is.
  • short term options:
    • patch in loader:
      1. look at all chunks and add corresponding overlap chunks
      2. look at overlap chunks and add missing empty chunk table
    • patch in partitioner: it could generate (empty) file for every empty chunk table and every empty overlap chunk table
    • decided to to the former. See DM-2058
  • longer term, we need code that enforces data consistency. That code should also create special things like the 0123456789 chunks (which is now done in loader). Idea: we could have a manifest file that defines a set of rules. Deal with it as part of larger "consistency checking" epic, after W15. See DM-2065
  • also, we need use case to test for missing empty overlap chunk tables and/or empty chunk tables. See DM-2066

DM-1841

  • AndyH working on it already

DM-213

  • lots of progress

DM-1675, DM-1145

  • in progress, will wrap up this week
  • Fritz is looking at spherical geom, so will review the DM-216 instead of AndyS

DM-202, DM-201

  • will work on that next week

1674 (ingest)

  • Serge doing urgent non-LSST work now
  • relatively simple, good chance will be done before end of February

1570

  • on track

XrdSsi

  • add ticket: revisit memory ownership of xrdssi objects. See DM-2067