Page History

...

•  RC2 (and DC2) "can" run in Gen3 (to zeroeth order as well as Gen2)
  •  DC2 data works in Gen3
    •  Ingestion
    •  QuantumGraph generation
  •  afw table environment is stable
  •  Full pipeline runs in both Gen2 and Gen3
    •  jointcal (DM-24300)
    •  pipe_analysis tasks "can" work (i.e. stability is sufficient that development necessary to bring these tasks forward can occur)
  •  Multi-node processing possible on NCSA HTCondor pool with shared filesystem and shared database (NCSA staff)
  •  Run a 3 tract RC2 in under a week.
  •  Ability to use the intermediate outputs of an RC2, as the inputs to some one-off processing (not just repo chaining, but different users) 
•  Multi-node processing possible on NCSA HTCondor pool with shared filesystem and shared database (friendly users) (i.e., a ctrl_pool replacement)   
  LIST STILL BEING GENERATED
  •  Use new pipeline description yaml instead of previous command-line
    
  •  Friendly-user HTCondor Pool
    
  •  Shared r/w Oracle schema with friendly user authentication
  •  Noteable features from single-node processing that must continue to be available:
    •  A log must be written out somewhere (at this point it is not a requirement to be anything other than file(s))
    •  The configs with which the tasks were run must be written out somewhere
    •  The stack version must be written out somewhere
    •  Task metadata must be written out somewhere
    •  The input repos must be written out somewhere (waiting clarification of this requirement)
  •  

    Extra HTCondor user documentation (e.g., know why job isn’t starting or why job was killed)

•  Raw files can be auto ingested into friendly-user shared Gen3 repositories usable on LSP (and HTCondor Pool)
  •  Single-file ingestion
  •  Chained repositories (?  Users will not have write access to raw file storage area.)
•  Raw files can be auto ingested into friendly user Gen3 OODS (Maybe this moved to Deprecation Begins?)
  •  TBD 

...

•  Note: There will still be Gen3 ToDo's, but they should not be known blockers to the removal of Gen2.
•  Removal of Gen2 ingest of raw images into DBB
  •  Deprecate any Gen2 ingestion specific code
  •  Remove Gen2 repository
•  Removal of Gen2 ingest of raw images into OODS
•  Stop running RC2 and DC2 in Gen2
  •  Remove production Gen2 repositories (users must remove their own)
•