Operating Assumptions

  • The only option in this version is to use MySQL (SQLite branch is done but will be active in v3.3)
  • Simulations are stored in a single MySQL DB (one per hostname) where each run is identified by hostname.sessionID (multiple runs in one DB)
  • All completed runs will have a MAF standard analysis (sstarDriver.py + ??) performed, which executes on the SQLite DB file of the run, and recorded in Mantis/showMaf central repository trackingDB
  • A system for tracking and locating files needs to be finalized and implemented
  • Not possible yet to run entire pipeline automatically because of manual intervention needed for visitTime, vistExpTime and Universal tags; 
  • Factored database makes it more efficient to run simulations, but requires using longer names and joins in order to do real time queries (Use Case 1) as opposed to MAF drivers (Use Case 2).

Bug Fixes (all completed)

  • OPSIM-371 removed maxAirmass influence into the query that selects the fields for the night. Only maxReach is used now.
  • OPSIM-328 create RA absolute limits parameters
  • OPSIM-388 Now the simulation aborts if nRun parameter is not present in the LSST.conf configuration file.
  • OPSIM-404 - Fix for simulator dies when no active proposals. Changed ObsScheduler.py
  • OPSIM-328 Fixed RA absolute limits behavior for WeakLensing.
  • OPSIM-504 Fixed IDLE state handling of a sequence, for the case of interruption during the very first deep-drilling event of a multi-subseq
  • Adding data SQL files for Cloud, Field, Seeing from opsim-install package
  • Using requests library rather than urllib2

Enhancements & Interface Changes (to be implemented)

  • (Done) Uses palpy instead of slalib
  • (Done) Scheduled downtime correction (1.15% more downtime) 
  • Do we need a script to run all steps with maybe stdin for manual steps?
  • New default config files will be added (Kem to describe new additions - new library; updated parameter values)
  • Replacing Mantis functionality
    • define where showMaf.,py will be run as the index for all runs completed by the OpSim team (ops1 and ops2) / implement
    • define how the sessionID gets recorded/stored in a central repository along with pointers to output files? should it connect to showMAF.py AND Mantis? / implement
  • Conversion tool:
    • add Universal tag to Config table
    • visitTime & visitExpTime values + inserting Universal tag - document process for manual insertion
    • ensure both schemas are the same (so that real time analysis on mysql DB is the same as using MAF or real time queries on sqlite DB)
      • postponed to v3.3 

    • strip down conversion script to keep only essential functions; locate in an accessible place (commit to simulator codebase?)

      • postponed to v3.3
    • define how simulations sessionIDs are recorded regardless of their completion or usefulness status (see Mantis)
    • define where the conversion tool gets called / or run? / implement  (where is the code actually located?)
    • define where do / how do output files from conversion tool get moved to repository or wherever MAF will be run / implement

MAF

Create a workable replacement for SSTAR, sstarDriver/showMaf:  Complete evaluation and revisions of content 

  • TBD - iteration in progress
  • Offline viewing of selected runs tool

 

Fastest way to Release v3.2 (Nov 19, 2014 email to opsim-lsst-dev)

1) Conversion tool

                - WFD tag is either added here as an argument (only one can be done at a time) or left as included in Config table

         - remove any output data files not needed by the code/scripts

                - remove any unneeded scripts / files

                - commit to stash under opsim (under "tools"?)

2) Edit main.py to reflect v3.2

3) Configuration Files

                - edit conf files with new defaults and new library directory structure

                - commit conf files

                - commit new downtime file (part of conf directory structure)

4) VisitExpTime & VisitTime

                - is this correct in ObsHistory in MySQL, output_ table in .sql and Summary table in  .sqlite

 

Assumptions

  - sphinx documentation will be incomplete/wrong (e.g. installation references CVS)

  - SQLite and MySQL schemas are different

  - need to cd /lsst/opsim/src to execute ./main.py

 - need to cd /lsst/opsim/tools to execute ./modifySchema.sh (conversion tool to SQLite)

 - WFD tag is written to Config table ( check) but MAF will have to be altered to make use of this

 

  • No labels