Attendees : Srini, Simon, Peter, Cathy

The following proposal is Srini's plan in order to address the current needs expressed by Simon, Peter and Cathy. This proposal is not a decision just a suggestion for discussion. The proposal is divided into near term goals and longer term goals. Near term in my mind is the next sprint or the one following it. Longer term is all the sprints after that.

ASSUMPTIONS

1. OpSim will maintain 2 branches, 1 will be the development branch for schema changes and moving to sqlite and the other will be the production branch (master) that will work with the current pipeline (OpSim + SSTAR).
2. The current pipeline will be retired on MAF's delivery date. OpSim at that date will produce the output required in the format that MAF most likely will be used i.e. sqlite.
3. OpSim will provide a nominal value for all important parameters associated with an observation (nominal position, dithered position, sky brightness, 5 sigma limiting depth, cloud cover, etc.), and that comparisons to other ways of calculating these values should be implemented within the MAF framework (not necessarily provided by MAF). - Simon

NEAR TERM

The OpSim needs to provide the MAF team a single sqlite DB for one run. There have been minor name changes in the "output" table that have been moved to the longer term goals due to the direct effect it has to the running of the SSTAR.

1. The current pipeline (OpSim + SSTAR) will provide an sqlite dump including the various other .dat & .sql dumps to MAF and the community during the development of the MAF. This will streamline the working with MAF while OpSim transitions from MySQL completely to sqlite.

Deliverables

1. .dat files for all tables for a sessionID, tar.gz file called opsimX_Y_datexport.tar.gz. This dat file will contain the "output" table i.e. the materialized join table, Field table, Seeing table & the Cloud table.
2. .sql files for all tables for a sessionID, tar.gz file called opsimX_Y_sqlexport.tar.gz. This sql file will contain the "output" table i.e. the materialized join table, Field table, Seeing table & the Cloud table.
3. A single sqlite file, called opsimX_Y_sqlite.tar.gz. This sqlite file will be a singular sqlite DB file that will contain all tables for a sessionID, the materialized join table, Field table, Seeing table & the Cloud table.

LONGER TERM i.e. non blocking items

1. Schema changes to ObsHistory - For the short term the following columns will exist in the output table in the form of "5sigma", "hexdithRA", "hexdithDec". These columns will be calculated at insertion time of the Observation
   1. filt5sigma - new column that will calculate the 5sgima limiting magnitude with filtSkyBright
   2. ditheredRA - new RA column that will incorporate the example dithering algorithm.
   3. ditheredDEC - new DEC column that will incorporate the example dithering algorithm.
2. OpSim will move to sqlite completely.
3. OpSim will generate the "summary" table which will be a materialized join table that currently is the "output" table with the name change and a few column name changes.
4. visitTime & visitExpTime issues
5. moonPhase is the illumination
6. moonillum name has to be changed in order to reflect its use.
7. MAF will provide a way to calculate different sky brightness and further post processing i.e. 5sigma and coadded depth calculation. So in the longer term perry's sky brightness values will not be available in the summary table because they will not be calculated during insertion time. There is no pre-processing layer or interface layer in the long term. MAF in the long term is doing all post processing i.e. the scope that currently is with SSTAR.

Deliverables

1. OpSim produces a single sqlite file, called opsim_IDENTIFIER_sqlite.dat that will contain all tables for a simulation, the materialized join table, Field table, Seeing table & the Cloud table.
2. This will contain the schema changes, name changes, additional calculations