Information resources

• Detailed schema in DM-7053 - Getting issue details... STATUS
  
• The link to the SDSS Stripe 82 main page on the project's site
• LSST Data Challenge Report - the document published in September 2013 with the summary of the LSST DM Date Challenge (LSST DocuShare)
• DC planning and testing documents (LSST TRAC)
• Technical instructions on the DC data processing and database ingestion. The document provides a detailed explanation of specific steps taken during the DC, LSST stack version, tools, etc.
• Additional instructions on the final stage of ingesting and de-duping of forced sources (LSST TRAC)
• Related information on the Winter 2013 predecessor processing (LSST DMTN-035): DC W13 handbook and report

• Database schema of catalogs (LSST GutHub)

• Stripe82 case03 at GitHub
• Stripe82 raw data: lsst-dev:/lsst7/stripe82/dr7/runs
  
  

Catalogs

Since the DC processing happened simultaneously in two sites (NCSA and IN2P3) the resulting MySQL catalogs ended up divided into halves (with an overlap in an interval of: +5 < R.A. < +10) in between 12 databases hosted by two MySQL servers (one per site):

lsst-db.ncsa.illinois.edu

daues_SDRP_Stripe82_ncsa

  61

  294,188,533

daues_SDRP_dedupe_byfilter_0

 683

1,729,599,837

daues_SDRP_dedupe_byfilter_1

 702

1,902,999,739

daues_SDRP_dedupe_byfilter_2

 703

1,904,627,357

daues_SDRP_dedupe_byfilter_3

1400

3,658,881,418

daues_SDRP_dedupe_byfilter_4

 692

1,751,879,716

ccdb02.in2p3.fr

lsst_prod_DC_2013_2

  62

  312,642,627

lsst_prod_dedupe_byfilter_u

 760

2,058,168,075

lsst_prod_dedupe_byfilter_g

 780

2,114,378,412

lsst_prod_dedupe_byfilter_r

 779

2,079,018,858

lsst_prod_dedupe_byfilter_i

 781

2,117,611,454

lsst_prod_dedupe_byfilter_z

 780

2,113,759,673

NCSA database names at NCSA are based on the following filter identifier-to-name mapping:

mysql> SELECT * FROM lsst_prod_DC_2013_2.Filter;
+----------+------------+----------+--------+
| filterId | filterName | photClam | photBW |
+----------+------------+----------+--------+
|        0 | u          |        0 |      0 |
|        1 | g          |        0 |      0 |
|        2 | r          |        0 |      0 |
|        3 | i          |        0 |      0 |
|        4 | z          |        0 |      0 |
|      -99 | DD         |        0 |      0 |
+----------+------------+----------+--------+

Other notes:

• data sizes were reported for the data stored in tables not including indexes.

Calibrated exposures

NCSA

A subset of 1104591 out of 1403385 images (161 run out of 207) was achieved on NCSA HPSS service after the DC. Greg Daues has kindly restored the images at the following location (at NCSA):

% find /nfs/scratch/daues/restore/dpool/scratch/daues/SDRP_2014_NCSA/calexp_dir/sci-results/*/ -name 'calexp-*.fits.gz' | wc -l
1104591


% du -hs /nfs/scratch/daues/restore/dpool/scratch/daues/SDRP_2014_NCSA/calexp_dir/sci-results/
15T	/nfs/scratch/daues/restore/dpool/scratch/daues/SDRP_2014_NCSA/calexp_dir/sci-results/

IN2P3

The images are stored in the iRODS server:

$ ils /lsst-fr/data/DC_2013/calexps/sci-results
/lsst-fr/data/DC_2013/calexps/sci-results:
  C- /lsst-fr/data/DC_2013/calexps/sci-results/1752
  C- /lsst-fr/data/DC_2013/calexps/sci-results/1755
  ...
  C- /lsst-fr/data/DC_2013/calexps/sci-results/7202

$ ils /lsst-fr/data/DC_2013/calexps/sci-results/1752/1/g/calexp
/lsst-fr/data/DC_2013/calexps/sci-results/1752/1/g/calexp:
  bkgd-calexp-001752-g1-0040.fits
  ...
  calexp-001752-g1-0229.fits
  calexp-001752-g1-0230.fits

The files still need to be transferred to NCSA. It's estimated to be about 40 TB of data.

Coadded images