Information resources

Catalogs

Since the DC processing happened simultaneously in two sites (NCSA and IN2P3) the resulting MySQL catalogs ended up divided into halves (with an overlap in an interval of: +5 < R.A. < +10) in between 12 databases hosted by two MySQL servers (one per site):

hostdatabasedata [GB]num. rows
lsst-db.ncsa.illinois.edu
daues_SDRP_Stripe82_ncsa
  61
  294,188,533
daues_SDRP_dedupe_byfilter_0
 683
1,729,599,837
daues_SDRP_dedupe_byfilter_1
 702
1,902,999,739
daues_SDRP_dedupe_byfilter_2
 703
1,904,627,357
daues_SDRP_dedupe_byfilter_3
1400
3,658,881,418
daues_SDRP_dedupe_byfilter_4
 692
1,751,879,716
ccdb02.in2p3.fr
lsst_prod_DC_2013_2
  62
  312,642,627
lsst_prod_dedupe_byfilter_u
 760
2,058,168,075
lsst_prod_dedupe_byfilter_g
 780
2,114,378,412
lsst_prod_dedupe_byfilter_r
 779
2,079,018,858
lsst_prod_dedupe_byfilter_i
 781
2,117,611,454
lsst_prod_dedupe_byfilter_z
 780
2,113,759,673

NCSA database names at NCSA are based on the following filter identifier-to-name mapping:

mysql> SELECT * FROM lsst_prod_DC_2013_2.Filter;
+----------+------------+----------+--------+
| filterId | filterName | photClam | photBW |
+----------+------------+----------+--------+
|        0 | u          |        0 |      0 |
|        1 | g          |        0 |      0 |
|        2 | r          |        0 |      0 |
|        3 | i          |        0 |      0 |
|        4 | z          |        0 |      0 |
|      -99 | DD         |        0 |      0 |
+----------+------------+----------+--------+

Other notes:


Calibrated exposures

NCSA

A subset of 1104591 out of 1403385 images (161 run out of 207) was achieved on NCSA HPSS service after the DC. Greg Daues has kindly restored the images at the following location (at NCSA):

% find /nfs/scratch/daues/restore/dpool/scratch/daues/SDRP_2014_NCSA/calexp_dir/sci-results/*/ -name 'calexp-*.fits.gz' | wc -l
1104591


% du -hs /nfs/scratch/daues/restore/dpool/scratch/daues/SDRP_2014_NCSA/calexp_dir/sci-results/
15T	/nfs/scratch/daues/restore/dpool/scratch/daues/SDRP_2014_NCSA/calexp_dir/sci-results/


IN2P3

The images are stored in the iRODS server:

$ ils /lsst-fr/data/DC_2013/calexps/sci-results
/lsst-fr/data/DC_2013/calexps/sci-results:
  C- /lsst-fr/data/DC_2013/calexps/sci-results/1752
  C- /lsst-fr/data/DC_2013/calexps/sci-results/1755
  ...
  C- /lsst-fr/data/DC_2013/calexps/sci-results/7202

$ ils /lsst-fr/data/DC_2013/calexps/sci-results/1752/1/g/calexp
/lsst-fr/data/DC_2013/calexps/sci-results/1752/1/g/calexp:
  bkgd-calexp-001752-g1-0040.fits
  ...
  calexp-001752-g1-0229.fits
  calexp-001752-g1-0230.fits

The files still need to be transferred to NCSA. It's estimated to be about 40 TB of data.

Coadded images