Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagebash
cd /path/to/working/directory
git clone httphttps://devgithub.lsstcorp.org/git/contrib/com/lsst-dm/tutorials.git
export PHODEMO_BIN=$PWD/tutorials/phoSimTutorial/python

...

We will use the SDSS Stripe 82 Standard Star Catalog of Ivezic et al. (2007 AJ, 134, 973) which contains u,g,r,i,z photometry of over 1 million stars with r-band standard errors < 0.05 mag. Download the catalog (67 MB) and unpack it in a working directory. 

Code Block
languagebash
themeConfluence
languagebash
cd /path/to/phosim/working/directory
mkdir phoSimData
cd phoSimData
curl -O http://www.astro.washington.edu/users/ivezic/sdss/catalogs/stripe82calibStars_v2.6.dat.gz
gunzip stripe82calibStars_v2.6.dat.gz

...

For this example we will use the range -0.5 < RA < 0.5 and -0.5 < Dec < 0.5, which is larger than the extent of the central Sensor. For simplicity a flat SED is used for all catalog stars, although the brightnesses will be taken from the reference catalog per filter. The visit ID is arbitrary, but it is handy to encode a date and some sort of sequence to aid bookkeeping. Create separate trim files for each filter of interest: 

Code Block
languagebash
themeConfluence
languagebash
$PHODEMO_BIN/refCalCat.py --filter 'u' --ID 2014040 --path flatSED \
    --ralim -0.5 0.5 --declim -0.5 0.5 \
    stripe82calibStars_v2.6.dat > S82_u.trim
$PHODEMO_BIN/refCalCat.py --filter 'g' --ID 2014041 --path flatSED \
    --ralim -0.5 0.5 --declim -0.5 0.5 \
    stripe82calibStars_v2.6.dat > S82_g.trim
$PHODEMO_BIN/refCalCat.py --filter 'r' --ID 2014042 --path flatSED \
    --ralim -0.5 0.5  --declim -0.5 0.5 \
    stripe82calibStars_v2.6.dat > S82_r.trim
$PHODEMO_BIN/refCalCat.py --filter 'i' --ID 2014043 --path flatSED \
    --ralim -0.5 0.5  --declim -0.5 0.5 \
    stripe82calibStars_v2.6.dat > S82_i.trim
$PHODEMO_BIN/refCalCat.py --filter 'z' --ID 2014044 --path flatSED \
    --ralim -0.5 0.5  --declim -0.5 0.5 \
    stripe82calibStars_v2.6.dat > S82_z.trim

...

In order to perform astrometric calibration, index files must be available for the astrometry_net_data package to enable WCS solutions for science images. Since we are using the SDSS Stripe 82 Standard Star Catalog (i.e., the same reference catalog that was used for the Summer 2013 Data Release Production of Stripe 82 data), the index files have already been built; we need only to download and install the index files and setup the appropriate package to make use of them

Code Block
languagebash
themeConfluence
languagebash
curl -O http://lsst-web.ncsa.illinois.edu/sdss-s12/sdss-2012-05-01-0.tgz 
tar xzf sdss-2012-05-01-0.tgz
setup -r sdss-2012-05-01-0

...

Setup the following packages and set some handy environment variables, including the path to the PhoSim installation directory, and another to contain the number of cores available on your machine to enable parallel processing: 

Code Block
languagebash
themeConfluence
languagebash
setup cfitsio
setup fftw
export PHOSIM_DIR=/path/to/phoSim
export NCORES=$((sysctl -n hw.ncpu || (test -r /proc/cpuinfo && grep processor /proc/cpuinfo | wc -l) || echo 2) 2>/dev/null)

For this example we will simulate images for a single Sensor (1,1) in the central Raft (2,2) of the LSST focal plane. Create the output directory and start the PhoSim processing: 

Code Block
languagebash
themeConfluence
languagebash
# Use the following sensor specification for the entire central raft:
# export SENSORS="R22_S00|R22_S01|R22_S02|R22_S10|R22_S11|R22_S12|R22_S20|R22_S21|R22_S22"
# A single sensor takes considerably less processing time:
export SENSORS="R22_S11"
 
# Invoke the following commands for each filter, substituting the output and filter name as needed:
mkdir work
mkdir 2014042.out
python $PHOSIM_DIR/phosim.py $PWD/S82_r.trim \
    -w $PWD/work \
    -o $PWD/2014042.out \
    -c $PWD/clean.params \ 
    -p $NCORES \ 
    -s $SENSORS >& phosim_r_log.txt

Now create images in the other colors, taking care to match the trim file name to the passband, and creating subdirectories for the output. 

Code Block
languagebash
themeConfluence
languagebash
mkdir 2014040.out
python $PHOSIM_DIR/phosim.py $PWD/S82_u.trim -w $PWD/work -o $PWD/2014040.out -c $PWD/clean.params-p $NCORES -s $SENSORS >& phosim_u_log.txt
mkdir 2014041.out
python $PHOSIM_DIR/phosim.py $PWD/S82_g.trim -w $PWD/work -o $PWD/2014041.out -c $PWD/clean.params-p $NCORES -s $SENSORS >& phosim_g_log.txt
mkdir 2014043.out
python $PHOSIM_DIR/phosim.py $PWD/S82_i.trim -w $PWD/work -o $PWD/2014043.out -c $PWD/clean.params-p $NCORES -s $SENSORS >& phosim_i_log.txt
mkdir 2014044.out
python $PHOSIM_DIR/phosim.py $PWD/S82_z.trim -w $PWD/work -o $PWD/2014044.out -c $PWD/clean.params-p $NCORES -s $SENSORS >& phosim_z_log.txt

...

The images in the *.out subdirectories do not follow the LSST Stack convention for data organization or file nomenclature. Thus, they need to be re-named and placed into a repository for subsequent processing by the LSST pipeline tasks. This can be accomplished with the tutorials script rename_images.pyNow run the script for each *.out directory, looping with your favorite shell: 

Section
Column
Code Block
languagebash
themeConfluence
languagebash
title(t)csh loop
cd /your/phoSim/working/directory
foreach i (*.out)
  cd $i 
  python $PHODEMO_BIN/rename_images.py ../imSim *E000.fits.gz
  cd ../
  end
Column
Code Block
languagebash
themeConfluence
languagebash
titlebash loop
cd /your/phoSim/working/directory
for i in $( ls -d *.out); do
    cd $i
    python $PHODEMO_BIN/rename_images.py ../imSim *E000.fits.gz
    cd ../
done

...

Source the LSST environment ($INSTALL_DIR refers the directory where the LSST Stack was installed), and setup the following packages to run the processing pipeline and related tasks:

Code Block
languagebash
themeConfluence
languagebash
setup pipe_tasks
setup obs_lsstSim --keep  # to process LSST Sims

Now create the input registry for pipeline processing: 

Code Block
languagebash
themeConfluence
languagebash
export SIM_BIN=$OBS_LSSTSIM_DIR/bin
echo "lsst.obs.lsstSim.lsstSimMapper.LsstSimMapper" >> imSim/_mapper
$SIM_BIN/genInputRegistry.py ./imSim -o ./imSim/registry.sqlite3

...

Run the pipeline to process the simulated images. This will generate catalogs of source measurements. 

Code Block
languagebash
themeConfluence
languagebash
export SIM_DIR=$PWD
$SIM_BIN/processEimage.py $SIM_DIR/imSim --output $SIM_DIR/calexp_dir --clobber-output --id visit=2014040..2014044

...

When running the script to create the merged catalog, the visit selector (last argument) may be a single visit number, a sequence of visits joined by carrots (2004040^2004044), or a range of visits that are contiguous in sequence, separated by two dots, as in the following: 

Code Block
languagebash
themeConfluence
languagebash
$PHODEMO_BIN/makeCat.py ./calexp_dir  2014040..2014044 > mergeCat.txt

...