Page History

The code driving our variability models is a part of the sims_phoUtils package.  Specifically, it is saved in

sims_photUtils/python/lsst/sims/photUtils/Variability.py

This code defines two mixins, VariabilityStars and VariabilityGalaxies.  These mixins provide getters for the columns 'delta_lsst_u', 'delta_lsst_g', etc. and, in the case of VarabilityGalaxies, 'delta_uAgn', 'delta_gAgn', etc.  When the photometry getters (see this page) calculate the magnitudes 'lsst_u' or 'uAgn', they check to see if the InstanceCatalog contains getters for 'delta_lsst_u' or 'delta_uAgn'.  If it does, these values are calculated and added to the baseline magnitudes before output.  Thus, in order to include variability in a catalog, users only have to make sure that the InstanceCatalog daughter class inherits from either VariabilityStars or VariabilityGalaxies.  Similarly, if users wish to calculate magnitudes in a different, non-LSST system (e.g. calculate columns such as 'myCustomMagnitude'), they just need to make sure that getter methods are defined for columns named 'delta_myCustomMagnitude' (or whatever the baseline columns are named).  Assuming that the getters for 'myCustomMagnitude' are written according to the formalism specified on the Simulated Photometry page, CatSim will know what to do.

Below, we discuss how the provided variability models behind 'delta_lsst_u', 'delta_lsst_g', etc. are actually encoded. the class Variability, a mixin which, when included in an InstanceCatalog daughter class, allows the catalog to output the columns lsst_u_var, sigma_lsst_u_var, uAgn_var, sigma_uAgn_var, etc. (which is to say, all of the columns provided by PhotometryStars and PhotometryGalaxies above, but with the suffix 'var' indicating that they account for source variability).  In the case of stars, magnitude lsst_u_var is the magnitude lsst_u_var with a delta_magnitude added because of variability.  In the case of galaxies, variability only occurs in the AGN component, though this is taken into account when calculating lsst_u_var as well.  The uncertainties sigma_lsst_u_var and sigma_uAgn_var are calculated the same way that sigma_lsst_u and sigma_uAgn are calculated, except using the variable magnitudes in place of the mean magnitudes.  There is also a stellar_magNorm_var and a magNorm_total_var which accounts for variability in the magNorm used by phoSim. 

How does Variability calculate the modified magnitudes?

There are multiple models of variability stored in the catalog simulations framework (one for each unique physical source of variability).  The Variability class keeps track of these different models by building a registry of available models and accessing them based on the contents of the database.  If one examines the source code in Variability.py, one sees that the class Variability is declared as:

@register_class
class Variability(PhotometryBase):

The @register_class is a decorator defined in sims_catalogs_measures/python/lsst/sims/catalogs/measures/instance/decorators.py.  When the class Variability is declared (not when it is instantiated by the user, but when it is declared by Python in preparation for a user instantiation), this decorator looks at the class and, if it does not have an attribute _methodRegistry, it creates an empty dictionary _methodRegistry.  The decorator then walks through the methods defined in Variability (or whatever class @register_class has been applied to) and adds any method with an attribute _registryKey to the dictionary _methodRegistry.  This is how the catalog simulator stores its physical variability models.

The actual variability models are at the end of Variability.py and they are declared something like

    @register_method('applyAgn')
    def applyAgn(self, params, expmjd_in):

@register_method is another decorator (also defined in decorators.py).  This decorator modifies any method to which it is applied by adding a member attribute _registryKey and setting it equal to the argument of @register_method.  Thus, when @register_class walks through the methods of Variability, it will log all of the methods marked by @register_method and store them in _methodRegistry.  Put another way, after the class has been declared, a call to

_methodRegistry['applyAgn'](arguments)

will call the method applyAgn defined above.

Variability is stored in the database as a json-encoded dictionary varParamStr such that

• varParamStr['varMethodname'] = the name of the variability method to be called (e.g. 'applyAgn') for this database row
• varaParamStr['pars'] = a dictionary of the parameters needed by that variability method

The catalog simulator applies variability by using json.loads(varParamStr) to rebuild that dictionary.  It then calls the appropriate method, passing the dictionary of parameters, i.e.

varCmd = json.loads(varParams)

method = varCmd['varMethodName']
params = varCmd['pars']
expmjd=self.obs_metadata.mjd #because the variability models all need to know the date of the observation

output = self._methodRegistry[method](self, params,expmjd)

where varParams is a stand-in for the varParamStr for a single database row.  This code was taken from Variability.applyVariability, which returns the delta_magnitudes that need to be added to the mean magnitudes to give the total variable magnitudes.

An example

Below is code that creates a fake database of 10 RRLyrae stars and then outputs them to a catalog.  Hopefully this illustrates how variability works internally (note how varParamStr is constructed in the method makeRRLyTestDB) and how to access variability (see how the class RRLyCatalog is written):

Currently, all of the provided variability models are light curve models:  example light curves (magnitude as a function of time) are stored for each of the LSST bands.  When a variability model is called, it is passed in parameters initializing the light curve and an MJD (mean Julian date) for the observation.  The variability model returns the change in magnitude relative to the mean magnitude for that model at that MJD.  Every variable object stored in the LSST database has an attribute 'varParamStr' which is a string specifying what specific variability model corresponds to that object and what parameters need to be given to correctly initialize the model.  This string is read in as an ordinary database column and passed to the method applyVariability which belongs to the class Variability, from which VariabilityStars and VariabilityGalaxies inherit.  Below, for example, is the getter for 'delta_lsst_u', 'delta_lsst_g', etc. in the mixin VariabilityStars

    @compound('delta_lsst_u', 'delta_lsst_g', 'delta_lsst_r',
             'delta_lsst_i', 'delta_lsst_z', 'delta_lsst_y')
    def get_stellar_variability(self):
        """
        Getter for the change in magnitudes due to stellar
        variability.  The PhotometryStars mixin is clever enough
        to automatically add this to the baseline magnitude.
        """
        varParams = self.column_by_name('varParamStr')

        output = numpy.empty((6,len(varParams)))

        for ii, vv in enumerate(varParams):
            if vv != numpy.unicode_("None") and \

import os, numpy, sqlalchemy, sqlite3, json
from lsst.sims.catalogs.generation.db import CatalogDBObject, ObservationMetaData
from lsst.sims.catalogs.measures.instance import InstanceCatalog, compound
from lsst.sims.photUtils.Photometry import PhotometryStars
from lsst.sims.photUtils.Variability import Variability

#list three actual sed files from sims_sed_library
sedFiles = ['km50_7500.fits_g25_7600','km30_5500.fits_g10_5600', 'km25_6000.fits_g15_6040']

#list three actual RRLyrae variability light curves from sims_sed_library
lcFiles = ['rrly_lc/RRab/962043_per.txt', 'rrly_lc/RRab/2089796_per.txt', 'rrly_lc/RRab/1164401_per.txt']

def makeRRLyTestDB(size=10, **kwargs):
    """
    Make a test database to serve information to the mflarTest object
    """
    conn = sqlite3.connect('testDatabase.db')
    c = conn.cursor()
    try:
        c.execute('''CREATE TABLE testRRLy
               self.obs_metadata is not None and \
 (id  int, varParamStr text, sedfilename text)''')
        conn.commit()
    except:
self.obs_metadata.mjd is not None:

        raise RuntimeError("Error creating database.")
    
 deltaMag = self.applyVariability(vv)
 for i in xrange(size):
        
    output[0][ii] = deltaMag['u']
  #randomly pick one of our SED files
        sedFileoutput[1][ii] = sedFiles[numpy.random.randint(0,3)deltaMag['g']
        
        #randomly choose a start MJD for the variability
output[2][ii] = deltaMag['r']
                tStart = numpy.random.random_sample()*1.0e4+3.0e4

output[3][ii] = deltaMag['i']
           #randomly choose one of our RRLyrae light curves
 output[4][ii] = deltaMag['z']
           lcName      output[5][ii] = lcFiles[numpy.random.randint(0,3)]
deltaMag['y']

            else:
           #set  varParamStr using 'applyRRly' as our varMethodName
 output[0][ii] = 0.0
                output[1][ii] = #0.0
          #consulting    the method applyRRly in Variability.py output[2][ii] = 0.0
        #we   see that it takes as parameters 'tStartMjd' and
 output[3][ii] = 0.0
         #'filename',  thus these are the keys that must be output[4][ii] = 0.0
        #included in the dictionary 'pars'
        output[5][ii] = 0.0

        return output

Variability.applyVariability interprets the varParamStr and calls the correct variability model, returning a dict of delta_magnitudes.

How are the variability models encoded?

In addition to applyVariability, the Variability class contains method for each of the supported variability light curve models (e.g. 'applyMflare' or 'applyRRly').  These methods are stored in a registry dict Variability._methodRegistry.  The keys of this dict are included in varParamStr.  When applyVariability is called, it extracts the appropriate key from varParamStr and calls the corresponding method.  Here is the source code for applyVariability

    def applyVariability(self, varParams):_raw = {'varMethodName':'applyRRly', 'pars':{'tStartMjd':tStart, 'filename':lcName}}
        """
        varParams #usewill jsonbe tothe encodevarParamStr thecolumn varParamStarfrom the data base
        varParamStrThis method =uses json.dumps(varParamStr_raw)
        
 to convert that into a machine-readable object
        it #insertuses the rowvarMethodName to intoselect the data base correct variability method from the

        qstr = '''INSERT INTO testRRLy VALUES (%i, '%s', '%s')''' % (i, varParamStr,sedFile)
        c.execute(qstr)dict self._methodRegistry

        it uses then feeds the pars array to that method, under the assumption
        
that the parameters needed  conn.commit()
    conn.close()


#a toy DBObject class for our RRLyrae
class RRLyDB(CatalogDBObject):
    objid = 'RRLyTest'
    tableid = 'testRRLy'by the method can be found therein

        @param [in] varParams is a string object (readable by json) that tells
    dbAddress  = 'sqlite:///testDatabase.db'
    idColKey = 'id' us which variability model to use

    #this must be included because@param the[out] columnoutput inis thea test
dict of magnitude offsets #databasekeyed is 'sedfilename', but to the PhotometryStarsfilter mixinname
    #(and most of the InstanceCatalog software) excpects
    #'sedFilename'
    #
e.g. output['u'] is the magnitude offset in the u band
     columns  = [('sedFilename','sedfilename',str,40)]

#a toy catalog class for our RRLyrae
#this class needs to inherit from PhotometryStars in order to calculate the magnitude lsst_u
#it needs to inherit from Variability in order to calculate the magnitude lsst_u_var
class RRLyCatalog(InstanceCatalog,PhotometryStars,Variability):
    catalog_type = 'RRLyCatalog'
    column_outputs = ['id','lsst_u','lsst_u_var']
    
    default_columns=[('magNorm',14.0,float)]
    

#delete testDatabase.db if it exists
if os.path.exists('testDatabase.db'):
    os.unlink('testDatabase.db')


makeRRLyTestDB()

myObsMetaData = ObservationMetaData(mjd=52000.0)

myDB = CatalogDBObject.from_objid('RRLyTest')
myCatalog = myDB.getCatalog('RRLyCatalog',obs_metadata=myObsMetaData)

myCatalog.write_catalog("RRLyTestCatalog.dat",chunk_size=10) """

        if self.variabilityInitialized == False:
            self.initializeVariability(doCache=True)

        varCmd = json.loads(varParams)
        method = varCmd['varMethodName']
        params = varCmd['pars']
        expmjd=self.obs_metadata.mjd
        output = self._methodRegistry[method](self, params,expmjd)
        return output

The registry dict Variability._methodRegistry is constructed using decorators.  If one examines the source code in Variability.py, one sees that the class Variability is declared as:

@register_class
class Variability(PhotometryBase):

The @register_class is a decorator defined in sims_catalogs_measures/python/lsst/sims/catalogs/measures/instance/decorators.py.  When the class Variability is declared (not when it is instantiated by the user, but when it is declared by Python in preparation for a user instantiation), this decorator looks at the class and, if it does not have an attribute _methodRegistry, it creates an empty dictionary _methodRegistry.  The decorator then walks through the methods defined in Variability (or whatever class @register_class has been applied to) and adds any method with an attribute _registryKey to the dictionary _methodRegistry.  This is how the catalog simulator stores its physical variability models.

The actual variability models are at the end of Variability.py and they are declared something like

    @register_method('applyAgn')
    def applyAgn(self, params, expmjd_in):

@register_method is another decorator (also defined in decorators.py).  This decorator modifies any method to which it is applied by adding a member attribute _registryKey and setting it equal to the argument of @register_method.  Thus, when @register_class walks through the methods of Variability, it will log all of the methods marked by @register_method and store them in _methodRegistry.  Put another way, after the class has been declared, a call to

_methodRegistry['applyAgn'](arguments)

will call the method applyAgn defined above.

Return to main catalog simulations documentation page.