At the SST F2F meeting on 5 November 2018, we reviewed the contents of the Source and DIASource tables. It was not possible to capture all the decisions about the columns in these tables in the meeting. The tables below contain the contents of the Source and DIASource tables transcribed from the DPDD. Three options were identified for each column, keep as is, keep but move to another (currently unspecified) table, or remove entirely.
We should also consider the possibility of using a Gaiastyle approach of storing everything and choosing what to publish
MWV: Overall Comments.
 The SST and the DPDD are about what the user sees and can access as a column. That's separate from how they are stored or generated. The lowerlevel Science Data Model will benefit from much of these discussions.
 I do think that
trail
* anddip
* columns need to be revisited. Here is a case where implementation is much more closely tied to the column definitions and where accepted practice is much less settled.trail
* More work should be done to come up with a planned model of how to do this photometry and related measurements and then that can be turned into a proposal for what the columns should be fore the DPDDdip
* These columns are the least certain to me. They are largely for characterizing our ability to do image subtraction.
Definition of Time in the Source/Object tables
Robert Lupton to clarify the meaning of time in the object table. (See DMSST action 20181105 DM SST F2F Agenda and Meeting notes)
RHL: I believe that the object table field in question is {{psRadecTai}}. I propose that we replace this field with an epoch that applies to all measurements in the object table. The epoch could be c. 2024 to coincide with the start of operations, or 2030 to minimise the amount of extrapolation required; we could use a different epoch for different releases, but that would probably be a little confusing.
LPG: Gaia choose this approach also and also has a different reference epoch for each DR. The reference epoch for Gaia DR3 is J2016.0. It is J2015.5 for Gaia DR2 and J2015.0 for Gaia DR1.
RHL: For DIASource table, midPointTai
also needs to be defined. We know the time profile for the shutter at every point in the focal plane from the camera's Hall effect sensors, and I propose that we define the midPointTai to be the mean of this time profile on the assumption that the source does not vary during the exposure. If we take data using snaps, this would be the average of the two halfexposures, and I propose that we do not weight these two measurements by the flux in the two snaps as this would require us to measure fluxes on the two snaps, which is not the current plan, and also change the definition of midPointTai
depending on whether the data is taken with or without snaps (and thus potentially depending on the band).
DIASource Table
Column  Description  Retain/Move/Delete? (Robert Lupton )  Comments 

diaSourceId  Unique source identifier 
 
ccdVisitId  ID of CCD and visit where this source was measured 
 
diaObjectId  ID of the DIAObject this source was associated with if any 
 
ssObjectId  ID of the SSObject this source has been linked to if any 
 RHL: I haven't thought about how to normalise/denormalise these tables, and would expect the ssObjectId to come via a join to diaObject LPG: I thought a DIASource was associated with either a DIAObject or an SSObject, but not both? 20220304: SSSource Table join on diaSourceId MWV: If SSObjects are not DIAObjects (which is my understanding of the current written plan), then both of these columns need to be retained. 
parentDiaSourceId  ID of the parent DIASource this object has been deblended from if any 
 RHL: I'm hoping not to have to deal with blended diaSources, but for completeness we need this. 
midPointTai  Time of midexposure for this DIASource. The visit midexposure time generally depends on the position of the source relative to the shutter blade motion trajectory. 
 RHL: or move to a lookup based on the known shutter timing. 
radec  Centroid 
 
radecCov  radec covariance matrix. 
 
xy  Column and row of the centroid. 
 
xyCov  Centroid covariance matrix. 
 
apFlux  the flux difference between the template and the visit image. 
 RHL define radius to be used. Is one enough? 
apFluxErr  Estimated uncertainty of apFlux. 
 
SNR  The signaltonoise ratio at which this source was detected in the difference image. 
 RHL this is not that well defined. I'd expect to use the S/N in the PSF flux (correct for point sources such as SNe and asteroids; not for comets), so we don't need this too MWV: Keep. But define what it means. Phase 1 detection threshold? Phase 2 detection threshold? abs(psFlux)/psFluxErr? LPG: The detection threshold will be in provenance and the same for all source detections. The signaltonoise ratio at which this source was detected is abs(psFlux)/psFluxErr and can be computed from the following 2 columns. We will remove this. ECB: note that the detection threshold is not abs(psFlux)/psFluxErr–detection SNR and measured SNR are different. See DM31811 
psFlux  Calibrated flux for point source model. Note this actually measures the flux difference between the template and the visit image. 
 LPG: Change to psfFlux if not done already. 
psFluxErr 
 RHL not present in the DPDD MWV: Agreed. Users will want this very simple quantity, even if it can be extracted from LPG: psfFluxErr if not done already  
psRadec  Centroid for point source model. 
 RHL We already have raDec MWV: Keep for now, but if implementation makes this redundant with LPG: Potentially confusing  will be the same forpoint sources. 
psCov  Covariance matrix for point source model parameters. 
 RHL In theory we need the full position x flux covariance, but in practice I think that we can separate them (and then they're in already) MWV: I think we should keep, even if some of the first implementations are only diagonal or block diagonal. LPG: psfCov. Not currently computed. Could compute but do not publish 
psLnL  Natural log likelihood of the observed data given the point source model. 
 RHL This was intended to be some sort of S/G number, I suspect, but I expect to use extendedness instead. This covers the next two entries too (psChi2, psNdata) MWV: I agree that LnL and Chi2 need some further thought. It's not clear to me that LnL is what we will want quote – are we including an astrophysical prior on the point source vs. extended source distribution as part of our point source model? ECB: I think all of these are trying to enable model comparison/goodness of fit for the point source vs trail vs dipole models. We should convey that information somehow... 
psChi2  chi^2 statistic of the model fit. 
 MWV: I would argue to keep. 
psNdata  The number of data points (pixels) used to fit the model. 
 MWV: I'm ambivalent. I can't figure out if this is usefully and consistently defined. Is this useful to understand performance of deblended objects? I certainly can't really imagine using this column across different visits. 
trailFlux  Calibrated flux for a trailed source model. A Trailed Source Model attempts to fit a (PSFconvolved) model of a point source that was trailed by a certain amount in some direction (taking into account the twosnap nature of the visit 
 RHL We want some sort of trailed number, probably. However, I'm not sure that we need it for every entry in the diaSource table, and I'm not convinced that we need all these numbers. The orientation is given by the second moments (Ixx etc. below), so a flux, fluxError and length is probably enough. So I'm voting for retain, but not to be taken literally. RHL We also need trailFluxErr MWV: I agree. More work needed here to develop a specific plan for trailing measurements and then return and figure out what all of these LPG: All tail* fields need to be reviewed with Mario 
trailRadec  Centroid for trailed source model. 
 RHL use raDec 
trailLength  Maximum likelihood fit of trail length (Note that we'll likely measure trailRow and trailCol) 
 
trailAngle  Maximum likelihood fit of the angle between the meridian through the centroid and the trail direction (bearing 
 
trailCov  Covariance matrix of trailed source model parameters. 
 
trailLnL  Natural log likelihood of the observed data given the trailed source model. 
 
trailChi2  chi^2 statistic of the model fit. 
 
trailNdata  The number of data points (pixels) used to fit the model. 
 
dipMeanFlux  Maximum likelihood value for the mean absolute flux of the two lobes for a dipole model. A Dipole Model attempts to fit a (PSFconvolved) model of two point sources of opposite signs. 
 RHL Remove all the dipole measurements. We will need some numbers to drive true/bogus measurements LPG: I understood we would move all dipole measurements to another table as they were useful for debugging (Simon Krughoff ?) MWV: I think I agree that the dipole measurements likely should be removed from the DPDD DiaSource Table. This isn't a particularly stable or welldefined quantity. LPG: Remove all dipole measurements frome the DPDD. Check with UW as to whether this is still useful and if it is the afw tables. ECB: are we sure there are no science uses for these? early identification of proper motion stars? 
dipFluxDiff  Maximum likelihood value for the difference of absolute fluxes of the two lobes for a dipole model. 
 
dipRadec  Centroid for dipole model. 
 
dipLength  Maximum likelihood value for the lobe separation in dipole model. 
 
dipAngle  Maximum likelihood fit of the angle between the meridian through the centroid and the dipole direction (bearing 
 
dipCov  Covariance matrix of dipole model parameters. 
 
dipLnL  Natural $log$ likelihood of the observed data given the dipole source model. 
 MWV: I think LnL for dipole makes particularly no sense. 
dipChi2  chi^ statistic of the model fit. 
 
dipNdata  The number of data points (pixels) used to fit the model. 
 
totFlux  Calibrated flux for point source model measured on the visit image centered at the centroid measured on the difference image (forced photometry flux) 
 RHL: unmeasurable in crowded fields. I believe that this was added to break degeneracies between dipole separation and amplitude. ECB: No, there's a science usewe need some way to go from difference image flux to total flux for variable stars. Forced photometry on the visit image seems preferable to me to the only alternative I'm aware of, which is figuring out the flux at the position of the DIASource in the template and adding it. The latter requires the same operations, has the same crowding problem, and creates more complications. MWV: This is a doable measurement. It just doesn't necessarily always mean what you might want it to mean. But we should still do and provide it, even though it means something different in crowded fields. LPG: Retain but change name to totPsfFlux 
totFluxErr  Estimated uncertainty of totFlux. 
 LPG: Retain but change name to totPsfFluxErr 
diffFlux  Calibrated flux for point source model centered on radec but measured on the difference of snaps comprising this visit. This flux can be used to detect sources changing on timescales comparable to snap exposure length ~15 
 LPG: Snaps are still in the baseline & the DPDD must agree with the baseline. Change name to snapDiffPsfFlux. Will remove if snaps are removed. 
diffFluxErr  Estimated uncertainty of diffFlux. 
 LPG: Change to snapDiffPsfFluxErr 
fpBkgd  Estimated background at the position (centroid) of the object in the template image. 
 LPG: What is fp? 
fpBkgdErr  Estimated uncertainty offpBkgd. 
 
Ixx  Adaptive second moment of the source intensity. See Bernstein & Jarvis [2] for detailed discussion of all adaptivemoment related quantities47 . 
 
Iyy  Adaptive second moment of the source intensity. See Bernstein & Jarvis [2] for detailed discussion of all adaptivemoment related quantities47 . 
 
Ixy  Adaptive second moment of the source intensity. 
 
Icov  Ixx, Iyy, Ixy covariance matrix 
 
IxxPSF  Adaptive second moment for the PSF 
 
IyyPSF  Adaptive second moment for the PSF. 
 
IxyPSF  Adaptive second moment for the PSF. 
 
IxxDeconv, IxyDeconv, IyyDeconv  Estimate of PSFdeconvolved second moments. Initial implementation may be quadrature difference of second moments and 
 LPG: Follow up with SNAPs/SSSC about the usefulness of these. 
extendedness  A measure of extendedness, computed using a combination of available moments, or from a likelihood ratio of point/trailed source models (exact algorithm TBD). extendedness = 1 implies a high degree of confidence that the source is extended. extendedness = 0 implies a high degree of confidence that the source is pointlike. 
 
spuriousness  A measure of spuriousness, computed using information from the source and image characterization, as well as the information on the Telescope and Camera system (e.g., ghost maps, defect maps, etc.). 
 RHL some sort of true/bogus number, I expect. Maybe more than one. LPG Measures (>1) to separate morph from other measures. Write desciprtion better. 
Flags  Various useful flags. 

Source Table
Column  Description  Retain/Move/Delete? (Robert Lupton )  Comments 

sourceId  Unique source identifier 
 
ccdVisitId  ID of CCD and visit where this source was measured 
 
objectId  ID of the Object this source was associated with, if any 
 
ssObjectId  ID of the SSObject this source has been linked to, if any 
 LPG: Same reasoning as above  can join tables 
parentSourceId  ID of the parent Source this source has been deblended from, if any 
 
xy  Position of the object (centroid) 
 LPG: See discussion above 
xyCov  Covariance matrix for xy 
 LPG: See discussion above 
radec  Calibrated RA, Dec 
 
radecCov  Covariance matrix for radec. 
 
apFlux  Calibrated aperture flux. 
 LPG: Calibration ap flux? Flux to which all else is tied? check with Eli/Robert. E.g. compensated aperture flux. 
apFluxErr  Estimated uncertainty of apFlux. 
 
sky  Estimated background (sky) surface brightness at the position (centroid) of the source. 
 LPG: change to fpBkgd / fpBkgdErr. Remove sky from description. Majke same as description in DIASource. 
skyErr  Estimated uncertainty of sky. 
 LPG: change to fpBkgd / fpBkgdErr 
psRadec  Point source model ra/dec 
 LPG: See above 
psFlux  Calibrated point source model flux. 
 LPG: Change to psfFlux/psfFluxErr 
psFluxErr  Calibrated point source model flux uncertainty. 
 MWV: Breaking out the LPG: Change to psfFlux/psfFluxErr? To reconsider. 
psCov  Pointsource model covariance matrix. 
 LPG: psfCov. Not currently computed. Could compute but do not publish. Will revise at a later date 
psLnL  Natural log likelihood of the observed data given the point source model. 
 MWV: What priors will we use for the log likelihood? Are we including astrophysical priors? Astrophysical priors that are a function of Galactic (l, b)? I think the answer has to be no, but if it is, then this isn't a likelihood. 
psChi2  $\chi^2$ statistic of the model fit. 
 
psNdata  The number of data points (pixels) used to fit the model. 
 LPG: See above 
Ixx  Adaptive second moment of the source intensity. See \citet{2002AJ....123..583B} for detailed discussion of all adaptivemoment related quantities\footnote{Or \url{http://ls.st/5f4} for a brief summary.}. 
 
Iyy  Adaptive second moment of the source intensity. 
 
Ixy  Adaptive second moment of the source intensity. 
 
Icov  Ixx, Iyy, Ixy covariance matrix 
 
IxxPSF  Adaptive second moment for the PSF. 
 
IyyPSF  Adaptive second moment for the PSF. 
 
IxyPSF  Adaptive second moment for the PSF. 
 LPG: Add deconvolved PSF  as per above. 
apNann  Number of elliptical annuli (see below). 
 MWV: I think this name is confusing. My brain keeps pulling a LPG: Not currently calculating elliptical apertures. Put forward changing elliptical to circular. Change to apNAnn. Need to check where the geometry of all the annulus is stored. 
apMeanSb  Mean surface brightness within an annulus. 
 LPG: 1 for each annulus. Flux v SB ? Assueme we want SB 
apMeanSbSigma  Standard deviation of apMeanSb 
 LPG: 1 for each annulus 
extendedness  A measure of extendedness (exact algorithm TBD). $extendedness=1$ implies a high degree of confidence that the source is extended. $extendedness=0$ implies a high degree of confidence that the source is pointlike. 
 
flags  Various useful flags. 
