Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...


DEEP & UDEEPWIDETotal node-hours
singleFrameDriver

2758.08

skymap
  •   slurm job ID: 229995
  •   slurm job ID: 229996
0.02
jointcal

3466.34

fgcmcal83.45
skyCorrection

369.50

coadd

3735.56

multiband

20792.75

post-processing

152.68

forcedPhotCcd (low priority)
  •  DM-23867  slum job IDs: 246937,247134,247135,247264,247740,247762,247852

3050.69

matchedVisitMetrics (validate_drp)

1233.11

the new validateDrp.py?
visitAnalysis

4298.16

CompareVisitAnalysis (low priority)
  •  DM-23580 slurm job IDs: 247627,247769
724.71
colorAnalysis
  •  DM-23866 slurm job IDs: 47430,247453,247456

26.06

coaddAnalysis

2231.57

matchVisits (qa_explorer)
  •  DM-23831   slurm job IDs: 245159,246742,246820

15.42

...

  • DM-10755: 354 failed with errors of with lsst.pipe.base.TaskError("Reference %s doesn't exist" % (dataId,))   
  • 85 failed with lsst.daf.persistence.butlerExceptions.NoResults: No locations for get: datasetType:fgcm_photoCalib
  • 2 failed with lsst.daf.persistence.butlerExceptions.NoResults: No locations for get: datasetType:jointcal_wcs

    See DM-23867 for the data IDs


Compute Time

See the Job Summary Table for the breakdowns. 

The total compute for HSC-PDR2 is 31205.7 node-hours up to multiband processing (that is, no forcedPhotCcd, no post-processing, no QA pipelines of any kind, -- same as in the S18 PDR1 run for a fair comparison). Before the execution it was estimated by a simple scaling of multiplying PDR1 by 3 times = 9227.15*3 = 27681.45; that was only ~13% more.  

All non-QA pipelines sum to 34409.07 node-hours. 

All jobs sum to 42938.10 node-hours