DRP Metrics Monitoring 2020-09-18

Postponed from 9/7/2020

12pm - 12:50pm EDT

Yusra's Zoom: https://princeton.zoom.us/my/yusra

1.1.1. Attendees:

Yusra, Eli, Merlin, Jeff, Keith, Lauren, Hsin-Fang, Arun, Lee, Eric, Dan, Clare, Monika, John, Andres, Morgan, Fred, Leanne, Sophie, Tim, Jim, Robert

1.1.2. Agenda:

• Review Action items from last week
  • See RC3 dataset wish list, considerations

• Review "w_2020_34" and w_2020_36:
  • What was noteworthy in this rerun?  
  • What changed? Annotate.
    • We don’t have to debug live, if we spend more than 3 minutes on a question we’ll assign someone or group to look into it for next time.
• What do we expect from w_2020_38? That's tonight!
• What new metric can we expect next time?
• AOB

Minutes:

We will move this meeting from Friday in the future.

Welcome to Eric Morganson. Astronomer at NCSA. Taking care of Gen2 reprocessing. Monika will focus on Gen3.

Monika and Eric have been using manual config overrides. w38 gets tagged tonight. Can we get a volunteer to review ticket today. Lauren volunteers. Is there a Jenkins test? It is integration test with RC2. ci_hsc_gen2 explciitly does not read in those configs. Will need to be explicitly overridden. We need to look into this today.

Everyone send RC3 wishes to Jeff and Lauren. Nominal deadline has passed. Still taking feedback. Speak now or forever hold your peace. Does RC3 include fakes? We should consider this! Can we include in the standard reprocessing. What specific fake source should we put in? Bright stars. Want a few of different magnitudes. Don't need more than we have with RC2. Need to get size limits for RC3 from NCSA.

Sophie wanted to bring people together to talk about different fake catalogs. Maybe we should get together to form a plan and then discuss at this meeting in a month. Put in #science-pipelines.

Fake stats have not made in to dashboard

Start running matched visit metrics on DC2. Change due date to weekly 40. Assignee Eric.

Running the pipeline is hard. 25 different steps. Logs go in different places currently. Can we create a logs repo? Yes, let's make a logs repo.

Starting to make comparisons between validate_drp metrics and new framework for science verification analyses. Able to run on all three tracts of RC2 as of this morning.

What's new in w_2020_36?

FGCM fixes should be in. Metrics didn't make it to dashboard.

9813 z band  didn't work. When re-running discovered real code error. Tim is only user. Real error (as oppose to slurm error) in coadds for 9813 z band. Eric will file a ticket.

New stellar locus has a September entry. Does not appear for other tracts.

validate_drp metrics didn't make it, but stellar locus metric did make it. We should annotate what happened. It was only 9813 was affected.

Two problems aware now have to be solved. Not clear that they will go away by using w_2020_38. In principle can work on in parallel. dispatch verifiy color analysis, dispatch verify matched visit, one of the coadd steps. Let's get coadd driver issue sorted out today. Multiband driver is last processing step. All the QA tasks after that can be changed on the fly.

Known bug in coadd driver. Will take all weekend to run. Not much resources because just a single tract.

Before the weekly gets tagged tonight:

• FGCM configs are in, so don't need as many overrides
• Coadd driver bug
• Merge the matched visits bug (Lauren has ticket)

Getting a 38 is more useful. Starting 38 higher priority than fixing 36. We don't want COSMOS to fail again. We don't like putting in tickets mid-week.

rho statistics from Arun. Dashboard squash-sandbox. Look for pipe_analysis.rhoStatistics

rho statistics are metrics that evaluate PSF modeling, assess impact on 2pt correlation functions used for weak lensing. A bunch of two point correlation functions between ellipticities and size and ellipticity residuals and size residuals.

see Zuntz 2017 Section 3.3 for formal definition.

Benefits from having a large contiguous area because then can measure on larger angular scales.

rho statistics are a function, but dashboard can only compute scalars. Arun is taking averages of small scales (<1 arcmin) and large scales (>1 arcmin).

Need star catalog and PSF model. Calculations for all stars and for stars used for PSF modeling only.

Dependence on how ellipticities are measured. HSM is one algorithm.

w_2020_38 should have the fakes metrics. What are the fakes metrics measuring? Difference and spread of PSF magnitudes for stars between input and output. Galaxy photometry is probably next to add.

Expect a Doodle poll soon for times.

w_2020_36 would be useful to have converted to Gen 3.

1.1.3. Action Items