Page History

...

• • • Gen3
      • Fairly OK. Couple dataset lookup errors.  On step4. 13000 multi-tract not supported. But there are no overlaps in RC2.
      • issue with step3 submission (tract ids were omitted). Resolved? Yes. 
  • What changed? Annotate
  • What do we expect from w_2021_38? 
    • Changes toward Gen2-Gen3 parity:
      • Eli Rykoff has put in a significant fix towards gen2/gen3 parity (

        Jira
        server JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 key DM-31505

        )...fingers crossed.
      • It may be that we can't hope for parity with jointcal/fgcm without imposing a deterministic ordering of inputs in gen3 middleware, so 

        Jira
        server JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 key DM-31701

         was created.
    • What new metric can we expect next time
      • Need Cross-band astrometry
      • Why do AM1/PA1 look better in Gen3 between w22 and w26 in HSC RC2?
        • Change in the faro algorithm that selects bright stars. 
        • DM-31673, DM-25789
      • Why is gen2to3 astrometry/photometry metrics so terrible? Could be the afterburner or faro?  
        •  Eli Rykoff Jeffrey Carlin will look into which is causing problems. 
• Review DC2
  • w_2021_36
    • Gen2:  
      • back to SFM parity   Can now use this run to probe any gen2/gen3 differences in vanilla coaddition
      • only processed tract 3829
      • an ArithmeticError was observed in scarlet, 

        Jira
        server JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 9da94fb6-5771-303d-a785-1b6c5ab0f2d2 key DM-31713

         created and assigned to Fred Moolekamp to investigate.
    • Gen3:
      • On step4. for w36 when running step4 you need DRP.yaml#step4,getTemplate
      • Needed 120 GB RAM to measure wPerp in faro. 
      • faro is not using parquet tables. This should help the memory usage.  
      • Yusra: Look into Source Tables for DC2 Gen3. Update: this was indeed a gen2-only issue as the config overrides in mind are for pipe_drivers task (Lauren MacArthur apologizes for the noise…I may have mislead myself looking into a gen3 repo that was actually converted from gen2, so also did not have the visit-level parquet files).

...