Coordination Standup notes 2014-12-09

• Not a lot to report
• The major achievement is that Jim fixed the segFault in meas_base.
  • The was the lat blocker prior to Release.
• John continued to work on understanding and documenting build processes and documenting the process in Confluence. In the process uncovering a few other bugs like the difference in the tests between the results in Mac vs Linux which he's adopted and will push through resolution.

• They have a new hire who started on Monday and he's already up to speed in many ways. He will be working on the qserv core.
  • Jacek will send an official welcome once he gets his slac email address.
• They had another interesting Data Access meeting and Jacek will post the notes as soon as he finds 5 or 10 minutes - so sometime today or tomorrow.
• They are also meeting with the Fermi Gamma Ray datacam people; things are going well there. No blockers there.
• Jacek is collecting input for the National Academy of Sciences meeting that he's going to next week.
  • He requests comments on his earlier email to DMLT list.
• Peter Darch is at Slac monitoring how they communicate within the group and within the project.

• Not much to report
• He's been helping where he can: on Release, on debug, on Questions about Alerts - but perhaps Mario and Simon have looked at his comments to ensure they made sense.
  • Mario will look 'right now'.
• He's working down his ToDo list so that he can get back to working on the butler.

• Not too much to report
• Don has been working on the DES continuation proposal so not much work has been done on the Data Center Security plans
• There was a standup meeting last Friday with Frossie and K-T to talk about Infrastructure with regards to what they're planning, what they hope to deliver.
• Margaret is hopeful that their two new hires will wrap up the hiring hurdles and start in January. If possible, they will be attending the DM All Hands Meeting in February.
• Steve is having some technical difficulties - OpenStack is not done with maintenance - so they will be delayed a bit on this week's progress.

• Mario has built a stack on OSX 10.10 including the qserv stack. At the moment he's been fixing various configuration files on the qserv side . He'll have a patch for someone to review by the end of the day. (This activity has been infill during boring meetings.)
  • K-T: The goal is not to build qserv on Macs. A single node qserv is not going to provide much more value over mysql.
  • Mario: It's to exercise building with clang on BSD. Already, just doing this build has uncovered a couple of suspect things. For example, some unit tests fail on clang.
    
    • Jacek: send the patch to Fabrice for review when it's done.
• Other than that, he's not done anything else which is of interest to this group.

• Frossie finished a whole bunch of JIRA struff. She will send out a summary but she is holding it hostage until Mario responds to an email from her.
• She tagged a 10.0 Release Monday night. The build she initiated after the tagging took over 7 hours to complete. She was waiting during the meeting for another build to finish. If the build is clean, she wants folks to test it. She hoping for a soft release later on this week.
• Robyn and Frossie are looking at the machine disposition at NCSA to start spreading out the load rather than doing everything on lsst-dev. There are a bunch of machines which don't do a lot. So it seems a bit of a shame.
• Frossie is beginning to be skeptical that the builds are slow because of EUPS. She is going to try (this weekend) to copy stack and build onto Moya to see if it really is a EUPS problem. She suspects it will go faster. On lsst-dev, 'find' takes forever, 'du' takes forever, ...it's just laboring.
  • K-T: so you really think it's an NFS problem
  • Frossie: yes, I filed a report with lsst-admin but Bill and Mike don't think so.
• Frossie is going to get Coughlin off of lsst-dev since he definitely does a lot of I/O. We should probably, in the new year, think about having some of the lsst-dev users do more of their work on their own machines and spreading the load around. Perhaps taking Buildbot off, too.
  • Robert: I tried to reproduce this problem on another machine with thousands of tags and packages but couldn't make it go slowly.
  • Frossie: I actually don't think it is EUPS. 7 hours is ridiculous. Maybe DM's use of the filesystem is hitting some issue with the filesystem.
  • Mario: If you build on SSD is it still slow?
  • Frossie: I did that. Well, maybe not all the right data was moved  
• Frossie is going to move the relevant directories over to a machine on which she has root so she can troubleshoot.
• Frossie is starting to think that we need to upgrade the OS on lsst-dev. Unless anyone has an objection, she'll ask for a Centos 7 upgrade over Christmas when it's a little bit quieter..
  • Mario: We need to support Centos 6 or RH 6 for clusters since clusters are still limited to that OS. If we move our main development machine to Centos 7 it may be difficult to make sure we are still maintaining the cluster capability.
  • Frossie: Who runs 'bleeds' on the cluster right now?
  • Simon:   MAF/Sims
  • After much back and forth, Frossie decides to put out an RFC for the upgrade.
• Simon says that he will be moving the Sim stuff off of lsst-dev as soon as they can create a new Sim release.

• Simon: We are hiring, so I spent time looking at applications and building candidate lists.
• Simon wrestled with a bug where numpy array() returned in incorrect object. It went away with anaconda 2.1. He didn't have enough time to look into it (nor does he think anyone else does either) but he is going to write a unit test on afw.math which will at least fail when this happens.
  • Robert: You are aware that there is a transposition problem then, aren't you? yxx y representation.
  • Simon: Yes, I was just comparing sums.
  • Robert: That shouldn't be an issue.
  • ...more discussion followed. Talk to Simon/Robert for details.
• Simon started on the camera tool. He's passed it by JimC and Merlin.
  • Robert: Merlin is getting corrupt headers from Connor's test stand so don't worry about that.

• The group is hiring.
• happy to work on critical eups slowdowns, but couldn't reproduce them on his laptop
• Thinking of putting a simplified version of the RFC mechanism in place on the HSC side, so we can generate proper LSST RFCs before they appear on the LSST side of things.
  • Frossie will publish the JIRA schema so Paul can use it for the HSC-side update.

 I was late a few minutes and could not get in.
Here is the IPAC team status.

• No blocker. One version of JDBC can be used to access Qserv.
• Had a very good meeting with SLAC team on Monday.
• We are working on Firefly separation from IRSA applications for open source.
  • It is more work than previously thought (all the details).