The preliminary ETA for the new hardware (15 nodes) is the end of November (heard at the same meeting)
Issues:
the network configuration of these 6 nodes prevents apps from connecting to the Internet
had to use a proxy
the real performance of the proxy while ingesting 40 TB of DP02 was limited by 200 MB/s
in the latest change of networking configuration, direct access to the Internet was enabled. Though, all external traffic now goes via a software router, limiting the available bandwidth to something like 10 Gbps or less. According to Yee, it was decided by Amedeo P and SLAC Cybersecurity Platoon. Yee thinks that in the future all ingested products should be put onto the SLAC/USDF-local data sources (shared POSIX filesystems, object stores).
the Replication system's algorithm for populating the director indexes has poor performance (has restricted implementation preventing it from utilizing available resources). This will be addressed in:
Colin Slater has it been enabled in the TAP service of qserv-prod? Are the truth tables of qserv-prod behaving the way it's expected (like we saw this at qserv-int)?
Igor Gaponenko on the status of the latest version of the remaining tables at IN2P3:
they got the "green light" to proceed with updating/ingesting these tables
according to my last comms with Gabriele Mainetti, this will happen shortly
the short hostnames for the Replication/Ingest workers that were introduced recently do not seem to work for qserv-ingest due to Kubernetes namespaces.
issues with the LSST Logger configuration reported in this development: