Multisite Campaign Notes

This page contains notes on setting up and running a multisite processing campaign

including getting going with Rucio and rucio-butler file transfer tools.

Multisite campaign processing works as follows:

Three butlers with Repos are setup at the three main Rubin processing sites.

To connect to Rucio, one needs to have the rucio-clients package installed.

One may find instructions for this at rucio.cern.ch.

1. setup a current stack (for instance w_2024_12)
2. pip install rucio-clients.  (you may need to update your version of pip)
3. rucio ping. (this will report an error as you don't have a rucio.cfg file).
4.  Make a rucio config rucio.cfg file which looks like this:

   [client]
   # Rubin Hosts
   # production rucio: (currently in use)
   rucio_host = https://rubin-rucio.slac.stanford.edu:8443
   auth_host = https://rubin-rucio.slac.stanford.edu:8443
   # development rucio:
   #rucio_host = https://rubin-rucio-dev.slac.stanford.edu:8443
   #auth_host = https://rubin-rucio-dev.slac.stanford.edu:8443
   # change to your account/user name
   account = yanny
   auth_type = ssh
   # send your *public* version of your ssh RSA key to Rubin Rucio Admin (i.e. Yuyi Guo)
   ssh_private_key = /pbs/home/b/byanny/.ssh/id_rsa
   request_retries = 3
   
   

5. Change the account = yanny line to match 
6. export RUCIO_CONFIG=rucio.cfg 
7. try rucio ping again and also 'rucio whoami'. once your rsa key is installed by a rucio admin, you should rucio whoami i again:
8. try rucio ping again. It should return the version of the rucio server (something like 34.0 or higher)

Once you have a working version of rucio-clients installed these common commands

Some common ones are:

rucio -h (help)

rucio list-scopes

rucio list-dids <scope>:*  (list all datasets)

rucio list-dids <scope>:* --filter 'type=all'.  (list all datasets and all files in each dataset)

rucio list-rses   (list all rucio storage endpoints)

rucio list-file-replicas <scope>:<name>  (show which replicas in which rse a did with <scope>:<name> is located in)

rucio get-metadata <scope>:<name>. (list metadata for a scope:name)

To make a new scope:

rucio-admin scope add --account yanny --scope user.amado

To register objects that are in a butler into a RUCIO rse, one uses the dm_rucio_register command available from github