wiki:GestionDesDonnees
close Warning: Can't synchronize with repository "(default)" (/SVN/atlas does not appear to be a Subversion repository.). Look in the Trac log for more information.

Version 39 (modified by /C=FR/O=CNRS/OU=UMR5821/CN=Sabine Crepe-Renaudin/emailAddress=crepe@…, 6 years ago) (diff)

--

Data management on IN2P3-LPSC_LOCALGROUPDISK

IN2P3-LPSC_LOCALGROUPDISK is the place where you can store the data you need on the grid. These data can be the one you produce yourself or data produced by any other ATLAS people. It's a "space token" (a kind of virtual space booked on grid disks for a particular ATLAS use) opened to ATLAS LPSC (and Fr) members located on our T2 disks.
Currently (12/2015), 75 To are available on that space. If you consider you will need more space in the future, please contact Sabine.

Important notice: this space is shared and managed by the ATLAS LPSC group that means by you ! It's thus important that you:

  1. check that enough space is available before transferring your data
  2. check that datasets stored there are correctly registered on the grid (which is automatic if you use ATLAS standard tools)
  3. delete regularly the datasets that you don't need anymore

This twiki gives some hints on how to do this. Don't hesitate to contact Sabine if something is missing or not clear, you can also update the documentation by yourself.

What is the space left ?

See LOCALGROUPDISK plot on the DDM dashboard

Note! these plots are updated only once a day and several persons from the group may intend to transfer data at the same time. If you see that the available space starts to be small (less than 5-10 To), please inform the group.

Is my rucio account properly defined ?

  • Set up the ATLAS environment (setupATLAS)

for csh shell:

setenv ATLAS_LOCAL_ROOT_BASE /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source $ATLAS_LOCAL_ROOT_BASE/user/atlasLocalSetup.csh

for sh:

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source $ATLAS_LOCAL_ROOT_BASE/user/atlasLocalSetup.sh

  • To enable rucio command do:

    lsetup rucio

  • Connect to ATLAS voms server to identify yourself

    voms-proxy-init -voms atlas:/atlas -valid 96:0

  • You can check if you are registered in Rucio with the correct country with the command

    rucio-admin account list-attributes --account myaccount

the output should contain "country-fr: user" or "country-fr: admin"

  • To check where you have quota:

    rucio list-account-limits myaccount

  • To check what quota you have used:

    rucio list-account-usage myaccount

How to transfer my data to IN2P3-LPSC_LOCALGROUPDISK

  • for existing data on the grid, use r2d2 and define a new rule for the dataset you want to transfer
  • you can send directly the dataset you are producing on the grid to our localgroupdisk using the --destSE IN2P3-LPSC_LOCALGROUPDISK option

Data management in IN2P3-LPSC_LOCALGROUPDISK

  • To list the datasets in IN2P3-LPSC_LOCALGROUPDISK:

    rucio list-rules --account=USERNAME

  • To know the owner of a dataset replica, the last time it was used, etc:

    rucio get-metadata name_of_the_file

  • To list the replicas of a dataset

    rucio list-rules DATASETNAME

  • To remove your datasets replicas (rucio rules):

    rucio delete-rule --rse_expression IN2P3-LPSC_LOCALGROUPDISK DATASETNAME

Note: no output means success

Documentation