wiki:GestionDesDonnees
close Warning: Can't synchronize with repository "(default)" (/SVN/atlas does not appear to be a Subversion repository.). Look in the Trac log for more information.

Version 16 (modified by /C=FR/O=CNRS/OU=UMR5821/CN=Sabine Crepe-Renaudin/emailAddress=crepe@…, 6 years ago) (diff)

--

Data management on IN2P3-LPSC_LOCALGROUPDISK

IN2P3-LPSC_LOCALGROUPDISK is the place where you can store the data you need on the grid. These data can be the one you produce yourself or data produced by any other ATLAS people. It's a "space token" (a kind of virtual space booked on grid disks for a particular ATLAS use) opened to ATLAS LPSC (and Fr) members located on our T2 disks.
Currently (12/2015), 75 To are available on that space. If you consider you will need more space in the future, please contact Sabine.

Important notice: this space is shared and managed by the ATLAS LPSC group that means by you ! It's thus important that you:

  1. check that enough space is available before transferring your data
  2. check that datasets stored there are correctly registered on the grid (which is automatic if you use ATLAS standard tools)
  3. delete regularly the datasets that you don't need anymore

This twiki gives some hints on how to do this. Don't hesitate to contact Sabine if something is missing or not clear, you can also update the documentation by yourself.

What is the space left ?

See LOCALGROUPDISK plot on the http://dashb-atlas-ddm-acc.cern.ch/dashboard/request.py/ddmaccounting#button=srmview&sites[=IN2P3-LPSC&sitesCat[]=All+Countries&sitesSort=2&sitesCatSort=0&start=&end=&timerange=lastWeek&granularity=Daily&gen_spacetoken=0&gen_streamname=0&gen_project=0&gen_datatype=0&sortby=10&series=All DDM dashboard]

Note! these plots are updated only once a day and several person from the group may intend to transfer data at the same time. If you see that the available space stats to be small (less than 5 To), please inform the group.

Is my rucio account properly defined

  • You can check if you are registered in Rucio in the correct country with the command

rucio-admin account list-attributes --account myaccount
the output should contain "country-xx: user" or "country-xx: admin"

  • To check where you have quota: rucio list-account-limits myaccount
  • To check what quota you have used: rucio list-account-usage myaccount

Data management in IN2P3-LPSC_LOCALGROUPDISK

To list the datasets in IN2P3-LPSC_LOCALGROUPDISK:

To know the owner of a dataset replica:

To list your datasets replicas:

To list their size:

awk -F"," -v a=0,b=0 '{print $6; a=a+$6; print "total = " a; b=a/1000000000; print "total en Giga = " b}' mesdatasets.txt

To remove your datasets replicas: