Disk quota on /tmp and /scratch
Introduction
Please have a look to our
T3 usage policies.
Note that on t3ui08 Urs wanted /scratch
thresholds 98% 99%
Puppet module quota-fs-users-in-ldap
In a Nutshell:
- The Puppet module on AFS
quota-fs-users-in-ldap
mounts automatically with usrquota
the /tmp
or the /scratch
filesystem.
- Inside of it there is script
quota-ldapusers-filesystem.sh
to enforce users quotas ( users retrieved from our LDAP ) on a filesystem type ext3
or xfs
.
- We use a custom RPM
quota-3.17-1.2.5
to make SSL LDAP connections vs our LDAP. The RPM comes from our own repository.
Basically you'll run
quota-ldapusers-filesystem.sh
on each UI or WN with
--fs=/tmp
or
--fs=/scratch
when:
- A new LDAP user is created, so you need to apply the quota for him/her, the script will do it for everybody so for him/her as well.
- To increase or decrease the
--soft
or the --hard
quota of everyone: then please update the T3 usage policies.
- You are installing from scratch a UI or a WN, so
/tmp
and /scratch
are mounted with usrquota
but any user quota is defined.
Bash script quota-fs-users-in-ldap.sh
Here follows a script invocation:
[root@t3ui02 ~]# /root/USERS-QUOTA/quota-ldapusers-filesystem.sh
Script to enforce users quotas on a generic filesystem. Users are retrieved from an LDAP server.
Here some info for users: https://wiki.chipp.ch/twiki/bin/view/CmsTier3/Tier3Policies
Usage: quota-ldapusers-filesystem.sh [-h|--help] --fs filesystem --ldaps ldapserver --quota quota
-h, --help Print the help message and exit
--ldaps SSL LDAP server listening on port 636 from where to retrieve the users list.
--soft Soft Quota in % such that 0 < Soft Quota < Hard Quota < 100.
--hard Hard Quota in % such that 0 < Soft Quota < Hard Quota < 100.
--fs Users quotas will be applied to this filesystem. So far just ext3 and xfs were tested.
The script will saves its logs into: /tmp/quota-ldapusers-filesystem.sh.log
An example of invocation might be : /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 40 --hard 50 --fs /tmp
Applying the quotas
[root@t3admin01 ~]# cexec ui: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 40 --hard 50 --fs /tmp
[root@t3admin01 ~]# cexec wn: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 40 --hard 50 --fs /tmp
[root@t3admin01 ~]# cexec ui: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 80 --hard 90 --fs /scratch
[root@t3admin01 ~]# cexec wn: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 80 --hard 90 --fs /scratch
/etc/warnquota.conf
This file is managed by the Puppet module
quota-fs-users-in-ldap
, inside you find the warnquota e-mail template and the
LDAP details:
[root@t3ui02 ~]# grep -v \# /etc/warnquota.conf
MAIL_CMD = "/usr/sbin/sendmail -t"
FROM =
SUBJECT = NOTE: You are exceeding your allocated disk space limits
CC_TO = "cms-tier3@lists.psi.ch"
SUPPORT = "cms-tier3@lists.psi.ch"
PHONE = "+41 056 310 36 12"
MESSAGE = Dear T3 User||your disk usage has exceeded the agreed limits\
on this server,|have a look to this page to check the actual T3 usage policies:|https://wiki.chipp.ch/twiki/bin/view/CmsTier3/Tier3Policies||Please delete any unnecessary files on following filesystems:|
SIGNATURE = The T3 Administrators.
GROUP_MESSAGE = Hello, a group '%s' you're member of use too much space.|\
I chose you to do the cleanup.|Delete group files on following filesystems:|
GROUP_SIGNATURE = See you!| Your admin|
LDAP_MAIL = true
LDAP_URI = ldaps://t3ldap01.psi.ch:636
LDAP_BASEDN = dc=cmst3,dc=psi,dc=ch
LDAP_SEARCH_ATTRIBUTE = uid
LDAP_MAIL_ATTRIBUTE = mail
LDAP_DEFAULT_MAIL_DOMAIN = psi.ch
/etc/quotatab
This file is managed by the Puppet module
quota-fs-users-in-ldap
, inside you find the maps between /dev/... and where is mounted that block device, basically:
[root@t3ui02 ~]# grep -v \# /etc/quotatab
/dev/sda7 : The /tmp filesystem
/dev/sda8 : The /scratch filesystem
Pay attention because the UIs have different disk layouts so Puppet will manage that file according to the specific UI.
/etc/sudoers for repquota command
Users need to observe reciprocally their disk usages to delete files or notify someone about an abuse, so they need to run repquota on UIs and WNs with the
root
rights
> =/etc/sudoers
file must to have these lines:
[root@t3wn23 ~]# grep QUOTA /etc/sudoers
Host_Alias QUOTA = t3ui[0-1][0-9],t3wn[0-9][0-9],t3vmui01
%cms QUOTA=(ALL) NOPASSWD: /usr/sbin/repquota -s /tmp, /usr/sbin/repquota -s /scratch,/usr/sbin/repquota -v -s /tmp,/usr/sbin/repquota -v -s /scratch, /usr/sbin/repquota -s -v /tmp,/usr/sbin/repquota -s -v /scratch
Here an example of the
repquota
command with the
/scratch
users quotas enforced:
[root@t3ui02 ~]# repquota -s /scratch/
*** Report for user quotas on device /dev/sda8
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
root -- 4 0 0 3 0 0
cmssgm -- 3692M 183G 206G 78496 0 0
fronga -- 12867M 183G 206G 3680 0 0
arizzi -- 35178M 183G 206G 2502 0 0
thea -- 1804 183G 206G 8453 0 0
pnef -- 0 183G 206G 3 0 0
leo -- 16255M 183G 206G 6 0 0
sdevissc -- 897M 183G 206G 1 0 0
bortigno -- 13641M 183G 206G 40936 0 0
buchmann -- 300M 183G 206G 19 0 0
casal -- 39777M 183G 206G 28 0 0
martinelli_f -- 0 183G 206G 1 0 0
mtakahashi -- 56222M 183G 206G 38638 0 0
peruzzi -- 7826M 183G 206G 33 0 0
nmohr -- 6102M 183G 206G 85 0 0
paktinat -- 0 183G 206G 2 0 0
--
FabioMartinelli - 2012-01-11