Tags:
view all tags
<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> %TOC% ---+ Disk quota on /tmp and /scratch ---++ Introduction Please have a look to our [[https://wiki.chipp.ch/twiki/bin/view/CmsTier3/Tier3Policies][T3 usage policies]]. ---++ Puppet module quota-fs-users-in-ldap In a Nutshell: * The Puppet module on AFS =quota-fs-users-in-ldap= mounts automatically with =usrquota= the =/tmp= or the =/scratch= filesystem. * Inside of it there is script =quota-ldapusers-filesystem.sh= to enforce users quotas ( users retrieved from our LDAP ) on a filesystem type =ext3= or =xfs=. * We use a custom RPM =quota-3.17-1.2.5= to make SSL LDAP connections vs our LDAP. The RPM comes from [[PSI Yum repository for Tier3][our own repository]]. Basically you'll run =quota-ldapusers-filesystem.sh= on each UI or WN with =--fs=/tmp= or =--fs=/scratch= when: * A new LDAP user is created, so you need to apply the quota for him/her, the script will do it for everybody so for him/her as well. * To increase or decrease the =--soft= or the =--hard= quota of everyone: then please update the [[https://wiki.chipp.ch/twiki/bin/view/CmsTier3/Tier3Policies][T3 usage policies]]. * You are installing from scratch a UI or a WN, so =/tmp= and =/scratch= are mounted with =usrquota= but any user quota is defined. ---++ Bash script quota-fs-users-in-ldap.sh Here follows a script invocation: <pre> [root@t3ui02 ~]# /root/USERS-QUOTA/quota-ldapusers-filesystem.sh Script to enforce users quotas on a generic filesystem. Users are retrieved from an LDAP server. Here some info for users: https://wiki.chipp.ch/twiki/bin/view/CmsTier3/Tier3Policies Usage: quota-ldapusers-filesystem.sh [-h|--help] --fs filesystem --ldaps ldapserver --quota quota -h, --help Print the help message and exit --ldaps SSL LDAP server listening on port 636 from where to retrieve the users list. --soft Soft Quota in % such that 0 < Soft Quota < Hard Quota < 100. --hard Hard Quota in % such that 0 < Soft Quota < Hard Quota < 100. --fs Users quotas will be applied to this filesystem. So far just ext3 and xfs were tested. The script will saves its logs into: /tmp/quota-ldapusers-filesystem.sh.log An example of invocation might be : /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 40 --hard 50 --fs /tmp </pre> ---+++ Applying the quotas <pre> [root@t3admin01 ~]# cexec ui: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 40 --hard 50 --fs /tmp [root@t3admin01 ~]# cexec wn: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 40 --hard 50 --fs /tmp [root@t3admin01 ~]# cexec ui: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 80 --hard 90 --fs /scratch [root@t3admin01 ~]# cexec wn: /root/USERS-QUOTA/quota-ldapusers-filesystem.sh --ldaps t3ldap01.psi.ch --soft 80 --hard 90 --fs /scratch </pre> ---++ /etc/warnquota.conf This file is managed by the Puppet module =quota-fs-users-in-ldap=, inside you find the warnquota e-mail template and the LDAP details: <pre> [root@t3ui02 ~]# grep -v \# /etc/warnquota.conf MAIL_CMD = "/usr/sbin/sendmail -t" FROM = SUBJECT = NOTE: You are exceeding your allocated disk space limits CC_TO = "cms-tier3@lists.psi.ch" SUPPORT = "cms-tier3@lists.psi.ch" PHONE = "+41 056 310 36 12" MESSAGE = Dear T3 User||your disk usage has exceeded the agreed limits\ on this server,|have a look to this page to check the actual T3 usage policies:|https://wiki.chipp.ch/twiki/bin/view/CmsTier3/Tier3Policies||Please delete any unnecessary files on following filesystems:| SIGNATURE = The T3 Administrators. GROUP_MESSAGE = Hello, a group '%s' you're member of use too much space.|\ I chose you to do the cleanup.|Delete group files on following filesystems:| GROUP_SIGNATURE = See you!| Your admin| LDAP_MAIL = true LDAP_URI = ldaps://t3ldap01.psi.ch:636 LDAP_BASEDN = dc=cmst3,dc=psi,dc=ch LDAP_SEARCH_ATTRIBUTE = uid LDAP_MAIL_ATTRIBUTE = mail LDAP_DEFAULT_MAIL_DOMAIN = psi.ch </pre> ---++ /etc/quotatab This file is managed by the Puppet module =quota-fs-users-in-ldap=, inside you find the maps between /dev/... and where is mounted that block device, basically: <pre> [root@t3ui02 ~]# grep -v \# /etc/quotatab /dev/sda7 : The /tmp filesystem /dev/sda8 : The /scratch filesystem </pre> Pay attention because the UIs have different disk layouts so Puppet will manage that file according to the specific UI. ---++ /etc/sudoers for repquota command Users need to observe reciprocally their disk usages to delete files or notify someone about an abuse, so they need to run repquota on UIs and WNs with the =root= rights => =/etc/sudoers= file must to have these lines: <pre> [root@t3wn23 ~]# grep QUOTA /etc/sudoers Host_Alias QUOTA = t3ui[0-1][0-9],t3wn[0-9][0-9],t3vmui01 %cms QUOTA=(ALL) NOPASSWD: /usr/sbin/repquota -s /tmp, /usr/sbin/repquota -s /scratch,/usr/sbin/repquota -v -s /tmp,/usr/sbin/repquota -v -s /scratch, /usr/sbin/repquota -s -v /tmp,/usr/sbin/repquota -s -v /scratch </pre> Here an example of the =repquota= command with the =/scratch= users quotas enforced: <pre> [root@t3ui02 ~]# repquota -s /scratch/ *** Report for user quotas on device /dev/sda8 Block grace time: 7days; Inode grace time: 7days Block limits File limits User used soft hard grace used soft hard grace ---------------------------------------------------------------------- root -- 4 0 0 3 0 0 cmssgm -- 3692M 183G 206G 78496 0 0 fronga -- 12867M 183G 206G 3680 0 0 arizzi -- 35178M 183G 206G 2502 0 0 thea -- 1804 183G 206G 8453 0 0 pnef -- 0 183G 206G 3 0 0 leo -- 16255M 183G 206G 6 0 0 sdevissc -- 897M 183G 206G 1 0 0 bortigno -- 13641M 183G 206G 40936 0 0 buchmann -- 300M 183G 206G 19 0 0 casal -- 39777M 183G 206G 28 0 0 martinelli_f -- 0 183G 206G 1 0 0 mtakahashi -- 56222M 183G 206G 38638 0 0 peruzzi -- 7826M 183G 206G 33 0 0 nmohr -- 6102M 183G 206G 85 0 0 paktinat -- 0 183G 206G 2 0 0 </pre> -- Main.FabioMartinelli - 2012-01-11
Edit
|
Attach
|
Watch
|
P
rint version
|
H
istory
:
r3
<
r2
<
r1
|
B
acklinks
|
V
iew topic
|
Raw edit
|
More topic actions...
Topic revision: r2 - 2012-01-12
-
FabioMartinelli
CmsTier3
Log In
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
User Pages
Main Page
Policies
Monitoring Storage Space
Monitoring Slurm Usage
Physics Groups
Steering Board Meetings
Admin Pages
AdminArea
Cluster Specs
Home
Site map
CmsTier3 web
LCGTier2 web
PhaseC web
Main web
Sandbox web
TWiki web
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
Edit
Attach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback