<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups # * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup # * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> %TOC% ---+ May-20 * Security updates/measures * set 'nosuid' flag for shared file-systems mount points (/t3home, /work and /pg-backup) * EGI Trust Anchor release 1.105 -> 1.106 * frontier-squid 4.10-1.1 -> 4.11-2.1 * as preventative measure yum/kernel updates on all user facing computers * request to "ssh-key passwdordless" users to add passphrase - completed for all accounts * excluded suid/sgid binaries on UIs/CNs by checks from EGI security * !EOS * addition of /eos subprojects (/eos/cms, etc) on login nodes * update of eos-client 4.5.9 -> 4.7.7 (and eos-xrootd 4.11.2 -> 4.11.3) on UIs * decommissioning of !EOS test partition Slurm (no single usage since ~2.5 months) and return idle CPU to other queues * Grid host cert * !QuoVadis subscription re-registered to cms-tier3-alerts@lists.psi.ch * instruction documented on t3wiki https://wiki.chipp.ch/twiki/bin/view/CmsTier3/GridHostCert * Misc * regular check/cleaning of old ZFS snapshots to release user quota space (when night update script failed to do this automaticaly due to "cannot destroy snapshot ... dataset is busy" error) * setup dcache xrootd movers uniformly upto 1000/pool, works stably (underpinned by 10*2NIC Bonding) * user management: new account for !UniZ student * return temporary t3ui04,07 to batch as t3wn49,50 * re-installation of t3wn48 due to odd (test) partition table and puppet run failure ---+ April-20 * Slurm * memory is configured as consumable resource (default !DefMemPerCPU is 2GB/CPU) to prevent out of memory situations caused by users jobs * added to client nodes LNAG enviromental variables to /etc/locale.conf to shut out LC_CTYPE/UTF-8 errors of ssh-sessions * Monitoring: * added Slurm CPU/GPU metric collection scripts and plots: https://wiki.chipp.ch/twiki/bin/view/CmsTier3/SlurmUtilisation * added Admins monitoring list: https://wiki.chipp.ch/twiki/bin/view/CmsTier3/MonitoringList * Miscellaneous: * CRIC/SRR storage monitoring ticket closed: storage descriptor is configured on t3dcachedb03 * updates of EGI Trust Anchor release 1.105-1 * users question to install phython3/root6 locally not needed, since availble in /cvmfs/sft.cern.ch/lcg/... * migration of puppet filecopy location to common for all t3admins gitlab place * user accounts/data cleaning (jfernan2, thaarres), creating of new !UniZ accounts (sliechti, yverma) ---+ March-20 * dCache Upgrade Follow-ups: * add CMS TFC config to xrootd door on SE node (https://www.dcache.org/downloads/xrootd4j/index.shtml) * implementation of Postgres Backup script to copy DB to t3nfs02:/zfs/data01/swshare/postgres * dcache after upgrade became too verbose and filled out /var/log partition; to fix the problem dcache restart was done on Sun Mar 15 (without user activity) * Storage Cleaning due to almost no free space on dcache: * deletion of leftover user data took several days. Too many (hundred thousands) files in single directories: dcache can't handle it * overal clenup brought ~ 30% free space; next step is needed - check and clean ~150TB of mc, data dirs * Slurm: * add !QoS (500 cpu/user) to quick partition * !EOS test configuration (enabled on Worker Nodes and UIs): since February no user feedback * Monitoring: * manually added non-standard /work server t3nfs02 to ganglia * solved the problem with SELinux (the reason of http access error) on ganglia server; works stably * dcache space monitoring added to t3wiki: https://wiki.chipp.ch/twiki/bin/view/CmsTier3/StoragePlots * all configuration changes saved on hiera/puppet/gitlab * most of this list was done remotely from home with no drop in efficiency in compare to work from PSI office
This topic: CmsTier3
>
WebHome
>
WebPreferences
>
AdminArea2019
>
T3Status20
Topic revision: r4 - 2020-05-28 - NinaLoktionova
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback