How to access, set up, and test your account
Preliminary steps
All the documentation is maintained in the T3 twiki pages:
https://wiki.chipp.ch/twiki/bin/view/CmsTier3/WebHome . Please take a look at it to explore T3's capabilities.
Information about T3 mailing-lists:
- Subscribe to the cms-tier3-users list using this web interface (list archives):
cms-tier3-users@lists.psi.ch
. This mailing-list is used to communicate information on Tier-3 matters (downtimes, outages, news, etc.) or for discussions among users and admins.
- To privately contact the CMS Tier-3 administrators, write to:
cms-tier3@lists.psi.ch
- NOTE: Both lists are read by the administrators and are archived. You can submit support requests to either of them. Mails to the user list have the added advantage that they can be read by everyone.
Please also take a look at our policies about usages, quotas, etc..
Tier3Policies.
T3 Linux Groups
All the T3 users are partitioned in several primary groups and all the them belong also to the secondary group
cms that's used for common files like the CMS files uploaded at T3 by
PhEDEx:
ETHZ |
UniZ |
PSI |
ethz-ecal |
uniz-higgs |
psi-bphys |
ethz-bphys |
uniz-pixel |
psi-pixel |
ethz-ewk |
uniz-bphys |
|
ethz-higgs |
|
|
ethz-susy |
|
|
The T3 Linux Groups:
- allow a faster and simpler understanding of the
/pnfs
space usage by leveraging on the gid
setting while in the past having all the files assigned to the global group cms
was making this accounting harder.
- allow to make accounting in the Sun Grid Engine batch system also according to the jobs
gid
setting.
- allowed the creation of a set of dedicated "group dirs"
/pnfs/psi.ch/cms/trivcat/store/t3groups/
where just the group members can write or delete.
- also simplify the
/shome
, /pnfs
, /scratch
or /tmp
file/dirs space usage accounting.
Following an ETHZ user with his primary and his secondary group:
$ id auser
uid=571(auser) gid=532(ethz-higgs) groups=532(ethz-higgs),500(cms)
Following the users dirs
/pnfs/psi.ch/cms/trivcat/store/user/
:
$ ls -l /pnfs/psi.ch/cms/trivcat/store/user | grep -v cms
total 56
drwxr-xr-x 2 alschmid uniz-bphys 512 Feb 21 2013 alschmid
drwxr-xr-x 5 amarini ethz-ewk 512 Nov 7 15:37 amarini
drwxr-xr-x 2 arizzi ethz-bphys 512 Sep 16 17:49 arizzi
drwxr-xr-x 5 bean psi-bphys 512 Aug 24 2010 bean
drwxr-xr-x 5 bianchi ethz-higgs 512 Sep 9 09:40 bianchi
drwxr-xr-x 98 buchmann ethz-susy 512 Nov 5 20:36 buchmann
...
Following the dedicated "group dirs"
/pnfs/psi.ch/cms/trivcat/store/t3groups/
; since they're "group dirs" they belong to
root
but the group has the
w
permission:
$ ls -l /pnfs/psi.ch/cms/trivcat/store/t3groups/
total 5
drwxrwxr-x 2 root ethz-bphys 512 Nov 8 15:18 ethz-bphys
drwxrwxr-x 2 root ethz-ecal 512 Nov 8 15:18 ethz-ecal
drwxrwxr-x 2 root ethz-ewk 512 Nov 8 15:18 ethz-ewk
drwxrwxr-x 2 root ethz-higgs 512 Nov 8 15:18 ethz-higgs
drwxrwxr-x 2 root ethz-susy 512 Nov 8 15:18 ethz-susy
drwxrwxr-x 2 root psi-bphys 512 Nov 8 15:18 psi-bphys
drwxrwxr-x 2 root psi-pixel 512 Nov 8 15:18 psi-pixel
drwxrwxr-x 2 root uniz-bphys 512 Nov 8 15:18 uniz-bphys
drwxrwxr-x 2 root uniz-higgs 512 Nov 8 15:18 uniz-higgs
drwxrwxr-x 2 root uniz-pixel 512 Nov 8 15:18 uniz-pixel
Basic setup
We offer the following user interfaces ( UIs ) assigned according to the following rules; generally speaking the very old
SL5 t3ui0*
servers will be stopped during 2014/2015 so your tasks should be executed on the
SL6 t3ui1*
servers.
Access to Login nodes is based on the institution
The access is not restricted to allow for some freedom, but you are requested to use the UI dedicated to your institution.
UI Login node |
for institution |
HW specs |
t3ui01.psi.ch |
ETHZ, PSI |
132GB RAM , 72 CPUs core (HT), 5TB /scratch |
t3ui02.psi.ch |
All |
132GB RAM , 72 CPUs core (HT), 5TB /scratch |
t3ui03.psi.ch |
UNIZ |
132GB RAM , 72 CPUs core (HT), 5TB /scratch |
- Try to log into a
t3ui*
machine ; -Y
or -X
flag for working with X applications; you can also try to connect by NX client, which allows to work efficiently with graphical applications
ssh -Y username@t3ui02.psi.ch
- If you are an external user and you don't have a standard PSI account, you'll have to change your initial password the first time you log in; use the standard
passwd
utility.
- Copy your grid credentials to the standard places, i.e. to
~/.globus/userkey.pem
and ~/.globus/usercert.pem
and make sure that both files permissions are set correctly:
-rw-r--r-- 1 feichtinger cms 2961 Mar 17 2008 usercert.pem
-r-------- 1 feichtinger cms 1917 Mar 17 2008 userkey.pem
For details about how to extract those files from your CERN User Grid-Certificate please read https://gridca.cern.ch/gridca/Help/?kbid=024010.
- source the grid environment associated to your login shell:
source /swshare/psit3/etc/profile.d/cms_ui_env.sh # for bash
source /swshare/psit3/etc/profile.d/cms_ui_env.csh # for tcsh
- You have to complete the CMS "Virtual Organization" subscription or the following command
voms-proxy-init -voms cms
won't work. CERN details about that, e.g. who is your representative.
- Try to create a proxy certificate for CMS
voms-proxy-init -voms cms
If this fails, run the command with an additional -debug
flag, and the error message will usually be sufficient for us to point out the problem.
- Test your access to the PSI Storage element with the
test-dCacheProtocols
command. You should see output like this (no failed tests):
$ test-dCacheProtocols -n t3se01.psi.ch -p /pnfs/psi.ch/cms/testing -i "GSIDCAP-write SRMv1-adv-del SRMv1-adv-del1 SRMv1-write SRMv1-get-meta SRMv1-read SRMv1-adv-del2"
Test directory: /tmp/dcachetest-20131118-1202-1970
TEST: GSIDCAP-write ...... [IGNORE]
TEST: SRMv1-adv-del ...... [IGNORE]
TEST: GFTP-write ...... [OK]
TEST: GFTP-ls ...... [OK]
TEST: GFTP-read ...... [OK]
TEST: DCAP-read ...... [OK]
TEST: SRMv1-adv-del1 ...... [IGNORE]
TEST: SRMv1-write ...... [IGNORE]
TEST: SRMv1-get-meta ...... [IGNORE]
TEST: SRMv1-read ...... [IGNORE]
TEST: SRMv1-adv-del2 ...... [IGNORE]
TEST: SRMv2-write ...... [OK]
TEST: SRMv2-ls ...... [OK]
TEST: SRMv2-read ...... [OK]
TEST: SRMv2-rm ...... [OK]
- Be aware of the CSCS CMS User Page
- The
test-dCacheProtocols
tool can also be used to test a remote element access (use the -h
flag to get more info about it): e.g. to test CSCS:
$ test-dCacheProtocols -n storage01.lcg.cscs.ch -p /pnfs/lcg.cscs.ch/cms/testing -i "GSIDCAP-write DCAP-read SRMv1-adv-del SRMv1-adv-del1 SRMv1-write SRMv1-get-meta SRMv1-read SRMv1-adv-del2"
Test directory: /tmp/dcachetest-20131118-1201-1784
TEST: GSIDCAP-write ...... [IGNORE]
TEST: SRMv1-adv-del ...... [IGNORE]
TEST: GFTP-write ...... [OK]
TEST: GFTP-ls ...... [OK]
TEST: GFTP-read ...... [OK]
TEST: DCAP-read ...... [IGNORE]
TEST: SRMv1-adv-del1 ...... [IGNORE]
TEST: SRMv1-write ...... [IGNORE]
TEST: SRMv1-get-meta ...... [IGNORE]
TEST: SRMv1-read ...... [IGNORE]
TEST: SRMv1-adv-del2 ...... [IGNORE]
TEST: SRMv2-write ...... [OK]
TEST: SRMv2-ls ...... [OK]
TEST: SRMv2-read ...... [OK]
TEST: SRMv2-rm ...... [OK]
Changing your account details
It's possible to get changed your login shell, e.g. from
bash
to
tcsh
, your group, also your account name ; often users requested to change their
Grid cert subject:
, e.g. because they were moving from a country to an other where they got a new certificate.
AFS CERN Ticket
You should use the following sequence of commands at T3:
kinit ${Your_CERN_Username}@CERN.CH
aklog cern.ch
The first command gets you a kerberos ticket, the second command uses that ticket to obtain an authentication token from CERN's AFS service
t3ui* SSH pub keys
Hackers on Internet are constantly waiting for a user mistake, even a mispelled
letter like in this example:
$ ssh t3ui02.psi.sh
The authenticity of host 't3ui02.psi.sh (62.210.217.195)' can't be established.
RSA key fingerprint is c0:c5:af:36:4b:2d:1f:88:0d:f3:9c:08:cc:87:df:42.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 't3ui02.psi.sh,62.210.217.195' (RSA) to the list of known hosts.
at3user@t3ui02.psi.sh's password:
We can't prevent a T3 user from confusing .ch
with a .sh
so pay attention to these cases ! you can also simply define these aliases:
More... Close
$ grep alias ~/.bash_profile | grep t3ui
alias ui2='ssh -X at3user@t3ui02.psi.ch'
alias ui3='ssh -X at3user@t3ui03.psi.ch'
alias ui4='ssh -X at3user@t3ui04.psi.ch'
alias ui5='ssh -X at3user@t3ui05.psi.ch'
alias ui6='ssh -X at3user@t3ui06.psi.ch'
alias ui7='ssh -X at3user@t3ui07.psi.ch'
alias ui8='ssh -X at3user@t3ui08.psi.ch'
alias ui9='ssh -X at3user@t3ui09.psi.ch'
alias ui12='ssh -X at3user@t3ui12.psi.ch'
alias ui15='ssh -X at3user@t3ui15.psi.ch'
alias ui16='ssh -X at3user@t3ui16.psi.ch'
alias ui17='ssh -X at3user@t3ui17.psi.ch'
alias ui18='ssh -X at3user@t3ui18.psi.ch'
alias ui19='ssh -X at3user@t3ui19.psi.ch'
More subdole attacks are the
SSH man in the middle attacks ; to discover them you have to register in
/$HOME/.ssh/known_hosts
each
t3ui*
SSH RSA public key by running these steps on each laptop/desktop/server ( also
lxplus
) that you'll use to connect at T3:
cp -p /$HOME/.ssh/known_hosts /$HOME/.ssh/known_hosts.`date +"%d-%m-%Y"`
mkdir /tmp/t3ssh/
for X in 19 18 17 16 15 12 09 08 07 06 05 04 03 02 ; do TMPFILE=`mktemp /tmp/t3ssh/XXXXXX` && ssh-keyscan -t rsa t3ui$X.psi.ch,t3ui$X,`host t3ui$X.psi.ch| awk '{ print $4}'` | cat - /$HOME/.ssh/known_hosts | grep -v 'psi\.sh' > $TMPFILE && mv $TMPFILE /$HOME/.ssh/known_hosts ; done
rm -rf /tmp/t3ssh
for X in 02 03 04 05 06 07 08 09 12 15 16 17 18 19 ; do echo -n "# entries for t3ui$X = " ; grep -c t3ui$X /$HOME/.ssh/known_hosts ; grep -Hn --color t3ui$X /$HOME/.ssh/known_hosts ; echo ; done
echo done
last
for
will report if there are duplicated rows in
/$HOME/.ssh/known_hosts
for a
t3ui*
server ; if there are you're suppose to preserve the correct occurrence and delete the others ; to delete you can use
sed -i
or
vim
; once you'll get just one row per
t3ui
server run this command and carefully compare your output with this output:
More... Close
$ ssh-keygen -l -f /$HOME/.ssh/known_hosts | grep t3ui
2048 |
6a:b6:0c:dc:1c:44:3b:4f:e8:da:f5:3c:c6:05:ef:00 |
t3ui02.psi.ch,t3ui02,192.33.123.29 (RSA) |
2048 |
a5:d1:41:e9:16:d7:42:90:14:ae:48:14:59:f5:a1:12 |
t3ui03.psi.ch,t3ui03,192.33.123.85 (RSA) |
2048 |
34:c7:fe:09:5c:2c:0b:51:e1:e7:1d:04:93:c6:c3:08 |
t3ui04.psi.ch,t3ui04,192.33.123.86 (RSA) |
2048 |
7d:78:03:9c:da:e3:ce:83:a4:05:95:84:74:3e:9f:e0 |
t3ui05.psi.ch,t3ui05,192.33.123.82 (RSA) |
2048 |
ec:f7:8d:f0:64:21:0d:d0:82:40:d4:c0:f1:36:90:99 |
t3ui06.psi.ch,t3ui06,192.33.123.83 (RSA) |
2048 |
e3:a4:ce:68:a2:6b:4d:cd:88:6b:ec:94:16:eb:6b:e6 |
t3ui07.psi.ch,t3ui07,192.33.123.84 (RSA) |
2048 |
9e:96:7c:5a:d3:63:96:b2:47:a7:8c:fd:ff:ab:be:2d |
t3ui08.psi.ch,t3ui08,192.33.123.61 (RSA) |
2048 |
cc:62:86:5e:28:b2:f6:50:7e:d7:66:40:a7:9b:a9:f7 |
t3ui09.psi.ch,t3ui09,192.33.123.62 (RSA) |
2048 |
d0:9c:a0:e9:8f:9c:3f:b2:f1:88:6c:15:32:07:fc:a0 |
t3ui12.psi.ch,t3ui12,192.33.123.132 (RSA) |
2048 |
77:1b:27:5e:c8:74:64:86:f8:50:f6:58:e6:6f:41:65 |
t3ui15.psi.ch,t3ui15,192.33.123.135 (RSA) |
2048 |
35:bb:d6:be:64:86:8d:db:1d:57:43:ef:05:39:72:c8 |
t3ui16.psi.ch,t3ui16,192.33.123.136 (RSA) |
2048 |
27:d1:57:f0:ac:da:1d:db:54:11:5c:46:4d:93:63:59 |
t3ui17.psi.ch,t3ui17,192.33.123.137 (RSA) |
2048 |
b1:56:06:5b:d3:da:1a:79:60:e9:02:16:be:82:fe:f7 |
t3ui18.psi.ch,t3ui18,192.33.123.138 (RSA) |
2048 |
73:fe:97:b2:e7:54:df:99:50:dc:19:3d:6f:cd:01:11 |
t3ui19.psi.ch,t3ui19,192.33.123.139 (RSA) |
eventually force your ssh client to always check if the server you're connecting to is already present in
/$HOME/.ssh/known_hosts
and request your consensus for servers that are absent by adding this line in
/$HOME/.ssh/config
:
StrictHostKeychecking ask
your
/$HOME/.ssh/config
can be more complex than just that line, study the
ssh_config man page or contact us; ideally you should put
StrictHostKeychecking yes
but in real life it's impractical.
now your ssh client will be able to discover the
SSH man in the middle attacks and if so it will report :
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
we commit to preserve the
t3ui
SSH RSA public and private keys even when we'll completely reinstall a
t3ui
server, so the case
It is also possible that the RSA host key has just been changed
will never be true.