Tags:
tag this topic
create new tag
view all tags
<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> ---+ SGE 6.1 Interactive Queue on t3ce01 --- %TOC% --- It's useful introduce an [[http://wikis.sun.com/display/GridEngine/Submitting+Interactive+Jobs][interactive queue]] in the t3ce01 SGE configuration for 2 main purposes; * to allow users to develop SW exploiting the WN computational power. * to inspect the /scratch dir hosted in each WN during and after a job execution. To achieve those we need to modify the configuration both CE and WN side and exploit the [[http://osdir.com/ml/clustering.gridengine.users/2007-11/msg00394.html][SGE-SSH integration]]; basically, for the "develop SW case", in response of an interactive queue request the less loaded WN is selected and an SSHd daemon is started on a TCP port ( not 22 ), later the CE open an SSH connection vs the couple ( WN, TCP port ); for the "inspect /scratch case" the user can point directly to the WN where his computation is running or where it ran; so first prepare an executable script called qlogin.sh: <pre> [root@t3ce01 n1ge6]# cat /swshare/sge/n1ge6/bin/lx24-amd64/qlogin.sh #!/bin/sh HOST=$1 PORT=$2 /usr/bin/ssh -XY -p $PORT $HOST [root@t3ce01 n1ge6]# </pre> ---++ Interactive queue Be sure that your interactive queue is really declared INTERACTIVE like showed here: <pre> [root@t3ce01 n1ge6]# qconf -sq interactive qname interactive ... qtype INTERACTIVE ... </pre> ---++ qconf tuning and modify the global SGE configuration to respect the 2 qlogin lines reported below: <pre> [root@t3ce01 n1ge6]# qconf -sconf global: execd_spool_dir /var/spool/sge mailer /bin/mail xterm /usr/bin/X11/xterm load_sensor none prolog none epilog none shell_start_mode posix_compliant login_shells sh,ksh,csh,tcsh min_uid 0 min_gid 0 user_lists none xuser_lists none projects none xprojects none enforce_project false enforce_user auto load_report_time 00:00:40 max_unheard 00:05:00 reschedule_unknown 00:00:00 loglevel log_warning administrator_mail none set_token_cmd none pag_cmd none token_extend_time none shepherd_cmd none qmaster_params none execd_params none reporting_params accounting=true reporting=true \ flush_time=00:00:15 joblog=true sharelog=00:00:00 finished_jobs 100 gid_range 50700-50800 qlogin_command /swshare/sge/n1ge6/bin/lx24-amd64/qlogin.sh qlogin_daemon /usr/sbin/sshd -f /etc/ssh/sshd_config_sge -i rlogin_daemon /usr/sbin/in.rlogind max_aj_instances 2000 max_aj_tasks 75000 max_u_jobs 0 max_jobs 0 auto_user_oticket 0 auto_user_fshare 100 auto_user_default_project none auto_user_delete_time 86400 delegated_file_staging false reprioritize 0 [root@t3ce01 n1ge6]# </pre> ---++ Specific sshd_config for SGE - WN side Instead to use the global file /etc/ssh/sshd_config it's worth to use a different file to specify a different Syslog facility and distinguish between administrative SSH login on TCP port 22 vs SGE SSH login, here we selected LOCAL5: <pre> [root@t3wn19 ~]# cat /etc/ssh/sshd_config_sge # /etc/ssh/sshd_config # The default configuration provided by the ssh module. Protocol 2 SyslogFacility LOCAL5 PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials yes UsePAM yes X11Forwarding yes Subsystem sftp /usr/libexec/openssh/sftp-server [root@t3wn19 ~]# </pre> Later configure syslogd to send LOCAL5 to a file like /var/log/secure-sge ---++ qlogin session Now try a qlogin session: <pre> [martinelli_f@t3ce01 ~]$ qlogin -q interactive local configuration t3ce01.psi.ch not defined - using global configuration Your job 684534 ("QLOGIN") has been submitted waiting for interactive job to be scheduled ... Your interactive job 684534 has been successfully scheduled. Establishing /swshare/sge/n1ge6/bin/lx24-amd64/qlogin.sh session to host t3wn19.psi.ch ... martinelli_f@t3wn19.psi.ch's password: </pre> WN side you can see the SSHd daemon started by SGE: <pre> [root@t3wn19 ~]# ps fax | grep -A 2 -B 2 sshd ... 16238 ? S 0:00 \_ sge_shepherd-684534 -bg 16239 ? Ss 0:00 \_ sshd: martinelli_f [priv] 16240 ? S 0:00 \_ sshd: martinelli_f [net] </pre> That SSHd process is not listening on any TCP port after the connection, this improves the WN security. ---++ Firewall on the WN + Hostbased Authentication ??? Hence users are allowed to login by SSH into the WN but there is also a process SSHd listening on the port 22, so a user could login by SSH outside the SGE control; to prevent that we can setup an iptables on the WN ( or hosts.deny ?? ) to allow a SYN on TCP 22 just from well known hosts like t3admin01 and Adminstrators laptops. Also introducing an other password during a qlogin session hurts more than one user, we can setup an SSH Hostbased Authentication UI => WN that's going to be respected just during the qlogin request because its dynamic SSHd management. -- Main.FabioMartinelli - 2011-03-18
E
dit
|
A
ttach
|
Watch
|
P
rint version
|
H
istory
: r2
<
r1
|
B
acklinks
|
V
iew topic
|
Ra
w
edit
|
M
ore topic actions
Topic revision: r2 - 2011-03-20
-
FabioMartinelli
CmsTier3
Log In
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
User Pages
Main Page
Policies
Monitoring Storage Space
Monitoring Slurm Usage
Physics Groups
Steering Board Meetings
Admin Pages
AdminArea
Cluster Specs
Home
Site map
CmsTier3 web
LCGTier2 web
PhaseC web
Main web
Sandbox web
TWiki web
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
E
dit
A
ttach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback