<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> ---+ SGE 6.1 Interactive Queue on t3ce01 --- %TOC% --- It's useful introduce an [[http://wikis.sun.com/display/GridEngine/Submitting+Interactive+Jobs][interactive queue]] in the t3ce01 SGE configuration for 2 main purposes; * to allow users to develop SW exploiting the WN computational power. * to inspect the /scratch dir hosted in each WN during and after a job execution. To achieve those we need to modify the configuration both CE and WN side and exploit the [[http://osdir.com/ml/clustering.gridengine.users/2007-11/msg00394.html][SGE-SSH integration]]; basically, for the "develop SW case", in response of an interactive queue request the less loaded WN is selected and an SSHd daemon is started on a TCP port ( not 22 ), later the CE open an SSH connection vs the couple ( WN, TCP port ); for the "inspect /scratch case" the user can point directly to the WN where his computation is running or where it ran; so first prepare an executable script called qlogin.sh: <pre> [root@t3ce01 n1ge6]# cat /swshare/sge/n1ge6/bin/lx24-amd64/qlogin.sh #!/bin/sh HOST=$1 PORT=$2 /usr/bin/ssh -XY -p $PORT $HOST [root@t3ce01 n1ge6]# </pre> ---++ Interactive queue Be sure that your interactive queue is really declared INTERACTIVE like showed here: <pre> [root@t3ce01 n1ge6]# qconf -sq interactive qname interactive ... qtype INTERACTIVE ... </pre> ---++ qconf tuning and modify the global SGE configuration to respect the 2 qlogin lines reported below: <pre> [root@t3ce01 n1ge6]# qconf -sconf global: execd_spool_dir /var/spool/sge mailer /bin/mail xterm /usr/bin/X11/xterm load_sensor none prolog none epilog none shell_start_mode posix_compliant login_shells sh,ksh,csh,tcsh min_uid 0 min_gid 0 user_lists none xuser_lists none projects none xprojects none enforce_project false enforce_user auto load_report_time 00:00:40 max_unheard 00:05:00 reschedule_unknown 00:00:00 loglevel log_warning administrator_mail none set_token_cmd none pag_cmd none token_extend_time none shepherd_cmd none qmaster_params none execd_params none reporting_params accounting=true reporting=true \ flush_time=00:00:15 joblog=true sharelog=00:00:00 finished_jobs 100 gid_range 50700-50800 qlogin_command /swshare/sge/n1ge6/bin/lx24-amd64/qlogin.sh qlogin_daemon /usr/sbin/sshd -f /etc/ssh/sshd_config_sge -i rlogin_daemon /usr/sbin/in.rlogind max_aj_instances 2000 max_aj_tasks 75000 max_u_jobs 0 max_jobs 0 auto_user_oticket 0 auto_user_fshare 100 auto_user_default_project none auto_user_delete_time 86400 delegated_file_staging false reprioritize 0 [root@t3ce01 n1ge6]# </pre> ---++ Specific sshd_config for SGE - WN side Instead to use the global file /etc/ssh/sshd_config it's worth to use a different file to specify a different Syslog facility and distinguish between administrative SSH login on TCP port 22 vs SGE SSH login, here we selected LOCAL5: <pre> [root@t3wn19 ~]# cat /etc/ssh/sshd_config_sge # /etc/ssh/sshd_config # The default configuration provided by the ssh module. Protocol 2 SyslogFacility LOCAL5 PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials yes UsePAM yes X11Forwarding yes Subsystem sftp /usr/libexec/openssh/sftp-server [root@t3wn19 ~]# </pre> Later configure syslogd to send LOCAL5 to a file like /var/log/secure-sge ---++ qlogin session Now try a qlogin session: <pre> [martinelli_f@t3ce01 ~]$ qlogin -q interactive local configuration t3ce01.psi.ch not defined - using global configuration Your job 684534 ("QLOGIN") has been submitted waiting for interactive job to be scheduled ... Your interactive job 684534 has been successfully scheduled. Establishing /swshare/sge/n1ge6/bin/lx24-amd64/qlogin.sh session to host t3wn19.psi.ch ... martinelli_f@t3wn19.psi.ch's password: </pre> WN side you can see the SSHd daemon started by SGE: <pre> [root@t3wn19 ~]# ps fax | grep -A 2 -B 2 sshd ... 16238 ? S 0:00 \_ sge_shepherd-684534 -bg 16239 ? Ss 0:00 \_ sshd: martinelli_f [priv] 16240 ? S 0:00 \_ sshd: martinelli_f [net] </pre> That SSHd process is not listening on any TCP port after the connection, this improves the WN security. ---++ Firewall on the WN + Hostbased Authentication ??? Hence users are allowed to login by SSH into the WN but there is also a process SSHd listening on the port 22, so a user could login by SSH outside the SGE control; to prevent that we can setup an iptables on the WN ( or hosts.deny ?? ) to allow a SYN on TCP 22 just from well known hosts like t3admin01 and Adminstrators laptops. Also introducing an other password during a qlogin session hurts more than one user, we can setup an SSH Hostbased Authentication UI => WN that's going to be respected just during the qlogin request because its dynamic SSHd management. -- Main.FabioMartinelli - 2011-03-18
This topic: CmsTier3
>
WebHome
>
AdminArea
>
SGE61InteractiveQueueOnT3ce01
Topic revision: r2 - 2011-03-20 - FabioMartinelli
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback