IssueNxSessionStartupFailure < CmsTier3

Tags: view all tags

<!-- keep this as a security measure:
   #uncomment if the subject should only be modifiable by the listed groups 
   # * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup
   # * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup
   #uncomment this if you want the page only be viewable by the listed groups
   # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup
-->

---+!! %TOPIC%

%TOC%

---++ Symptoms
Summary: %FORMFIELD{"Symptom summary"}%

---++ Occurrences
At what times did this problem occur (used to estimate frequency):

| 2011-12-09 |

---++ Observations
<!--
   #collect here the information which may help to better understand the state of the system or services, e.g.
   #log excerpts, strace output, etc.
   #this also may help to identify the problem if similar conditions arise again
-->

<pre>
NX> 707 SSL tunneling: 1
/usr/bin/nxserver: line 1283: 10560 Terminated              sleep $AGENT_STARTUP_TIMEOUT
NX> 105 NX> 596 Session startup failed.
NX> 1004 Error: NX Agent exited with exit status 1. To troubleshoot set SESSION_LOG_CLEAN=0 in node.conf and investigate "/shome/feichtinger/.nx/F-C-t3ui04-1000-4CF830B0D9497CF30EAA80B396801B43/session". You might also want to try: ssh -X myserver; /usr/bin/nxnode --agent to test the basic functionality. Session log follows:
Can't open /var/lib/nxserver/db/running/sessionId{4CF830B0D9497CF30EAA80B396801B43}: No such file or directory.
mv: cannot stat `/var/lib/nxserver/db/running/sessionId{4CF830B0D9497CF30EAA80B396801B43}': No such file or directory
NX> 1006 Session status: closed
NX> 1001 Bye.
</pre>

Activating SESSION_LOG_CLEAN=0 in the =node.conf= file on this server node results in the session directory surviving in the client environment.

e.g. in /shome/${USER}/.nx/F-C-t3ui04-1000-3B202252EDD84AF476FFC5881A216871

<pre>
cat session

Error: Aborting session with 'Cannot establish any listening sockets - Make sure an X server isn't already running'.
Session: Aborting session at 'Fri Dec  9 11:53:23 2011'.
Session: Session aborted at 'Fri Dec  9 11:53:23 2011'.
</pre>

---++ Solution or Workaround

It turned out that there were some leftover X lock files on the server. These seem to have caused the problems. After removing all of these files, it was possible again to start up graphical NX sessions.
<pre>
ls -l /tmp/.X1*
-r--r--r-- 1 leo  cms    11 Nov 29 14:05 /tmp/.X1001-lock

/tmp/.X11-unix:
total 0
srwxrwxrwx 1 leo cms 0 Nov 21 22:30 X1000
srwxrwxrwx 1 leo cms 0 Nov 29 14:05 X1001
</pre>

NOTE: Usually these lock files get cleared after the session is dissociated. Usually the sessions start at X000. Maybe the problem resulted because based on /tmp/.X1001-lock, the NX server assumed X1000 to be free, but when looking in /tmp/.X11-unix, it found already a file /tmp/.X11-unix/X1000 present.


---++ Monitoring for this condition
<!--
   #how can this condition be recognized automatically, if at all?
-->


-- Main.DerekFeichtinger - 2011-12-09

IssueForm
Affected Service	NX
Symptom summary	after connection is establiished NX fails to start up X
Reason Understood	yes
Solution Exists	workaround
Obsolete	no

Topic revision: r1 - 2011-12-09 - DerekFeichtinger

CmsTier3

User Pages
Main Page
Policies

Physics Groups
Steering Board Meetings

Admin Pages
AdminArea
Cluster Specs