(r1) T3News201304291448 < CmsTier3

Tags: view all tags

<!-- keep this as a security measure:
   #uncomment if the subject should only be modifiable by the listed groups 
   * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup
   * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup
   #uncomment this if you want the page only be viewable by the listed groups
   # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup
-->

*CMS Tier-3 News Item*

<!--
  # Use the attached form to define title and summary of this news item. The details you can fill in directly on this wiki page.
-->

---+ %FORMFIELD{"Title"}%

---++ Summary
%FORMFIELD{"Summary"}%

---++ Details
A new and powerful VMWare cluster has been installed at PSI with 2013 HW plus a recent VMWare installation; 

as a consequence of this change our T3 VMs have to be migrated from the current VMWare cluster to the new environment.<br>

In terms of operations and services impact this will mean:
   * Around 9:30 am I'll send a 1st e-mail announcing the beginning of this downtime.
   * I'll stop and migrate the VMs: =t3ce02,t3se01,t3dcachedb03,t3frontier01 ( our local CMS Fontier ),t3cmsvobox02 ( our local Phedex )=
   * Servers =t3ui0[1-9]*= will be left ON all the time but during the downtime their =/pnfs/psi.ch/cms= mount point will be not accessible for a while.
   * Users can submit jobs before the downtime but during the downtime I'll switch in =disabled= mode the batch queues and %RED%I'll kill all the running jobs%ENDCOLOR%
   * Following the migration I'll enable the batch queues and the pending jobs will start to run as usual.  
   * It will follow a 2nd and final e-mail announcing the end of this downtime. 
 
_Hint_: <br>
If you want to preserve your jobs from being killed then you can put in "hold" your pending jobs by running the command [[http://arc.liv.ac.uk/SGE/htmlman/htmlman1/qhold.html][qhold]] and after the downtime you can enable again your jobs by running the command [[http://arc.liv.ac.uk/SGE/htmlman/htmlman1/qrls.html][qrls]]; 
ideally you should try to get in the state where all your pending jobs are "in hold" and there are not running jobs, so the T3 downtime will be transparent for you; 
your CMS proxy hasn't to expire during the downtime. 
   

  

-----------------

%FORMFIELD{"User"}% %FORMFIELD{"Date"}%

T3NewsForm
Title	T3 Scheduled Downtime on Monday 13th May ~9:30 am
Summary	We need to stop for ~ `2h` several VMs that have to be migrated into the new PSI VMware cluster.
User	FabioMartinelli
Date	29. 04. 2013