Arrow left Go to previous page / next page of Tier3 site log MOVED TO...

26. 07. 2012 Enforcing flexible memory limits on SGE

Up to now we didn't apply memory limits on SGE but because we got 5 servers crashes in the last days now it's time to enforce them; said that I propose to:

  • Assign h_vmem values to each host according to its memory capacity, so for instance 25G to t3wn[10-29] and 50G to t3wn[30-40]. We'll run for each WN:
    • qconf -se WN
    • qconf -aattr exechost complex_values h_vmem=25G WN
    • qconf -se WN
  • Configure the h_vmem complex like a forced consumable with a default value; so switching:
    • its consumable property from NO to YES
    • its requestable property from YES to FORCED
    • its default value from 0 to 3G.
    • Run qconf -sc to see the complexes before and after the change.
  • For each queue Q configure the hard limit h_vmem like 4G, so users can ask more than the default 3G but <= 4G.
    • The limit h_vmem is per server slot and it's an effective method to enforce an upper limit on h_vmem but even better we might program a JVS to reject the unsatisfiable job requests instead to leave them queued forever.
    • Our JSV can be installed like a forced check into /gridware/sge/util/sge_request.
  • We modify the file /etc/security/limits.conf to enforce the memory the limit '@cms as 4500000' like a final security mechanism would be SGE stop to respect its h_vmem deal or becasue of an SGE Admin misconfiguration.

-- FabioMartinelli - 2012-07-26


Arrow left Go to previous page / next page of Tier3 site log MOVED TO...

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2012-07-26 - FabioMartinelli
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback