Tags:
tag this topic
create new tag
view all tags
<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> ---+ Custom YUM repositories for the Tier3 %TOC% ---++ Introduction PSI runs a customized version of [[http://ait.web.psi.ch/services/linux/][Scientific Linux]] + some local RPMs repositories ( e.g. including IBM GPFS ) ; the T3 uses these PSI repositories but it also has *its own YUM repositories* organized by SL version: * =/afs/psi.ch/software/linux/dist/scientific/57/Tier3/= * =/afs/psi.ch/software/linux/dist/scientific/6/Tier3= M. Gasser commented in this way our request to create our own YUM repositories: <pre> Hi Fabio, Go-to /afs/psi.ch/software/linux/dist/scientific/57/ There, you create a directory such as tier3 (not in the psi subdirectory). We use so called all subdirectories to put all versions of an RPM in. From there we just take the latest ones and put corresponding symlinks to snapshot directories, the date of creation is the name of such a snapshot. I distinguish between unstable, testing and stable, this is quite handy, because when I try new things, I can point unstable to another snapshot as testing or stable. However, it's up to you. To populate your repo you just can copy things or you can use 'repomanage' (package yum-utils, see /afs/psi.ch/software/linux/dist/scientific/57/scripts/create_new_snapshots.sh or the manpage for an example how to use it). When your repo is populated, you have to run the command 'createrepo' (package createrepo), for instance in the directory you have the latest RPMS which you wanna install. The subdirectory repodata is made, where the metadata of your RPMS are kept. On the client you have to edit /etc/yum.conf accordingly, to address this repo, or you add a separate repo config file to /etc/yum.repos.d/ (see the existing yum.conf as an example). Important, each time you change something in your repo you have to run createrepo again, even if you copy the same RPM again to the same place, otherwise you'll get an error on the client. On the client it's often a good idea to 'yum clean all' to clear the yum cache, before running yum install or whatever. So you are sure the repo data on your client corresponds to the latest repo data in your repo. > > Hi Marc > > could you please create, or tell me how to, a directory like: > /afs/psi.ch/software/linux/dist/scientific/57/psi/Tier3 > > Fabio </pre> to do what Marc said we need this RPM: <pre> Installed Packages Name : createrepo Arch : noarch Version : 0.4.11 Release : 3.el5 Size : 216 k Repo : installed Summary : Creates a common metadata repository URL : http://linux.duke.edu/projects/metadata/ License : GPLv2 Description: This utility will generate a common metadata repository from a directory of : rpm packages </pre> ---++ HTTP protection of the YUM repository *Be aware of the =htaccess-special.acl= files*, they prevent Internet access to the PSI =yum= repositories: <pre> [martinelli_f@t3ui02 scientific]$ cat /afs/psi.ch/software/linux/dist/scientific/57/htaccess-special.acl #################################################################### # # PSI SLP REPO ACCESS # #################################################################### Order deny,allow deny from all allow from 129.129 10.1.1.0/24 172.16.0.0/12 192.33.120 192.33.121 192.33.123 192.33.126 192.168.6 192.33.118 192.33.127 [martinelli_f@t3ui02 scientific]$ cat /afs/psi.ch/software/linux/dist/scientific/6/htaccess-special.acl #################################################################### # # PSI SLP REPO ACCESS # #################################################################### Order deny,allow deny from all allow from 129.129 10.1.1.0/24 172.16.0.0/12 192.33.120 192.33.121 192.33.123 192.33.126 192.33.127 192.168.6 192.33.118 195.176.0 </pre> ---++ YUM pointer files *Usually Puppet will take care of this file* ---+++ SL5 On the SL5 servers where you want to deploy our own T3 RPMs it must to be present this YUM pointer: <pre> # https://wiki.chipp.ch/twiki/bin/view/CmsTier3/PSIYumRepositoryForTier3 [Tier3] name=Tier3 repo baseurl=http://linux.web.psi.ch/dist/scientific/57/Tier3/stable enabled=1 gpgcheck=0 # top priority priority=1 </pre> ---+++ SL6 The equivalent =YUM= pointer for =SL6= is: <pre> [root@t3fs13 ~]# cat /etc/yum.repos.d/Tier3.repo # Puppet Managed File # # https://wiki.chipp.ch/twiki/bin/view/CmsTier3/PSIYumRepositoryForTier3 [Tier3] name=Tier3 repo baseurl=http://linux.web.psi.ch/dist/scientific/6/Tier3/stable enabled=1 gpgcheck=0 # top priority priority=1 </pre> ---++ How to add new RPMs into the YUM repository To add a new RPM(s) in our =YUM= repository you have to: * login into a =t3ui*= server * run =klog= to create a Kerbers ticket * copy the new RPM(s) into =/afs/psi.ch/software/linux/dist/scientific/57/Tier3/all= *or* =/afs/psi.ch/software/linux/dist/scientific/6/Tier3= * =cd /afs/psi.ch/software/linux/dist/scientific/57/scripts= *or* cd =/afs/psi.ch/software/linux/dist/scientific/6/scripts= * Create a new snapshot by running: =./create_new_snapshots.sh Tier3= * Update the =testing/stable= symbolic links; probably just =testing= since =stable= points to =testing= *Example:* <pre> [martinelli_f@t3ui03 scripts]$ ./create_new_snapshots.sh Tier3 ### begin ./create_new_snapshots.sh ### Sourcing configuration file /scripts/dist-config Running ./create_new_snapshots.sh, this will take some time ... ALL_DIRS is psi others update.i386 update.x86_64 epelp dagp nagios nonfree cluster fastbugs.i386 fastbugs.x86_64 The following directories will be processed: ### Tier3 Create snapshot for Tier3 ... creating directory /afs/psi.ch/software/linux/dist/scientific/57/Tier3/20111219/ ... ------------------------------------------------------------------------ Using /afs/psi.ch/software/linux/dist/scientific/57/Tier3/all to populate /afs/psi.ch/software/linux/dist/scientific/57/Tier3/20111219 ... (all RPMS will be copied as soft links) ------------------------------------------------------------------------ Find newest rpms, please wait ... done. Create softlinks for each ../all/TARGET.rpm in /afs/psi.ch/software/linux/dist/scientific/57/Tier3/20111219/: ganglia-3.0.7-1.src.rpm ganglia-devel-3.0.7-1.x86_64.rpm ganglia-gmetad-3.0.7-1.x86_64.rpm ganglia-gmond-3.0.7-1.x86_64.rpm ganglia-web-3.0.7-1.noarch.rpm quota-3.17-1.2.5.x86_64.rpm sun-sge-bin-linux24-x64-6.2-5.x86_64.rpm sun-sge-common-6.2-5.noarch.rpm done. Run createrepo in /afs/psi.ch/software/linux/dist/scientific/57/Tier3/20111219/ ... 8/8 - sun-sge-common-6.2-5.noarch.rpm Saving Primary metadata Saving file lists metadata Saving other metadata done. ### end ./create_new_snapshots.sh ### [martinelli_f@t3ui03 scripts]$ </pre> Like showed above, the Marc's =create_new_snapshots.sh= script creates a new repository, in this case =20111219= ; to use it you need to change the =testing/stable= soft link: <pre> [martinelli_f@t3ui02 Tier3]$ pwd /afs/psi.ch/software/linux/dist/scientific/57/Tier3 [martinelli_f@t3ui02 Tier3]$ ll total 10 drwxr-xr-x 3 martinelli_f cms 2048 Dec 14 11:19 20111214 drwxr-xr-x 3 martinelli_f cms 2048 Dec 14 11:54 20111214-1154 drwxr-xr-x 3 martinelli_f cms 2048 Dec 19 17:21 20111219 drwxr-xr-x 3 martinelli_f cms 2048 Dec 20 12:27 all lrwxr-xr-x 1 martinelli_f cms 3 Dec 14 12:01 testing -> 20111214 [martinelli_f@t3ui02 Tier3]$ rm -f testing && ln -s 20111219 testing [martinelli_f@t3ui02 Tier3]$ ll total 10 drwxr-xr-x 3 martinelli_f cms 2048 Dec 14 11:19 20111214 drwxr-xr-x 3 martinelli_f cms 2048 Dec 14 11:54 20111214-1154 drwxr-xr-x 3 martinelli_f cms 2048 Dec 19 17:21 20111219 drwxr-xr-x 3 martinelli_f cms 2048 Dec 20 12:27 all -rw-r--r-- 1 martinelli_f cms 503 Dec 14 12:08 htaccess-intranet.acl lrwxr-xr-x 1 martinelli_f cms 8 Dec 20 12:38 testing -> 20111219 </pre> Basically case by case, and server by server, you have to decide if to use a new soft link or to recycle a past one. Common case is 2 soft links =testing= newer, or the same, than =stable= and the development servers pointing to =testing= while the production servers pointing to =stable=. ---++ How to install an RPM from the YUM repository If the YUM pointer file is properly configured as well as the soft link on the =/afs/= filesystem then you simply execute =yum= like in the following example ( we're installing a custom quota RPM that understands =ldaps= ): <pre> [root@t3vmui01 ~]# yum install quota Loaded plugins: kernel-module Excluding Packages from 54 update Finished Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package quota.x86_64 1:3.17-1.2.5 set to be updated --> Finished Dependency Resolution Beginning Kernel Module Plugin Finished Kernel Module Plugin Dependencies Resolved =========================================================================================================================================================================== Package Arch Version Repository Size =========================================================================================================================================================================== Installing: quota x86_64 1:3.17-1.2.5 Tier3 358 k Transaction Summary =========================================================================================================================================================================== Install 1 Package(s) Update 0 Package(s) Remove 0 Package(s) Total download size: 358 k Is this ok [y/N]: y Downloading Packages: quota-3.17-1.2.5.x86_64.rpm | 358 kB 00:00 Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Installing : quota 1/1 Installed: quota.x86_64 1:3.17-1.2.5 Complete! [root@t3vmui01 ~]# </pre> ---++ How to consolidate the YUM repository *I discourage this step* If you feel confident about the stability of the T3 RPM(s) you can apply the =createrepo= command directly to the =all= subdirectory, but it's usually safer to work by using the nice snapshots mechanisms suggested by Marc.
E
dit
|
A
ttach
|
Watch
|
P
rint version
|
H
istory
: r6
<
r5
<
r4
<
r3
<
r2
|
B
acklinks
|
V
iew topic
|
Ra
w
edit
|
M
ore topic actions
Topic revision: r6 - 2014-06-23
-
FabioMartinelli
CmsTier3
Log In
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
User Pages
Main Page
Policies
Monitoring Storage Space
Monitoring Slurm Usage
Physics Groups
Steering Board Meetings
Admin Pages
AdminArea
Cluster Specs
Home
Site map
CmsTier3 web
LCGTier2 web
PhaseC web
Main web
Sandbox web
TWiki web
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
E
dit
A
ttach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback