Tags:
tag this topic
create new tag
view all tags
<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups # * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup # * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> ---+ !!Pool Migration %TOC% ---++ External documentation * dcache book: http://www.dcache.org/manuals/Book/cookbook/cb-pool-migration.shtml ---++ Example for copying pool t3fs05_cms to t3fs07_cms using the 'migration copy' command ---+++ Set source pool to read only The dcache book documentation describes setting the pool to readonly by using the =pool disable -rdonly= command of the pool cell. I noticed that this led to a whole number of stuck transfers, but was not able to analyze the situation adequately. However, one can also set a pool to readonly inside the !PoolManager configuration or by using =psu set pool rdonly= in the !PoolManager cell. This only affects the routing of the requests. *I prefer to do it in both places*! We have a shelltool for setting the PoolManager pool configuration (as most shelltools, it can take a list of arguments on stdin, therefore the leading echo): <pre> echo t3fs05_cms | dc_set_pools_readonly.sh -f </pre> Enter the admin shell and switch to the source pool's cell: <pre> [t3se01.psi.ch] (local) admin > cd t3fs05_cms [t3se01.psi.ch] (t3fs05_cms) pool disable -rdonly </pre> You can check which pools are disabled and which pools are set to readonly, by using <pre> dc_get_pool_list.sh -o d dc_get_pool_list.sh -o ro </pre> ---+++ Start the migration <pre> [t3se01.psi.ch] (t3fs05_cms) migration copy t3fs07_cms </pre> You can list the active migrations <pre> [t3se01.psi.ch] (t3fs05_cms) migration ls </pre> This yields 1 for our example You can check on the progress of the migration <pre> [t3se01.psi.ch] (t3fs05_cms) migration info 1 </pre> You can set the number of parallel copy processes to two for our migration job by using <pre> [t3se01.psi.ch] (t3fs05_cms) migration concurrency 1 2 </pre> ---+++ Check whether all files have been migrated Simple way using the shelltools Get the contents of both pools <pre> dc_get_rep_ls.sh t3fs05_cms > t3fs05_cms.lst dc_get_rep_ls.sh t3fs07_cms > t3fs07_cms.lst </pre> Run a simple loop to find the missing entries on the target pool <pre> for n in $(cat t3fs05_cms.lst); do grep -q $n t3fs07_cms.lst; if test $? -ne 0; then echo $n >> missing-on-t3fs07.lst; fi; done </pre> If there are missing entries, use the shelltools to find out more about them, e.g. map to pnfs IDs: <pre> dc_get_pnfsname_from_IDlist.sh missing-on-t3fs07.lst </pre> It may well be that you find that all of these files lack a mapping in pnfs (which can happen if during the migration these entries were deleted by users. Since the source pool is set to read only, the physical file will remain on the disk until the pool is made read/write again). ---+++ Remove files on the source pool <pre> dc_rep_rm_list.sh t3fs05_cms.lst t3fs05_cms.lst </pre> ---++ Using migration move Instead of copying the files, checking the correct copying and then removing the source files one can also use the *migration move* command instead of the *migration copy*. Then dcache itself will remove the copied source files. -- Main.DerekFeichtinger - 2010-03-17
E
dit
|
A
ttach
|
Watch
|
P
rint version
|
H
istory
: r3
<
r2
<
r1
|
B
acklinks
|
V
iew topic
|
Ra
w
edit
|
M
ore topic actions
Topic revision: r3 - 2010-04-07
-
DerekFeichtinger
CmsTier3
Log In
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
User Pages
Main Page
Policies
Monitoring Storage Space
Monitoring Slurm Usage
Physics Groups
Steering Board Meetings
Admin Pages
AdminArea
Cluster Specs
Home
Site map
CmsTier3 web
LCGTier2 web
PhaseC web
Main web
Sandbox web
TWiki web
CmsTier3 Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
E
dit
A
ttach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback