<!-- keep this as a security measure: #uncomment if the subject should only be modifiable by the listed groups # * Set ALLOWTOPICCHANGE = Main.TWikiAdminGroup,Main.CMSAdminGroup # * Set ALLOWTOPICRENAME = Main.TWikiAdminGroup,Main.CMSAdminGroup #uncomment this if you want the page only be viewable by the listed groups # * Set ALLOWTOPICVIEW = Main.TWikiAdminGroup,Main.CMSAdminGroup,Main.CMSAdminReaderGroup --> ---+ !!Pool Migration %TOC% ---++ External documentation * dcache book: http://www.dcache.org/manuals/Book/cookbook/cb-pool-migration.shtml ---++ Example for copying pool t3fs05_cms to t3fs07_cms using the 'migration copy' command ---+++ Set source pool to read only The dcache book documentation describes setting the pool to readonly by using the =pool disable -rdonly= command of the pool cell. I noticed that this led to a whole number of stuck transfers, but was not able to analyze the situation adequately. However, one can also set a pool to readonly inside the !PoolManager configuration or by using =psu set pool rdonly= in the !PoolManager cell. This only affects the routing of the requests. *I prefer to do it in both places*! We have a shelltool for setting the PoolManager pool configuration (as most shelltools, it can take a list of arguments on stdin, therefore the leading echo): <pre> echo t3fs05_cms | dc_set_pools_readonly.sh -f </pre> Enter the admin shell and switch to the source pool's cell: <pre> [t3se01.psi.ch] (local) admin > cd t3fs05_cms [t3se01.psi.ch] (t3fs05_cms) pool disable -rdonly </pre> You can check which pools are disabled and which pools are set to readonly, by using <pre> dc_get_pool_list.sh -o d dc_get_pool_list.sh -o ro </pre> ---+++ Start the migration <pre> [t3se01.psi.ch] (t3fs05_cms) migration copy t3fs07_cms </pre> You can list the active migrations <pre> [t3se01.psi.ch] (t3fs05_cms) migration ls </pre> This yields 1 for our example You can check on the progress of the migration <pre> [t3se01.psi.ch] (t3fs05_cms) migration info 1 </pre> You can set the number of parallel copy processes to two for our migration job by using <pre> [t3se01.psi.ch] (t3fs05_cms) migration concurrency 1 2 </pre> ---+++ Check whether all files have been migrated Simple way using the shelltools Get the contents of both pools <pre> dc_get_rep_ls.sh t3fs05_cms > t3fs05_cms.lst dc_get_rep_ls.sh t3fs07_cms > t3fs07_cms.lst </pre> Run a simple loop to find the missing entries on the target pool <pre> for n in $(cat t3fs05_cms.lst); do grep -q $n t3fs07_cms.lst; if test $? -ne 0; then echo $n >> missing-on-t3fs07.lst; fi; done </pre> If there are missing entries, use the shelltools to find out more about them, e.g. map to pnfs IDs: <pre> dc_get_pnfsname_from_IDlist.sh missing-on-t3fs07.lst </pre> It may well be that you find that all of these files lack a mapping in pnfs (which can happen if during the migration these entries were deleted by users. Since the source pool is set to read only, the physical file will remain on the disk until the pool is made read/write again). ---+++ Remove files on the source pool <pre> dc_rep_rm_list.sh t3fs05_cms.lst t3fs05_cms.lst </pre> ---++ Using migration move Instead of copying the files, checking the correct copying and then removing the source files one can also use the *migration move* command instead of the *migration copy*. Then dcache itself will remove the copied source files. -- Main.DerekFeichtinger - 2010-03-17
This topic: CmsTier3
>
WebHome
>
AdminArea
>
DcachePoolMigration
Topic revision: r3 - 2010-04-07 - DerekFeichtinger
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback