Upgrading dcache storage hardware
This is a procedure to follow when you don't wish to create a new pool but instead copy an old one to new hardware and have it otherwise remain the same.
Preparing for move
- Attach the new hardware to your system, make volume/filesystem as needed. On our Dell systems it is fairly simple to use omconfig to make a volume (determine the physical disk ids by running "omreport storage pdisk"):
omconfig storage controller action=createvdisk controller=1 \
raid=r6 size=max \
pdisk=1:1:0,1:1:1,1:1:2,1:1:3,1:1:4,1:1:5,1:1:6,1:1:7,1:1:8,1:1:9,1:1:10,1:1:11,1:1:12,1:1:13,1:1:14 \
stripesize=512kb readpolicy=ra writepolicy=wb
- Label and mount the new volume someplace (we'll be changing the label later). For this example, we use /dcache_new presuming we are migrating /dcache.
- Mark the old pool read only using the admin interface on head01:
ssh -c blowfish -p 22223 -l admin localhost
[head01.aglt2.org] (local) admin > cd PoolManager
[head01.aglt2.org] (PoolManager) admin > psu set pool umfs07_2 rdonly
- Wait at least 30 minutes for transfers to finish (NOTE: Document how we check they are finished?)
- Remount old partition read-only: "mount -o remount,ro /dev/sdb"
- Run a mirrordir from old to new. I chose to include parameters to copy and preserve access times but it's probably not that important. Best to run this in a "screen" session and then type "CTRL-a d" to detach and let it run as long as it takes:
"mirrordir --access-times --restore-access /dcache /dcache_new &"
- Unmount the new volume, and assign the same label as the old (assume you are using labels in fstab, don't even try to guess hardware names please).
- Disable gridftp/dcap doors for system to shutdown:
ssh -c blowfish -p 22223 -l admin localhost
[head01.aglt2.org] (local) admin > cd LoginBroker
[head01.aglt2.org] (LoginBroker) admin > disable gridftp-umfs07Domain
[head01.aglt2.org] (LoginBroker) admin > disable dcap-umfs07Domain
- Shutdown, replace hardware, and power on. New hardware should mount where the old was if you set the label right
- Edit /opt/d-cache/etc/pool_path and set the size to the size of your new volume - 1. Example line for 24213 GB volume:
/dcache 24212 no
Volume size in GB can be determined with "df --block-size=1G".
- Edit pool setup file (example: /dcache/pool/setup). Change this line to increase the size (use a size about 50G less than physical size):
set max diskspace 24160g
- Restarting dcache seems to not be necessary. However you do need to reset the pool to notrdonly, and also remember to re-enable the gridftp door. Use "enable" instead of "disable" in dcache LoginBroker admin command and "notrdonly" instead of rdonly in PoolManager command.
-- BenMeekhof - 04 Nov 2009