Upgrading dcache storage hardware

Preparing for move

1. Attach the new hardware to your system, make volume/filesystem as needed. On our Dell systems it is fairly simple to use omconfig to make a volume (determine the physical disk ids by running "omreport storage pdisk"):

   omconfig storage controller action=createvdisk controller=1  \ 
   raid=r6 size=max  \
   pdisk=1:1:0,1:1:1,1:1:2,1:1:3,1:1:4,1:1:5,1:1:6,1:1:7,1:1:8,1:1:9,1:1:10,1:1:11,1:1:12,1:1:13,1:1:14 \ 
   stripesize=512kb readpolicy=ra writepolicy=wb
   

2. Label and mount the new volume someplace (we'll be changing the label later). For this example, we use /dcache_new presuming we are migrating /dcache.
3. Mark the old pool read only using the admin interface on head01:

    ssh -c blowfish -p 22223 -l admin localhost
   [head01.aglt2.org] (local) admin > cd PoolManager
   [head01.aglt2.org] (PoolManager) admin > psu set pool umfs07_2 rdonly
   

4. Wait at least 30 minutes for transfers to finish (NOTE: Document how we check they are finished?)
5. Remount old partition read-only: "mount -o remount,ro /dev/sdb"
6. Run a mirrordir from old to new. I chose to include parameters to copy and preserve access times but it's probably not that important. Best to run this in a "screen" session and then type "CTRL-a d" to detach and let it run as long as it takes:
   "mirrordir --access-times --restore-access /dcache /dcache_new &"
7. Unmount the new volume, and assign the same label as the old (assume you are using labels in fstab, don't even try to guess hardware names please).
8. Disable gridftp/dcap doors for system to shutdown:

   ssh -c blowfish -p 22223 -l admin localhost
   [head01.aglt2.org] (local) admin > cd LoginBroker
   [head01.aglt2.org] (LoginBroker) admin > disable gridftp-umfs07Domain
   [head01.aglt2.org] (LoginBroker) admin > disable dcap-umfs07Domain
   
   

9. Shutdown, replace hardware, and power on. New hardware should mount where the old was if you set the label right
10. Edit /opt/d-cache/etc/pool_path and set the size to the size of your new volume - 1. Example line for 24213 GB volume:

    /dcache 24212 no 
    

    Volume size in GB can be determined with "df --block-size=1G".
11. Edit pool setup file (example: /dcache/pool/setup). Change this line to increase the size (use a size about 50G less than physical size):

    set max diskspace 24160g
    

12. Restarting dcache seems to not be necessary. However you do need to reset the pool to notrdonly, and also remember to re-enable the gridftp door. Use "enable" instead of "disable" in dcache LoginBroker admin command and "notrdonly" instead of rdonly in PoolManager command.
-- BenMeekhof - 04 Nov 2009