Upgrading dcache storage hardware

This is a procedure to follow when you don't wish to create a new pool but instead copy an old one to new hardware and have it otherwise remain the same.

Preparing for move

  1. Attach the new hardware to your system, make volume/filesystem as needed. On our Dell systems it is fairly simple to use omconfig to make a volume (determine the physical disk ids by running "omreport storage pdisk"):
    omconfig storage controller action=createvdisk controller=1  \ 
    raid=r6 size=max  \
    pdisk=1:1:0,1:1:1,1:1:2,1:1:3,1:1:4,1:1:5,1:1:6,1:1:7,1:1:8,1:1:9,1:1:10,1:1:11,1:1:12,1:1:13,1:1:14 \ 
    stripesize=512kb readpolicy=ra writepolicy=wb
  2. Label and mount the new volume someplace (we'll be changing the label later). For this example, we use /dcache_new presuming we are migrating /dcache.
  3. Mark the old pool read only using the admin interface on head01:
     ssh -c blowfish -p 22223 -l admin localhost
    [head01.aglt2.org] (local) admin > cd PoolManager
    [head01.aglt2.org] (PoolManager) admin > psu set pool umfs07_2 rdonly
  4. Wait at least 30 minutes for transfers to finish (NOTE: Document how we check they are finished?)
  5. Remount old partition read-only: "mount -o remount,ro /dev/sdb"
  6. Run a mirrordir from old to new. I chose to include parameters to copy and preserve access times but it's probably not that important. Best to run this in a "screen" session and then type "CTRL-a d" to detach and let it run as long as it takes:
    "mirrordir --access-times --restore-access /dcache /dcache_new &"
  7. Unmount the new volume, and assign the same label as the old (assume you are using labels in fstab, don't even try to guess hardware names please).
  8. Disable gridftp/dcap doors for system to shutdown:
    ssh -c blowfish -p 22223 -l admin localhost
    [head01.aglt2.org] (local) admin > cd LoginBroker
    [head01.aglt2.org] (LoginBroker) admin > disable gridftp-umfs07Domain
    [head01.aglt2.org] (LoginBroker) admin > disable dcap-umfs07Domain
  9. Shutdown, replace hardware, and power on. New hardware should mount where the old was if you set the label right
  10. Edit /opt/d-cache/etc/pool_path and set the size to the size of your new volume - 1. Example line for 24213 GB volume:
    /dcache 24212 no 
    Volume size in GB can be determined with "df --block-size=1G".
  11. Edit pool setup file (example: /dcache/pool/setup). Change this line to increase the size (use a size about 50G less than physical size):
    set max diskspace 24160g
  12. Restarting dcache seems to not be necessary. However you do need to reset the pool to notrdonly, and also remember to re-enable the gridftp door. Use "enable" instead of "disable" in dcache LoginBroker admin command and "notrdonly" instead of rdonly in PoolManager command.
  13. -- BenMeekhof - 04 Nov 2009
Topic revision: r3 - 17 Dec 2010, ShawnMcKee
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback