This is a preview release of the VDT installation dCache. This version is for testing only and is not supported by VDT. INSTRUCTIONS ------------ Note: These scripts should be run as user 'root'. ---- The scripts provide a fresh install of dCache 1.8.0-x or an upgrade of 1.7.0-x or 1.8.0-x to 1.8.0-y. To install dCache, do following: On any PC running linux, o Download the vdt-dcache package. o Untar the vdt-dcache package. o cd vdt-dcache-x.x/install o ./config-node.pl (creates the file site-info.def) o Edit the site-info.def file if site-dependent customizations are desired (see CUSTOMIZATIONS below). First on the node which will run pnfs, and then on all other dCache nodes, o Copy and untar the vdt-dcache package. o Copy site-info.def to the vdt-dcache-x.x/preview directory. o cd vdt-dcache-x.x/install o Make sure rpcinfo is in your path. o ./install.sh DIRECTORY STRUCTURE ------------------- Upon untarring the tarball, you should see following directory structure: RPMS - This directory contains dcache, postgresql, pnfs and zsh rpms. preview - This directory contains: the following installation scripts - config-node.pl find_java.sh install.sh install_dcache.sh install_java.sh install_pnfs.sh install_postgres.sh install_srmwatch.sh README - This file. site-info.def.template - template used by config-node.pla config - This directory contains LinkGroupAuthorization.conf, PoolManager.conf, and storage-authzdb. The first two serve as default configuration files. The last is an example authorization file. WARNING: If you are doing an upgrade, your previous PoolManager.conf will be replaced. A backup of the previous PoolManager.conf is saved to refer to when customizing PoolManager.conf. util - This directory contains file "config_file". This is Yaim Configuration and setup file for dCache 1.7.0-X DRYRUN ------------ If you would like to see what the installation script will do and NOT do the actual install, run install.sh with the option --dryrun. NOTES ABOUT INDIVIDUAL INSTALLATION SCRIPTS ------------------------------------------- Postgres -------- Postgres will be installed only if o this is a pnfs server node or o any of srm, billing,space manager, pin manager or replica manager service are running on this node. Note: Postgres will not be installed, if it already exists on the system. Please note that dCache strongly recommends to use atleast version 8 or higher Pnfs ------ Pnfs server software will be installed only if it is not already installed and o this is the pnfs server node. (defined by the value of variable $DCACHE_PNFS in site-info.def) o NFS server is not running on this node. In a single dCache installation, there can only be one pnfs server node. Note: Some pnfs rpms have a dependency on zsh. This is why zsh rpm is installed as part of pnfs installation. Note for existing PNFS installations: Here is what you need to know if have a previously installed PNFS - o Case 1 - PNFS is running when you run the installation script: The installation script will check that the pnfs export for the doors has value /pnfsdoors. If not, a warning is issued and dCache installation might not work. o Case 2 - PNFS is not running when you run the install script: You need to start PNFS first. The dCache installation will not work till you do so. Instructions on how to upgrade the PNFS Server code ---------------------------------------------------- Disclaimer Though an attempt has been made to cover all that is necessary, ----------- we do not take responsibility for loss of any data. Depending on your PNFS setup, you may need to perform additional steps. The following instructions have been tested while upgrading from pnfs-postgresql-3.1.10-3.i386.rpm to pnfs-postgresql-3.1.10-7.i386.rpm In any case, you need to be very careful while performing each step. Note Please DO NOT confuse this with the Postgres Database upgrade. ------ If you need to upgrade Postgres databases, then refer to instructions on official postgres website www.postgresql.org. Assumption The Postgres databases are located outside the PNFS installation ---------- directory. So upgrading the PNFS server code will not affect the Postgres databases in any way. Step 1 Backup the PNFS Database structure information (preferrably on different host) ------ This is very critical information and must be backed up, before you upgrade the PNFS server code. If this is lost, then the actual data stored in Postgres databases cannot be accessed. This location may be different for different sites, depending on where you had installed PNFS.Common locations are $PNFS_INSTALL_DIR/pnfsdb and $PNFS_INSTALL_DIR/pnfs/db [root@fapl034 src]# scp -r /opt/pnfsdb fapl035:/usr/local/pnfs-backup/ D-0000 100% |******************************| 51 00:00 databases 100% |******************************| 78 00:00 shmservers 100% |******************************| 2 00:00 D-0001 100% |******************************| 51 00:00 admin 100% |******************************| 0 00:00 data1 100% |******************************| 0 00:00 [root@fapl034 src]# Step 2 Backup the main PNFS Server configuration file (preferrably on different host) ------ This file is usually present at location /usr/etc/pnfsSetup [root@fapl034 src]# scp /usr/etc/pnfsSetup fapl035:/usr/local/pnfs-backup/ pnfsSetup 100% |******************************| 498 00:00 Step 3 Backup the existing PNFS Server rpm (preferrably on different host) ------ [root@fapl034 src]# scp ../RPMS/pnfs-postgresql-3.1.10-3.i386.rpm fapl035:/usr/local/pnfs-backup/ Step 4 Remove the old and get the new PNFS Server rpm ------ o Remove the old pnfs server rpm from $VDT_DCACHE_HOME/dcache/RPMS/ [root@fapl034 src]# rm ../RPMS/pnfs-postgresql-3.1.10-3.i386.rpm rm: remove regular file `../RPMS/pnfs-postgresql-3.1.10-3.i386.rpm'? y [root@fapl034 src]# o Copy the new rpm in $VDT_DCACHE_HOME/dcache/RPMS/ Step 5 Install the new PNFS Server rpm ------- o Stop PNFS (if it's running) [root@fapl034 src]# /opt/pnfs/bin/pnfs stop o Stop Postmaster (if it's running) [root@fapl034 src]# /etc/init.d/postgresql stop o Make sure that old PNFS Server rpm is not installed. If it is, erase it. [root@fapl034 src]# rpm -qa | grep pnfs pnfs-postgresql-3.1.10-3 [root@fapl034 src]# rpm -e pnfs-postgresql-3.1.10-3 o Uninstalling pnfs does not unregister its RPC service (a bug in pnfs). You should do it manually as follows: [root@fapl034 src]# rpcinfo -p | grep nfs | grep -v grep 100003 2 udp 2049 nfs [root@fapl034 src]# rpcinfo -d nfs 2 o Run the install script with dryrun option to make sure that things are setup right for PNFS to be installed again. [root@fapl034 src]# ./install.sh -s site-info.def --dryrun o Start Postmaster if it is not already running [root@fapl034 src]# /etc/init.d/postgresql start o Run the install script (without dryrun option). This will install new PNFS Server rpm. [root@fapl034 src]# ./install.sh -s site-info.def Step 6 Restore the backed up configuration and data to the new installation ------ Two options: Option 1 -------- o Stop PNFS server [root@fapl034 src]# /opt/pnfs/bin/pnfs stop o Copy the backed up PNFS Server configuration file to /usr/etc/pnfsSetup [root@fapl034 vdt-dcache-neha]# scp fapl035:/usr/etc/pnfsSetup /usr/etc/pnfsSetup o Copy the backed up PNFS Database structure to /opt/pnfsdb [root@fapl034 vdt-dcache-neha]# scp -r fapl035:/usr/local/pnfs-backup/pnfsdb /opt/ Option 2 -------- o Stop PNFS server [root@fapl034 src]# /opt/pnfs/bin/pnfs stop o Copy the backed up PNFS Server configuration file to /usr/etc/pnfsSetup [root@fapl034 vdt-dcache-neha]# scp fapl035:/usr/etc/pnfsSetup /usr/etc/pnfsSetup o Make sure in file /opt/pnfs.3.1.10/pnfs/etc/pnfs_config, the variable PNFS_OVERWRITE is set as follows: PNFS_OVERWRITE = no This basically means to not overwrite the pnfs database specified by $PNFS_DB, if it exists already. Here make sure value of $PNFS_DB point to right location. Step 7 Start the PNFS Server ------ [root@fapl034 src]# /opt/pnfs/bin/pnfs start dCache ------ The dCache install portion unpacks the dcache-server rpm if necesary and modifies dCache configuration files. o Services will be turned on and pools created according to the information in site-info.def. o The pnfs node should be installed and started before any gridftp nodes. o pnfsManager will run by default on the pnfs node. o Authorization files are not set up by the install script. Configure the files used by gPlazma before starting dCache. o Files etc/node_config.delta and etc/dCacheSetup.delta are created, which show exactly what lines in etc/node_config or config/dCacheSetup will be set. These files can be used for custom settings as well (see CUSTOMIZATIONS below). NODE CONFIGURATION ------------------ The config-node.pl script will turn the site-info.def.template file into a site-info.def file. If an existing site-info.def file is found, it will use those values as defaults. Internally, this script invokes the find_java.sh script which finds java installed on your system in following order: o First, checks if java exists in $PATH. If it does, it chooses that version (provided its a valid installation) o Second, checks if $JAVA_HOME is defined. If it is, it chooses that version o Finally, it checks in various standard locations and if multiple java installations are found, the most recent version is selected. To give you an idea how this order works, consider following situation: You have a /usr/bin/java that points to a JRE installation and have another JDK installation as well. In this case, after you run the config_node.pl script, $JAVA_LOCATION in your site-info.def will point to the JRE install (Note that dCache requires JDK). In this case, you either can include the java from JDK version in $PATH (before the JRE version), or explicitly set $JAVA_LOCATION in site-info.def file to point to JDK version. Next, config-node.pl script will ask a number of questions for the user to answer. It first will ask how many non-pool and non-door nodes you have. Based on this it makes a recommended configuration using up to 5 nodes. Then it asks which services should run on each of these nodes (Note: node #1 is considered the DCACHE_ADMIN node.) The list of services should be entered on a single line seperated by white space. The complete list of choices are: lmDomain poolManager pnfsManager dirDomain adminDoor httpDomain utilityDomain gplazmaService infoProvider srm replicaManager statistics Next the door nodes are configured. One instance each of dcap, gsidcap, and gridftp will be made to run on each door node. And last, the pools with the directory locations on those nodes will be set. By default, pool sizes will be made to be the size of the entire partition the pool is are on. If another size is desired for a pool, edit the file site-info.def, replacing the text "all" with the desired size. The site_info.def file ---------------------- In the site-info.def file, the node which shall run each service is defined. If a service is required and no definition is given, it will run on the admin node. The definition for services which may run on more than one node may be a space-separated list of fully-qualified hostnames. If a service needs further information, such as pool size and location, the may appear in colon-separated fields after the hostname. CUSTOMIZATIONS -------------- Some dCache options not covered by the config-node.pl script may be set by editing site-info.def after config-node.pl has been run. MAX_FTP_STREAMS The maximum number of client streams allowed by gridftp. GRIDFTP_PRIVATE_NETWORK The network portion of the ip on the private network of the pools. For example, if the pools ip's are 192.168.1.x, enter "192.168.1". USE_MULTI_MOVER_QUEUES Sets up default,WAN, and LAN queues on the pools, and directs gridftp to use WAN and dcap and gsidcap to use LAN. If desired, set to "yes". BILLING_USE_DB Causes billing info to be copied to a database as well as the log file. If desired, set to "yes". DCACHE_LOG_DIR Set to a non-default log directory for dCache domain output logs. Directory must exist before dCache is started. There is one customization that is always done, REMOTE_GSI_FTP_MAX_TRANSFERS=2000 This value corrects a mismatch between gridftp and srm paramters that exists in the default settings of dCache. In addition, etc/node_config.delta and etc/dCacheSetup.delta can be used to change any other lines in etc/node_config or config/dCacheSetup. To change or add any other lines in these files, before running install.sh add the line or lines to the *.delta file (creating the file if necesary). CLEANUP --------- You should follow instructions in this section, ONLY if you tried to do the installation and all of below mentioned conditions are true o installation didnot finish successfully o you want to do a clean up o you want to run installation script again Postgres -------- This will erase all postgresql rpms and delete any existing postgres databases and installations !! Make a backup of your databases and anything else, you think you might need later. o Stop the postmaster /etc/init.d/postgresql stop o Erase following postgresql rpms rpm -e postgresql postgresql-libs postgresql-server o Delete the /var/lib/pgsql directory o As additional check, make sure following files do not exist /bin/postmaster /sbin/postmaster /usr/bin/postmaster /usr/sbin/postmaster /usr/local/bin/postmaster PNFS ----- This will erase pnfs-server rpm and delete all pnfs directories created in /opt !! Make a backup of your databases and anything else, you think you might need later. o Erase pnfs server rpm rpm -e pnfs-postgresql o Erase zsh rpm rpm -e zsh o Delete various pnfs directories (depending on where your install failed, all of below might not have created) rm -rf /opt/pnfs.3.1.10 rm -rf /opt/pnfs rm -rf /opt/pnfsdb o Uninstalling pnfs does not unregister its RPC service (a bug in pnfs). You should do it manually as follows: [root@fapl034 src]# rpcinfo -p | grep nfs | grep -v grep 100003 2 udp 2049 nfs [root@fapl034 src]# rpcinfo -d nfs 2 dCache ------- o Erase dcache-server rpm rpm -e dcache-server-1.7.0-19.noarch If this returns a "scriptlet failed" error, try running with the --noscripts option. o Delete the d-cache directory rm -rf /opt/d-cache SUPPORT ------- o For more detailed information, refer the dCache Book at http://www.dcache.org/manuals/Book/ o Questions? vdt-support@opensciencegrid.org