System Install Checklist (UM Systems)

Attached to this document is a tarball containing reasonable examples, many of which can probably be used with no modification.
Don't forget to start and chkconfig on services after configuration is finished.

1. Use the following install options for SLC 4.4 x86_64 (this list needs a little work yet).
   • Custom installation
   • Software and Kernel Development
   • Compatibility
   • SELinux disabled
   • Firewall enabled, ports open for SSH, services as appropriate
   • Unselect things like multimedia/sound, printing support (usually), other things that won't be needed
2. Desktop partitions (few local users):
   • /boot 200M
   • / 20GB
   • /var 10GB
   • /tmp 20GB (per cpu core if condor desktop).
   • /scratch - remaining space, world writable
   
   (May reduce by half /var and /tmp if <200GB)
3. Document connections (spreadsheet, OCS will document network interfaces).
4. Update kernel, check drivers.
   • kernel-2.6.20-20UL5
5. YUM configuration - add umatlas repo to /etc/yum.repos.d:

      [umatlas]
      name=UM-ATLAS
      baseurl=http://linat05.grid.umich.edu/pub/SLC/4x/custom/
      enabled=1
   

6. Firewall Configuration - as needed. Come up with standard configurations and attach to this document or otherwise manage:
   • Restricted hosts

         -A RH-Firewall-1-INPUT  -m state --state NEW -m tcp -p tcp --dport 22 -i eth0 -s 141.211.101.0/24 -j ACCEPT
         -A RH-Firewall-1-INPUT  -m state --state NEW -m tcp -p tcp --dport 22 -i eth0 -s 141.211.96.0/22 -j ACCEPT
         -A RH-Firewall-1-INPUT  -m state --state NEW -m tcp -p tcp --dport 22 -i eth0 -s 192.41.230.0/23 -j ACCEPT
         -A RH-Firewall-1-INPUT  -m state --state NEW -m tcp -p tcp --dport 22 -i eth0 -s 141.211.43.96/27 -j ACCEPT
         

   • Hosts backed up via Amanda client from atback1 (could be tighter)

          # amanda 
          #-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10080 -j ACCEPT
          #  rule above is not sufficient for amanda backups, what would be?
          -A RH-Firewall-1-INPUT -s 141.211.43.99 -j ACCEPT
          

   • UMOPT1 rocks client
   • Fileservers, NFS, AFS (additions for NFS below):

         # ipmi
        -A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
        -A RH-Firewall-1-INPUT -p tcp -m tcp --dport 199 -j ACCEPT         
        # snmp
         -A RH-Firewall-1-INPUT -p udp -m udp --dport 161 -j ACCEPT
        # Ganglia
        -A RH-Firewall-1-INPUT -p udp -m udp --dport 8649 -j ACCEPT
        -A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
        # Gridftp
        -A RH-Firewall-1-INPUT -p tcp --dport 2811 -j ACCEPT
        # Globus-gatekeeper
        -A RH-Firewall-1-INPUT -p tcp --dport 2119 -j ACCEPT
        # NFS ports
        -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 991 -j ACCEPT
        -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 991 -j ACCEPT
        -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 32777 -j ACCEPT
        -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 32777 -j ACCEPT
        -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 1001 -j ACCEPT
        -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1001 -j ACCEPT     
         

   • grid servers
7. NFS tuning
   • add to /etc/sysconfig/nfs:

         LOCKD_TCPPORT=32777
         LOCKD_UDPPORT=32777
         RQUOTAD_PORT=991
         MOUNTD_PORT=1001
         RPCNFSDCOUNT=128
         

8. NIS/YP configuration (/etc/yp.conf)
   • add "NISDOMAIN=um-atlas-grid" to /etc/sysconfig/network
9. Automount config (attached)
   • /etc/auto.master, auto.net, auto.atlas
10. OCSNG installation
    • Once yum is configured: "yum install OCSNG_LINUX_CLIENT"
11. Verify OpenAFS rpm version (openafs-1.4.4-4)

     yum install openafs-client openafs openafs-krb5 openafs-kernel-smp openafs-compat openafs-authlibs

12. Kerberos configuration - ATLAS.UMICH.EDU, CERN.CH, FNAL.GOV, BNL.GOV (/etc/krb5.conf).
    /etc/pam.d/system-auth/ needs the krb5afs line modified added:

     
    auth        required      /lib/security/$ISA/pam_env.so
    auth        sufficient    /lib/security/$ISA/pam_unix.so likeauth nullok
    auth        sufficient    /lib/security/$ISA/pam_krb5afs.so     use_first_pass tokens
    auth        required      /lib/security/$ISA/pam_deny.so
    

13. Syslog forwarding, install syslog-ng (/etc/syslog-ng/syslog-ng.conf). Forward to 141.211.43.109 (atgrid.grid.umich.edu).
14. SNMP configuration (install net-snmp package). rocummunity should be usatlasgrid.
15. IPMI - install OpenIPMI if system has hardware
16. Network tuning and config, standard sysctl.conf. (Rocks clients have one, desktops can use default probably).
17. Mail configuration - default should be fine for local mail to root that we probably won't read (with RHEL, the default config runs an instance of sendmail that only listens on the loopback interface). Headnodes should relay for nodes, and nodes should use the headnode as relay host (ROCKS default config).
18. Install/Request host grid certificate as needed.
-- BenMeekhof - 17 Apr 2007