Proof on demand Quick start This section briefly describes how to setup a proof on demand system. Follow these steps: * Setup a root version or skip this step...
Creation of gate04.aglt2.org, the SL6 gatekeeper Core dump style... Not bothering for now to make this pretty, just recording the actions taken 2 cd /etc/yu...
NB, condor is now in umatlas repo, and it along with yum priorites are now installed via cf3 epel should be installed as well root@gate02 ~ # cat /etc/...
The PanDA Auto Exclusion process for ANALY_AGLT2 Introduction Procedures here were documented by D. van der Ster in this talk. To see this you will need a CERN ...
Procedures followed to bring gate03 online as a test gate keeper NOTE: This page is changing as the procedures and tests evolve. This note will be removed once te...
Update Kerberos on our Servers The kerberos servers were installed long ago when DES was the primary encryption. We need to change to using newer more secure algo...
OpenAFS and Kerberos on Windows Software prerequisites Kerberos for windows. The current release of OpenAFS 1.7.4 recommends the Heimdal Kerberos implementation. ...
Notes on upgrade of Lustre to 2.1.6 from 1.8.4 With the implementation of SL6.4 everywhere it became necessary to also upgrade Lustre from 1.8.4, which was not re...
Configuring new storage This page describes how to configure newly purchased storage. 1 Start server 1 At BIOS prompt for storage controller setup, press Ct...
Upgrading from GPlazma1 to GPlazma2 on dCache 2.2.12 In preparation for upgrading to dCache 2.6 we need to reconfigure our dCache GPlazma configuration to go from...
Updating and Upgrading dCache Headnodes for AGLT2 June 2013 The dCache headnodes head01.aglt2.org and head02.aglt2.org were transitioned to VMware VMs in 2012. As...
Index of other pages ForemanPuppetInitialSetup unorganized notes from initial setup. Mostly you won't need these. HOWTO: Build new host with foreman Mostly se...
Puppet Infrastructure Setup from our repo: To bootstrap foreman and puppetmaster from our svn code: Initial setup notes Installed from puppetlabs repo: http://y...
Using Slony to Replicate dCache Postgresql DBs We have been running postgresql 9.0.x on our dCacheheadnodes for almost two years. Currently we have have the follo...
Proxy server This system has no access to outside world, and in general systems can't talk to non edu locations. To be able to download things, use proxy server. ...
Configuring to PXE boot Servers from the Rocks 5.5 HeadNodes From Ben on 10/10/2012, the following procedure can be used to PXEboot a machine into SL6 via Rocks 5...
ROCKS This is the main local page for the ROCKS cluster software. Subpages: * BuildingRocksRolls * RocksAglReleases Notes on configs used in production ...
How to add a new pool node to dCache/Chimera * Provision node via ROCKS (or other method) * Install correct kernel (if not done above): currently 2.6.38 UL5...
Lustre Backup Following: http://wiki.lustre.org/manual/LustreManual18_HTML/BackupAndRestore.html Snapshots On umfs15 there are regularly scheduled hourly and dai...
Lustre Configuration and Setup for AGLT2 In March 2010 we revisited our exploration of Lustre for use at AGLT2. This was motivated in part by the release of Lus...
VMWare vSphere Upgrade at AGLT2 In September 2012 VMware release a signficant upgrade to vSphere: 5.1. There are a number of nice features that we want to benefit...
Network Issues at AGLT2 This page is intended to capture the network related issues at AGLT2 Network Issues after UltraLight Router at Starlight (R04CHI) was Ret...
Implementing Network QoS at AGLT2 Recently we have seen periods where our LANs have been congested and packets are dropped. This has resulted in some of the monit...
Testing Glusterfs For testing purposes only we used the Redhat Storage Appliance demo which has gluster tools pre installed. Docs are here: http://docs.redhat.com...
Operational notes Setup zfs filesystems and quotas for users Mkdir as usual: mkdir /atlas/data19/bmeekhof chown bmeekhof:umatlas /atlas/data19/bmeekhof Set quota...
Tests of ZFS on SunX4540 running FreeBSD 9.0 For setup notes see SunX4540ConfigFreeBSD For nearest comparison of similar system running Solaris see BenchmarkOnX4...
MDTChambers MDT status application Application public link (CERN login required): https://atlasop.cern.ch/atlas point1/muon/MDTchambers/ Following are some i...
Hardware notes here...little to say about setup. As the manual also says, hit appropriate keys during boot and configure the management/IPMI card using an exter...
IO performance test with tuning (reset readahead) Here are some IO performance (we focus on the tow IO patterns :read and write)test with tuning of the readahead ...
All these test are run by IOZONE, and the max file size are set about twice of the RAM size of each server. IOZONE parameters: a run a automatic model R generat...
This is now obsolete thor01 now runs FreeBSD. See SunX4540ConfigFreeBSD. Zpool configuration Destroy existing pools (pool1 through pool4): zpool destroy pool1 T...
Installation, Setup, and Usage reference for Sun hardware and Solaris SolarisOnPE2950 Installation of Solaris 10 (08/07) on umfs11 SunX4540ConfigFreeBSD In...
Info about actually using the 4540 is at SunX4540ConfigSolaris Migrating pre installed X4540 Solaris 10 to boot from flash drive NOTE: There's probably no need ...
Basics steps to install dCache First, install a package from dcache.org. As of this writing we run on 1.9.4 3. cf.dcache in cfengine can manage config files for ...
Tier3 for Users For information on using ATLAS software please see this section of our index page: WebHome#AGLT2_User_Information Information here includes how to...
DNS in ROCKS References: * http://rscott.org/dns/ DNS Oversimplified This page written for ROCKS 4.3 with the update given below. ROCKS will manage the config ...
Setting up Geant4 A 64 bit gcc 4.1.2 Geant4 installation for SL5 is located at /afs/atlas.umich.edu/opt/geant4. All available data packages are included. For a li...
Dark Sector Event Generation This section describes getting Itay Yavin's dark sector event generation files into a format usable for ATLAS simulation. Itay's cod...
Using CVMFS at AGLT2 General Information CVMFS is a new method of distributing ATLAS software that relies on using central repositories of software on servers lo...
Notes on Building ShawnGenerator Below are the series of steps I used to create my "ShawnGenerator". This generator is based upon source code from Loek Hooft va...
WLCG Accounting for Tier 2 Sites This page contains some plots showing WLCG Tier 2 accounting results for Tier 2's worldwide. Currently the plots are only availa...
VMware perfSONAR Plots for Debugging Network Issues As part of AGLT2's VMware setup, we have created three ESX host specific perfSONAR instances running OWAMP (on...
Setup and Configuration of the AGLT2 MD3600i July 25, 2011 This details our installation and configuration of our new MD3600i (UMVMSTOR02) plus MD1200 shelf. W...
Intro Here are listed releases or "tags" of the ROCKS installation. Issues with each can be added here. Summer 2011 making an attempt to maintain this page go...
UMVM0x (ESX) Network Reconfiguration June 2011 The details about the network changes on the UMVM0x nodes are below organized by host with before and after NIC/por...
Building a ROCKS client worker node Build Server Status The software revision for ROCKS and CFEngine that are active on the ROCKS frontend are shown here, the ne...
Evaluation and testing of Nexsan SATABeast with B60E expansion Unpacking and Installation See photos and some comments here: https://picasaweb.google.com/ben.mee...
Using TortoiseSVN with Putty/Pageant for SVN access to AGLT2 SVN server Prerequisites Install TortoiseSVN from http://tortoisesvn.net/downloads.html Setup putty a...
Site Blacklisting via DQ2 Commands If you want to exclude your site from DQ2 you need to use the dq2 set location status command. The specific command is: dq2 set...
FTS Channel Management Instructions The FTS channels for AGLT2 and be managed using the glite software. You will need an ATLAS VOMS production role (voms proxy i...
Installing pCache and LSM at AGLT2 We are interested in setting up both a Local Site Mover (LSM) and pCache on our worker nodes. The goals are: * Reduce the I...
Details on Configuring the /etc/multipath.conf for our Nexsan Setup The multipath v2 d with our configuration gives: root@umfs03 ~ # multipath v2 d create: m...
Subversion Subversion is a software revision control system designed to be an improvement on CVS. It generally replicates the features of CVS. References * T...
Planning for the production network. NetworkHardwareInfo Near term To Do List Here is a list of network related items that need doing as of February 4, 2011: ...
Creation of New dCache Headnodes (Dell R610) in January 2011 As part of our Fall 2010 procurements we purchased 2 Dell R610 nodes to host the dCache services (he...
Upgrading dcache storage hardware This is a procedure to follow when you don't wish to create a new pool but instead copy an old one to new hardware and have it o...
Setting up LVS (Linux Virtual Server) for use with dCap Newer linux kernels have LVS built in (as well as our UltraLight kernels). See http://kb.linuxvirtualserv...
Migrating files to newly added OST so as to balance content It is desirable to distribute access over as many Lustre OSS as possible, so when a new OSS umdist04.a...
Room/Site Infrastructure Monitoring at MSU Liebert Air Handlers The two Liebert System/3 Air Handler units have Intellislot Web / 485 cards. See: * http://ww...
lighttpd service is running on client during install. it matches URLs that have HOST == 127.0.0.1. Then does a redirect of "/install/(. )$" = rocks by.py?filena...
Lustre Reinstall Notes After testing Lustre "in production" (mostly tests by Tiesheng) we have decided to go ahead with our plans to utilize Lustre to replace the...
High Available Lustre MDS failover nodes with Redhat cluster tools Background: The Lustre Meta Data Server is integral to using Lustre. If it is not available...
AGLT2 IP Addresses Information on IP addresses for the Tier2 For detail list of IPs at MSU see ask Tom for msu ips.ods or see configs/msu/network/msu ips.csv For ...
Monitor Disk Activity with iostat and Ganglia The iostat utility from the sysstat package provides information about disk operations and throughput. It works in ...
Track Transfers to dCache Just some quick notes on tracking specific transfers to dCache. * First, make sure the SRM logging is verbose enough (going to catali...
How msurxx was setup Create config files in SVN In the ROCKS SVN repo, below hostconfigs, copy msurxii.aglt2.org to msurxx.aglt2.org. Checkout (nominal location...
Introduction A prototype build of a file server in Rocks 5 is described, along with caveats and difficulties. In this instance, the file server is destined to be...
ROCKS 5.3 * R53abMSURXX Setup of msurxx * R5abFileServer Building a file server and configuring with cfengine * UMRocksiSetup Taking umrocksi.aglt2.org f...
Procedure to Migrate dCache headnodes (head01/head02.aglt2.org) to new hardware and operating system. During fall 2009 and winter 2010, AGLT2 is migrating all lin...
Testing of New dCache Storage Node Build Here is the testing result for a new storage node (summary: it works). After running \x91install_dcache.sh\x92 on UMFS16...
Frontend Config in SVN Have a scheme to track frontend config in SVN. A directory structure is created at /var/svn, below here modified configurations are copied....
Updating umrocksi.aglt2.org to the most recent SL5.4 rpm set Following is a description of the process followed to ready a Rocks 5.3 client node built with the SL...
Install on Frontend Setup EPEL yum repo On test ROCKS5 frontend, install puppet server from EPEL repo. Note that redhat.com is not reachable from aglt2.org. But...
ROCKS 5 Routing ROCKS 5 has a flexible scheme for setting routing rules. Similar to the attribute scheme, routes can be set on a global, OS, appliance and host l...
Configuring the Frontends Record of configuration done to frontends. Updated Dec 16th for msurx build. Connecting You can ssh to the frontend as root to perform...
Host Configs in SVN Have modified host config files tracked in SVN. Also have ROCKS DB entries tracked. Pull these out and put on the frontend. Storage area We...
Installing the Frontends The frontends are installed on VMware clients. Note that you must have a valid resolvable IP address and name or the install will fail. ...
LOST all my edits by a key sequence...(from Emacs)... * First read dCache book for info: http://www.dcache.org/manuals/Book/start/in install.shtml http://www....
How To Add New Pools to Dcache This page is obsolete for dCache/Chimera. See HowToAddNewPoolsToChimera instead. On our existing dcache system ,I add some New ...
PC6248 Reformat Note that using the command show dir seems be a good stress test. Even switches that pass the check disk commands below can fail running the show...
First begin by installing your Frontend by following the directions in the Rocks users guide, make sure you include the service pack roll, as it's required and fi...
Re Adding to Database If a host needs to be re added to the database (it was erased, or the front end is being rebuilt), get the host info from SVN. root@msurxi ...
VMware Hosting of Frontend Wish to run the frontends in VMware ESXi. The primary benefit is server consolidation. This also provides a good way to make a full b...
Power Connect SNMP Lots of info is available via SNMP from the PowerConnects. References * http://wiki.xdroop.com/space/snmp/Switching Tables * http://for...
Test Frontend Wish to be able to separate production and test use of frontend. The idea is that the normal production frontend can be maintained with a well defi...
Intro This is a guide for using Athena on the Tier 3 that all users should follow. Useful webpages: * ATLAS computing workbook * nice introdution, a bit ol...
0. Useful webpages This webpage is a mix between a tutorial and reference. If you are just interested in a quick overview of useful condor commands, just google "...