You are here: Foswiki>AGLT2 Web>UmAtlasRepo (11 Jun 2015, BenMeekhof)EditAttach

UMATLAS yum repository

The umatlas repo primary mirror is sysprov01. It is aliased as http://mirror.aglt2.org/pub/umatlas or http://mirror.local/pub/umatlas. Rsync is enabled on this machine so it can be mirrored to http://mirror.msulocal/mirror/umatlas (http://msut3-rx6.aglt2.org/mirror/umatlas). The /etc/rsync.conf file is managed by cobber and re-written during a cobbler sync. The rsync URL is rsync://sysprov01.local/cobbler-repos/umatlas. The actual directory is /var/www/cobbler/repo_mirror/umatlas.

In cobbler we reference it as a repository that is not mirrored locally. In our URL we use a variable that will be replaced by the IP of the cobbler server used in the kickstart. If kickstarted via private IP it'll be the server's private IP, same for public IP. Example: http://@@http_server@@/pub/umatlas/sl5/x86. We could also use the unaliased path at /cobbler/repo_mirror/umatlas/sl5 (or sl6). The /pub aliases are setup via /etc/httpd/conf.d/cobbler-aliases.conf. We use AliasMatch? directives so the $releasever variable can be used in client yum configs (ie, umatlas/sl6.5 or RHEL 6Server all point at umatlas/sl6). Cfengine manages that file.

The repository is arranged by distribution, then arch. Noarch packages are symlinked by a script into both architectures. There are also some packages applicable to all distributions kept in an "all" directory and symlinked into each distribution by arch. The script that does all this is /usr/local/RepoTools/createrepo (cfengine installs it).

For example: Packages from umatlas/all/x86_64 ----> umatlas/sl6/x86_64 and umatlas/sl5/x86_64.

For another example: Packages in umatlas/sl6/noarch ----> umatlas/sl6/x86_64 and umatlas/sl6/i386

Here are a few examples of where packages go in the hierarchy.

***** IMPORTANT *** Do not put packages directly in those paths. Work in an SVN working directory and use the sync-repo.sh script to put packages into the repository. Read more in the next section.

Applicable systems Location in SVN working dir
All x86_64 dists packages/all/x86_64
All i386 dists packages/all/i386
All dists, any arch packages/all/noarch
SL6, any arch packages/sl6/noarch
SL6, x86_64 packages/sl6/x86_64

....and so on, presumably you get the idea. Reading /usr/local/RepoTools/createrepo will provide more detail.

Adding or Removing Packages

Login to sysprov01 as yourself and create your own directory under /home/packagers to work in. All work in this regard should happen in an SVN working directory. Packages added directly to /var/www/cobbler/repo_mirror/umatlas will be overwritten by this workflow.

Let's imagine we're adding a new java package to our repo and removing an out of date one. There's lots of ways to get it there but often I just wget it from wherever it is hosted. Always delete packages you don't need anymore - we can always very easily get them back from SVN (if not from a public mirror). There's no reason to ever keep a package around in the repository.

  1. First checkout the repository. The space /home/packagers is provided as a working space for this purpose on sysprov01. Everyone in the "packagers" group can create a working directory here. The following command will checkout the repository into a directory named "umatlas" in your local directory. You could also end the command with a . (a period) to dump the contents of umatlas directly in your local dir.
    
    kinit (to use kerberos credentials to authenticate, ssh key will also work)
    
    svn co svn+ssh://ndt.aglt2.org/repos/yum/umatlas
    
  2. cd umatlas/packages/sl6/x86_64
    
     (remove package we are updating - easier to do with wildcards if we do it before downloading anything, but be sure to be safe with your wildcards!)
    
    svn delete java-1.7.0*
    
     wget http://mirror.mcs.anl.gov/pub/scientific-linux/6x/x86_64/updates/security/java-1.7.0-openjdk-1.7.0.45-2.4.3.2.el6_4.x86_64.rpm
     wget http://mirror.mcs.anl.gov/pub/scientific-linux/6x/x86_64/updates/security/java-1.7.0-openjdk-devel-1.7.0.45-2.4.3.2.el6_4.x86_64.rpm
    
    svn add java-1.7.0*
    
    
  3. Now we can sync to the working repository. The sync-repo.sh script is at the top level of the svn working dir so you will want to cd there from wherever you were adding packages.
    ./sync-repo.sh --yes
    
    You'll be asked for your sudo password (required to set group ownership and permissions). To do a trial run just leave off the "--yes"
  4. Once again, be at the top level of the working directory to catch all changes you may have made. Check your changes back into svn:
     svn ci -m "Update java-openjdk to 1.7.0.45" 

***** IMPORTANT *** If this is an urgent change that needs to go into effect immediately, you need to manually force the other mirror to update. Otherwise it will do so on a nightly schedule. Please see how in the next section.

Propagating to mirrors

Yum repo configuration references both available mirrors. If the mirror at MSU is not updated then your packages may not end up being installed. It is automatically updated nightly but sometimes you may need it faster.

Update MSU mirror on msu3.

Log into msu4.msulocal

/exports/vmware/mirror/mirrorumatlas.sh

-- BenMeekhof - 22 Nov 2013
Topic revision: r6 - 11 Jun 2015 - 17:57:44 - BenMeekhof
 

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback