There are several reasons you may want to create a local repository. The first is that you want to save on bandwidth if you have multiple Ubuntu machines to update. For example if you had 25 Ubuntu machines that all needed updating at least once a week, you would significantly save bandwidth because you could do all but the repository locally. Most organizations have decent bandwidth for their network gateways but this bandwidth is a precious commodity that needs to be used wisely. Many organizations still have routers with 10MB or 100MB limits at the gateway but 1 GB network connections internally so bandwidth could be better used internally. The second reason for creating your own repository is that you can control what applications are loaded on your internal Ubuntu machines. You can remove any applications your organization does not want to use on the local network from the repository that updates the machines. Even better, you can create a test box and test applications and versions before you allow them to roll out into your network assuring security and stability.
apt-get install apt-mirror
/etc/apt/mirror.list
############# config ##################
#
# set base_path /var/spool/apt-mirror
#
# if you change the base path you must create the directories below with write privileges
#
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch <running host architecture>
# set postmirror_script $var_path/postmirror.sh
set run_postmirror 0
set nthreads 20
set _tilde 0
#
############# end config ##############
deb http://archive.ubuntu.com/ubuntu karmic main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu karmic-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu karmic-updates main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu karmic-proposed main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu karmic-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu karmic main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu karmic-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu karmic-updates main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu karmic-proposed main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu karmic-backports main restricted universe multiverse
clean http://archive.ubuntu.com/ubuntu
Need help configuring an Ubuntu Server….take a look at the Online Ubuntu Server Course.
Files in apt-mirror
The main configuration file is /etc/apt/mirror.list which contains a list of files that you should be aware of in case you have any issues.
/etc/cron.d/apt-mirror
This is the cron configuration template which you can modify. The example which is not active as it is commented out, starts a cron job at 4 AM each day and sends the activity into the log as listed. Some administrators may want to only update once a week and others will prefer twice a day to be up to date.
#
# Regular cron jobs for the apt-mirror package
#
#0 4 * * * apt-mirror /usr/bin/apt-mirror > /var/spool/apt-mirror/var/cron.log
/var/spool/apt-mirror/mirror
This is the location of the mirrors that you create. Be aware, that the /var directory can fill up and these mirrors will take 15-20 GB each so verify you have the space to set them up.
/var/spool/apt-mirror/skel
The location for the indexes which will list the applications that are available.
/var/spool/apt-mirror/var
The log files, MD5 sums, URLS and clean.sh are placed here as you can see below.
ALL archive-log.15 archive-log.5 archive-urls.11 archive-urls.19 archive-urls.9 index-log.6 MD5
archive-log.0 archive-log.16 archive-log.6 archive-urls.12 archive-urls.2 clean.sh index-urls.0 NEW
archive-log.1 archive-log.17 archive-log.7 archive-urls.13 archive-urls.3 index-log.0 index-urls.1 postmirror.sh
archive-log.10 archive-log.18 archive-log.8 archive-urls.14 archive-urls.4 index-log.1 index-urls.2
archive-log.11 archive-log.19 archive-log.9 archive-urls.15 archive-urls.5 index-log.2 index-urls.3
archive-log.12 archive-log.2 archive-urls.0 archive-urls.16 archive-urls.6 index-log.3 index-urls.4
archive-log.13 archive-log.3 archive-urls.1 archive-urls.17 archive-urls.7 index-log.4 index-urls.5
archive-log.14 archive-log.4 archive-urls.10 archive-urls.18 archive-urls.8 index-log.5 index-urls.6
Download Repositories
Once you have your set up apt-mirror you can start your download of the repository with this command.
apt-mirror /etc/apt/mirror.list
Downloading 7 index files using 7 threads…
Begin time: Tue Jan 19 20:21:07 2010
[7]… [6]… [5]… [4]… [3]… [2]… [1]… [0]…
End time: Tue Jan 19 20:23:17 2010
Proceed indexes: [P]
6.6 GiB will be downloaded into archive.
Downloading 6612 archive files using 20 threads…
Begin time: Tue Jan 19 20:23:19 2010
The apt-mirror provides multithreaded downloads and allows for multiple architectures for your repositories. As the download progresses you will see applications saved and space being used.
ls mirror/archive.ubuntu.com/ubuntu/pool/main/
a b d f g h k l libc libm libx m o p r t u
Link Repository to Web Interface
Because apt-mirror saves everything in /var/spool/apt-mirror you need to create a link to make it available to users as the /var/spool is not available to the public. The first link created provides a link to the Packages.gz which lists the available packages in the repository. This is the line you will place in the /etc/apt/sources.list for the client to access this list.
sudo ln -s /var/spool/apt-mirror/skel/archive.ubuntu.com/ubuntu/ /var/www/ubuntu
The additional symlinks provide access to the public to individual applications to download. Of course the symlinks are representative of the repositories you will use.
sudo ln -s /var/spool/apt-mirror/mirror/archive.ubuntu.com /var/www/archive-ubuntu
sudo ln -s /var/spool/apt-mirror/mirror/archive.canonical.com /var/www/archive-canonical
sudo ln -s /var/spool/apt-mirror/mirror/security.ubuntu.com /var/www/security-ubuntu
Lock File Problem
There is a bug in apt-mirror so that it runs fine the first time but then it does not remove the lock file so it will not run the second time, telling you it is already running. If you see that remove the lock file and it will work fine.
sudo rm /var/spool/apt-mirror/var/apt-mirror.lock
Client Set Up for apt
You will need to modify the /etc/apt/sources.list file in order to have the client connect only to your local repository.
deb http://192.168.5.96/ubuntu/ karmic main
Note that the example above has only the main repository installed so as to provide an evaluation of the set up. Remember, this takes a lot of space for these repositories.
Now you can update and then use apt-get to install programs.
sudo apt-get update
Get:1 http://192.168.5.96 karmic Release.gpg [189B]
Ign http://192.168.5.96 karmic/main Translation-en_US
Get:2 http://192.168.5.96 karmic Release [65.9kB]
Get:3 http://192.168.5.96 karmic/main Packages [1,353kB]
Fetched 1,419kB in 1s (1,322kB/s)
Reading package lists… Done
Here is an example of a website address set up for a local repository.
http://192.168.5.96/ubuntu/dists/karmic/main/binary-i386/
If you want to download manual individual packages you can use this URL.
http://192.168.5.96/archive-ubuntu/ubuntu/pool/main/
{ 4 comments }
Cool! Not that I will have any use for this at the moment (only a private user), but its really amazing to see how powerful Ubuntu is on administrative tasks in general. It doesn’t even seem very hard, as long as you have the basics under control.
Thanks
Great article! I just wanted to point out something similar if you only want to download packages once even if they are to be installed on more than 1 computer. Also, unlike apt-mirror, you don’t download the entire repository, instead you download the packages you actually use. It’s called apt-cacher. It’s very easy to use and only has to be installed on 1 computer. You then change your sources.list file on all your other computers to point at the one running apt-cacher and you’re done. It works great, only downloads packages you will actually use, and saves bandwidth.
A nice to know but.. I just update one machine to test out the patches, then rsync /var/cache/apt back to a “spray” server and then rsync to all the other machines and then run apt-get upgrade. Beauty of *nix many ways to get your fix.
Great! I’m tired have to open and close my DVD tray just to install 20 packages. Hopefully this tutorial will solve it.
{ 1 trackback }