Previous | Contents | Next

Chapter 4: Advanced Server Configuration

4.1 Vocabulary

This chapter introduces some terminology which is needed to understand the functionality of apt-cacher-ng; it's recommended to understand it before continuing with the advanced configuration.

4.2 Configuration file types

By default, the /etc/apt-cacher-ng directory contains all config files, HTML page templates, the stylesheet and other text-based support files used by apt-cacher-ng. The contents may vary depending on the installation of apt-cacher-ng, refer to the package documentation for Linux Distribution packages.

The main configuration files are located in one single directory. Three types files are recognized (by suffix) and interpreted by the following schema.

*.conf files are assumed to contain configuration directives in the form of "key: value" pairs. The package comes with a commented example configuration file. apt-cacher-ng reads all files matching *.conf in alphabetical order and merges the contents. For options documentation, see commented example file shipped with apt-cacher-ng (conf/ directory in original source).

Lists of remote package repositories (i.e. mirrors) can be specified in one of two formats:

Apt-cacher-ng autodetects the format of the list.

4.3 Repositories and URL mapping

IMPORTANT: it should be clear from the beginning that the URL remapping is an optional feature. You do not have to use it, but you can. It is not a strong prerequisite for apt-cacher-ng's work and in trouble, this feature can be disabled for debugging purposes. On the other hand, it's also possible to forbid users to download from non-remapped locations using the ForceManaged option.

4.3.1 Basic use of URL remapping

URL remapping has several uses. First, it allows your apt-cacher-ng based proxy to masquerade as a Debian or Ubuntu mirror; it can be made to appear to contain the same directory structure and files the client would find on a real mirror. Instead of referencing a real mirror, the client's sources.list file would contain a line like deb http://name.of.proxy.host/debian stable main. Your backend definition in apt-cacher-ng's configuration determines which actual Debian mirror will be used to fetch the files.

This keeps the client configuration clean, simple and straightforward. Without URL remapping, the sources.list file on the client would have to read something like deb http://name.of.proxy.host/ftp.de.debian.org/debian stable main or alternatively additional setting of HTTP proxy for APT would be required. Note that, depending on the remapping configuration, it is possible that apt-cacher-ng will actually use a mirror other than the one requested by the client (see below).

A second use is obscuring the real location of a repository from your clients. This allows you to change the real location easily, without having to modify the configuration of all clients; you just have to edit the backend configuration in one single place.

What happens behind the scenes can be demonstrated in few examples. In all examples below, fetching from http://proxy.host/some-url is the same as fetching from http://some-url having APT-Configuration with proxy-host as HTTP proxy. The exact meaning of the configuration directive(s) is explained below.

Example 1, no remapping is used:

When a client asks for http://proxy.host/ftp.de.debian.org/debian/something, apt-cacher-ng fetches http://ftp.de.debian.org/debian/something.

Example 2:

Remap-debrep: ftp.de.debian.org/debian ftp.at.debian.org/debian

This is a simple case. When a client asks for http://proxy.host/ftp.de.debian.org/debian/something, apt-cacher-ng fetches http://ftp.de.debian.org/debian/something. When another client asks for http://proxy.host/ftp.at.debian.org/debian/something (i.e. the same "something" from another mirror) then the file is delivered from the local cache (if cached, otherwise it would be fetched from ftp.at.debian.org). Which means, apt-cacher-ng uses the file which was originally downloaded from another mirror; they are considered equal as advised by the Remap-... directive above.

Example(s) 3, slightly more complex:

Remap-ubuntu: /ubuntu ; http://us.archive.ubuntu.com/ubuntu Remap-medibuntu: /medibuntu ; http://packages.medibuntu.org

These two examples specify trivial mappings. Whenever a client asks for http://proxy.host/ubuntu/foo, apt-cacher-ng will retreive http://us.archive.ubuntu.com/ubuntu/foo. When a client wants http://proxy.host/medibuntu/bar, apt-cacher-ng fetches http://packages.medibuntu.org/bar. This is still relatively simple.

This game can be continued: adding a path before semicolon adds more URL variants "visible" by the client (see Example 2), and it's also possible to add multiple backend URLs (after semicolon). When multiple backends are set, they are probed in the specified order until the download succeeds.

Side note: "success" means a started download or a non-critical failure in this context, i.e. a "404 File not found" status is not a critical failure, since APT (client) sometimes looks remote file's existence and reacts to such status response accordingly.

4.3.2 Details of remapping syntax

A "Remap-" line contains three pieces of information:

1. A "repository name": unique internal identifier for the remapping ruleset specified by the line; in the examples above, "ubuntu" and "medibuntu". These identifiers have no meaning; we could just as well have chosen "blarg" and "deedledum". Restrictions: the identifier must be valid as a filename; it mustn't begin with an underscore; and it mustn't contain whitespace. It should also not be "apt-cacher" for historical reasons.

This is the part of the line between "Remap-" and the first colon (':').

2. A list of path (first URL parts) to apply remapping to. This is the part of the line between the first colon and the first semicolon (';'). The simple way is specifying the URL paths directly, separated by spaces. To keep the configuration line short with more than few mirrors, they can be listed in a separate file and this list file can be specified in the Remap line instead of the URL path list. I.e. you can use "file:filename" to read a list of pathnames from "filename". Filename can be absolute, or relative to the configuration directory. You can mix files and literal pathnames in the list. Pathnames mustn't contain wildcards, but "file:" specifications may. apt-cacher-ng can decompress .gz and .bz2 files. A leading "http://" in pathnames is replaced by a single "/".

3. A list of backend servers to fetch the files from, whenever clients request a file from a remapped path. Backend lists are specified the same way as path lists.

Complex example:

Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian

This causes apt-cacher-ng to read a list of debian mirrors from all files matching "deb_mirror*.gz" and construct a list of pathnames consisting of entries like /ftp.de.debian.org/debian /ftp.kfki.hu/linux/debian and so on. The last element of the list will be simply /debian. All these paths will be remapped to paths read from backends_debian, which can contain the base URL of a single Debian mirror, or several URLs, one per line. The same RFC822-like format deb_mirrors.gz is supplied in is also supported, so you could just make backends_debian a subset of deb_mirrors.gz; however the use of good local mirrors is recommended.

Client sources.list files could specify either of the following and still actually use the mirrors from backends_debian:

Side notes:


Comments to blade@debian.org
[Eduard Bloch, Tue, 20 Nov 2007 00:03:24 +0100]