Compiling Squid

Compiling Squid is quite easy: you need the right tools to do the job, though. First, let's go through getting the tools, then you can extract the source code package, include optional Squid components (using the configure command) and then actually compile the distributed code into a binary format.

A word of warning, though: this is the stage where most people run into problems. If you haven't compiled source before, try and follow the next section in order - it shouldn't be too bad. If you don't manage to get Squid running, at least you have gained experience.

Compilation Tools

All GNU utilities mentioned below are avaliable via FTP from the official GNU ftp site or one of it's mirrors. A list of mirrors is available at, or download them directly from

The GNU compiler is only distributed as source (creating a chicken-and-egg problem if you do not have a compiler) you may have to do an Internet search (using one of the standard search engines) to try and find a binary copy of the GNU compiler for your system. The Squid source is distributed in compressed form. First a standard tar file is created. This file is then compressed with the GNU gzip program. To decompress this file you need a copy of gzip. GCC (The Gnu C Compiler) is the recommended compiler: the developers wrote Squid with it, and it is available for almost all systems.

You will also need the make program, of which there is also a GNU version easily available.

If possible, install a C debugger: the GNU debugger (GDB) is available for most platforms. Though a debugger is not necessary for installation, but is very useful in the case of software bugs (as discussed in chapter 13).

Unpacking the Source Archive

Earlier we looked at the tree structure of the /usr/local/squid directory. I suggest extracting the Squid source to the /usr/local/squid/src directory. So, create the directory and copy the downloaded Squid tar.gz file into it.

First let's decompress the file. Some versions of tar can decompress the file in one step, but for compatability's sake we are going to do it in two steps. Decompress the tar file by running gzip -dv squid-version.tar.gz. If all has gone well you should have a file called squid-version.tar in the current directory. To get the files out of the "tarball", run tar xvf squid-version.tar.

Tar automatically puts the files into a subdirectory: something like squid-2.1.PRE2. Change into the extracted directory, and we can start configuring the Squid source.

Compilation options

Squid features are enabled (or disabled) with the configure shell script. Some Squid features have to be specifically enabled when Squid is compiled, which can mean that you have to recompile at a later stage. There are two reasons that a feature can be disabled by default:

You may be wondering why there simply aren't config file options for these less used features. For most of the features there really isn't a reason other than (?minimalisim?). Why have code sitting in the executable that isn't actually used? You can include the features that you might use at some time in the future without detrimental effects (other than a slightly larger binary), so as to avoid having to recompile the Squid source later on.

The configure program also has a second function: with some source code you have to edit a header file which tell the compiler which function calls to use on the system. This very often makes source compilation difficult. With Squid, however, the GNU configure script checks what programs, libraries and function calls are available on your system. This simplifies setup dramatically.

To make configure as generic as possible, it's actually a Bourne Shell /bin/sh script. If you have replaced your /bin/sh shell with a less Posix-capable shell (like ash) you may not be able to run configure. If this is the case you will have to change the first line of the configure script to run the full shell.

all source inclusion options are set with the command './configure option'. On most systems root doesn't have a '.' in their search path for security reasons, so you have to fully specify the path to the binary (hence the '/').

To turn more than one configuration option on at once you simply append each option to the end of the command line. You can, for example, change the prefix install directory and turn Async-IO on with a command like the following (more on what each of these options is for shortly).

 ./configure --prefix=/usr/people/staff/oskar/squid --enable-async-io

Note that only the commonly used configuration options are included here. To get a complete list of options you can run './configure --help'. Many of the resulting options are standard to the GNU configure script that Squid uses, and are used for some things like cross compilation.

If you wish to find out about some of the more obscure options you may have to ask someone on one of the relevant mailing lists, or even read the source code!

Asynchronous IO

Squid 2.0 includes a major performance increase in the form of Async-IO.

It's important to remember that Squid is one processes. In many Internet daemons, more than one copy runs at a time, so if one process is by a system call, it does not effect the other running copies.

Squid is only one process. If the main loop stops running for some reason, all connections are slowed. In all versions of Squid, the main loop uses the select and poll system calls to decide which connections to service. As Squid receives data from the server, it writes the data to disk and to the client.

To write data to disk, a file has to be opened on the cache drive. When lots of clients are opening and closing connections to a busy cache, the main loop has to make lots of calls to open and close network and disk filehandles (note that the word filehandle can refer to both a network connection and an on-disk file). These two functions block the flow of all data through the cache. While waiting for open to return, Squid cannot perform any other functions.

When you enable Async-IO, Squid 2.0 uses threads to open and close filedescriptors. A thread is part of the main Squid program in most ways, except that if it makes use of a blocking system call (such as open), only the thread stops, not the main loop or other threads. Note that there is not one thread per connection.

Using threads to make calls to blocking function calls reduces the latency that a cache adds to each request. (People sometimes worry about the latency that caches add, but if you have a fast enough cache the latency is not an issue - the client sees no noticeable overhead. Network overhead normally outweighs Squid overhead). Async-IO drastically reduces cache overhead when you have a loaded cache.

Unfortunately Posix threads aren't available on all operating systems. This ties your hardware choice into your choice of operating system, since if your operating system does not support threads there may be no choice but to use a faster system, or even to split the load between multiple machines. (? need a table of machines that work ?)

You should probably try and run Squid with Async-IO enabled if you have a few thousand requests per hour. Some systems only support threads properly with a fair amount of initial setup. If your load is low and Async-IO doesn't work straight away you can leave Squid in the default configuration.

Use the --enable-async-io configure option to include the async-io code into Squid.

Running configure

Now that you have decided which options to use, it's time to run configure. Here's an example:

 ./configure --enable-err-language=Bulgarian --prefix=/usr/local

Running ./configure with the options that you have chosen should go smoothly. In the unlikely event that configure returns with an error message, here are some suggestions that may help.

Compiling the Squid Source

Now that you have configured Squid, you need to make the Squid binaries. You should simply have to run make in the extracted source directory, and a binary will be created as src/squid.

cache:/ # cd /usr/local/squid/src/squid-2.2.RELEASE
cache:/usr/local/squid/src/squid-2.2.RELEASE # make

If the compilation fails, it may be because of conflicting configure options as described in the configure section. Follow the same instructions described there to find the offending option. (You should run make clean between configure runs, to ensure that old binaries are removed) As a start, try running configure without any options at all and then see if make completes. If this works, try additional configure options one at a time to see which one causes the problem.

Installing the Squid binary

The make command creates the binary, but doesn't install it.

Running make install creates the /usr/local/squid/bin and /usr/local/squid/etc subdirectories, and copies the binaries and default config files in the appropriate directories. Permissions may not be set correctly, but we will work through all created directories and set them up correctly shortly.

This command also copies the relevant config files into the default directories. The standard config file included with the source is placed in the etc subdirectory, as are the mime.types file and the default Squid MIB file (squid.mib).

If you are upgrading (or reinstalling), make install will overwrite binary files in the bin directory, but will not overwrite your painfully manipulated configuration files. If the destination configuration file exists, make install will instead create a file called filename.default. This allows you to check if useful options have been added by comparing config files.

If all has gone well you should have a fully installed (but unconfigured) Squid system setup.