BigAdmin System Administration Portal
Community Submitted Article
Print-friendly VersionPrint-friendly Version
This content is submitted by a BigAdmin user. It has not been reviewed for technical accuracy by Sun Microsystems, though it may have been lightly edited to improve readability. If you find an error or would like to comment on the article, please contact the submitter or use the comment field at the bottom of the article. Community submissions may not follow Sun trademark guidelines. For information on Sun trademarks, please see http://www.sun.com/suntrademarks/.
 
 

Using Squid as a Robust Transparent Proxy on Linux-Based Systems

By Muhammad Talha

Squid is a high-performance proxy caching server for web clients, supporting FTP, gopher, and HTTP data objects. With Squid, you can make your proxy transparent with WCCP version 1 support so that no configuration is needed at the client end. The Web Cache Communication Protocol (WCCP), developed by Cisco Systems, specifies interactions between one or more routers (or Layer 3 switches) and one or more web caches.

Following are the steps needed to use Squid web proxy cache as a transparent proxy with WCCP v1on a Linux-based system:

  1. Operating System Installation
  2. Patching the Kernel for WCCP v1 Support
  3. Changing and Optimizing Operating System Parameters
  4. Compiling Squid
  5. Configuring Squid
  6. Starting Squid
  7. Troubleshooting

1. Operating System Installation

Operating system installation is not covered in this installation procedure. We use Red Hat Linux as our operating system. Usually, it is recommended to install the OS on a different hard drive than the one with the cache partition.


2. Patching the Kernel for WCCP v1 Support

The following steps are needed:

1. Download the ip_wccp.c from squid-cache.org. Compile ip_wccp.c using the following command:

gcc -D__KERNEL__ -I/usr/src/linux-2.4.18-3/include -Wall -Wstrict-prototypes \
-O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce \
-m386 -DCPU=386 -DMODULE -DMODVERSIONS -include \
/usr/src/linux-2.4.18-3/include/linux/modversions.h -c ip_wccp.c

After this, the compiled ip_wccp.o will be created.

2.Copy ip_wccp.o to /lib/modules/2.4.18-3/kernel/net/ipv4/ip_wccp.o. Then edit /lib/modules/2.4.18-3/modules.dep.

Add the line:

/lib/modules/2.4.18-3/kernel/net/ipv4/ip_wccp.o:
3. Test the module with:
/sbin/modprobe ip_wccp
/sbin/depmod -a -e

This should report no errors.


3. Changing and Optimizing Operating System Parameters

The following steps are needed.

A. Increasing Number of File Descriptors in System

Linux kernel supports an "unlimited" number of open files without patching. However, you still need to take some actions, as the kernel defaults to only allow processes to use up to 1024 file descriptors, and Squid picks up the limit at build time.

1. Edit /usr/include/bits/types.h to define __FD_SETSIZE to at least the number of file descriptors you'd like to support.

2. Before configuring Squid, run ulimit -HSn **** (where **** is the number of file descriptors you need to support). Make sure your script for starting Squid contains the above ulimit command to raise the file descriptor limit.

B. ReiserFS

Simply switching your cache directories to ReiserFS file system can increase Squid performance by about 20 percent. ReiserFS, a journaling file system, comes by default with many Linux distributions, such as Red Hat. We have found ReiserFS significantly faster than ext3 when dealing with thousands of small files.

Here's how to build the ReiserFS file system. First, unmount the desired file system:

#mkreiserfs /dev/sda2

Put following line in /etc/fstab:

/dev/sda2      /cache        reiserfs  notail,noatime   1   2  

     #mount   /cache

C. Enabling IP Forwarding

To enable IP forwarding, use:

echo 1 > /proc/sys/net/ipv4/ip_forward

Put it in /etc/rc.local so it will not be lost after reboot.

D. Enable Transparent HTTP Redirection

You probably have a more sophisticated NetFilter setup. However, there is one line that you need to redirect inbound traffic on port 80 to Squid on port 3128:

iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128

Verify with following command:

iptables -t nat –L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere tcp dpt:http redir ports 3128 
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination 
Chain OUTPUT (policy ACCEPT)
target prot opt source destination

E. TCP Tuning

For servers that are serving up huge numbers of concurrent sessions, there are some TCP options that should probably be enabled. In order to optimize TCP performance for this situation, I would suggest tuning the following parameters:


echo 1024 65000 > /proc/sys/net/ipv4/ip_local_port_range

F. Network Card Setting

Use mii-tool to check speed and duplex setting.

eth0: 100 Mbit, full duplex, link ok
   

G. Tuning Syn Backlog

Set the Syn backlog to a high number so Squid will never be starved due to a kernel-limiting network connection. The default value is 1024.

echo NUMBER  > /proc/sys/net/ipv4/tcp_max_syn_backlog

4. Compiling Squid

First download the latest version of Squid: Squid version 2.5.

Untar the Squid distribution:

#  tar -zxvf squid-2.5.STABLE4.tar.gz

We use the following compile options:

#  ./configure  --prefix=/usr/local/squid --enable-async-io \
                --enable-storeio=ufs,aufs --enable-snmp

Other compilation option can be seen with ./configure –help.

Note: Before running the configure script, be sure to run ulimit -HSn 16384 to raise the number for file descriptors, otherwise Squid will not support more than 1024 file descriptors.

Then use:

# make  

and next

# make install  

5. Configuring Squid

Now after all of these things, you will have a Squid installation that is capable of handling a much higher load than the default compile. But, we still need to do a few more things before we have a web cache running full tilt. No further patching or compiling should be necessary.

The first thing to do is configure Squid's squid.conf file.

The configure file is located at /usr/local/squid/etc/squid.conf.

The important configuration options for performance follow:

cache_mem

Example: cache_mem 32 GB

cache_dir

Example: cache_dir aufs /cache 10240 16 256

Also, the following may be set to improve performance marginally:

half_closed_clients off
maximum_object_size 1024 KB
cache_swap_high 100%
cache_swap_low 80%

The other important things to set up in squid.conf are as follows:

visible_hostname        CacheCS

(This is the host name you advertise for the cache.)

cache_mgr    webmaster@yourdomain.com

(Put here the email address of the manager.)

cache_effective_user    nobody

Note: If you must start Squid as root, find a safe user and group to run as after startup (typically "nobody" and "nogroup"). Do not use "root", for security reasons.

Also, set the access control list that will be used to allow your clients access:

acl myacl src 192.168.1.0/255.255.255.0          # IPs your clients              
http_access allow myacl

Parameters Needed to Run Squid as Transparent Proxy

Configure the /usr/local/squid/etc/squid.conf file to run as transparent proxy. The following changes should be made:

httpd_accel_host virtual
httpd_accel_port 80

httpd_accel_with_proxy on

httpd_accel_uses_host_header on

wccp_router 

Example:

wccp_router  192.168.1.1

6. Starting Squid

Use this command to create cache directories:

 
/usr/loca/squid/sbin/squid  -z

# ulimit -HSn 16384

# /usr/local/squid/sbin/squid

7. Troubleshooting

Read the Squid log files in case of a problem. The file locations are as follows:

/var/log/squid/access.log
/var/log/squid/cache.log

You can also run Squid in debug mode for testing:

/usr/local/squid/sbin/squid -NCd1

References

We strongly recommend that you read the excellent Squid Frequently Asked Questions, especially the information and related links available from the section on 17.11 WCCP - Web Cache Coordination Protocol. Further, we advise you to read Cisco's WCCP documentation to get a better understanding of the protocol.

 


The information and links on this page have been provided by a BigAdmin user. The submitter is solely responsible for such information and links. Sun is not responsible for the availability of external sites or resources, and does not endorse and is not responsible or liable for any content, advertising, products, or other materials on or available from such sites or resources. Sun will not be responsible or liable, directly or indirectly, for any actual or alleged damage or loss caused by or in connection with use of or reliance on the information posted here, or goods or services available on or through any external site or resource.


BigAdmin