Increase internet response speed by using a local Squid proxy

There are many reasons to start hosting an outgoing proxy. Mine was slow upstream servers. But if, for whatever reason, you wish to use an outgoing proxy, this blogpost might be helpful.

The steps are as follows:

  1. Install the software (squid on Debian Linux in my case)
  2. Configure the software (squid on Debian Linux)
  3. Configure your various software to accept the proxy intercepting the TLS secure connections
  4. Redirect all traffic through the proxy, with some exceptions if needed. (Websockets are not supported, so you will need to put some exception in for that)

Install

  1. Debian doesn't include tls interception (called ssl-bump in squid) support in the default package. If you want this, you need to install squid-openssl.
    apt install squid-openssl

Configure

  1. Prepare the Certificate Authority (CA)
    mkdir -p /etc/squid/cert/
    cd /etc/squid/cert/
    # This puts the private key and the self-signed certificate in the same file
    openssl req -new -newkey rsa:4096 -sha256 -days 3650 -nodes -x509 -keyout myCA.pem -out myCA.pem
  2. Create a config file /etc/squid/squid.conf
    acl dst_localnet dst 0.0.0.1-0.255.255.255  # RFC 1122 "this" network (LAN)
    acl dst_localnet dst 10.0.0.0/8     # RFC 1918 local private network (LAN)
    acl dst_localnet dst 100.64.0.0/10      # RFC 6598 shared address space (CGN)
    acl dst_localnet dst 169.254.0.0/16     # RFC 3927 link-local (directly plugged) machines
    acl dst_localnet dst 172.16.0.0/12      # RFC 1918 local private network (LAN)
    acl dst_localnet dst 192.168.0.0/16     # RFC 1918 local private network (LAN)
    acl dst_localnet dst 127.0.0.1/8
    acl dst_localnet dst fc00::/7           # RFC 4193 local private network range
    acl dst_localnet dst fe80::/10          # RFC 4291 link-local (directly plugged) machines
    acl SSL_ports port 443
    acl Safe_ports port 80      # http
    acl Safe_ports port 21      # ftp
    acl Safe_ports port 443     # https
    acl Safe_ports port 70      # gopher
    acl Safe_ports port 210     # wais
    acl Safe_ports port 1025-65535  # unregistered ports
    acl Safe_ports port 280     # http-mgmt
    acl Safe_ports port 488     # gss-http
    acl Safe_ports port 591     # filemaker
    acl Safe_ports port 777     # multiling http
    acl CONNECT method CONNECT
    acl step1 at_step SslBump1  # SNI host is discovered here
    acl step2 at_step SslBump2  # Server cert details are discovered here
    acl step3 at_step SslBump3
    acl tunnelips dst "/etc/squid/tunnelips.list"
    acl tunnelsites ssl::server_name_regex -i "/etc/squid/tunnelsites.list"
    http_access deny !Safe_ports
    http_access deny CONNECT !SSL_ports
    http_access allow localhost manager
    http_access deny manager
    include /etc/squid/conf.d/*
    http_access allow localnet
    http_access allow localhost
    http_access deny all
    http_port 3128
    http_port 3129 intercept
    https_port 3131 intercept ssl-bump tls-cert=/etc/squid/cert/myCA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=8MB
    ssl_bump peek DiscoverSNIHost
    ssl_bump splice step2 dst_localnet  # No need to cache local resources
    ssl_bump splice step2 tunnelips  # IP based override
    ssl_bump splice step2 tunnelsites  # domain name based override (better)
    ssl_bump bump  # Bump everything else
    sslcrtd_program /usr/lib/squid/security_file_certgen -s /var/spool/squid/ssl_db -M 4MB
    sslcrtd_children 5
    sslproxy_cert_error deny all
    cache_mem 2500 MB
    memory_cache_mode always
    maximum_object_size_in_memory 5 MB
    coredump_dir /var/spool/squid
    refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
    refresh_pattern -i .(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
    refresh_pattern -i .(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private
    refresh_pattern -i .(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
    refresh_pattern -i .(css|js)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
    refresh_pattern -i .index.(html|htm)$ 0 40% 10080
    refresh_pattern -i .(html|htm)$ 1440 40% 40320
    refresh_pattern . 0 40% 40320
    global_internal_static off
    forwarded_for delete
  3. Create the certificate database by issuing
    /usr/lib/squid/security_file_certgen -c -s /var/spool/squid/ssl_db -M 4MB

    and correct access rights with

    chmod -R proxy:proxy  /var/spool/squid/ssl_db
  4. Restart squid with systemctl restart squid

Configure client software

  1. Prepare the CA certificate for use by browsers.
    # This can be added to browsers
    openssl x509 -in /etc/squid/cert/myCA.pem -outform DER -out /tmp/myCA.der
  2. Add the squid CA to the system wide ca-certificates
    # Copy the CA to the ca-certificates directory
    sudo openssl x509 -in /etc/squid/cert/myCA.pem -out /usr/share/ca-certificates/myCA.crt
  3. Add the line myCA.crt to /etc/ca-certificates.conf and run sudo update-ca-certificates
  4. For firefox (this is what I use), open the settings, find security devices and add /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so as a security device. This means Firefox will also trust the system wide CAs.
  5. For other browsers, you can import the /tmp/myCA.der as a Certificate Authority.

Force all traffic through the proxy

  1. Install packages for firewall management if not already installed: sudo apt install netfilter-persistent iptables-persistent
  2. Configure the firewall rules
    sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -m owner --uid-owner root -j RETURN
    sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -m owner --uid-owner proxy -j RETURN
    sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -m owner --uid-owner root -j RETURN
    sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -m owner --uid-owner proxy -j RETURN
    sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129
    sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3131
  3. Save the rules: sudo netfilter-persistent save

Have a cup of tea

Because you're done. You might need to relogin/reboot for the p11-kit stuff to take effect.

This article is my 10th oldest. It is 845 words long