Categories
  • Linux 7
  • Automation 1
  • Life 2
Home automation, Linux, tinkering
  • Posts
  • Categories
  • Increase internet response speed by using a local Squid proxy

    There are many reasons to start hosting an outgoing proxy. Mine was slow upstream servers. But if, for whatever reason, you wish to use an outgoing proxy, this blogpost might be helpful.

    The steps are as follows:

    1. Install the software (squid on Debian Linux in my case)
    2. Configure the software (squid on Debian Linux)
    3. Configure your various software to accept the proxy intercepting the TLS secure connections
    4. Redirect all traffic through the proxy, with some exceptions if needed. (Websockets are not supported, so you will need to put some exception in for that)

    Install

    1. Debian doesn't include tls interception (called ssl-bump in squid) support in the default package. If you want this, you need to install squid-openssl.
      apt install squid-openssl

    Configure

    1. Prepare the Certificate Authority (CA)
      mkdir -p /etc/squid/cert/
      cd /etc/squid/cert/
      # This puts the private key and the self-signed certificate in the same file
      openssl req -new -newkey rsa:4096 -sha256 -days 3650 -nodes -x509 -keyout myCA.pem -out myCA.pem
    2. Create a config file /etc/squid/squid.conf
      workers 4
      acl localnet dst 0.0.0.1-0.255.255.255  # RFC 1122 "this" network (LAN)
      acl localnet dst 10.0.0.0/8     # RFC 1918 local private network (LAN)
      acl localnet dst 100.64.0.0/10      # RFC 6598 shared address space (CGN)
      acl localnet dst 169.254.0.0/16     # RFC 3927 link-local (directly plugged) machines
      acl localnet dst 172.16.0.0/12      # RFC 1918 local private network (LAN)
      acl localnet dst 192.168.0.0/16     # RFC 1918 local private network (LAN)
      acl localnet dst fc00::/7           # RFC 4193 local private network range
      acl localnet dst fe80::/10          # RFC 4291 link-local (directly plugged) machines
      acl localnet dst 2a02:a44e:5504::/56
      acl tunnelips dst "/etc/squid/tunnelips.list"
      acl SSL_ports port 443
      acl SSL_ports port 8443
      acl SSL_ports port 8843
      acl SSL_ports port 5281
      acl SSL_ports port 9091
      acl Safe_ports port 80      # http
      acl Safe_ports port 21      # ftp
      acl Safe_ports port 443     # https
      acl Safe_ports port 8443    # https
      acl Safe_ports port 8843    # https
      acl Safe_ports port 70      # gopher
      acl Safe_ports port 210     # wais
      acl Safe_ports port 1025-65535  # unregistered ports
      acl Safe_ports port 280     # http-mgmt
      acl Safe_ports port 488     # gss-http
      acl Safe_ports port 591     # filemaker
      acl Safe_ports port 777     # multiling http
      acl Safe_ports port 9091    # transmission
      acl CONNECT method CONNECT
      acl step1 at_step SslBump1
      acl step2 at_step SslBump2
      acl step3 at_step SslBump3
      acl tunnelsites ssl::server_name_regex -i "/etc/squid/tunnelsites.list"
      http_access deny !Safe_ports
      http_access deny CONNECT !SSL_ports
      http_access allow localhost manager
      http_access deny manager
      include /etc/squid/conf.d/*
      http_access allow localnet
      http_access allow localhost
      http_access allow all
      http_access deny all
      on_unsupported_protocol tunnel all
      http_port 3128 ssl-bump tls-cert=/etc/squid/cert/SQUID_CA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=8MB
      http_port 3129 intercept
      https_port 3131 intercept ssl-bump tls-cert=/etc/squid/cert/SQUID_CA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=8MB
      https_port 443 intercept ssl-bump tls-cert=/etc/squid/cert/SQUID_CA.pem generate-host-certificates=on dynamic_cert_mem_cache_size=8MB
      tls_outgoing_options min-version=1.2 cafile=/etc/ssl/certs/ca-certificates.crt
      ssl_bump splice tunnelsites
      ssl_bump splice step1 localnet
      ssl_bump splice step1 tunnelips
      ssl_bump peek step1
      ssl_bump splice tunnelsites
      ssl_bump bump
      sslcrtd_program /usr/lib/squid/security_file_certgen -s /var/spool/squid/ssl_db -M 40MB
      sslcrtd_children 16 startup=5 idle=3
      sslproxy_cert_error deny all
      server_idle_pconn_timeout 10 minutes
      cache_mem 256 MB
      memory_cache_mode always
      maximum_object_size_in_memory 5 MB
      logformat bumpsquid      %ts.%03tu %6tr %>a %Ss/%03>Hs %ssl::bump_mode %<st %rm %ru %[un %Sh/%<a %mt
      access_log daemon:/var/log/squid/access.log bumpsquid
      coredump_dir /var/spool/squid
      refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
      refresh_pattern -i .(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern -i gstatic\.com 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern -i .(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern -i .(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff|InRelease|Packages.gz)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern -i .(css|js)$ 10080 90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern -i .index.(html|htm)$ 0 40% 10080 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern -i .(html|htm)$ 1440 40% 40320 override-expire ignore-no-cache ignore-no-store ignore-private ignore-auth
      refresh_pattern . 0 40% 40320
      shutdown_lifetime 1 seconds
      global_internal_static off
      dns_v4_first on
      forwarded_for delete
      pipeline_prefetch on
      max_filedesc 204800
      fqdncache_size 2048
      ipcache_size 2048
      ipcache_low 95
      ipcache_high 98
      cache_swap_low 95
      cache_swap_low 98
      quick_abort_min 0
      quick_abort_max 0
      quick_abort_pct 95
      range_offset_limit -1
      request_header_max_size 200 KB
      reply_header_max_size 200 KB
      memory_pools off
      buffered_logs off
      log_icp_queries off
      logfile_rotate 1
      icp_hit_stale on
      query_icmp off
      reload_into_ims on
      negative_ttl 2 minutes
      vary_ignore_expire on
      half_closed_clients off
      high_page_fault_warning 2
      nonhierarchical_direct on
      prefer_direct off
      cachemgr_passwd none all
      client_db on
      forwarded_for on
      via on
      max_stale 1 month
    3. Create the certificate database by issuing
      /usr/lib/squid/security_file_certgen -c -s /var/spool/squid/ssl_db -M 4MB

      and correct access rights with

      chmod -R proxy:proxy  /var/spool/squid/ssl_db
    4. Restart squid with systemctl restart squid

    Configure client software

    1. Prepare the CA certificate for use by browsers.
      # This can be added to browsers
      openssl x509 -in /etc/squid/cert/myCA.pem -outform DER -out /tmp/myCA.der
    2. Add the squid CA to the system wide ca-certificates
      # Copy the CA to the ca-certificates directory
      sudo openssl x509 -in /etc/squid/cert/myCA.pem -out /usr/share/ca-certificates/myCA.crt
    3. Add the line myCA.crt to /etc/ca-certificates.conf and run sudo update-ca-certificates
    4. For firefox (this is what I use), open the settings, find security devices and add /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so as a security device. This means Firefox will also trust the system wide CAs.
    5. For other browsers, you can import the /tmp/myCA.der as a Certificate Authority.

    Force all traffic through the proxy

    1. Install packages for firewall management if not already installed: sudo apt install netfilter-persistent iptables-persistent
    2. Configure the firewall rules
      sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -m owner --uid-owner root -j RETURN
      sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -m owner --uid-owner proxy -j RETURN
      sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -m owner --uid-owner root -j RETURN
      sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -m owner --uid-owner proxy -j RETURN
      sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129
      sudo iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3131
    3. Save the rules: sudo netfilter-persistent save

    Have a cup of tea

    Because you're done. You might need to relogin/reboot for the p11-kit stuff to take effect.

    Posted 2 years ago by Vincent.
  • etherape wireshark ping and other tools that need sudo

  • distro-sync for Debian

  • uBlock Origin: Block certain website from appearing in search results

  • Network Manager Auto VPN on wireless

  • wireguard dkms raspberry segfault

← Previous
© 2023 Home automation, Linux, tinkering. All rights reserved.
  • RSS
  • @idiot
  • Admin area
  • Home