MitMing SSL/TLS Connections

There are many reasons for intercepting SSL/TLS connections. For companies, it's the basis for Data Loss Prevention (DLP) programs. For pentesters, it's to understand and attack an application. And for me, it's a way to implement a new feature on Gauntlet.io that will allow users to add custom headers to scanners' requests.

With custom headers, you will be able to override an existing header or add a new one. There are many reasons why users may want to do this, such as:

1. Add authentication headers (Basic or JWT, for example)
2. Add an extra header to track all requests performed by Gauntlet
3. Add additional headers that are necessary for the application to work

That said, how can we do this?

1. We could rely on each scanner to add a custom header, but it's unlikely that scanners support such a feature.
2. We can add a layer on top of scanner requests that will intercept and modify them, which sounds much more plausible.

To intercept those requests, we can use a proxy, a program specifically built to intercept requests and responses, such as squid. However, squid is focused on content filtering and caching rather than request/response manipulation, although it can add headers to a certain extent.

The problem is that squid doesn't support SSL interception out of the box. You need to recompile it with certain flags. While looking for the source on the official website, I was surprised to find that the squid version available in the Ubuntu package repository is much lower than the latest version (from 3.0 to 3.5).

Okay, I downloaded the latest version, added flags, compiled it, and started squid, passing https_port 3129 in the configuration file. However, this port didn't show up no matter what. Only the http_port 3128, which is the port to intercept HTTP traffic, appeared. I could try again, check the flags and everything else, but the documentation for squid, although popular software, is very messy compared to documentation for programming languages and APIs. It's weird because squid has been popular for a long time, but it's still hard to figure out some things.

And then I realized that there is a proxy specifically built to manipulate requests and responses. That's what I was looking for in the first place. That's mitmproxy, a program built in Python.

Just a few commands to get it up and running:

sudo apt-get install python-pip python-dev libffi-dev libssl-dev libxml2-dev libxslt1-dev libjpeg8-dev zlib1g-dev
sudo pip install mitmproxy
mitmdump
# mitmproxy is an interactive console
# mitmdump is basically "mitm capabilities for tcpdump" that comes with mitmproxy

And to test:

curl -x http://127.0.0.1:8080 http://some-http-site.com 
# -x points to the proxy

What about HTTPS sites? There is a catch.

HTTPS connections are encrypted end-to-end, so the only way to intercept them is to generate a certificate on the MITM software that is trusted by a CA present on the requester's device. Since we're using the same machine, we have to make our operating system trust the CA cert from mitmproxy.

# run mitmproxy once to generate the file below (mitmproxy-ca-cert.cer) and exit
mitmproxy

# add cert to trusted certs on your O.S.
cp ~/.mitmproxy/mitmproxy-ca-cert.cer /usr/local/share/ca-certificates/mitmproxy-ca-cert.crt
/usr/sbin/update-ca-certificates

And it's done. Now we can try HTTPS websites:

curl -x https://127.0.0.1:8080 https://dadario.com.br

Okay, but how do we manipulate requests?

We can simply run mitmdump, passing a script file, e.g., mitmdump -s add_headers.py. The content of add_headers.py is as follows:

def request(context, flow):
    flow.request.headers["newheader"] = "foo"