mitmproxy: X-Ray Vision for Web Developers

Last week I found myself needing to debug a PHP upload error. The problem: PHP was reporting a UPLOAD_ERR_PARTIAL without giving specifics. For starters, I was curious what data was being uploaded. This turned out to be trickier to discover than I thought.

First, I tried using the file_get_contents('php://input') trick to grab the bytes being uploaded. This doesn't work, because as the manual reports, php://input is not available with enctype="multipart/form-data". In other words: you're out luck if you're dealing with file uploads.

Undeterred, I decided to move up the stack a bit. If PHP wasn't going to reveal the raw data, then I'd simply ask Apache to log this information for me. This lead me to try mod_dumpio, mod_dumpost and mod_security. mod_security was the closest to a viable solution, as it didn't just log the data but also gave me the option of extracting and storing the data being uploaded. The problem with mod_security is its complexity; it does so much. I found this post pointed me in the right direction to getting mod_security at least partially configured. I still wasn't happy with this solution.

I decided to take a step even further back: could I run some sort of proxy server that could sit between my browser and Apache?

I can, and I did! I give you the well named: Man-in-the-Middle Proxy, or mitmproxy for short.

I was skeptical about how much work setting up mitmproxy would be. I cheated and grabbed a binary download for linux. I then made a single tweak to apache: instead of listening on port 80, I changed it to listen on port 8080.

# Listen 80
Listen 8080

I restarted apache and then kicked off mitmproxy from the command line like so:

$ sudo ./mitmproxy -p 80 \
        --set keep_host_header=true \
        --mode reverse:http://localhost:8080

I pointed my web browser to the usual location and started accessing my web app. To my surprise, everything Just Worked. Flipping back to the mitmproxy screen, I saw my browser requests streaming in. At this point I started to appreciate the interactive side of mitmproxy. I realized I could choose to inspect individual requests on the fly, including those with multi-part file uploads. Finally, I could easily inspect these mysterious failed PHP uploads.

I've used Proxy Servers in the past to aid with debugging, but mitmproxy takes this to a new level. It's definitely going into my bag of tricks for future debugging sessions.