See how facelessuser compares to other vendors in security performance
Summary
The CSS selector parser in soupsieve (the CSS selector engine for Beautiful Soup 4) contains a regular expression vulnerable to catastrophic backtracking. When processing an attribute selector with an unterminated quoted value, the VALUE regex pattern in cssparser.py enters exponential backtracking. A payload of only 300 bytes causes the regex engine to hang for over 3 seconds, enabling a trivial Regular Expression Denial of Service (ReDoS) attack.
To be completely transparent, AI tools helped surface this issue. However, this was independently reproduced and carefully validated.
Any application that passes untrusted CSS selector strings to soupsieve.compile() or Beautiful Soup's .select() / .selectone() is affected.
Details
Affected code: soupsieve/cssparser.py, line ~121 - REVALUES / VALUE regex pattern
The soupsieve CSS parser uses a compiled regular expression to tokenise attribute selector values. This pattern matches both quoted strings ("value" or 'value') and unquoted identifiers. The regex contains alternation branches for:
1. Double-quoted strings: "[^"\\](?:\\.[^"\\])" 2. Single-quoted strings: '[^'\\](?:\\.[^'\\])' 3. Unquoted identifiers
When an attribute selector contains an unterminated quoted value - e.g., [a="xxxx... (opening " but no closing ") -” the regex engine attempts to match the quoted-string branch. After that branch fails (no closing quote), the engine backtracks and attempts to match the remaining input against subsequent alternation branches and parent patterns. The structure of the pattern causes catastrophic backtracking where the number of backtracking steps grows exponentially with the length of the content between the opening quote and the end of the string.
Root cause: The regex pattern does not anchor or guard against the case where a quoted string is never terminated. The overlapping character classes across alternation branches create exponential backtracking when the quoted-string branch fails on long input.
Key characteristics: - Input size: Only 300 bytes are needed to trigger a >3 second hang - Amplification: Each additional character approximately doubles the backtracking time - No memory impact: The attack consumes CPU only (regex backtracking is compute-bound)
Proof of Concept
python import time import soupsieve as sv
PAYLOADLEN = 300
Control: well-formed selector with terminated quote (completes instantly) wellformed = '[a="' + ('x' PAYLOADLEN) + '"]' start = time.perfcounter() try: sv.compile(wellformed) except Exception: pass controltime = time.perfcounter() - start print(f"Well-formed selector ({len(wellformed)} bytes): {controltime:.4f}s")
Exploit: unterminated quote triggers catastrophic regex backtracking malformed = '[a="' + ('x' PAYLOADLEN) start = time.perfcounter() try: sv.compile(malformed) # WARNING: This will hang for >3 seconds except Exception: pass exploittime = time.perfcounter() - start print(f"Malformed selector ({len(malformed)} bytes): {exploittime:.4f}s")
slowdown = exploittime / max(controltime, 1e-9) print(f"Slowdown: {slowdown:.0f}x")
Expected output: Well-formed selector (306 bytes): ~0.001s Malformed selector (304 bytes): >3.0s (may need to be killed) Slowdown: >3000x NOTE: On some systems the malformed selector may hang indefinitely. Use a timeout mechanism (signal.alarm, threading.Timer) when testing.
Safe testing variant with timeout:
python import signal import soupsieve as sv
def timeouthandler(signum, frame): raise TimeoutError("ReDoS confirmed: regex backtracking exceeded timeout")
PAYLOADLEN = 300 malformed = '[a="' + ('x' PAYLOADLEN)
signal.signal(signal.SIGALRM, timeouthandler) signal.alarm(3) # 3-second timeout
try: sv.compile(malformed) print("Selector compiled (not vulnerable)") except TimeoutError as e: print(f"VULNERABLE: {e}") except Exception as e: print(f"Other error: {e}") finally: signal.alarm(0) # Cancel the alarm
Impact
Severity: High
An attacker can cause CPU exhaustion on any server-side Python application that compiles user-supplied CSS selectors via soupsieve. The attack is particularly dangerous because:
1. Tiny payload: Only 300 bytes are needed - well within typical URL parameter, form field, or API request limits 2. No special characters: The payload consists entirely of printable ASCII characters ([a="xxx...) 3. Exponential scaling: Each additional byte approximately doubles the backtracking time, making the attack easily tuneable 4. Thread blocking: The regex engine blocks the calling thread with no opportunity for interruption (except via OS signals)
| Parameter | Value | |---|---| | Input size | 300 bytes | | CPU time consumed | >3 seconds (exponential with payload length) | | Memory consumed | Negligible (CPU-only attack) | | Authentication required | None | | User interaction required | None |
Deployment impact: In threaded or async web applications, a single malicious request blocks a worker thread for the duration of the backtracking. An attacker can submit multiple concurrent requests to exhaust all available workers, causing complete service denial. The small payload size makes the attack easy to deliver and difficult to detect via request size limits.
Downstream exposure: soupsieve is an automatic dependency of beautifulsoup4, one of the most widely installed Python packages. Any web application, API, or service that accepts CSS selectors from users is potentially affected.
---
Credit
The vulnerability was discovered by a security research team from the University of Sydney, whose focus is detecting open source software vulnerabilities. Liyi Zhou: https://lzhou1110.github.io/ Ziyue Wang: https://zyy0530.github.io/ Strick: https://str1ckl4nd.github.io/ Maurice: https://maurice.busystar.org/ Chenchen Yu: https://7thparkk.github.io/
Summary
The CSS selector parser in soupsieve (the CSS selector engine for Beautiful Soup 4) allocates unbounded memory when compiling large comma-separated selector lists. An attacker who can supply a crafted CSS selector string to soupsieve.compile() or Beautiful Soup's .select() / .selectone() can cause the application to allocate hundreds of megabytes of heap memory from a relatively small input, leading to memory exhaustion and denial of service.
To be completely transparent, AI tools helped surface this issue. However, it was independently reproduced and carefully validated. Researchers follow responsible disclosure practices and originally shared this report privately.
A 500 KB selector string triggers allocation of approximately 244 MB of heap memory - a 488x— amplification ratio.
Details
Affected code: soupsieve/cssparser.py, lines ~204, 925, 1106
The soupsieve CSS parser splits comma-separated selector lists and creates one CSSSelector object per list item. Each CSSSelector object contains parsed selector data structures including SelectorList, Selector, and associated tag/attribute/pseudo-class metadata.
When a selector string such as a,a,a,... (with 250,000 comma-separated items) is passed to sv.compile(), the parser:
1. Tokenises the entire string and identifies each comma-delimited segment (line ~1106) 2. Parses each segment into a full Selector object with all associated metadata (line ~925) 3. Stores all parsed selectors in a SelectorList (line ~204)
Root cause: No limit is enforced on the number of selectors in a comma-separated list. The parser will attempt to parse and store an arbitrary number of selectors, with each selector object consuming approximately 976 bytes of heap memory. The total allocation scales linearly with the number of list items, but the amplification ratio (output memory / input bytes) is extremely high because each single-character selector like a expands into a complex object graph.
Attack surface: Any application that passes user-supplied CSS selectors to soupsieve.compile() or Beautiful Soup's .select() / .selectone().
Proof of Concept
python import tracemalloc import soupsieve as sv
tracemalloc.start()
Build a 500 KB selector string: "a,a,a,...,a" (250,000 items) count = 250000 selector = ",".join("a" for in range(count)) print(f"Selector string size: {len(selector):,} bytes ({len(selector) / 1024:.0f} KB)")
Compile the selector — this allocates ~244 MB compiled = sv.compile(selector)
current, peak = tracemalloc.gettracedmemory() tracemalloc.stop()
print(f"Compiled selector count: {len(compiled.selectors):,}") print(f"Current memory: {current / 1024 / 1024:.1f} MB") print(f"Peak memory: {peak / 1024 / 1024:.1f} MB") print(f"Amplification ratio: {peak / len(selector):.0f}x")
Expected output: Selector string size: 499,999 bytes (488 KB) Compiled selector count: 250,000 Current memory: ~244 MB Peak memory: ~244 MB Amplification ratio: ~488x
Impact
Severity: High
An attacker can exhaust available memory on any server-side Python application that compiles user-supplied CSS selectors via soupsieve. This can cause:
- OOM kills in containerised deployments (Kubernetes pods, Docker containers) with memory limits - Swap thrashing on bare-metal servers, degrading performance for all co-located processes - Process termination via Python's MemoryError exception if the system runs out of addressable memory
| Parameter | Value | |---|---| | Input size | ~500 KB selector string | | Memory allocated | ~244 MB | | Amplification ratio | ~488× | | Per-object overhead | ~976 bytes per selector | | Authentication required | None | | User interaction required | None |
Scalability of attack: The memory allocation scales linearly - doubling the selector count doubles memory usage. An attacker can tune the payload to exactly exhaust a target's memory limits. Multiple concurrent requests multiply the effect.
Downstream exposure: soupsieve is an automatic dependency of beautifulsoup4, one of the most widely installed Python packages. Any web application accepting CSS selectors from users (e.g., web scraping APIs, content filtering tools, CMS preview features) is potentially affected.
--- Credit
Discovered by a security research team from the University of Sydney, focused on detecting open source software vulnerabilities. Liyi Zhou: https://lzhou1110.github.io/ Ziyue Wang: https://zyy0530.github.io/ Strick: https://str1ckl4nd.github.io/ Maurice: https://maurice.busystar.org/ Chenchen Yu: https://7thparkk.github.io/
Summary
pymdownx.snippets has a regression of the CVE-2023-32309 / GHSA-jh85-wwv9-24hv fix. With restrictbasepath: True (the default), the current filename.startswith(base) containment check does not enforce a directory boundary. As a result, a markdown snippet directive can read files from sibling paths that share the same prefix as basepath, such as docs vs docsinternal.
The regression was introduced in PR #2039 / commit 7c13bda5b7793b172efd1abb6712e156a83fe07d, which replaced the original directory-identity check with a plain string-prefix comparison.
Details
The regression was introduced in commit 7c13bda5b7793b172efd1abb6712e156a83fe07d (2023-05-15, #2039 "Fix regression of snippets nested deeply under specified base path"), which relaxed the original os.path.samefile(base, os.path.dirname(filename)) check to a plain startswith(base).
SnippetPreprocessor.getsnippetpath() in pymdownx/snippets.py:
python if self.restrictbasepath: filename = os.path.abspath(os.path.join(base, path)) # If the absolute path is no longer under the specified base path, reject the file if not filename.startswith(base): continue
base is os.path.abspath(b) and has no trailing separator. str.startswith(base) is True for any filename whose string representation begins with the same characters as base, regardless of whether those characters end at a directory boundary.
Concrete example:
base = "/x/docs" path = "../docssecret/leak.txt" (inside the markdown snippet directive) os.path.join(base, path) → "/x/docs/../docssecret/leak.txt" os.path.abspath(...) → "/x/docssecret/leak.txt" filename.startswith(base) → True, because "/x/docssecret/..." begins with the literal string "/x/docs".
All releases from 10.0.1 (2023-05-15) through 10.21.2 (current) are affected.
Impact
Arbitrary file read within the host the build runs on, bounded by the prefix match. With basepath = /x/docs the attacker can read files from any sibling directory whose path begins with the literal string /x/docs followed by any non-separator character — for example /x/docsinternal/, /x/docs.bak/, /x/docs2/.
The threat model is the same as the original CVE-2023-32309: markdown content processed by the snippets preprocessor in a build pipeline (typical scenario: an MkDocs documentation site built in CI from PR contributions or otherwise less-trusted markdown) can read files outside the configured base. CI builds that publish the generated HTML expose the read file to the public; CI builds with secrets on disk leak those secrets.
Reproduction
Minimal local PoC, non-destructive:
python import os, shutil, tempfile, markdown
work = tempfile.mkdtemp(prefix="pmxpoc") try: base = os.path.join(work, "docs") sibling = os.path.join(work, "docssecret") os.makedirs(base) os.makedirs(sibling) with open(os.path.join(sibling, "leak.txt"), "w") as f: f.write("TOPSECRETFROMSIBLINGDIR\n")
out = markdown.markdown( '--8<-- "../docssecret/leak.txt"\n', extensions=["pymdownx.snippets"], extensionconfigs={ "pymdownx.snippets": { "basepath": [base], "restrictbasepath": True, "checkpaths": True, } }, ) print(out) # -> <p>TOPSECRETFROMSIBLINGDIR</p> finally: shutil.rmtree(work)
Default restrictbasepath: True is sufficient — no non-default option is required.
Suggested fix
Minimal change — require the separator after the base prefix:
diff - if not filename.startswith(base): + # Append os.sep so a sibling directory whose name shares a prefix + # (e.g. /x/docs vs /x/docsevil) cannot satisfy the check. + if not filename.startswith(base + os.sep): continue
This preserves the original intent (allow snippets nested at any depth under basepath) while restoring the directory-boundary check. It does not affect the os.path.isdir(base) branch where base is a file (that branch still uses os.path.samefile).
Alternative: os.path.commonpath([base, filename]) == base is equivalent and slightly more idiomatic, though it raises ValueError on different drives on Windows and would need a try/except. The startswith(base + os.sep) fix is the smaller diff.
Note: this fix does not change behaviour for symlinks inside basepath. The existing implementation uses os.path.abspath (not os.path.realpath), so a symlink within basepath pointing outside is still followed. That is a separate concern — symlinks require write access to basepath, a much higher bar than the current bypass — and matches the behaviour the CVE-2023 fix established.
Regression test
A regression test class TestSnippetsSiblingPrefix was added in tests/testextensions/testsnippets.py. It uses tests/testextensions/snippets/nested as basepath and a new fixture directory tests/testextensions/snippets/nestedsiblingevil/leak.txt. It asserts that the markdown directive --8<-- "../nestedsiblingevil/leak.txt" raises SnippetMissingError.
Without fix: test fails (AssertionError: SnippetMissingError not raised, sibling file is silently read). With fix: test passes.
Full suite: python -m pytest tests/ -q → 738 passed (737 baseline + 1 new regression test). No regressions.
Affected versions
>= 10.0.1, <= 10.21.2
Impact
This issue describes a ReDOS bug found within the figure caption extension (pymdownx.blocks.caption ).
In systems that take unchecked user content, this could cause long hangs when processing the data if a malicious payload was crafted.
Patches
This issue is patched in Release 10.16.1.
Workarounds
Some possible workarounds
If users are concerned about this vulnerability and process unknown user content without timeouts or other safeguards in place to prevent really large, malicious content being aimed at systems, the use of pymdownx.blocks.caption could be avoided until the library is updated to 10.16.1+.
References
The original issue https://github.com/facelessuser/pymdown-extensions/issues/2716.
Description
The original issue came through PyMdown Extensions' normal issue tracker instead of the typical security flow: https://github.com/facelessuser/pymdown-extensions/issues/2716. Because this came through the normal issue flow, it was handled as a normal issue. In the future, PyMdown Extensions will ensure such issues, even if prematurely made public through the normal issue flow, are redirected through the typical security process.
The regular expression pattern in question is as follows:
py REFIGNUM = re.compile(r'^(\^)?([1-9][0-9](?:.[1-9][0-9]))(?= |$)')
The POC was provided by @ShangzhiXu
py import re import time
regexpattern = re.compile(r'^(\^)?([1-9][0-9](?:.[1-9][0-9]))(?= |$)')
for i in range(50, 500, 50): longstring = '1' i + 'a' starttime = time.time() match = re.match(regexpattern, longstring) endtime = time.time() print(f"longstring execution time: {endtime - starttime:.6f} s")
The issue with the above pattern is that . was used, which accepts any character when we meant to use \.. The fix was to update the pattern to:
py REFIGNUM = re.compile(r'^(\^)?([1-9][0-9](?:\.[1-9][0-9]))(?= |$)')
Relevant PR with fix: https://github.com/facelessuser/pymdown-extensions/pull/2717
Version(s) & System Info
- Operating System: Any - Python Version: Any
Summary
Arbitrary file read when using include file syntax.
Details
By using the syntax --8<--"/etc/passwd" or --8<--"/proc/self/environ" the content of these files will be rendered in the generated documentation. Additionally, a path relative to a specified, allowed base path can also be used to render the content of a file outside the specified base paths: --8<-- "../../../../etc/passwd".
Within the Snippets extension, there exists a basepath option but the implementation is vulnerable to Directory Traversal. The vulnerable section exists in getsnippetpath(self, path) lines 155 to 174 in snippets.py.
base = "docs" path = "/etc/passwd" filename = os.path.join(base,path) # Filename is now /etc/passwd
PoC
py import markdown
payload = "--8<-- \"/etc/passwd\"" html = markdown.markdown(payload, extensions=['pymdownx.snippets'])
print(html)
Impact
Any readable file on the host where the plugin is executing may have its content exposed. This can impact any use of Snippets that exposes the use of Snippets to external users.
It is never recommended to use Snippets to process user-facing, dynamic content. It is designed to process known content on the backend under the control of the host, but if someone were to accidentally enable it for user-facing content, undesired information could be exposed.
Suggestion
Specified snippets should be restricted to the configured, specified base paths as a safe default. Allowing relative or absolute paths that escape the specified base paths would need to be behind a feature switch that must be opt-in and would be at the developer's own risk.