CVE-2026-72818: NLTK TweetTokenizer URL Pattern Backtracks Catastrophically on Naked-Domain-Like Input
The URLS regular expression in nltk/tokenize/casual.py, compiled into TweetTokenizer.WORDRE and applied by TweetTokenizer.tokenize, contains a naked-domain branch whose domain-label prefix [a-z0-9]+(?:[.\-][a-z0-9]+) is unbounded. Input consisting of many alternating label separators can be partitioned in exponentially many ways, and because the branch also requires a trailing top-level domain that such input never supplies, the engine explores those partitions before failing at each offset. A few kilobytes of input therefore consumes seconds to minutes of single-threaded CPU, and the HANGRE substitution performed before matching does not collapse the pattern. TweetTokenizer is intended for tokenizing untrusted social-media text, so any service that applies it, or the module-level casualtokenize, to submitted text can be stalled per request without authentication. Version 3.10.1 bounds the label repetition.
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
nltk/tokenize/casual.py (TweetTokenizer WORD_RE)to a version that resolves this vulnerability.Fixed in 3.10.1
Event History
Frequently Asked Questions
Which deployments are exposed to this denial-of-service issue?
Services that pass submitted or otherwise untrusted text to TweetTokenizer.tokenize or the module-level casual_tokenize are exposed. An attacker can trigger the issue remotely without authentication when they can supply text to a tokenization path.
What input is needed to trigger the excessive CPU use?
The triggering input consists of many alternating domain-label separators arranged to resemble a naked domain but without a valid trailing top-level domain. Only a few kilobytes can consume seconds to minutes of single-threaded CPU.
Are default preprocessing steps sufficient to prevent exploitation?
No. The HANG_RE substitution performed before matching does not collapse the problematic pattern, so it does not prevent the catastrophic backtracking.
What version addresses the vulnerable regular expression?
NLTK version 3.10.1 bounds the domain-label repetition in the URL pattern.