GHSA-8mpw-7fpc-4gqj: Pip/nltk vulnerability
Summary
Pl196xCorpusReader still parses whole TEI blocks with multiple lazy regexes over attacker-controlled text. A malformed file with many opening tags and no matching closing tags forces repeated rescans and produces quadratic CPU growth in public reader APIs.
Details
- Vulnerability type: Regular-expression denial of service - Affected component: nltk.corpus.reader.pl196x.TEICorpusView.readblock and Pl196xCorpusReader public methods - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Lazy .? whole-block regexes rescan untrusted XML-like blocks from each opening-tag position.
The parser uses regexes for paragraphs, sentences, and word tags across the whole <text> block. When the attacker supplies many unmatched opening tags, each attempt scans toward the end of the block and fails, then restarts from the next opening tag. There is near four-times runtime growth each time the number of malformed <p> tags doubled, through normal public calls such as words() and taggedwords().
PoC
Preconditions - The application parses attacker-influenced PL196X or TEI-like corpus files through public reader APIs.
Steps 1. Create a corpus file with a valid header followed by a <text> block that contains many opening tags and no matching closing tags. 2. Instantiate Pl196xCorpusReader on that corpus. 3. Call words() or taggedwords() and measure elapsed time as the malformed tag count doubles. 4. Observe near quadratic growth instead of near-linear behavior.
Minimal reproducible excerpt
text size=1000 0.014s size=2000 0.057s size=4000 0.231s size=8000 0.927s
Impact
A consumer that accepts attacker-influenced corpus files can be forced into heavy CPU use and parser-thread stalling before the application concludes the input contains no valid content.
Remediation
Replace the whole-block lazy-regex parser with a linear parser or bounded tokenizer, and add regression tests that assert near-linear behavior on malformed inputs with many unmatched tags.
Affected Software
Remediation
Recommended actions to resolve this vulnerability, in priority order.
- Upgrade
Upgrade
pip/nltkto a version that resolves this vulnerability.Fixed in 3.10.3
Event History
Frequently Asked Questions
Which applications are realistically exposed?
Applications that use NLTK's PL196x corpus reader to parse PL196x/TEI-like content influenced by an attacker are exposed. The affected public APIs include calls such as words() and tagged_words().
What does an attacker need to supply?
The attacker needs to provide a malformed input containing many opening tags without matching closing tags. This causes repeated whole-block regex scans and quadratic CPU growth.
Is a fixed NLTK release available?
No patched version is identified. The issue was reproduced in published version 3.9.4 and in current source v3.10.0-rc2.
How can I recognize exploitation or a vulnerable workload?
Look for unusually high CPU consumption while processing malformed TEI-like text with many unmatched opening paragraph, sentence, or word tags. Runtime may grow by nearly four times when the number of malformed opening <p> tags doubles.