Summary PSDImage.composite() (and .numpy()) allocate the output image buffer from the PSD's header geometry (width × height × channels × depth, and per-layer rectangles) before validating those values against the actual file contents. A tiny crafted PSD declaring huge dimensions causes a multi-gigabyte allocation. Critically, composite() then returns a (black) image with only a warning, no exception is raised, so a caller cannot detect or guard against it.

Impact On psd-tools 1.17.2 (latest), default usage, a 49-byte PSD makes composite() commit ~3 GB and return successfully (warning only); .numpy() reaches ~7.5 GB, and the per-layer rectangle is a second lever (up to ~32 GB), all from an input under 100 bytes (input-to-commit amplification over 1000×). Because the buffer is committed before validation and no exception is thrown, any service that composites untrusted PSDs is exposed to denial of service: on a host with less RAM than the attacker-declared geometry the allocation is an unrecoverable OOM-kill.

Steps to reproduce python pip install psd-tools==1.17.2 from psdtools import PSDImage psd = PSDImage.open("psd-psdtools-grammar-d23.psd") psd.composite() # commits ~3 GB from a 49-byte file and returns (warning only)

PoC (49 bytes), reconstruct with: sh base64 -d > psd-psdtools-grammar-d23.psd <<'EOF' OEJQUwABAAAAAAAAAAYAACg4AAAXTAAIAAMAAAAAAAAAAAAAAAAAAUNIUIFU+yQtDw== EOF Verify: 7d8ebf03a54393cb0359ecf4b676d1b08c9a8c6afdd06671ef406d6893cce826 psd-psdtools-grammar-d23.psd

Root cause The composite/numpy buffer is sized from the declared image (and per-layer) dimensions and channel/depth without checking them against the available data length or a sane maximum.

Suggested fix Validate the declared dimensions, channel count, and per-layer rectangles against the actual file length (and a configurable maximum pixel/byte budget) before allocating; raise an error on overflow instead of committing the buffer and returning a black image.

Summary

A security review of the psdtools.compression module (conducted against the fix/invalid-rle-compression branch, commits 7490ffa–2a006f5) identified the following pre-existing issues. The two findings introduced and fixed by those commits (Cython buffer overflow, IndexError on lone repeat header) are excluded from this report.

---

Findings

1. Unguarded zlib.decompress — ZIP bomb / memory exhaustion (Medium)

Location: src/psdtools/compression/init.py, lines 159 and 162

python result = zlib.decompress(data) # Compression.ZIP decompressed = zlib.decompress(data) # Compression.ZIPWITHPREDICTION

zlib.decompress is called without a maxlength cap. A crafted PSD file containing a ZIP-compressed channel whose compressed payload expands to gigabytes would exhaust process memory before any limit is enforced. The RLE path is not vulnerable to this because the decoder pre-allocates exactly rowsize × height bytes; the ZIP path has no equivalent ceiling.

Impact: Denial-of-service / OOM crash when processing untrusted PSD files.

Suggested mitigation: Pass a reasonable maxlength to zlib.decompress, derived from the expected width height depth // 8 byte count already computed in decompress().

---

2. No upper-bound validation on image dimensions before allocation (Low)

Location: src/psdtools/compression/init.py, lines 138 and 193

python length = width height max(1, depth // 8) # decompress() rowsize = max(width depth // 8, 1) # decoderle()

Neither width, height, nor depth are range-checked before these values drive memory allocation. The PSD format (version 2 / PSB) permits dimensions up to 300,000 × 300,000 pixels; a 4-channel 32-bit image at that size would require ~144 TB to hold. While the OS/Python allocator will reject such a request, there is no early, explicit guard that produces a clean, user-facing error.

Impact: Uncontrolled allocation attempt from a malformed or adversarially crafted PSB file; hard crash rather than a recoverable error.

Suggested mitigation: Validate width, height, and depth against known PSD/PSB limits before entering decompression, and raise a descriptive ValueError early.

---

3. assert used as a runtime integrity check (Low)

Location: src/psdtools/compression/init.py, line 170

python assert len(result) == length, "len=%d, expected=%d" % (len(result), length)

This assertion can be silently disabled by running the interpreter with -O (or -OO), which strips all assert statements. If the assertion ever becomes relevant (e.g., after future refactoring), disabling it would allow a length mismatch to propagate silently into downstream image compositing.

Impact: Loss of an integrity guard in optimised deployments.

Suggested mitigation: Replace with an explicit if + raise ValueError(...).

---

4. cdef int indices vs. Pyssizet size type mismatch in Cython decoder (Low)

Location: src/psdtools/compression/rle.pyx, lines 18–20

cython cdef int i = 0 cdef int j = 0 cdef int length = data.shape[0]

All loop indices are C signed int (32-bit). The size parameter is Pyssizet (64-bit on modern platforms). The comparison j < size promotes j to Pyssizet, but if j wraps due to a row size exceeding INTMAX (~2.1 GB), the resulting comparison is undefined behaviour in C. In practice, row sizes are bounded by PSD/PSB dimension limits and are unreachable at this scale; however, the mismatch is a latent defect if the function is ever called directly with large synthetic inputs.

Impact: Theoretical infinite loop or UB at >2 GB row sizes; not reachable from standard PSD/PSB parsing.

Suggested mitigation: Change cdef int i, j, length to cdef Pyssizet.

---

5. Silent data degradation not surfaced to callers (Informational)

Location: src/psdtools/compression/init.py, lines 144–157

The tolerant RLE decoder (introduced in 2a006f5) replaces malformed channel data with zero-padded (black) pixels and emits a logger.warning. This is the correct trade-off over crashing, but the warning is only observable if the caller has configured a log handler. The public PSDImage API does not surface channel-level decode failures to the user in any other way.

Impact: A user parsing a silently corrupt file gets a visually wrong image with no programmatic signal to check.

Suggested mitigation: Consider exposing a per-channel decode-error flag or raising a distinct warning category that users can filter or escalate via the warnings module.

---

6. encode() zero-length return type inconsistency in Cython (Informational)

Location: src/psdtools/compression/rle.pyx, lines 66–67

cython if length == 0: return data # returns a memoryview, not an explicit std::string

All other return paths return an explicit cdef string result. This path returns data (a const unsigned char[:] memoryview) and relies on Cython's implicit coercion to bytes. It is functionally equivalent today but is semantically inconsistent and fragile if Cython's coercion rules change in a future version.

Impact: Potential silent breakage in future Cython versions; not a current security issue.

Suggested mitigation: Replace return data with return result (the already-declared empty string).

---

Environment

- Branch: fix/invalid-rle-compression - Reviewed commits: 7490ffa, 2a006f5 - Python: 3.x (Cython extension compiled for CPython)