Unpickling is still RCE: where Python pickle still bites

  • python
  • deserialization
  • rce
  • mlops
  • web

Proof of concept on GitHub

Python's pickle runs code during deserialization by design: an object can define __reduce__, and pickle.load will call whatever it returns. So any pickle.load (or torch.load, joblib.load, pandas.read_pickle, a cache that stores pickles) reading data an attacker can influence is remote code execution as the process that loads it. That is old news. What keeps it alive is where the sink hides now, and how often the guard in front of it does not work. Three modern places it bites, then how to close it.

The mechanism in four lines

import os
class Exploit:
    def __reduce__(self):
        return (os.system, ("id > /tmp/pickle-proof",))   # harmless proof; real payloads vary
# pickle.dumps(Exploit()) is now a blob that runs `id` the moment anything unpickles it.

Everything below is this one fact, moved to a place a defender did not think of as a deserialization boundary.

Sink 1: a writable Django FileBasedCache directory

Django's FileBasedCache stores each cache entry as a pickle in a .djcache file. If that directory (from CACHES in settings.py, often /var/tmp/django_cache/) is writable by your foothold user but read by the web app running as another user, you overwrite an entry with a malicious pickle and the next cache.get on that key deserializes it, running your code as the gunicorn or www-data user. It is a clean lateral hop: find the cache dir, clobber a frequently-read entry preserving Django's framing (a pickled expiry followed by the pickled value), then hit the endpoint that reads it. Drop an SSH key into that user's home for stability.

Sink 2: ML artifact pipelines (ClearML, CVE-2024-24590)

MLOps platforms move pickles between users and schedulers constantly. In ClearML before 1.14.3, Artifact.get() calls pickle.load() on any pickle-typed artifact with no validation, so an artifact you upload to a project that a scheduled task or another analyst later opens runs your code as that identity. This is the important shift: it is delivery to a victim, not just self-RCE. The load-bearing details are operational, not clever: pin the client to the server version, match the project_name and tags the reviewer task filters on, and pass the live object to artifact_object= (a path uploads a plain blob that never gets unpickled). The same shape applies to any torch.load of a shared checkpoint and any notebook pipeline that reads a teammate's .pkl.

Sink 3: the "we scan pickles first" wrapper that does not

The most instructive one is a defense that fails. A root-run wrapper accepts a file you control, shells out to a scanner (fickling, a custom checker), parses the output, and only proceeds "if clean". The guard is only as good as its parse of that output. The canonical break: the scanner reports severity per item, but the guard tests a top-level field that is always null, so the match never fires and every payload is "clean":

if echo "$fickling_output" | jq -e 'select(.severity == "OVERTLY_MALICIOUS")'; then rm "$file"; fi
# .severity is null at the top level (real severities are nested per analysis),
# so the select matches nothing, jq -e exits 1, and the "malicious" branch never runs.

The lesson generalizes past pickle: a security gate is only as strong as its parse of the tool it trusts. Before building any evasion, feed the plainly-malicious file and watch whether the guard actually blocks it; often it never worked. (If it does block, note that many such guards delete the file on a hit and gate the real action on the file still existing, so a tripped payload leaves you with nothing.)

Defense

The fix is the same everywhere, and it is not "scan the pickle":

  • Do not unpickle untrusted or attacker-reachable data. Use a data-only format (JSON, or a schema like Protocol Buffers) for anything that crosses a trust boundary. For models, prefer formats that do not execute on load, such as safetensors.
  • Lock the file permissions. A cache or artifact directory read by a privileged process must not be writable by a lower-privileged one. This single ACL closes Sink 1.
  • If you must scan, parse the verdict correctly and fail closed. Test the field the scanner actually populates, treat a parse error or empty result as malicious, not clean, and do not gate on a side effect like the file still existing. A unit test that feeds a known-bad pickle and asserts the guard blocks it is worth more than the scanner.
  • Patch the platform. ClearML 1.14.3 and later validate artifact types; keep MLOps tooling current, because these pipelines pass pickles between identities by default.

Detection

A Python web or worker process spawning a shell, sh, or a network tool right after a cache read or an artifact fetch is the signal; so is a non-privileged user writing into a cache or model directory owned by a service. Log execve with parent process for your app's service accounts, and alert on a .djcache, .pkl, or checkpoint file modified by the wrong uid.

Use

Authorized and educational use only: your own systems, a lab you control, or an engagement with written permission. The proof payload here runs id; this explains the sink so defenders stop trusting pickle across trust boundaries, not for use against systems you do not own. The writable-directory angle connects to living off the land on Linux.

Proof of concept on GitHub

Back to all techniques