python-patterns

# Python Development Patterns

Idiomatic Python patterns and best practices for building robust, efficient, and maintainable applications.

## When to Activate

- Designing Python package structure and module boundaries
- Choosing between dataclasses, Pydantic models, and TypedDicts for a data structure
- Applying idiomatic Python (list comprehensions, generators, context managers, decorators)
- Setting up type hints and mypy for a new module or existing codebase
- Structuring a Python service with dependency injection or hexagonal architecture
- Deciding when to use ABCs vs protocols for interface design

## Core Principles

### 1. Readability Counts

Python prioritizes readability. Code should be obvious and easy to understand.

```python
# Good: Clear and readable
def get_active_users(users: list[User]) -> list[User]:
    """Return only active users from the provided list."""
    return [user for user in users if user.is_active]


# Bad: Clever but confusing
def get_active_users(u):
    return [x for x in u if x.a]
```

### 2. Explicit is Better Than Implicit

Avoid magic; be clear about what your code does.

```python
# Good: Explicit configuration
import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Bad: Hidden side effects
import some_module
some_module.setup()  # What does this do?
```

### 3. EAFP - Easier to Ask Forgiveness Than Permission

Python prefers exception handling over checking conditions.

```python
# Good: EAFP style
def get_value(dictionary: dict, key: str) -> Any:
    try:
        return dictionary[key]
    except KeyError:
        return default_value

# Bad: LBYL (Look Before You Leap) style
def get_value(dictionary: dict, key: str) -> Any:
    if key in dictionary:
        return dictionary[key]
    else:
        return default_value
```

> **When to use which:** Use EAFP for external resources (files, APIs, DB) where failures are expected and informative. Use LBYL for simple attribute checks on known objects where a conditional read is cheaper than exception handling.

## Type Hints

### Type Annotations (Python 3.10+)

Use built-in generic types directly (`list[str]`, `dict[str, Any]`, `X | None`) — no `typing` imports for basic annotations:

```python
from typing import Any

def process_user(user_id: str, data: dict[str, Any], active: bool = True) -> User | None:
    return User(user_id, data) if active else None

def process_items(items: list[str]) -> dict[str, int]:
    return {item: len(item) for item in items}
```

### Type Aliases and Generics (Python 3.12+)

```python
from typing import Any

# Type alias (PEP 695 — Python 3.12+)
type JSON = dict[str, Any] | list[Any] | str | int | float | bool | None

def parse_json(data: str) -> JSON:
    return json.loads(data)

# Generic functions with new syntax (Python 3.12+)
def first[T](items: list[T]) -> T | None:
    """Return the first item or None if list is empty."""
    return items[0] if items else None

# Generic classes (Python 3.12+)
class Stack[T]:
    def __init__(self) -> None:
        self._items: list[T] = []

    def push(self, item: T) -> None:
        self._items.append(item)

    def pop(self) -> T:
        return self._items.pop()
```

### Protocol-Based Duck Typing

```python
from typing import Protocol

class Renderable(Protocol):
    def render(self) -> str:
        """Render the object to a string."""

def render_all(items: list[Renderable]) -> str:
    """Render all items that implement the Renderable protocol."""
    return "\n".join(item.render() for item in items)
```

## Error Handling Patterns

### Specific Exception Handling

```python
# Good: Catch specific exceptions
def load_config(path: str) -> Config:
    try:
        with open(path) as f:
            return Config.from_json(f.read())
    except FileNotFoundError as e:
        raise ConfigError(f"Config file not found: {path}") from e
    except json.JSONDecodeError as e:
        raise ConfigError(f"Invalid JSON in config: {path}") from e

# Bad: Bare except
def load_config(path: str) -> Config:
    try:
        with open(path) as f:
            return Config.from_json(f.read())
    except:
        return None  # Silent failure!
```

### Custom Exception Hierarchy

```python
class AppError(Exception):
    """Base exception for all application errors."""
    pass

class ValidationError(AppError):
    """Raised when input validation fails."""
    pass

class NotFoundError(AppError):
    """Raised when a requested resource is not found."""
    pass

# Usage
def get_user(user_id: str) -> User:
    user = db.find_user(user_id)
    if not user:
        raise NotFoundError(f"User not found: {user_id}")
    return user
```

## Context Managers

Always use `with` for resource management (files, DB connections, locks). Prefer `@contextmanager` for simple cases; use `__enter__`/`__exit__` classes for stateful resources.

### Custom Context Managers

```python
from contextlib import contextmanager

@contextmanager
def timer(name: str):
    """Context manager to time a block of code."""
    start = time.perf_counter()
    yield
    elapsed = time.perf_counter() - start
    print(f"{name} took {elapsed:.4f} seconds")

# Usage
with timer("data processing"):
    process_large_dataset()
```

### Context Manager Classes

```python
class DatabaseTransaction:
    def __init__(self, connection):
        self.connection = connection

    def __enter__(self):
        self.connection.begin_transaction()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is None:
            self.connection.commit()
        else:
            self.connection.rollback()
        return False  # Don't suppress exceptions

# Usage
with DatabaseTransaction(conn):
    user = conn.create_user(user_data)
    conn.create_profile(user.id, profile_data)
```

## Comprehensions and Generators

```python
# List comprehension — prefer over manual loops
names = [user.name for user in users if user.is_active]

# Generator expression — lazy, no intermediate list (O(1) memory)
total = sum(x * x for x in range(1_000_000))

# Generator function — stream large data without loading all into memory
def read_large_file(path: str) -> Iterator[str]:
    with open(path) as f:
        for line in f:
            yield line.strip()
```

Keep comprehensions simple — if a comprehension needs more than one `if` or a nested loop, use a regular function instead.

## Data Classes

```python
from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class User:
    """Automatic __init__, __repr__, __eq__. Use __post_init__ for validation."""
    id: str
    name: str
    email: str
    created_at: datetime = field(default_factory=datetime.now)
    is_active: bool = True

    def __post_init__(self):
        if "@" not in self.email:
            raise ValueError(f"Invalid email: {self.email}")
```

> For advanced patterns — `NamedTuple` with methods, function decorators, parameterized decorators, class-based decorators, concurrency (threading, multiprocessing, async/await), hexagonal architecture with FastAPI, and memory optimization — see `python-patterns-advanced`.

## Anti-Patterns

### Using a Mutable Default Argument

**Wrong:**
```python
def append_item(item, collection=[]):  # list is created once at definition time
    collection.append(item)
    return collection

append_item("a")  # ["a"]
append_item("b")  # ["a", "b"]  — unexpected: shares the same list
```

**Correct:**
```python
def append_item(item, collection=None):
    if collection is None:
        collection = []
    collection.append(item)
    return collection
```

**Why:** Default argument values are evaluated once when the function is defined; mutable defaults accumulate state across calls.

---

### Catching `Exception` (or bare `except`) and Swallowing It

**Wrong:**
```python
def load_user(user_id: str) -> User | None:
    try:
        return db.find(user_id)
    except Exception:
        return None  # hides programming errors, network failures, everything
```

**Correct:**
```python
def load_user(user_id: str) -> User | None:
    try:
        return db.find(user_id)
    except UserNotFoundError:
        return None
    except DatabaseError as e:
        raise ServiceUnavailableError("Database unreachable") from e
```

**Why:** Catching `Exception` silently swallows bugs and infrastructure failures, making them impossible to observe or recover from correctly.

---

### Building Strings with `+` in a Loop

**Wrong:**
```python
def build_csv(rows: list[dict]) -> str:
    result = ""
    for row in rows:
        result += ",".join(str(v) for v in row.values()) + "\n"
    return result
```

**Correct:**
```python
def build_csv(rows: list[dict]) -> str:
    lines = [",".join(str(v) for v in row.values()) for row in rows]
    return "\n".join(lines) + "\n"
```

**Why:** String concatenation in a loop is O(n²) because each `+` creates a new string; `str.join` is O(n).

---

### Using `Optional[X]` Instead of `X | None` in Modern Python

**Wrong:**
```python
from typing import Optional

def find_user(user_id: str) -> Optional[User]:
    ...
```

**Correct:**
```python
def find_user(user_id: str) -> User | None:
    ...
```

**Why:** `X | None` (PEP 604, Python 3.10+) is the idiomatic modern syntax; `Optional` requires a `typing` import and is now considered legacy style.

---

### Using `type(x) == SomeClass` Instead of `isinstance`

**Wrong:**
```python
def process(value):
    if type(value) == int:
        return value * 2
```

**Correct:**
```python
def process(value):
    if isinstance(value, int):
        return value * 2
```

**Why:** `type(x) == SomeClass` breaks for subclasses and does not respect the Python type hierarchy; `isinstance` handles inheritance correctly.

> For advanced patterns — concurrency (threading, multiprocessing, async/await), hexagonal architecture with FastAPI (full working code, Protocol ports, DI wiring, RFC 7807 error handling, tests), memory optimization, tooling (`pyproject.toml`, ruff, mypy), and anti-patterns — see skill: `python-patterns-advanced`.