python-packaging-patterns

# Python Packaging Patterns

Structure Python projects for reliable distribution and dependency management.

## Project Layout

### The `src` Layout (Recommended)

```
my-project/
├── pyproject.toml
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── core.py
│       └── cli.py
├── tests/
│   ├── conftest.py
│   └── test_core.py
└── README.md
```

**Why src layout:** Prevents accidental imports from the working directory. Forces installation before testing, catching packaging errors early.

### Flat Layout (Simple Projects)

```
my-project/
├── pyproject.toml
├── my_package/
│   ├── __init__.py
│   └── core.py
└── tests/
```

Acceptable for internal tools and single-organ repos where distribution is not a concern.

## pyproject.toml Configuration

### Minimal Configuration

```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-package"
version = "0.1.0"
description = "A concise description"
requires-python = ">=3.11"
license = "MIT"
dependencies = [
    "httpx>=0.27",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "ruff>=0.5",
    "mypy>=1.10",
]
```

### Build Backend Selection

| Backend | When to Use |
|---------|-------------|
| **hatchling** | Default choice. Fast, minimal config, good monorepo support |
| **setuptools** | Legacy projects, C extensions, complex build needs |
| **flit** | Pure Python, minimal config, simple projects |
| **poetry-core** | When using Poetry for dependency management |

### Optional Dependency Groups

Organize optional dependencies by use case:

```toml
[project.optional-dependencies]
dev = ["pytest>=8.0", "ruff>=0.5", "mypy>=1.10"]
docs = ["sphinx>=7.0", "myst-parser"]
dashboard = ["fastapi>=0.110", "uvicorn"]
metrics = ["prometheus-client"]
```

Install specific groups: `pip install -e ".[dev,dashboard]"`

## Entry Points

### Console Scripts

```toml
[project.scripts]
my-cli = "my_package.cli:main"
```

### Plugin Entry Points

```toml
[project.entry-points."my_app.plugins"]
csv = "my_package.plugins.csv:CsvPlugin"
json = "my_package.plugins.json:JsonPlugin"
```

## Version Management

### Single Source of Truth

```toml
# In pyproject.toml
[project]
dynamic = ["version"]

[tool.hatch.version]
path = "src/my_package/__init__.py"
```

```python
# In __init__.py
__version__ = "0.1.0"
```

### CalVer for System Packages

For infrastructure packages where semantic versioning adds little value:

```python
__version__ = "2026.03.1"  # YYYY.MM.patch
```

## Dependency Pinning Strategy

| Context | Strategy | Tool |
|---------|----------|------|
| Library | Loose bounds (`>=1.0,<2.0`) | pyproject.toml |
| Application | Exact pins | pip-compile / uv lock |
| CI | Lockfile | uv.lock / requirements.txt |

### Generating Lockfiles

```bash
# Using uv (recommended)
uv pip compile pyproject.toml -o requirements.txt
uv pip compile pyproject.toml --extra dev -o requirements-dev.txt

# Using pip-tools
pip-compile pyproject.toml -o requirements.txt
```

## Editable Installs

```bash
# Standard editable install
pip install -e .

# With dev dependencies
pip install -e ".[dev]"

# In a fresh venv
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
```

## Tool Configuration in pyproject.toml

### Ruff

```toml
[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.lint]
select = ["E", "F", "I", "N", "W", "UP"]
```

### Pytest

```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
asyncio_mode = "auto"
pythonpath = ["."]
```

### Mypy

```toml
[tool.mypy]
strict = true
python_version = "3.11"
```

## Publishing

### To PyPI

```bash
# Build
python -m build

# Upload (use trusted publishing when possible)
python -m twine upload dist/*
```

### Trusted Publishing (GitHub Actions)

```yaml
- uses: pypa/gh-action-pypi-publish@release/v1
  with:
    password: ${{ secrets.PYPI_API_TOKEN }}
```

## Common Patterns

### Namespace Packages

For multi-repo packages sharing a namespace:

```
# Repo A
src/organvm/engine/__init__.py

# Repo B
src/organvm/dashboard/__init__.py
```

Use implicit namespace packages (no `__init__.py` at namespace level).

### Conditional Dependencies

```toml
dependencies = [
    "tomli>=2.0; python_version < '3.11'",
    "typing-extensions>=4.0; python_version < '3.12'",
]
```

### Package Data

```toml
[tool.hatch.build.targets.wheel]
packages = ["src/my_package"]

[tool.hatch.build.targets.wheel.force-include]
"assets" = "my_package/assets"
```

## Anti-Patterns to Avoid

- **setup.py without pyproject.toml** — Always use pyproject.toml as the single config source
- **Pinning exact versions in libraries** — Use compatible ranges to avoid dependency conflicts
- **Importing from project root in tests** — Use src layout or ensure editable install
- **Multiple version sources** — Keep version in exactly one place
- **requirements.txt as sole dependency spec** — Use pyproject.toml; generate lockfiles from it