pytorch-deployment
Advanced sub-skill for PyTorch focused on model productionization and deployment. Covers TorchScript (JIT/Tracing), ONNX export, LibTorch (C++ API), and inference optimization (Quantization, Pruning).
Best use case
pytorch-deployment is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Advanced sub-skill for PyTorch focused on model productionization and deployment. Covers TorchScript (JIT/Tracing), ONNX export, LibTorch (C++ API), and inference optimization (Quantization, Pruning).
Teams using pytorch-deployment should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/pytorch-deployment/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How pytorch-deployment Compares
| Feature / Agent | pytorch-deployment | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Advanced sub-skill for PyTorch focused on model productionization and deployment. Covers TorchScript (JIT/Tracing), ONNX export, LibTorch (C++ API), and inference optimization (Quantization, Pruning).
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# PyTorch - Deployment & Production Engineering
Deploying a model in a high-performance environment often means removing the Python dependency. This guide covers how to serialize models into formats that can be loaded in C++, optimized for edge devices, or executed in high-throughput inference engines like TensorRT.
## When to Use
- Moving a model from a Jupyter Notebook to a production web server (FastAPI/Go/Rust).
- Embedding a neural network into a C++ application (LibTorch).
- Running inference on mobile devices (iOS/Android) or edge hardware (NVIDIA Jetson).
- Accelerating inference speed using specialized hardware backends (OpenVINO, TensorRT).
- Ensuring model reproducibility across different versions of PyTorch.
## Core Principles
### 1. Scripting vs. Tracing
- **Tracing**: PyTorch runs the model once with "example data" and records all operations. Fast, but ignores Python control flow (if, for).
- **Scripting**: PyTorch compiles the Python source code of the module. Slower to prepare, but preserves logic and control flow.
### 2. The ONNX Bridge
ONNX (Open Neural Network Exchange) is a cross-platform format. A model exported to ONNX can be run by Microsoft's ONNX Runtime, which is often faster than standard PyTorch for inference.
### 3. Quantization
Reducing weights from float32 (4 bytes) to int8 (1 byte). This shrinks the model size by 4x and can speed up inference by 2-3x on CPUs.
## Quick Reference: Export Patterns
```python
import torch
model = MyModel().eval()
example_input = torch.randn(1, 3, 224, 224)
# 1. Tracing (Most common)
traced_model = torch.jit.trace(model, example_input)
traced_model.save("model_jit.pt")
# 2. Scripting (For dynamic logic)
scripted_model = torch.jit.script(model)
scripted_model.save("model_script.pt")
# 3. ONNX Export
torch.onnx.export(model, example_input, "model.onnx",
input_names=['input'], output_names=['output'],
dynamic_axes={'input': {0: 'batch_size'}})
```
## Critical Rules
### ✅ DO
- **Call model.eval() before export** - This freezes BatchNorm and Dropout layers. Forgetting this leads to incorrect predictions.
- **Use torch.no_grad()** - Always wrap your export logic in a `no_grad` context to avoid saving unnecessary gradient-tracking metadata.
- **Define dynamic_axes in ONNX** - If your model will handle different batch sizes or image resolutions, you must specify them during export.
- **Verify Export Accuracy** - Always compare the output of the original Python model and the exported model using `torch.allclose()`.
- **Use torch.compile for Python Deployment** - If you are deploying within Python, use `torch.compile` (PyTorch 2.0+) instead of JIT for better performance.
### ❌ DON'T
- **Don't use JIT Tracing for models with if/else** - The tracer will only capture the branch taken during the example run.
- **Don't include preprocessing in the model (usually)** - Keep image resizing/normalization outside the core model for better flexibility, unless using TorchScript-compatible operations.
- **Don't ignore quantization warnings** - Some layers (like custom activations) don't support int8 and will fall back to float32, reducing gains.
## Advanced Optimization
### Post-Training Quantization (Static)
```python
import torch.quantization
# 1. Set backend (x86 or ARM)
model.qconfig = torch.quantization.get_default_qconfig('fbgemm')
# 2. Prepare and Calibrate (Run some data through the model)
model_prepared = torch.quantization.prepare(model)
# ... run calibration loop ...
# 3. Convert
model_int8 = torch.quantization.convert(model_prepared)
```
## LibTorch (C++ Deployment)
To load a TorchScript model in C++:
```cpp
#include <torch/script.h>
int main() {
// Load model
torch::jit::script::Module module = torch::jit::load("model_jit.pt");
// Create input tensor
auto input = torch::randn({1, 3, 224, 224});
// Run inference
at::Tensor output = module.forward({input}).toTensor();
std::cout << output.slice(1, 0, 5) << std::endl;
}
```
## Practical Workflows
### 1. Optimizing for Mobile (Lite Interpreter)
For mobile deployment, standard TorchScript is too heavy. Use the "Mobile" optimizer.
```python
from torch.utils.mobile_optimizer import optimize_for_mobile
optimized_model = optimize_for_mobile(traced_model)
optimized_model._save_for_lite_interpreter("model_mobile.ptl")
```
### 2. Deploying via ONNX Runtime
```python
import onnxruntime as ort
session = ort.InferenceSession("model.onnx", providers=['CUDAExecutionProvider'])
outputs = session.run(None, {"input": example_input.numpy()})
```
## Common Pitfalls and Solutions
### The "Missing Attribute" Error in JIT
TorchScript can't see attributes added to the model after initialization.
```python
# ✅ Solution: Define all needed attributes in __init__ or use @torch.jit.export
```
### Dynamic Shape Failures
If your model uses `x.shape[0]` in a calculation, tracing might hardcode that value.
```python
# ✅ Solution: Use Scripting or ensure calculations use tensor methods
# like .size(0) which JIT understands.
```
PyTorch Deployment is the bridge between science and the real world. Mastering these tools ensures that your discoveries don't just stay in a notebook, but power the next generation of intelligent systems.Related Skills
pytorch
Leading deep learning framework. Provides Tensors and Dynamic Computational Graphs with strong GPU acceleration. Widely used for research, neural networks, and differentiable programming.
pytorch-research
Advanced sub-skill for PyTorch focused on deep research and production engineering. Covers custom Autograd functions, module hooks, advanced initialization, Distributed Data Parallel (DDP), and performance profiling.
pytorch-geometric
Graph Neural Networks (GNN) for learning on graph-structured data. PyTorch Geometric (PyG) extends PyTorch with the MessagePassing framework — the core abstraction for all GNN layers — and provides standard convolutions (GCNConv, GATConv, GraphSAGEConv, GINConv), graph pooling, batching of variable-size graphs, and datasets. Use when: performing node classification (e.g., predicting labels on a citation network), graph classification (e.g., predicting molecular properties), link prediction (e.g., recommending new connections), learning representations on any graph-structured data (social networks, molecules, knowledge graphs, protein structures), implementing custom GNN architectures via the MessagePassing base class, working with heterogeneous graphs (multiple node/edge types), or any task where data has explicit relational structure that CNNs/RNNs cannot capture. Complements networkx (classical graph algorithms) and rdkit (molecular graphs) — PyG adds the deep learning layer on top.
xgboost-lightgbm
Industry-standard gradient boosting libraries for tabular data and structured datasets. XGBoost and LightGBM excel at classification and regression tasks on tables, CSVs, and databases. Use when working with tabular machine learning, gradient boosting trees, Kaggle competitions, feature importance analysis, hyperparameter tuning, or when you need state-of-the-art performance on structured data.
xarray
N-dimensional labeled arrays and datasets in Python. Built on top of NumPy and Dask. It introduces labels in the form of dimensions, coordinates, and attributes on top of raw NumPy-like arrays, making data analysis in physical sciences more intuitive and less error-prone. Use for working with multi-dimensional scientific data, NetCDF/GRIB/Zarr files, climate/weather/oceanographic datasets, remote sensing, geospatial imaging, large out-of-memory datasets with Dask, and labeled array operations.
transformers
State-of-the-art Machine Learning for PyTorch, TensorFlow, and JAX. Provides thousands of pretrained models to perform tasks on different modalities such as text, vision, and audio. The industry standard for Large Language Models (LLMs) and foundation models in science.
tqdm
A fast, extensible progress bar for Python and CLI. Instantly makes your loops show a smart progress meter with ETA, iterations per second, and customizable statistics. Minimal overhead. Use for monitoring long-running loops, simulations, data processing, ML training, file downloads, I/O operations, command-line tools, pandas operations, parallel tasks, and nested progress bars.
tensorflow
Comprehensive deep learning framework for building, training, and deploying neural networks. TensorFlow provides tf.keras high-level API for model construction, tf.data for efficient data pipelines, and tf.function for graph-mode optimization. Use when working with: neural network training and inference, image classification/detection/segmentation, NLP/text processing with embeddings or transformers, time series forecasting, generative models (VAE, GAN), transfer learning with pretrained models, custom training loops with GradientTape, GPU/TPU accelerated computation, or any deep learning task.
sympy
Comprehensive guide for SymPy - Python library for symbolic mathematics. Use for symbolic expressions, calculus (derivatives, integrals, limits, series), equation solving (algebraic, differential, systems), linear algebra, simplification, matrix operations, special functions, code generation, and mathematical proofs. Essential for analytical mathematics and computer algebra.
sunpy
The community-developed free and open-source software package for solar physics. Provides tools for data search and download, coordinate transformations specific to solar physics, and powerful image processing through the Map object. Use when working with solar data, solar images (EUV, magnetograms, white light), solar coordinates (Helioprojective, Heliographic), Fido data search, solar time series, differential rotation, limb fitting, or multi-instrument solar analysis (AIA, HMI, GOES).
statsmodels
Advanced statistical modeling and hypothesis testing. Complementary to SciPy's stats module, it provides classes and functions for the estimation of many different statistical models, as well as for conducting statistical tests and statistical data exploration. Use for linear regression, GLM, time series analysis, ANOVA, survival analysis, causal inference, and statistical hypothesis testing. Load when working with OLS, WLS, logistic regression, Poisson regression, ARIMA, SARIMAX, statistical diagnostics, p-values, confidence intervals, or R-style statistical analysis.
spacy-nltk
Natural Language Processing for text analysis, corpus linguistics, and production NLP pipelines. spaCy provides fast production-grade tokenization, POS tagging, NER, dependency parsing, and custom model training. NLTK provides classical corpus linguistics, linguistic analysis, VADER sentiment, collocation analysis, and access to standard linguistic corpora. Use when: processing and analyzing text data, extracting named entities (people, orgs, locations, dates), dependency parsing and syntactic analysis, building text classification pipelines, performing corpus-level linguistic analysis (frequency, collocations, readability), sentiment analysis, lemmatization and stemming, working with multilingual text, training custom NER or text classifiers, or any task requiring structured understanding of natural language beyond simple string operations.