drone-cv-expert

# Drone CV Expert

Expert in robotics, drone systems, and computer vision for autonomous aerial platforms.

## Decision Tree: When to Use This Skill

```
User mentions drones or UAVs?
├─ YES → Is it about inspection/detection of specific things (fire, roof damage, thermal)?
│        ├─ YES → Use drone-inspection-specialist
│        └─ NO → Is it about flight control, navigation, or general CV?
│                ├─ YES → Use THIS SKILL (drone-cv-expert)
│                └─ NO → Is it about GPU rendering/shaders?
│                        ├─ YES → Use metal-shader-expert
│                        └─ NO → Use THIS SKILL as default drone skill
└─ NO → Is it general object detection without drone context?
        ├─ YES → Use clip-aware-embeddings or other CV skill
        └─ NO → Probably not a drone question
```

## Core Competencies

### Flight Control & Navigation
- **PID Tuning**: Position, velocity, attitude control loops
- **SLAM**: ORB-SLAM, LSD-SLAM, visual-inertial odometry (VIO)
- **Path Planning**: A*, RRT, RRT*, Dijkstra, potential fields
- **Sensor Fusion**: EKF, UKF, complementary filters
- **GPS-Denied Navigation**: AprilTags, visual odometry, LiDAR SLAM

### Computer Vision
- **Object Detection**: YOLO (v5/v8/v10), EfficientDet, SSD
- **Tracking**: ByteTrack, DeepSORT, SORT, optical flow
- **Edge Deployment**: TensorRT, ONNX, OpenVINO optimization
- **3D Vision**: Stereo depth, point clouds, structure-from-motion

### Hardware Integration
- **Flight Controllers**: Pixhawk, Ardupilot, PX4, DJI
- **Protocols**: MAVLink, DroneKit, MAVSDK
- **Edge Compute**: Jetson (Nano/Xavier/Orin), Coral TPU
- **Sensors**: IMU, GPS, barometer, LiDAR, depth cameras

## Anti-Patterns to Avoid

### 1. "Simulation-Only Syndrome"
**Wrong**: Testing only in Gazebo/AirSim, then deploying directly to real drone.
**Right**: Simulation → Bench test → Tethered flight → Controlled environment → Field.

### 2. "EKF Overkill"
**Wrong**: Using Extended Kalman Filter when complementary filter suffices.
**Right**: Match filter complexity to requirements:
- Complementary filter: Basic stabilization, attitude only
- EKF: Multi-sensor fusion, GPS+IMU+baro
- UKF: Highly nonlinear systems, aggressive maneuvers

### 3. "Max Resolution Assumption"
**Wrong**: Processing 4K frames at 30fps expecting real-time performance.
**Right**: Resolution trade-offs by altitude/speed:
| Altitude | Speed | Resolution | FPS | Rationale |
|----------|-------|------------|-----|-----------|
| <30m | Slow | 1920x1080 | 30 | Detail needed |
| 30-100m | Medium | 1280x720 | 30 | Balance |
| >100m | Fast | 640x480 | 60 | Speed priority |

### 4. "Single-Thread Processing"
**Wrong**: Sequential detect → track → control in one loop.
**Right**: Pipeline parallelism:
```
Thread 1: Camera capture (async)
Thread 2: Object detection (GPU)
Thread 3: Tracking + state estimation
Thread 4: Control commands
```

### 5. "GPS Trust"
**Wrong**: Assuming GPS is always accurate and available.
**Right**: Multi-source position estimation:
- GPS: 2-5m accuracy outdoor, unavailable indoor
- Visual odometry: 0.1-1% drift, lighting dependent
- AprilTags: cm-level accuracy where deployed
- IMU: Short-term only, drift accumulates

### 6. "One Model Fits All"
**Wrong**: Using same YOLO model for all scenarios.
**Right**: Model selection by constraint:
| Constraint | Model | Notes |
|------------|-------|-------|
| Latency critical | YOLOv8n | 6ms inference |
| Balanced | YOLOv8s | 15ms, better accuracy |
| Accuracy first | YOLOv8x | 50ms, highest mAP |
| Edge device | YOLOv8n + TensorRT | 3ms on Jetson |

## Problem-Solving Framework

### 1. Constraint Analysis
- **Compute**: What hardware? (Jetson Nano = ~5 TOPS, Xavier = 32 TOPS)
- **Power**: Battery capacity? Flight time impact?
- **Latency**: Control loop rate? Detection response time?
- **Weight**: Payload capacity? Center of gravity?
- **Environment**: Indoor/outdoor? GPS available? Lighting conditions?

### 2. Algorithm Selection Matrix

| Problem | Classical Approach | Deep Learning | When to Use Each |
|---------|-------------------|---------------|------------------|
| Feature tracking | KLT optical flow | FlowNet | Classical: Real-time, limited compute. DL: Robust, more compute |
| Object detection | HOG+SVM | YOLO/SSD | Classical: Simple objects, no GPU. DL: Complex, GPU available |
| SLAM | ORB-SLAM | DROID-SLAM | Classical: Mature, debuggable. DL: Better in challenging scenes |
| Path planning | A*, RRT | RL-based | Classical: Known environments. DL: Complex, dynamic |

### 3. Safety Checklist
- [ ] Kill switch tested and accessible
- [ ] Geofence configured
- [ ] Return-to-home altitude set
- [ ] Low battery action defined
- [ ] Signal loss action defined
- [ ] Propeller guards (if applicable)
- [ ] Pre-flight sensor calibration
- [ ] Weather conditions checked

## Quick Reference Tables

### MAVLink Message Types
| Message | Purpose | Frequency |
|---------|---------|-----------|
| HEARTBEAT | Connection alive | 1 Hz |
| ATTITUDE | Roll/pitch/yaw | 10-100 Hz |
| LOCAL_POSITION_NED | Position | 10-50 Hz |
| GPS_RAW_INT | Raw GPS | 1-10 Hz |
| SET_POSITION_TARGET | Commands | As needed |

### Kalman Filter Tuning
| Matrix | High Values | Low Values |
|--------|-------------|------------|
| Q (process noise) | Trust measurements more | Trust model more |
| R (measurement noise) | Trust model more | Trust measurements more |
| P (initial covariance) | Uncertain initial state | Confident initial state |

### Common Coordinate Frames
| Frame | Origin | Axes | Use |
|-------|--------|------|-----|
| NED | Takeoff point | North-East-Down | Navigation |
| ENU | Takeoff point | East-North-Up | ROS standard |
| Body | Drone CG | Forward-Right-Down | Control |
| Camera | Lens center | Right-Down-Forward | Vision |

## Reference Files

Detailed implementations in `references/`:
- `navigation-algorithms.md` - SLAM, path planning, localization
- `sensor-fusion-ekf.md` - Kalman filters, multi-sensor fusion
- `object-detection-tracking.md` - YOLO, ByteTrack, optical flow

## Simulation Tools

| Tool | Strengths | Weaknesses | Best For |
|------|-----------|------------|----------|
| Gazebo | ROS integration, physics | Graphics quality | ROS development |
| AirSim | Photorealistic, CV-focused | Windows-centric | Vision algorithms |
| Webots | Multi-robot, accessible | Less drone-specific | Swarm simulations |
| MATLAB/Simulink | Control design | Not real-time | Controller tuning |

## Emerging Technologies (2024-2025)

- **Event cameras**: 1μs temporal resolution, no motion blur
- **Neuromorphic computing**: Loihi 2 for ultra-low-power inference
- **4D Radar**: Velocity + 3D position, works in all weather
- **Swarm autonomy**: Decentralized coordination, emergent behavior
- **Foundation models**: SAM, CLIP for zero-shot detection

## Integration Points

- **drone-inspection-specialist**: Domain-specific detection (fire, damage, thermal)
- **metal-shader-expert**: GPU-accelerated vision processing, custom shaders
- **collage-layout-expert**: Report generation, visual composition

---

**Key Principle**: In drone systems, reliability trumps performance. A 95% accurate system that never crashes is better than 99% accurate that fails unpredictably. Always have fallbacks.