document-rag-pipeline-step-1-database-schema
Sub-skill of document-rag-pipeline: Step 1: Database Schema (+5).
Best use case
document-rag-pipeline-step-1-database-schema is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Sub-skill of document-rag-pipeline: Step 1: Database Schema (+5).
Teams using document-rag-pipeline-step-1-database-schema should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/step-1-database-schema/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How document-rag-pipeline-step-1-database-schema Compares
| Feature / Agent | document-rag-pipeline-step-1-database-schema | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Sub-skill of document-rag-pipeline: Step 1: Database Schema (+5).
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# Step 1: Database Schema (+5)
## Step 1: Database Schema
```python
import sqlite3
from pathlib import Path
from datetime import datetime
def create_database(db_path):
"""Create SQLite database with full schema."""
conn = sqlite3.connect(db_path, timeout=30)
cursor = conn.cursor()
*See sub-skills for full details.*
## Step 2: PDF Text Extraction
```python
import fitz # PyMuPDF
def extract_pdf_text(pdf_path):
"""Extract text from PDF using PyMuPDF."""
try:
doc = fitz.open(pdf_path)
text_parts = []
for page_num in range(len(doc)):
*See sub-skills for full details.*
## Step 3: OCR for Scanned PDFs
```python
import fitz
import pytesseract
from PIL import Image
import io
def ocr_pdf(pdf_path, dpi=200):
"""OCR scanned PDF using Tesseract."""
try:
doc = fitz.open(pdf_path)
*See sub-skills for full details.*
## Step 4: Text Chunking
```python
def chunk_text(text, chunk_size=1000, overlap=200):
"""Split text into overlapping chunks."""
chunks = []
start = 0
text_len = len(text)
while start < text_len:
end = start + chunk_size
chunk = text[start:end]
*See sub-skills for full details.*
## Step 5: Embedding Generation
```python
from sentence_transformers import SentenceTransformer
import numpy as np
import pickle
import os
# Force CPU mode (for CUDA compatibility issues)
os.environ["CUDA_VISIBLE_DEVICES"] = ""
def create_embeddings(db_path, model_name='all-MiniLM-L6-v2', batch_size=100):
*See sub-skills for full details.*
## Step 6: Semantic Search
```python
def semantic_search(db_path, query, top_k=10, sample_size=50000):
"""Search for similar chunks using cosine similarity."""
# Force CPU mode
os.environ["CUDA_VISIBLE_DEVICES"] = ""
model = SentenceTransformer('all-MiniLM-L6-v2')
query_emb = model.encode(query, normalize_embeddings=True)
*See sub-skills for full details.*Related Skills
teams-meeting-pipeline
Operate the Teams meeting summary pipeline via Hermes CLI — summarize meetings, inspect pipeline status, replay jobs, manage Microsoft Graph subscriptions.
solidworks-to-blender-pipeline
Use when converting SolidWorks .sldprt/.sldasm geometry to Blender for rendering, animation, or visualization, including questions about STEP export settings, FreeCAD as a bridge, or which mesh format (STL/OBJ/GLTF) to choose.
multi-source-tax-document-reconciliation
Verify generated tax forms against source documents by line-by-line comparison, not just totals
multi-role-agent-contract-review-pipeline
Execute a 4-role agent team (Planner/Architect/Reviewer/Integrator) pipeline for self-reviewing knowledge artifacts before delivery
gtm-prospect-pipeline-phased-execution
Phased execution pattern for
documentation-contract-plan-hardening
Harden a documentation/contract plan before adversarial review by mapping every issue-scope requirement to independent acceptance criteria and tests, especially for routing/indexing contracts.
nextflow-pipelines
Run nf-core bioinformatics pipelines (rnaseq, sarek, atacseq) on sequencing data for gene expression, variant calling, and chromatin accessibility analyses.
ocr-and-documents
Extract text from PDFs and scanned documents. Use web_extract for remote URLs, pymupdf for local text-based PDFs, marker-pdf for OCR/scanned docs. For DOCX use python-docx, for PPTX see the powerpoint skill.
gmail-attachment-to-document
Download attachments from Gmail threads, parse their content (Excel, PDF), extract structured data, and save to target repos with proper legal scanning.
data-pipeline-processor
Process data files through transformation pipelines with validation, cleaning, and export. Use for CSV/Excel/JSON data processing, encoding handling, batch operations, and data transformation workflows.
knowledge-pipeline
Workflow for maintaining workspace-hub knowledge and learning pipelines across scripts/knowledge, scripts/learning, and docs/superpowers, including indexing, archive synthesis, issue updates, and pipeline troubleshooting.
document-rag-pipeline
Build complete document knowledge bases with PDF text extraction, OCR for scanned documents, vector embeddings, and semantic search. Use this for creating searchable document libraries from folders of PDFs, technical standards, or any document collection.