nltk-linguistics

NLP and corpus analysis via NLTK. Use when: user asks about tokenization, POS tagging, parsing, sentiment, or corpus statistics. NOT for: deep learning NLP or transformer models.

564 stars

Best use case

nltk-linguistics is best used when you need a repeatable AI agent workflow instead of a one-off prompt.

NLP and corpus analysis via NLTK. Use when: user asks about tokenization, POS tagging, parsing, sentiment, or corpus statistics. NOT for: deep learning NLP or transformer models.

Teams using nltk-linguistics should expect a more consistent output, faster repeated execution, less prompt rewriting.

When to use this skill

  • You want a reusable workflow that can be run more than once with consistent structure.

When not to use this skill

  • You only need a quick one-off answer and do not need a reusable workflow.
  • You cannot install or maintain the underlying files, dependencies, or repository context.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/nltk-linguistics/SKILL.md --create-dirs "https://raw.githubusercontent.com/beita6969/ScienceClaw/main/skills/nltk-linguistics/SKILL.md"

Manual Installation

  1. Download SKILL.md from GitHub
  2. Place it in .claude/skills/nltk-linguistics/SKILL.md inside your project
  3. Restart your AI agent — it will auto-discover the skill

How nltk-linguistics Compares

Feature / Agentnltk-linguisticsStandard Approach
Platform SupportNot specifiedLimited / Varies
Context Awareness High Baseline
Installation ComplexityUnknownN/A

Frequently Asked Questions

What does this skill do?

NLP and corpus analysis via NLTK. Use when: user asks about tokenization, POS tagging, parsing, sentiment, or corpus statistics. NOT for: deep learning NLP or transformer models.

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

SKILL.md Source

# NLTK Linguistics

Natural language processing and corpus analysis using NLTK.

## Setup

```python
import nltk
for pkg in ['punkt_tab', 'averaged_perceptron_tagger_eng', 'maxent_ne_chunker_tab',
            'words', 'vader_lexicon', 'wordnet', 'stopwords']:
    nltk.download(pkg, quiet=True)
```

## Tokenization

```python
from nltk.tokenize import word_tokenize, sent_tokenize
sentences = sent_tokenize(text)
words = word_tokenize(text)
```

## POS Tagging

```python
from nltk import pos_tag
from nltk.tokenize import word_tokenize
tagged = pos_tag(word_tokenize(text))  # list of (word, tag) tuples
# Tags: NN=noun, VB=verb, JJ=adjective, RB=adverb, DT=determiner
```

## Named Entity Recognition

```python
from nltk import ne_chunk, pos_tag, word_tokenize
tree = ne_chunk(pos_tag(word_tokenize(text)))
for subtree in tree:
    if hasattr(subtree, 'label'):
        entity = " ".join(word for word, tag in subtree.leaves())
        print(f"{subtree.label()}: {entity}")
```

## Sentiment Analysis (VADER)

```python
from nltk.sentiment.vader import SentimentIntensityAnalyzer
sia = SentimentIntensityAnalyzer()
scores = sia.polarity_scores(text)
# Returns: {'neg': 0.0, 'neu': 0.5, 'pos': 0.5, 'compound': 0.6369}
# compound: -1 (most negative) to +1 (most positive)
```

## Frequency Distributions and Concordance

```python
from nltk import FreqDist, Text
from nltk.tokenize import word_tokenize
fdist = FreqDist(word_tokenize(text.lower()))
fdist.most_common(20)                       # top 20 words
t = Text(word_tokenize(text))
t.concordance('language', width=80)         # keyword-in-context
t.collocations()                            # frequent bigrams
```

## WordNet Lookups

```python
from nltk.corpus import wordnet as wn
synsets = wn.synsets('bank')                # all senses
defn = synsets[0].definition()              # definition string
sim = wn.synset('dog.n.01').wup_similarity(wn.synset('cat.n.01'))  # Wu-Palmer similarity
synonyms = [l.name() for s in wn.synsets('good') for l in s.lemmas()]
hypernyms = wn.synset('dog.n.01').hypernyms()
```

## Stopword Filtering

```python
from nltk.corpus import stopwords
stop_words = set(stopwords.words('english'))
filtered = [w for w in tokens if w.lower() not in stop_words]
```

## Best Practices

1. Always download required NLTK data before first use.
2. Use `word_tokenize` over `split()` for proper tokenization.
3. VADER works best on short social-media-style text.
4. For large corpora, consider streaming with `PlaintextCorpusReader`.
5. POS tag sets: use `nltk.help.upenn_tagset()` for tag reference.
6. WordNet similarity requires both synsets to share a common hypernym.

Related Skills

linguistics-analysis

564
from beita6969/ScienceClaw

Analyze language structures, typological features, and semantic change across languages

xurl

564
from beita6969/ScienceClaw

A CLI tool for making authenticated requests to the X (Twitter) API. Use this skill when you need to post tweets, reply, quote, search, read posts, manage followers, send DMs, upload media, or interact with any X API v2 endpoint.

xlsx

564
from beita6969/ScienceClaw

Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.

writing

564
from beita6969/ScienceClaw

No description provided.

world-bank-data

564
from beita6969/ScienceClaw

World Bank Open Data API for development indicators. Use when: user asks about GDP, population, poverty, health, or education statistics by country. NOT for: real-time financial data or stock prices.

wikipedia-search

564
from beita6969/ScienceClaw

Search and fetch structured content from Wikipedia using the MediaWiki API for reliable, encyclopedic information

wikidata-knowledge

564
from beita6969/ScienceClaw

Query Wikidata for structured knowledge using SPARQL and entity search. Use when: (1) finding structured facts about entities (people, places, organizations), (2) querying relationships between entities, (3) cross-referencing external identifiers (Wikipedia, VIAF, GND, ORCID), (4) building knowledge graphs from linked data. NOT for: full-text article content (use Wikipedia API), scientific literature (use semantic-scholar), geospatial data (use OpenStreetMap).

weather

564
from beita6969/ScienceClaw

Get current weather and forecasts via wttr.in or Open-Meteo. Use when: user asks about weather, temperature, or forecasts for any location. NOT for: historical weather data, severe weather alerts, or detailed meteorological analysis. No API key needed.

wacli

564
from beita6969/ScienceClaw

Send WhatsApp messages to other people or search/sync WhatsApp history via the wacli CLI (not for normal user chats).

voice-call

564
from beita6969/ScienceClaw

Start voice calls via the OpenClaw voice-call plugin.

visualization

564
from beita6969/ScienceClaw

Create publication-quality scientific figures and plots using Python (matplotlib, seaborn, plotly). Supports bar charts, scatter plots, heatmaps, box plots, violin plots, survival curves, network graphs, and more. Use when user asks to plot data, create figures, make charts, visualize results, or generate publication-ready graphics. Triggers on "plot", "chart", "figure", "graph", "visualize", "heatmap", "scatter plot", "bar chart", "histogram".

video-frames

564
from beita6969/ScienceClaw

Extract frames or short clips from videos using ffmpeg.