twitter-scraper
Search and scrape Twitter/X posts using Apify. Use when you need to find tweets, track brand mentions, monitor competitors on Twitter, or analyze Twitter discussions. Uses Twitter native search syntax (since:/until:) for reliable date filtering.
Best use case
twitter-scraper is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Search and scrape Twitter/X posts using Apify. Use when you need to find tweets, track brand mentions, monitor competitors on Twitter, or analyze Twitter discussions. Uses Twitter native search syntax (since:/until:) for reliable date filtering.
Teams using twitter-scraper should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/twitter-scraper/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How twitter-scraper Compares
| Feature / Agent | twitter-scraper | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Search and scrape Twitter/X posts using Apify. Use when you need to find tweets, track brand mentions, monitor competitors on Twitter, or analyze Twitter discussions. Uses Twitter native search syntax (since:/until:) for reliable date filtering.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# Twitter Scraper
Search Twitter/X posts using the Apify `apidojo/tweet-scraper` actor.
## Quick Start
Requires `APIFY_API_TOKEN` env var (or `--token` flag). Install dependency: `pip install requests`.
```bash
# Search with date range (recommended — uses Twitter native since:/until: operators)
python3 skills/twitter-scraper/scripts/search_twitter.py \
--query "YourCompany" --since 2026-02-15 --until 2026-02-23
# Quick summary of recent mentions
python3 skills/twitter-scraper/scripts/search_twitter.py \
--query "@yourhandle" --max-tweets 20 --output summary
# Search without date filtering
python3 skills/twitter-scraper/scripts/search_twitter.py \
--query "AI content marketing" --max-tweets 50
```
## Date Filtering
**Important:** The `apidojo/tweet-scraper` actor's built-in date parameters are unreliable.
This script embeds `since:YYYY-MM-DD` and `until:YYYY-MM-DD` directly into the search query
string, using Twitter's native advanced search syntax. This ensures date filtering works
correctly server-side.
## How the Script Works
1. Builds a search term with the query quoted and date operators appended
2. Calls the Apify `apidojo/tweet-scraper` actor via REST API
3. Polls until the run completes, then fetches the dataset
4. Deduplicates by tweet ID/URL
5. Applies optional keyword filtering (client-side)
6. Sorts by likes (descending) and outputs JSON or summary
## CLI Reference
| Flag | Default | Description |
|------|---------|-------------|
| `--query` | *required* | Search query (quoted in Twitter search) |
| `--since` | none | Start date YYYY-MM-DD (inclusive) |
| `--until` | none | End date YYYY-MM-DD (exclusive) |
| `--max-tweets` | 50 | Max tweets to scrape |
| `--keywords` | none | Additional filter keywords (comma-separated, OR logic) |
| `--output` | json | Output format: `json` or `summary` |
| `--token` | env var | Apify token (prefer `APIFY_API_TOKEN` env var) |
| `--timeout` | 300 | Max seconds to wait for the Apify run |
## Direct API Usage
```json
{
"searchTerms": ["\"YourCompany\" since:2026-02-15 until:2026-02-22"],
"maxTweets": 50,
"searchMode": "live"
}
```
## Output Format
Tweets are returned as JSON array sorted by likes. Each tweet has:
```json
{
"id": "...",
"text": "Tweet text...",
"fullText": "Full tweet text...",
"likeCount": 42,
"retweetCount": 5,
"replyCount": 3,
"viewCount": 1200,
"createdAt": "2026-02-18T12:00:00.000Z",
"author": {"userName": "handle", "name": "Display Name", ...},
"twitterUrl": "https://twitter.com/..."
}
```
## Common Workflows
### Competitor Monitoring
```bash
python3 skills/twitter-scraper/scripts/search_twitter.py \
--query "CompetitorName" --since 2026-02-15 --until 2026-02-23 --output summary
```
### Brand Mention Tracking
```bash
python3 skills/twitter-scraper/scripts/search_twitter.py \
--query "@YourHandle OR \"YourBrand\"" --max-tweets 100
```Related Skills
web-archive-scraper
Search the Wayback Machine for archived versions of websites. Extract cached pages, customer lists, testimonials, and partner directories from sites that have changed or gone offline. Uses the free CDX API — no API key needed.
review-scraper
Scrape product reviews from G2, Capterra, and Trustpilot using Apify. Single script with platform dispatch. Use when you need to monitor competitor reviews, track product sentiment, or gather customer feedback from review sites.
reddit-scraper
Scrape and search Reddit posts using Apify. Use when you need to find Reddit discussions, track competitor mentions, monitor product feedback, discover pain points, or analyze subreddit content. Supports keyword filtering, time-based searches, and subreddit-specific queries.
product-hunt-scraper
Scrape Product Hunt trending products using Apify. Use when you need to discover new product launches, track competitors on Product Hunt, or monitor the startup ecosystem for relevant launches.
meta-ad-scraper
Scrape competitor ads from Meta's Ad Library (Facebook, Instagram, Messenger, Threads, WhatsApp). Search by company name, Facebook Page URL, or keyword. Returns ad creatives, spend estimates, reach, impressions, and campaign details. Use for competitive ad research, messaging analysis, and creative inspiration.
linkedin-profile-post-scraper
Scrape recent posts from LinkedIn profiles using Apify. Use when you need to monitor what specific people are posting on LinkedIn, track founder/exec activity, or gather LinkedIn content for competitive intelligence.
linkedin-job-scraper
Scrapes LinkedIn job postings using the JobSpy library (python-jobspy). Use this skill whenever the user wants to find jobs on LinkedIn, search for open roles, pull job listings, build a job pipeline, source job targets for GTM research, or monitor hiring signals. Even if the user just says "find me some jobs" or "what roles is [company] hiring for", use this skill. It runs a local Python script that outputs a CSV of job postings with title, company, location, salary, job type, description, and direct URLs.
hacker-news-scraper
Search Hacker News stories and comments using the free Algolia API. No Apify token needed. Use when you need to find HN discussions, track mentions, discover Show HN launches, or monitor tech community sentiment.
google-ad-scraper
Scrape competitor ads from Google's Ads Transparency Center (Search, YouTube, Display, Gmail). Search by company name, domain, or advertiser ID. Returns ad creatives, formats, targeting regions, and campaign details. Use for competitive ad research and messaging analysis.
conference-speaker-scraper
Extract speaker names, titles, companies, and bios from conference websites. Supports direct HTML scraping and Apify web scraper fallback for JS-heavy sites. Use for pre-event research and outreach targeting.
blog-scraper
Scrape blog posts via RSS feeds (free, no API key) with Apify fallback for JS-heavy sites. Use when you need to monitor competitor blogs, track industry content, or aggregate blog posts by keyword.
signal-detection-pipeline
Detect buying signals from multiple sources, qualify leads, and generate outreach context