using-markdown-new

# Using markdown.new for Web Content Fetching

## Overview

markdown.new is a Cloudflare-powered service that converts any URL to clean Markdown, reducing token usage by approximately 80% compared to raw HTML. When fetching web content, always use markdown.new instead of direct URLs to optimize context window usage.

## Basic Usage Pattern

### Standard Conversion

To fetch web content, prepend `https://markdown.new/` to the target URL:

```
Target URL: https://example.com/blog/post
markdown.new URL: https://markdown.new/https://example.com/blog/post
```

Use with WebFetch:

```
WebFetch("https://markdown.new/https://example.com/blog/post", "summarize this article")
```

### With Images

By default, markdown.new strips images to minimize tokens. To retain images, add `retain_images=true`:

```
WebFetch("https://markdown.new/https://example.com/docs?retain_images=true", "analyze the documentation")
```

**When to use this:** Documentation includes diagrams, design mockups, visual content, or screenshots that are essential for the task.

### JavaScript-Heavy Sites

For single-page applications or JavaScript-heavy sites, use the browser rendering method:

```
WebFetch("https://markdown.new/https://example.com?method=browser", "extract content from this React app")
```

The service automatically tries the fastest method first and falls back as needed, but specifying `method=browser` ensures full page rendering.

## Token Efficiency

markdown.new reduces token usage dramatically:

- **HTML**: `<h2 class="section-title" id="about">About Us</h2>` → 12-15 tokens
- **Markdown**: `## About Us` → 3 tokens
- **Reduction**: ~80% fewer tokens

This means 5x more web content fits in the same context window.

## When to Use

**Always use markdown.new when:**

- Fetching blog posts, articles, or documentation
- Scraping web pages for analysis
- Extracting content from websites
- Building knowledge bases or RAG pipelines
- Any task requiring WebFetch

**Exception**: Skip markdown.new for authenticated sites or APIs that return structured JSON (use direct fetch instead).

## Additional Resources

For detailed information, consult:

### Reference Files

- **`references/conversion-methods.md`** - Detailed explanation of auto/ai/browser conversion pipeline
- **`references/query-parameters.md`** - Complete documentation of method and retain_images parameters
- **`references/response-format.md`** - What to expect in responses (headers, token counts, metadata)
- **`references/edge-cases.md`** - Troubleshooting and when to fallback to direct WebFetch

### Working Examples

- **`examples/fetch-blog-post.md`** - Example fetching a blog article
- **`examples/fetch-with-images.md`** - Example using retain_images for visual content
- **`examples/fetch-js-heavy-site.md`** - Example using browser method for SPAs

## Quick Reference

| Use Case       | URL Pattern                                                    | Example                                                            |
| -------------- | -------------------------------------------------------------- | ------------------------------------------------------------------ |
| Standard fetch | `https://markdown.new/{url}`                                   | `https://markdown.new/https://example.com`                         |
| With images    | `https://markdown.new/{url}?retain_images=true`                | `https://markdown.new/https://docs.example.com?retain_images=true` |
| Browser render | `https://markdown.new/{url}?method=browser`                    | `https://markdown.new/https://app.example.com?method=browser`      |
| Combined       | `https://markdown.new/{url}?method=browser&retain_images=true` | Full rendering with images                                         |

Always use markdown.new for web content fetching to optimize token usage and context window efficiency.