knods

# Knods

## Overview

Handle two Knods modes:

1. Interactive canvas chat via the polling gateway
2. Headless flow execution via the REST API

Use the polling bridge for Knods Iris/chat payloads. Use the headless API when the task is to discover a flow, inspect inputs, run it, wait, cancel, or fetch outputs programmatically.

## Mode Selection

- Use **polling gateway mode** when input arrives as a Knods chat envelope with `messageId`, `message`, and `history`, and the response must stream back with optional `[KNODS_ACTION]...[/KNODS_ACTION]`.
- Use **headless API mode** when the user wants to:
  - list flows
  - search flows by name/description
  - inspect a flow's input schema
  - start a run
  - poll until completion
  - cancel a run
  - retrieve outputs programmatically

## Workflow

### A. Polling Gateway Flow

1. Parse incoming payload fields.
- Treat `message` as the primary request.
- Use `history` for continuity.
- On first turn in a conversation, expect prepended context in `message` describing node types and action rules. **Always prefer the node catalog from this context over the defaults below.**
- Use `messageId` to map all response chunks to the correct message.

2. Choose whether to emit a canvas action block.
- Use `addNode` for single-node additions.
- Use `addFlow` for multi-node workflows or any request requiring edges.
- If the user only asks a question, respond with normal text and no action block.

3. Build strict action JSON.
- Wrap each action exactly as:
  - `[KNODS_ACTION]{"action":"addNode","nodeType":"FluxImage"}[/KNODS_ACTION]`
  - `[KNODS_ACTION]{"action":"addFlow","nodes":[...],"edges":[...]}[/KNODS_ACTION]`
- Use `"nodeType"` (not `"type"`) in node objects. Do NOT include `position` or `data` fields — Knods handles layout automatically.
- For `addFlow`, ensure every edge `source` and `target` references an existing node id.
- Always end flows with an `Output` node.
- Never connect two generator nodes directly; route through `Output`.
- Use stable node IDs (for example `n1`, `n2`, `n3`) so follow-up edits are easy.
- Avoid unknown keys in action JSON.

4. Stream response back to Knods.
- Send assistant text as delta chunks to `/respond` for the same `messageId`.
- Send `{"messageId":"...","done":true}` when complete.
- Keep first chunk quick to avoid timeout perception.

### B. Headless API Flow

1. Discover candidate flows.
- Run:
  - `python3 {baseDir}/scripts/knods_headless.py list`
  - or `python3 {baseDir}/scripts/knods_headless.py resolve --query "<text>"`

2. Inspect the selected flow.
- Run:
  - `python3 {baseDir}/scripts/knods_headless.py get --flow-id "<flowId>"`
- Read `inputs` and preserve every `nodeId` exactly.

3. Start a run.
- Build `inputs` as JSON array with `nodeId`, `content`, and `type`.
- Run:
  - `python3 {baseDir}/scripts/knods_headless.py run --flow-id "<flowId>" --inputs-json '[...]'`

4. Poll until terminal state.
- Prefer:
  - `python3 {baseDir}/scripts/knods_headless.py wait --run-id "<runId>"`
- Or use:
  - `python3 {baseDir}/scripts/knods_headless.py run-wait --flow-id "<flowId>" --inputs-json '[...]'`

5. Handle result.
- On `completed`, read `outputs`
- On `failed`, surface `error.message` and `error.nodeId` if present
- On timeout, optionally cancel the run

## Output Rules

- Return normal assistant text; do not wrap the full reply in a custom envelope.
- Include `[KNODS_ACTION]...[/KNODS_ACTION]` inline only when a canvas mutation is intended.
- Do not mention internal polling URLs/tokens in user-facing text.
- Keep action JSON valid and compact.

## Node Catalog

**IMPORTANT:** Every generator node listed below has a built-in prompt textarea. Do NOT add a DocumentPanel before a single generator — just connect the generator directly to an Output. Only use DocumentPanel when one shared prompt feeds multiple generators in parallel.

When the first message includes a node catalog context, **always use that list** over these defaults. The context catalog is always more up-to-date.

### Text Generators (output: text)
All text generators accept text + image input and have a built-in prompt textarea.
- `ChatGPT` — OpenAI models. Best all-rounder.
- `Claude` — Anthropic models. Great for reasoning and creative writing.

### Image Generators (output: image)
All image generators have a built-in prompt textarea and accept optional image input for image-to-image editing.
- `GPTImage` — OpenAI. Best at following complex instructions and text rendering.
- `FluxImage` — FLUX by Black Forest Labs. Industry-leading quality for portraits and artistic styles. Fast.
- `ImagePrompt` — Google Gemini. Great for photorealistic images and concept art.
- `ZImageTurbo` — Lightning-fast (<2 seconds). Best for rapid prototyping.
- `QwenImage` — Alibaba Qwen. Strong at anime, illustrations, and Asian-inspired aesthetics.
- `Seedream` — ByteDance. Dreamy, surreal compositions. Good at text rendering in images.
- `GrokImage` — xAI. Text-to-image and image editing.

### Video Generators (output: video)
All video generators below have a built-in prompt textarea and support both text-to-video and image-to-video (connect an ImagePanel for image-to-video).
- `Veo3FalAI` — Google Veo 3.1. Cinematic video up to 8s with native audio. Best overall quality.
- `Sora2Video` — OpenAI Sora 2. Realistic motion and physics, up to 12s.
- `Kling26Video` — Kling 2.6 Surreal Engine. Cinematic with audio, up to 10s.
- `KlingO3Video` — Kling 3.0. Latest generation, Standard/Pro quality, up to 10s.
- `Wan26Video` — Wan 2.6. Multi-shot videos, 720p/1080p, up to 15s.
- `LTXVideo` — LTX-2 Pro. High-fidelity cinematic with synchronized audio.
- `GrokVideo` — xAI. Video with native audio.

### Special Video Node
- `WanAnimateVideo` — Character animation. **REQUIRES two inputs**: a VIDEO (motion reference) + an IMAGE (character to animate). Does NOT have a text prompt. Only use when user wants to animate a character image using motion from another video.

### Input/Container Nodes
- `ImagePanel` — Upload or paste an image. Output: image. Use when user wants to provide a reference image or a starting frame for image-to-video.
- `DocumentPanel` — Editable text container. Output: text. Use ONLY when one shared prompt feeds multiple generators in parallel.
- `Output` — Displays generated results (text, image, video). REQUIRED at the end of every flow.

## Flow Design Rules

1. **Every generator has a built-in prompt textarea.** Never prepend a DocumentPanel to a single generator.
2. **Use DocumentPanel only** for one shared prompt feeding multiple generators in parallel.
3. **Use ImagePanel** when user wants to provide a reference image, a starting frame for video, or an image input for WanAnimateVideo.
4. **Always end flows with an Output node.**
5. **Never connect two generators directly.** Route through an Output node if chaining.
6. **Flows go left to right:** inputs → generators → Output.
7. **Use EXACT PascalCase node names** from the catalog. Do NOT invent node names.
8. **WanAnimateVideo** is the only node that requires a video input. Only suggest it when the user specifically wants to animate a character image using motion from a video.
9. Add `initialData` only when user intent clearly implies parameters.
10. Build the smallest flow that satisfies the request.

## Flow Examples

Single image generator (most common):
```
FluxImage → Output
```

Image from reference photo:
```
ImagePanel → GPTImage → Output
```

One prompt feeding two image generators:
```
DocumentPanel → FluxImage → Output
DocumentPanel → GPTImage → Output
```

Text-to-video:
```
Veo3FalAI → Output
```

Image-to-video (animate a still image):
```
ImagePanel → Veo3FalAI → Output
```

Character animation from video motion (WanAnimateVideo needs both video + image):
```
ImagePanel → WanAnimateVideo → Output
[video source] → WanAnimateVideo
```

Text generation:
```
ChatGPT → Output
```

## Gateway Behavior Constraints

- Poll interval target: about 1-2 seconds.
- Message claim timeout: about 2 minutes.
- Always preserve `messageId` across all chunk posts for a turn.
- Gateway auth uses `gw_...` token via query parameter `token`; never require Supabase JWT in this flow.

## Runtime Operations

When running a persistent poller service/process:

- Support either configuration style:
  - `KNODS_BASE_URL` already includes `/updates?token=...`
  - or `KNODS_BASE_URL` points to connection base and token is supplied separately (`KNODS_GATEWAY_TOKEN`)
- Derive `/respond` from the same connection root as `/updates`.
- Log handled `messageId` values and transport errors for debugging.

For headless API operations:

- Prefer `KNODS_API_BASE_URL` + `KNODS_API_KEY`
- `KNODS_API_BASE_URL` should look like `https://<instance>/api/v1`
- `KNODS_API_KEY` must have `knods:read` and `knods:run`
- If the API base URL is omitted, the packaged client can derive it from the same host as `KNODS_BASE_URL`

### Packaged Runtime (required)

This skill ships the runtime bridge and installer:

- `scripts/knods_iris_bridge.py`
- `scripts/knods_headless.py`
- `scripts/install_local.sh`

Install/deploy from the skill folder:

```bash
bash /home/rolf/.openclaw/skills/knods/scripts/install_local.sh
```

The installer deploys:

- `~/.openclaw/scripts/knods_iris_bridge.py`
- `~/.config/systemd/user/knods-iris-bridge.service`

Then runs:

- `systemctl --user daemon-reload`
- `systemctl --user enable --now knods-iris-bridge.service`

### Environment Variables

Set these in `~/.openclaw/.env`:

- Required for polling gateway mode:
  - `KNODS_BASE_URL`
- Required when `KNODS_BASE_URL` does not already include `?token=...`:
  - `KNODS_GATEWAY_TOKEN`
- Required for headless API mode:
  - `KNODS_API_KEY`
- Preferred for headless API mode:
  - `KNODS_API_BASE_URL`
- Optional:
  - `OPENCLAW_AGENT_ID` (default: `iris`)
  - `OPENCLAW_BIN` (default: `openclaw` on `PATH`)

### Service Operations

- Status:
  - `systemctl --user status knods-iris-bridge.service`
- Restart:
  - `systemctl --user restart knods-iris-bridge.service`
- Logs:
  - `journalctl --user -u knods-iris-bridge.service -f`

### Config Change Lifecycle (required)

After changing gateway URL/token env values, restart the running bridge process so it reloads config.

- Generic service form:
  - `systemctl --user restart <knods-bridge-service>`
- Generic process form:
  - stop old process
  - start poller again with updated env

Do not assume env changes are picked up live without restart.

## Reference

- Read `references/protocol.md` for canonical polling endpoints, payload schemas, and action examples.
- Read `references/headless-api.md` for the direct run/list/poll/cancel flow execution API.