desktop-find-element

Find desktop UI elements using UIAutomation (fast, accurate) or vision recognition (fallback). When you need to locate buttons/menus/icons, get element positions before clicking, or verify UI state. For browser webpage elements, use browser_* tools instead.

1,592 stars

Best use case

desktop-find-element is best used when you need a repeatable AI agent workflow instead of a one-off prompt.

Find desktop UI elements using UIAutomation (fast, accurate) or vision recognition (fallback). When you need to locate buttons/menus/icons, get element positions before clicking, or verify UI state. For browser webpage elements, use browser_* tools instead.

Teams using desktop-find-element should expect a more consistent output, faster repeated execution, less prompt rewriting.

When to use this skill

  • You want a reusable workflow that can be run more than once with consistent structure.

When not to use this skill

  • You only need a quick one-off answer and do not need a reusable workflow.
  • You cannot install or maintain the underlying files, dependencies, or repository context.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/desktop-find-element/SKILL.md --create-dirs "https://raw.githubusercontent.com/openakita/openakita/main/skills/system/desktop-find-element/SKILL.md"

Manual Installation

  1. Download SKILL.md from GitHub
  2. Place it in .claude/skills/desktop-find-element/SKILL.md inside your project
  3. Restart your AI agent — it will auto-discover the skill

How desktop-find-element Compares

Feature / Agentdesktop-find-elementStandard Approach
Platform SupportNot specifiedLimited / Varies
Context Awareness High Baseline
Installation ComplexityUnknownN/A

Frequently Asked Questions

What does this skill do?

Find desktop UI elements using UIAutomation (fast, accurate) or vision recognition (fallback). When you need to locate buttons/menus/icons, get element positions before clicking, or verify UI state. For browser webpage elements, use browser_* tools instead.

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

SKILL.md Source

# Desktop Find Element

查找桌面 UI 元素。优先使用 UIAutomation(快速准确),失败时用视觉识别(通用)。

## Parameters

| 参数 | 类型 | 必填 | 说明 |
|------|------|------|------|
| target | string | 是 | 元素描述,如 '保存按钮'、'name:文件'、'id:btn_ok' |
| window_title | string | 否 | 限定在某个窗口内查找 |
| method | string | 否 | 查找方法:auto(默认)、uia、vision |

## Supported Target Formats

- 自然语言:"保存按钮"、"红色图标"
- 按名称:"name:保存"
- 按 ID:"id:btn_save"
- 按类型:"type:Button"

## Find Methods

- `auto`: 自动选择(推荐)
- `uia`: 只用 UIAutomation
- `vision`: 只用视觉识别

## Returns

- 元素位置(x, y)
- 元素大小
- 元素属性

## Warning

如果操作的是浏览器内的网页元素,请使用 `browser_*` 工具。

## Related Skills

- `desktop-click`: 点击找到的元素
- `desktop-inspect`: 查看元素树结构

Related Skills

find-skills

1592
from openakita/openakita

Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.

desktop-window

1592
from openakita/openakita

Window management operations. When you need to list all open windows, switch to a specific window, minimize/maximize/restore windows, or close windows. Use title parameter for targeting specific window (fuzzy match).

desktop-wait

1592
from openakita/openakita

Wait for UI element or window to appear. When you need to wait for dialog to open, loading to complete, or synchronize with application state before next action. Default timeout is 10 seconds.

desktop-type

1592
from openakita/openakita

Type text at current cursor position in desktop applications. When you need to enter text in dialogs, fill input fields, or type in text editors. Supports Chinese input. For browser webpage forms, use browser_type instead.

desktop-scroll

1592
from openakita/openakita

Scroll mouse wheel in specified direction. When you need to scroll page/document content, navigate long lists, or zoom in/out with Ctrl. Directions - up/down/left/right.

desktop-screenshot

1592
from openakita/openakita

Capture Windows desktop screenshot with automatic file saving. When you need to show desktop state, capture application windows, or record operation results. IMPORTANT - must actually call this tool, never say 'screenshot done' without calling. Returns file_path for deliver_artifacts.

desktop-inspect

1592
from openakita/openakita

Inspect window UI element tree structure for debugging and understanding interface layout. When you need to debug UI automation issues, understand application structure, or find correct element identifiers.

desktop-hotkey

1592
from openakita/openakita

Execute keyboard shortcuts. When you need to copy/paste (Ctrl+C/V), save files (Ctrl+S), close windows (Alt+F4), undo/redo (Ctrl+Z/Y), or select all (Ctrl+A).

desktop-click

1592
from openakita/openakita

Click desktop elements or coordinates. When you need to click buttons/icons in applications, select menu items, or interact with desktop UI. Supports element description, name prefix, or coordinates. For browser webpage elements, use browser_click instead.

openakita/skills@yuque-skills

1592
from openakita/openakita

Manage Yuque (语雀) knowledge bases, documents, and team collaboration through API integration. Supports personal search, weekly reports, knowledge base management, document CRUD, and group collaboration workflows. Based on yuque/yuque-skills.

openakita/skills@youtube-summarizer

1592
from openakita/openakita

Summarize YouTube videos by extracting transcripts and generating structured notes. Use when the user wants to summarize a YouTube video, extract key points from a talk, create study notes from a lecture, or get timestamps for important moments. Supports multiple URL formats and languages.

openakita/skills@xlsx

1592
from openakita/openakita

Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.