CUPComputer Use Protocol
Schema

Node Structure

The schema for individual UI elements in the CUP tree.

Overview

Each node in the CUP tree represents a UI element. Nodes are recursive — they can contain child nodes.

Structure

{
  "id": "e14",
  "role": "button",
  "name": "Submit",
  "description": "Submit the form",
  "value": null,
  "bounds": { "x": 120, "y": 340, "w": 88, "h": 36 },
  "states": ["focused"],
  "actions": ["click"],
  "attributes": {
    "level": null,
    "placeholder": null,
    "orientation": null
  },
  "children": [],
  "platform": {
    "windows": {
      "controlType": "Button",
      "automationId": "submitBtn"
    }
  }
}

Core fields

FieldTypeDescription
idstringElement ID, pattern e\d+ (e.g., e0, e14)
rolestringOne of 59 canonical ARIA-derived roles
namestringAccessible name (max 200 chars)
descriptionstring?Secondary accessible description
valuestring?Current value for inputs, sliders
boundsobjectBounding rectangle { x, y, w, h }
statesarrayActive state flags (16 possible)
actionsarrayAvailable interaction verbs (15 possible)
childrenarrayNested child nodes

Attributes

Optional semantic attributes:

AttributeTypeDescription
levelint?Heading level (1-6)
valueMinnumber?Minimum range value
valueMaxnumber?Maximum range value
valueNownumber?Current range value
orientationstring?horizontal or vertical
rowIndexint?Table row index
colIndexint?Table column index
rowCountint?Total table rows
colCountint?Total table columns
posInSetint?Position in set (1-indexed)
setSizeint?Total set size
placeholderstring?Input placeholder text
urlstring?Link or image target URL
livestring?ARIA live region: polite, assertive, off
autocompletestring?inline, list, both, none
keyShortcutstring?Keyboard shortcut
roledescriptionstring?Custom role description

Platform-specific properties

The platform field preserves raw native properties for advanced use cases:

PlatformKey properties
WindowscontrolType, automationId, className, patterns, hwnd
macOSaxRole, axSubrole, axIdentifier, axActions
LinuxatspiRole, interfaces
WebtagName, ariaRole, selector, xpath, inputType
AndroidclassName, resourceId, packageName
iOSelementType, identifier, traits

On this page