CUPComputer Use Protocol
Platform Guides

Windows

CUP on Windows using UI Automation (UIA) via COM.

Overview

On Windows, CUP uses the UI Automation (UIA) framework via COM interop. This is the primary accessibility API on Windows 10 and later.

Requirements

  • Windows 10 or later
  • Python: comtypes (installed automatically)
  • TypeScript: PowerShell + C# interop (bundled)

No additional setup is needed — CUP auto-detects Windows and loads the UIA adapter.

How it works

The Windows adapter uses comtypes (Python) or PowerShell with inline C# (TypeScript) to connect to the UIA COM interface. It walks the automation tree, reading control types, names, patterns, and bounding rectangles.

Role mapping

Windows UIA control types map to CUP roles:

UIA ControlTypeCUP Role
Buttonbutton
CheckBoxcheckbox
ComboBoxcombobox
Edittextbox
Hyperlinklink
Imageimg
Listlist
ListItemlistitem
Menumenu
MenuItemmenuitem
ProgressBarprogressbar
RadioButtonradio
Sliderslider
Tabtablist
TabItemtab
Tabletable
Texttext
Windowwindow

Action mapping

CUP actions map to UIA control patterns:

CUP ActionUIA Pattern
clickInvokePattern
toggleTogglePattern
expandExpandCollapsePattern
selectSelectionItemPattern
setvalueValuePattern
scrollScrollPattern

Platform-specific properties

For the full platform-specific property schema, see Node Structure.

Use detail="full" to include platform-specific properties. In compact mode, only the canonical CUP properties are included.

On this page