Let the agent use your computer
Computer Use lets the agent operate your computer: take screenshots, read the accessibility tree, move and click the mouse, type, and manage windows and applications. Use it when a task lives in a desktop app that has no API or command line.
It's off by default. Enable it under Settings → Extensions → Computer Use. There's no panel. Turning it on makes the computer_use_* tools available to the agent in chat.
Decide when to use Computer Use
- Driving a legacy or GUI-only desktop application
- Reproducing a bug exactly as a user would see it
- Multi-app sequences that no CLI covers end to end
- Reading what's on screen when no file or API exposes it
Prefer a narrower tool when one exists: the Terminal for commands, in-app Browser tools for web pages, and Browser Use for your signed-in Chrome. Computer Use is the broadest and least contained. It doesn't reuse your Chrome session, and it isn't the in-app Browser.
Computer Use isn't a sandbox. Its tools can see your screen and send input to any local application. Keep it off when you don't need it. Your approval mode and your organization's policy still govern agent actions.
Turn on and use Computer Use
Enable the extension
Navigate to Settings → Extensions → Computer Use.
Grant operating-system permissions
On macOS, enable Accessibility and Screen & System Audio Recording under System Settings → Privacy & Security, then restart Glean Tau so the permission change is picked up. These grants are independent of the extension toggle, so turning the extension on isn't enough by itself.
On Windows, Glean Tau uses UI Automation and screen capture. Keep the app running as a standard user. Elevated windows, protected content, and endpoint security policy can prevent an operation. Ask your admin to check the required access rather than run Glean Tau as Administrator.
Ask for the outcome
Ask for the outcome in a chat and name the app and the goal, for example: "Open the VPN client, take a screenshot of the connection status, and tell me which profile is active."
Diagnose failures
If tools fail, ask Glean Tau to run the Computer Use health check. It reports on the driver executable, its connection, permissions, and whether any windows were found.
Computer Use actions remain subject to your approval mode and organization policy. Automatic review can ask you or deny an action. If your organization requires a confinement boundary that Computer Use can't satisfy, the feature is unavailable. Use the health check to diagnose permission or connection problems. See Agent approval modes.
Limitations
- Screen-dependent. Actions rely on what's visible and on the accessibility tree, so minimized windows, locked screens, and protected content can block a step.
See also
- Browse and automate web pages — work with web pages in the in-app browser or your signed-in Chrome
- Extensions — the full list of extensions and their default states