One of the most annoying things about giving an agent desktop control is the inevitable tug-of-war for your mouse and keyboard. You want to keep working, but the agent keeps hijacking your screen.

Hermes just solved that with a different approach to computer use.

Its computer_use tool can click, type, scroll, and drag inside native macOS, Windows, and Linux apps—all in the background. Your real cursor stays put, and your keyboard focus doesn't jump around. Hermes quietly works in another app without taking over your desktop.

This is a game-changer for workflows outside the browser:

• Searching through a desktop mail app
• Navigating native business software
• Automating GUI steps that have no API or CLI path
• Adding GUI actions to a larger agent workflow

Getting started is simple:

hermes computer-use install
hermes computer-use status
hermes computer-use doctor  # if something's off

Then enable it with:

hermes -t computer_use chat

Or just add computer_use to your enabled toolsets in the config.

Of course, the key is knowing when not to use it. If you're working entirely on the web, the browser toolset is cleaner. If there's an API, terminal command, or file operation that does the job, use that first.

Computer use earns its place when the GUI itself is the only practical path.

And that's what I love about this design—Hermes can operate the GUI without making your computer unusable while it works. Finally, an agent that doesn't fight you for control.