Computer Use
Computer Use lets Fluent do things in your Mac's other apps: it reads what's in an app's window, then clicks, types, scrolls and drags there with a pointer of its own. Your own mouse and keyboard stay yours, so you can keep working while it does.
Ask in the Smart Panel or the chat window, and name the app when more than one could do it.
Try Asking
| Ask | What Fluent does |
|---|---|
| "Fill in the shipping form in Safari with my address from Memory" | Finds the form's fields and types into each one |
| "Copy the totals from this PDF into the open Numbers sheet" (with the PDF attached) | Reads the PDF, then types each total into its cell |
| "In Music, find Kind of Blue and play it" | Searches the catalog and starts the album |
| "What do you see on my screen?" | Takes a screenshot of the display and describes it |
| "Fill in the rest of October in my timesheet in Chrome" | Works down the whole page, row by row, to its end |
How It Works
- Fluent reads the app's buttons, fields, lists and text through Accessibility, in the background: the app doesn't have to be in front.
- It acts with its own pointer, drawn over the app, then looks at the app again to check what changed.
- The answer's badge says what it's doing ("Working with Music"); click the badge for the details.
Some apps show a button only while the pointer is over an item, like Play on an album's artwork. macOS shows those under your own pointer only, not Fluent's, so Fluent opens the item's right-click menu instead, where apps usually offer the same command.
On a long web page, Fluent sees the rows in view first and scrolls for the rest, so it can fill in a whole timesheet or form.
Fluent can work in any app that's running, except Fluent itself. It works best in apps whose controls Accessibility can read, as most Mac apps' are. For a game or a drawing canvas, it can look at screenshots instead.
Turn It On
Computer Use is on by default. Click it in Settings → Integrations to open its page:
| Permission | Needed for | On its page |
|---|---|---|
| Accessibility | Reading and controlling apps | Allow Accessibility, which opens System Settings |
| Screenshots (Screen Recording) | Only when Fluent should look at a window or your screen as a picture | Allow Screenshots |
A permission that's granted shows Allowed. Below them, the Browser section sets up how it works in your browser, see In Your Browser below. To keep Computer Use out of your requests, turn it off with the switch at the top of its page, or its checkbox in the list of integrations.
Stop Or Approve
- Esc stops Fluent's pointer in the middle of a step, and Fluent is told that you stopped it. With the Smart Panel in front, Esc also closes the panel, which ends the request too unless Keep requests running in background is on.
- To end the whole request, click the ring in the prompt row, as for any answer.
- Computer Use doesn't ask before it acts. To confirm each step, tick Ask Before Running for its Interact tool, on the same page: Fluent then waits for Approve before each batch of clicks and keys.
In Your Browser
In a browser, Computer Use works as it does in any app, signed in to your sites as you are, which can bring the browser's window forward. To have it work in the background, set up one of these in the Browser section of Computer Use's page:
| Chrome extension (recommended) | Remote debugging | |
|---|---|---|
| Works in | Chrome and the browsers built on it, such as Arc, Brave and Edge | Chrome, Edge, Brave, Vivaldi, Arc, Helium and Chromium |
| In the background | In background tabs, even while you're away | In the tabs you have open; the browser asks for your approval after every restart, so not while you're away |
| Used by | Computer Use and your teammates | Computer Use only |
| Set it up | Click Add to Chrome (or your browser's name), then add Fluent from the Chrome Web Store. The row then says Connected. | Turn it on, click Open Browser Settings, and turn on Allow remote debugging on the page that opens. Approve Fluent when the browser asks. |
Neither is required. When a browser isn't connected, Computer Use says so once and keeps working in it the usual way. Remote debugging gives Fluent broad access to the browser, so turn it on only if you want Fluent working in your tabs.
The Extension's Menu
Click Fluent's button in the browser's toolbar:
| Item | What it does |
|---|---|
| Ask Fluent | Opens the Smart Panel with the page in its prompt |
| Add to Memory | Keeps the page's text in Memory's Inbox, read as you see it, signed in |
| Hire a Teammate | Starts a hire for the page in the chat window, for example a teammate that tells you when its price drops. See Team |
| Working Now | The tabs Fluent is working in, each with Stop |
| Fluent's Tabs | The tabs Fluent opened, kept together in a tab group named Fluent |
While Fluent works in a tab, the browser shows a bar saying it's being controlled. To stop it, click Stop in the menu, or cancel the bar.
If It Doesn't Work
| What happens | What to do |
|---|---|
| Fluent says it can't read or control the app | Allow Fluent in System Settings → Privacy & Security → Accessibility, or click Allow Accessibility in Computer Use's settings |
| It can't take a screenshot | Click Allow Screenshots and turn on Fluent under Screen Recording; reopen Fluent if macOS asks |
| The browser comes to the front | Add the Chrome extension, or turn on Remote debugging, in Computer Use's Browser section |
| The extension's menu says Fluent isn't running, or the browser isn't set up | Open Fluent; in Settings → Integrations → Computer Use, the Chrome extension row says what it needs, such as folder access for that browser |
| It works in the wrong app | Name the app in your request: "in Numbers, …" |