screen aware ai assistant
Screen-Aware AI Assistant
On-screen AI guidance grounded in what you're looking at right now. Not what you can remember to describe in a chat box.
Describing a UI from memory is lossy. You forget a menu name, misremember where the toggle moved, or spend three sentences on layout when the assistant just needs to see the button.
Screen-aware AI closes that gap: the model gets context from what's visible, not from your reconstruction of it.
GuideLayer captures on demand, reads the accessibility tree, and sends a reticle to the exact element. Not approximate coordinates.
What actually works
Accessibility-tree targeting
The guide cursor lands on the element node, not a guessed pixel region. If the button moved, it finds it again.
On-demand capture
ScreenCaptureKit snapshots when you trigger a question. Not continuous recording.
Works on any Mac app
Native apps, browser tabs, Electron tools. Not locked to one SaaS product's DOM.
Response cards over your workflow
Answers float above the app you're in. No full-screen takeover.
Workflow examples
Find a buried setting
System Settings moved the pane again. You know what you want, not where it lives.
- 01Open Settings. Ask GuideLayer how to change the setting.
- 02Screen context goes to the model. Reticle points at the control.
- 03Follow through without reading a 12-step help article.
Unfamiliar admin panel
Client's dashboard, vendor portal, internal tool. Docs don't match this instance.
- 01Ask about what's on screen: "where do I export users?"
- 02GuideLayer reads the live UI and points at the export button.
- 03Save the walkthrough if you'll need the same path again.
Cross-app workflow
Task starts in Mail, finishes in Preview, sends through a web portal.
- 01GuideLayer follows across apps as you switch.
- 02Each step grounded in the current window.
- 03Voice keeps your hands on the trackpad.
Screen-aware AI is only useful if the pointing is accurate. Approximate regions are just a slower way to guess.
Related guides
Try GuideLayer on your Mac
Screen-aware, private, local-first. Download for Apple Silicon with Claude Code and Codex already signed in.