What Actually Leaves Your Mac: Screenshots, OCR Text, and the AI Call

"Local-first" is easy to say and easy to overclaim. The skeptical version of the question — the one that shows up in every Reddit thread about screen-tracking tools — is more specific: if an AI classifies my screen activity, then something from my screen is going to an AI. What exactly? And what happens when a credit card number or a password is on that screen? This page answers that question at the pipeline level, including the parts that make the honest answer more nuanced than a slogan.

The short version

Screenshots never leave your Mac. For each classified activity, exactly one thing goes off-device: a masked text snippet extracted by on-device OCR, together with the app name and window title, sent for a single AI classification call. Before that snippet leaves your machine, personal data in it is replaced with tokens — and screens that look sensitive at all (banking, passwords, payments, identity) are skipped before capture, so there is no snippet to send in the first place.

Step 0: sensitive apps are skipped before anything is captured

Before Kloy takes a screenshot, AI looks at two pieces of metadata only — the app name and the window title — and decides whether the surface is sensitive: banking or brokerage apps, password managers, payment and checkout forms, one-time-code screens, government ID, medical records. If the answer is yes, the pipeline stops there. No screenshot, no OCR, no classification call. The activity still appears in your local timeline, marked as a protected app, so the record is honest about the gap.

You can also exclude any app yourself in Settings; your list is checked first and doesn't depend on the AI's judgment. This ordering matters: content protection that starts after capture is already one step too late for the screens that matter most.

Step 1: capture and OCR stay on-device

For screens that pass the gate, capture and text extraction run entirely on your Mac using Apple's on-device Vision OCR. The sharp capture is discarded after OCR; if you keep previews enabled, only a privacy-blurred thumbnail is stored locally for your own review. No image — sharp or blurred — is ever uploaded.

Step 2: personal data is masked before the text leaves

The OCR text is redacted locally before anything else touches it: checksum-validated card numbers become [CARD], Korean resident registration numbers become [RRN], emails and phone numbers become [EMAIL] and [PHONE], and any line carrying password/OTP-style labels is dropped whole as [REDACTED]. This masked version is the only version that exists downstream — it's what the AI call sees, and it's what your local activity log stores. The unmasked OCR text is never persisted and never transmitted.

An honest caveat belongs here rather than in fine print: pattern-based masking is never 100%. A sensitive number in an unusual format, or sensitive prose with no recognizable pattern, can get through. That's precisely why the sensitive-app gate runs first — masking is the second layer, not the only one.

Step 3: one classification call, nothing retained as raw data

The masked snippet, app name, and window title go out for a single AI classification call that returns a category, a relevance score, and a one-line reason. The input text is used for that call and is not stored on a server as raw data — what persists is the derived result, on your Mac. Your report is built from those derived results, which is why a shared report contains reasoning and numbers but never raw screen text.

Verify it instead of trusting it

Every activity in the Performance log has a "Sent for AI analysis" section showing the redacted OCR screen text that was included in that classification call — or a protected-app marker if the capture was skipped. The preview shows only the screen text (not the system prompt, project list, or telemetry). Open a banking app, then check the log: you'll see the block, not the content. Type a test card number in a notes app: the preview shows [CARD], not digits.

One deliberate design choice to know about: that preview lives in app memory only. It's never written to disk, and it clears when Kloy restarts. Keeping a permanent on-disk archive of everything sent to the AI would itself be a privacy liability — so transparency is available for the session, and retention is deliberately zero.

What this does and doesn't claim

It doesn't claim your machine is unhackable, that OCR masking is perfect, or that a determined person couldn't photograph their own screen. It claims something narrower and checkable: screenshots stay local, sensitive surfaces are skipped before capture, the only outbound data is a masked text snippet you can inspect per-activity, and nothing raw is retained server-side. For how those properties feed into reports a client can trust, see the report trust FAQ and the verified public link explainer — or download Kloy and watch the log yourself.