One afternoon in September I installed a personal AI on my Mac. It’s a venture-backed app with a simple promise: let it watch your screen and listen to your calls, and it builds a model of you that remembers everything and can act on your behalf.
By the evening it was gone. I’d revoked its access, deleted it and written to ask for my data to be erased.
This isn’t a hit piece. It’s built by smart people, and the problem it’s trying to solve is real. It’s the same problem I’ve been solving in my own business. That’s exactly why it was worth a proper look, and why what I found matters to anyone thinking about giving an AI their memory.
The problem it’s trying to solve
Running several businesses as one person means the bottleneck is never the doing. It’s remembering. Who I promised what to. Which thread moved on WhatsApp but never got written down. What I decided on a call last Tuesday.
My own answer to that is a system I call AFCS. It’s a set of agents that capture what happens, keep a ledger of every commitment, and write me a brief every morning. The agents draft, and I decide. Nothing gets sent without me.
The app I installed is the same idea built on different foundations. Its own documents say it plainly: the value isn’t a personal model of you, it’s the memory architecture feeding it. I agree with that completely. It’s the main thing I’ve learned building mine.
What I found
Five things, and any one of them would have been enough.
There was no way out. No public API, no documented way to connect it to anything else, and an export promise hedged to the point of meaning nothing. Anything it learned about me would have been stranded inside it. If you can’t take your memory with you, you don’t own it.
My data would leave the country with no stated protection. It processes everything in the US, and its privacy policy said nothing about UK or EU data protection. For a business that holds other people’s information, that ends the conversation.
It would have broken my most important rule. My system has a ring-fence. Client deal files and a legal matter never enter the general memory, and no automated process ever reads them. An app that records the whole screen, all day, can’t respect that. No setting fixes it, because watching everything is the product.
“Private” meant a desktop app, not a private one. Only a basic filter ran on my machine. Everything that passed it went to their servers.
It couldn’t hold the complexity. One person, one undifferentiated pile of memory, no way to keep one business separate from another. In use, it shot from the hip. It answered confidently from whatever it had, and a published review had it inventing details of the reviewer’s own life.
The finding that mattered more than the app
The app wasn’t really the lesson. My own system was.
On 2 September I’d run a health check on AFCS. It graded the part that takes things in at A minus and the part that makes sense of them at D, with an overall score of 64 out of 100. The summary line has stuck with me: every layer that writes is healthy, and every layer that closes, revises or promotes is missing or dead.
Screen recording adds input, and input was never my problem. Pointing the biggest, noisiest source ever connected at a system that already struggles to close loops would have made the D worse. The app is what maximum capture with no judgment layer looks like, at scale.
So the rule became: close first, capture second, and capture narrow.
The five ideas I kept
Rejecting the product didn’t mean rejecting the thinking. Five mechanisms were worth taking:
- Read the text, not the picture. Your Mac already knows the text on screen through its accessibility layer. Reading that is almost free, and on-device text recognition covers the gaps in a fraction of a second. No screenshot ever needs to reach a model.
- Capture on events. Record when you switch app, pause typing or copy something, rather than every couple of seconds. That’s the difference between something that runs for a year and something you switch off in a fortnight because it ate your battery.
- Rank by three things. Relevance, recency and what you’re working on right now, weighted roughly 40, 30 and 30.
- Gate writes at the boundary. Decide structurally what is allowed to write where, rather than trusting a model’s prose to behave.
- Give the agent its own address. An agent that acts for you should have its own identity, rather than borrowing yours.
What I built instead
By mid-October the problem was obvious in the numbers. Between 1 and 16 October my system logged about 370 status updates and 65 new commitments. I sent zero corrections back. Anything I did on WhatsApp, by phone or in person stayed “owed” and got asked about again the next morning. 43 commitments were more than a fortnight old, and 19 threads were marked “stalled” when many had moved weeks earlier.
The app solved this by watching my screen. I solved it with what was already written down.
Most of my working day is already text, on my own machine: what I type into my work sessions, the titles of the pages I visit, my calls, my calendar. So the new piece is a nightly pass that runs locally, reads only that, and answers two questions. What did I actually work on today, and does the ledger know? What did I say I’d do that never became a ledger line?
The rules that make it safe are the interesting part:
- Text only, never frames. No screenshots, no video, no image store.
- Only my words. It reads what I typed, never file contents or tool output, which is where client data and secrets live.
- The ring-fence holds by construction. Anything touching a confidential client matter is dropped before any text is read.
- Page titles, never full links, with banking, health and family sites filtered out first.
- Nothing is marked done unless I said so. It only closes a commitment on my own first-person words, such as “sent it” or “signed”, and every change is append-only and reversible.
- Silence over filler. If there’s nothing to report, it writes nothing.
- Deletable in an afternoon. A few small scripts and one scheduled job. No database, no daemon.
It was built on 23 October and had to pass a two-week gate to stay. Here’s how it did.
It ran on time on all 12 live nights. It found 6 commitments I’d made that the ledger had missed, which passed that half of the gate. It closed only 1 commitment from my own words, which failed the other half. I’ve kept it on, because finding what I said I’d do is worth having, but the closing rule is still open.
The wider system is better and still not finished. A fresh health check on 4 November put the overall score at 72, up from 64, and the part that makes sense of things moved from D to C plus. The morning brief came down from 4,250 words to about 2,400. But I have 300 live commitments, up from 192 in September, and 89 of them are more than a fortnight overdue. In October I filed 198 and closed 62.
So the finding from September still stands. Capturing is the easy part, and closing is the hard part. That’s the honest state of it.
What still needs me
All of it that matters. The system notices, drafts and proposes. I decide what’s true, what’s done and what goes out. The autonomy dial is set there on purpose, and that’s the design.
If you’re tempted by an AI that remembers everything
Ask four questions before you hand anything your memory:
- Can I get everything out, in a format I can use elsewhere?
- Where is my data processed, and under which country’s law?
- Can I fence off what must never be read, and is that fence structural rather than a setting?
- Does it close loops, or does it just capture more?
Most products fail at least one. The good news is that the narrow, boring version you build around your own work often beats the impressive one you install.
We dig into builds like this every fortnight in the Leanpreneur community: what operators are running, what broke, and what still needs a human.
