A second brain that talks back

In June I started writing a small Telegram bot to save notes into my Obsidian vault (a folder of plain-text Markdown notes), which I was using as the main source of context and memory for AI agents. Four months and a few thousand messages later, the bot has grown into a small personal assistant organized around a system of agents, which I call threads. Each thread is assigned a part of my life: my research group, my courses, my reviewing, my calendar. They save papers, keep my to-do list, prepare a short brief before meetings, read the emails I let them read and, most of the time, remember when I correct them. I talk to them mostly from my phone, through a web app called Arianna.

This is what it looks like today (the app is the real one; the data in the screenshots is made up).

The rest of this post explains how it works under the hood. I had two reasons to write it. One is to share it (and stop randomly talking about it at lunch). The other is more selfish: coding agents wrote most of the system, in long conversations with me, and explaining it end to end is a way to check that I still understand all of it.

Some numbers, as of October 2026: about 900 commits to the bot’s repository since June 19, around 100k lines of Python (tests included) plus a 7k-line TypeScript app, more than 2,300 unit tests, roughly a thousand notes in the vault, and about 1,200 exchanges since the current version of the model runner went live at the end of August.

The short version

Everything runs on a Mac Studio at home, as three programs that are always on. The vault app is the front door: it receives my messages, decides which thread they go to, and decides when anything is allowed to reach me. The runner is the place where agents actually run. A search service answers questions about my notes and past conversations. Everything that needs to last is kept in three places: the vault itself (a private Git repository of Markdown files), a small SQLite database for tasks, and some private state of the app.

Click around the map below. The button traces the path of one message.

Two ideas shape most of the design.

1. Models propose, plain code acts. A model can read anything, and it can edit files in its own private copy of the vault. Anything else, like changing a task, publishing changes, or giving itself more permissions, is only a request. Ordinary Python code checks the request and carries it out after the model has finished.
2. The software does the bookkeeping. Locks, retries, logs, restarts and the timing of messages are the application’s job, not mine and not the agents’. If I ever have to think about a lock file, something went wrong.

Threads

The basic unit is the thread: a conversation with an agent that never really ends, and that has a home and a job. There are six right now: Assistant (the front door, plus paperwork and personal life), Research (my group, students, papers, grants), Courses, Reviewing, Marvel Reading (yes, really¹), and System, which looks after the system itself.

Each thread is described by one Markdown note in the vault, its profile. The profile holds:

• what the thread is for, and its Instructions, in plain language;
• a Learned section with lessons from my corrections, each linked to where it came from;
• which model it uses and how hard it should think (Codex or Claude, which I can switch per thread, for a single exchange, or for a side conversation);
• the saved task views it follows, which is how it knows which tasks are “its own”;
• its standing orders, which I call duties: things it does without being asked, each with a schedule or trigger and an explicit permission;
• its hours, when it may write to me without being asked.

A conversation can’t grow forever, because a model can only take in so much text at once (its context window). So each thread keeps a short working brief. When a conversation fills 60% of the model’s context window, or 150k tokens, whichever comes first, the runner starts a fresh session that begins with the brief, and the thread carries on more or less where it left off. I can also open side conversations on a thread, with their own model and history, for tangents that shouldn’t clutter the main one.

Threads are of course not fixed, and I can create or remove them simply by talking to the system itself (e.g., to create a temporary thread to track a working trip or ongoing project).

The life of a message

Let’s follow one message from the moment I type it to the moment I read the answer. Press play, or click any stop.

Three things about this path are worth spelling out.

The app and the runner are separate programs. I can restart the front door, for instance to install a fix, while a model is halfway through a long task. When the app comes back, it picks up the stored result and delivers the reply as if nothing had happened. If the runner itself crashes while carrying out a turn’s changes, the outcome is marked unknown and held until someone checks. It is never repeated blindly, because sending the same reminder twice, or moving the same task twice, is worse than asking.

Models never touch the task database or Git. They get named tools instead (find_tasks, complete_task_with_followup, save_task_view, and so on). A tool only checks the request and puts it in a queue, and the app carries out the queue once the model’s turn is over. A welcome side effect is that every change can be audited, and most can be undone.

Each model provider has its own adapter. Codex is driven through its app-server (JSON-RPC over stdin/stdout), Claude through the Agent Client Protocol. The runner hides the difference, so a thread can move from one to the other and keep its identity, its brief and its history.

What the model actually sees

Every time a thread answers, the model receives a prompt built in layers, from the most general to the most specific. Most layers are the same every time. What changes is the profile, the brief, and of course my message.

Skills are just Markdown procedures kept in the vault, such as “Save a reading”, “Refresh the course slides” or “Draft a referee report”. There are about forty. A thread’s profile lists the ones it usually needs, and for anything else a small script picks the right skill for the request, so the prompt doesn’t have to include all forty.

Writing safely

At any moment several things may be writing to the vault: Research answering me, a side conversation in Courses, the email worker saving a review request, the job that exports tasks. Each of them has its own Git worktree, a separate working copy of the vault, and the model only edits files there. When a turn ends, the app takes a shared lock, commits only the files that turn changed, brings its copy up to date with the latest vault (a rebase), pushes the result to the Git server on a Raspberry Pi, and checks that the push actually arrived. Only then does the next writer get the lock.

For other repositories (this website, my course slides, my CV) the rule is stricter. The model has to end its answer with an explicit list of the files it changed, and only those get committed. No list, nothing is published. If two changes conflict, the conflict goes back to the same writer, which sorts it out on its next turn. Nothing tries to fix it automatically somewhere else.

Working on its own

So far I have only described what happens when I write. The other half of the system works without being asked. Each duty is a standing order that I gave a thread just by telling it: “every morning, look for new papers on these topics”, “after each lecture, ask me what to log”, “when a review request arrives by email, make a task with its deadline”. A standing order can be triggered by:

• a time (Field Watch, which looks for new papers, runs on Monday, Wednesday and Friday at 07:00);
• a change to a task (follow-ups are collected every minute and left to settle for a few minutes before the thread wakes up);
• a calendar event (a brief before a meeting);
• an email that matches a rule I confirmed;
• a correction I made (more on this below);
• a problem in the system (System investigates, once per incident).

Each order comes with a grant: which files it may write, which sources it may read, and whether it may use the web. A read-only order is saved right away. Anything that writes, or that asks for more than it had before, shows me a card to confirm, and only my tap counts. A model can’t approve its own permissions.

Results go into an outbox, and one rule decides when they reach me. If it’s within the thread’s hours and I haven’t asked for quiet, they go out right away. Otherwise they wait, with a visible reason, until the next allowed moment. Replies to my own messages and urgent problems are never held back, and a duty with nothing new to report sends nothing.

Try it below. Each row is something a thread might want to tell me during an ordinary day.

When several results are ready at once, a small model may merge two obvious duplicates or drop an empty message. It can’t hold anything back for more than two minutes, and if it fails, everything is sent anyway. For a while I had a much fancier delivery policy, with daily limits, gaps between messages and quiet buffers around meetings. It was hard to predict and harder to debug, so I removed it.

Memory and learning

People use “memory” to mean many things when they talk about agents. Here there are three kinds, and keeping them separate is one of the design decisions I’m happiest with.

Notes are knowledge. Papers, projects, students, courses: ordinary Markdown files in the vault, which I can read and fix myself.

Conversations are history. Every finished exchange is indexed, together with the sessions I run with coding agents on my desktop, and any thread can search it (“what did we decide about the exam format?”). The search combines exact word matching with embeddings, numerical fingerprints of meaning that find related passages even when the words differ (nomic-embed-text through Ollama, stored with sqlite-vec). The index is rebuilt every night. A summary of a conversation is always labelled as a summary, and never treated as an instruction.

Profiles are behaviour. This is where learning happens.

When I correct a thread in a way that should last (“don’t create a task for every paper I mention”), the reply shows Noted · Lesson, and my exact words are stored with a link to where I said them. Overnight, the thread reviews its new corrections and turns each one into a line of its Learned section, which I can undo. If there are no new corrections, no model is called at all. Changes to the Instructions, to skills, or to rules shared by all threads are only ever proposals, and I have to approve them.

Tasks

Tasks are the one thing a model never edits directly. They live in a single SQLite database, the only source of truth. Every change goes through one function that records who made it (me in the app, a thread through the app, a desktop session through a command-line tool), so the full history is always there. Every three minutes the tasks are also exported to Markdown so I can see them in Obsidian, but nothing is ever read back from those files.

Tasks don’t belong to any thread. They have an area and some tags, and threads follow saved views over them. Every open task has a “next look” date. A task can wait on someone or on another task, and it comes back by itself on its date. My own edits don’t trigger follow-ups from the threads. That sounds obvious, but it took a bug report to get right.

Arianna

For about three months Telegram was the only way to talk to the system: one group chat, with one topic per thread. Telegram is very good at moving messages around, but it’s a poor place to read (no tables, no equations, no sections you can fold away), and tasks and settings lived on a separate dashboard where I couldn’t reply.

Arianna replaces both.² It’s a web app that installs like a phone app (a progressive web app, written with React and Vite). The Mac serves it over Tailscale, a private network that only my own devices can join, and I have it on my phone and on my laptop. It’s a second door into the same conversations, not a new agent: a message sent from the app becomes exactly the same turn as one sent from Telegram, and the reply shows up in Telegram too.

Some things make it more than a chat client.

• Cards instead of commands. Questions, permission requests, suggested tasks (Add, Another day, and Undo for 15 minutes), the note or event a reply is about, and email drafts that I can edit and send. Send has a 10-second undo, and only my tap can send an email.
• Live steps. While a thread works, its steps (reading a note, searching, editing a file) appear in the chat as they happen. When it finishes, they fold into a single “Worked for 2 min” line that I can open.
• To do, Today, Activity, Ideas. The task list; a once-a-day summary with meetings, due tasks and what came in overnight; a screen showing what is running and what the duties did; and a feed of proposals from the threads, which I answer with Yes, Later or No.
• Input from anywhere. I can paste, drop or attach files, record voice notes (the Mac transcribes them), or share to Arianna from any Android app.

Security is kept deliberately simple. The app can only be reached from inside my private network. Every action needs a token issued by the running app, and sending messages or editing notes only works from a device I’ve paired with a one-time code sent over Telegram (the system stores only a hash of the code). The app has no shell, and the only thing it can start is a message to an existing thread.

Some examples

The diagrams show the machinery, but they say little about what using it is like. The screenshot at the top shows one ordinary exchange, saving a paper. There is no special code behind it: the “Save a reading” skill says where notes go, the model edits two files in its working copy, and the app publishes them. Here are a few more exchanges, simplified from real ones, with names removed.

Giving a standing order. I tell Courses something once, and it becomes a duty.

A quiet morning. Field Watch ran at 07:00, outside the hours of Research, so its result waited in the outbox. The meeting brief came from a calendar trigger. At 09:00 both arrive.

Correcting a habit. A correction turns into a lesson, in plain sight, and I can still undo it.

Fixing the system from the system. Since October, a message I send to System carries the same authority as a session at my desk. On the repositories it is allowed to work on, it can edit the code, run the tests, publish the change, and restart a service after replying.

This last one is based on a real fix from this week, and it’s the part of the system that I still find a bit uncanny.

What I’ve learned building it

Simplify whenever possible. The most useful changes of the last month removed something: messages between threads that nobody used, the complicated delivery policy, a second place to store tasks, focus modes in Telegram. Every mechanism has to be maintained, and the agents pay for it too, since they read about it on every turn.

Put the boundaries in code, not in prompts. “Please don’t run Git” is a request. A separate working copy, an app that commits only the listed files, and a sandbox with no way to reach the database are guarantees. The prompt tells the model the rules, and the code enforces them.

Make “I don’t know” a valid answer. A finished job and a delivered message are two different facts. When the runner crashes halfway through a turn, the outcome stays unknown and waits for me instead of being retried. A surprising amount of the code exists only so that the system can say “I’m not sure this happened, please check” instead of pretending it did.

Test by using it. At some point I stopped checking every change by hand. Small changes I try right away from my phone; the rest ship once their tests pass and go on a list of things to watch. Every week or two, the System thread checks that list against real conversations and logs, and tells me what didn’t behave as intended.

What’s next

Mostly, using it. On the list: a short daily review that compares what I asked for yesterday with what was actually saved and sent; a small routing model trained on my own corrections, once there are enough of them; push notifications from the app, so that Telegram can finally retire; and a way for people in my group to ask a thread about their own projects without seeing anything else.

Is it worth it? I don’t know yet, but it’s definitely fun. :-) What surprised me most is how unreliable agents still are in this setting. The system can work well for a whole day, and then send me spiraling after some bug because an agent didn’t do what I expected. On the positive side, working on a system that can adapt itself directly is a very interesting experience, and I think this kind of software will evolve a lot over the next year.

If you’re building something similar, I’d be happy to compare notes (my contacts are at the bottom of the page).