How I use Hermes and OpenClaw as a personal assistant

On a weekday morning my phone sometimes buzzes with a Telegram message from a program running on a small server at home. It says one email needs my attention before lunch, and it quotes the line from the email that makes it urgent. Most mornings there’s no message at all, and that’s on purpose.

This is the personal assistant I’ve been building for a while. I started it on OpenClaw and moved it to Hermes this month.

OpenClaw is an open-source personal AI assistant that runs on your own machine. It was called Clawdbot for a while. The main idea is a gateway: chat apps like Telegram or WhatsApp on one side, an agent with tools, skills and memory on the other, and the gateway in between so you can talk to the agent from your phone like it’s a contact.

My first setup lived on a laptop. A local gateway, Brave web search for looking things up, a markdown search tool called QMD for finding things in my notes, text to speech so it could read to me, and agent-to-agent messaging so one agent could hand something to another. It worked. It was also only awake when the laptop was.

Hermes Agent is Nous Research’s open-source agent framework. It has a CLI, a messaging gateway, cron jobs, skills, tool approvals and memory, and it supports MCP servers.

Earlier this month I moved everything over to Hermes on a small always-on home server. I reach it over Tailscale from wherever I am, and Telegram is still the main way I talk to it. The laptop is a laptop again.

What runs on a schedule

Honestly, the cron jobs are the part I use most. I chat with it less than I expected.

• Weekday working hours: urgent email alerts. Only urgent, only with a quote.
• Twice a day: an action digest. What needs doing, from mail and tasks, in a few lines.
• Afternoon: a follow-ups list. Who I’m waiting on, and who’s waiting on me.
• Friday: bills and renewals. Anything due or about to auto-renew.
• Sunday: a weekly review. What happened, what slipped.
• Once a week: a memory review, which reads through what the assistant has remembered about me and proposes edits. Proposes. I accept or reject them.

What it’s allowed to touch

Not much, on purpose.

Email is read-only. It comes in through Proton Mail Bridge over IMAP, read-only, so the assistant can read mail but cannot send, reply, delete or flag anything. The access simply doesn’t exist, which I trust more than a rule in a prompt.

Notion tasks are read-only too. It can see my list, it can’t add to it.

Mail classification runs locally. There’s Ollama on the server with a small open model, and that’s what reads and sorts the mail. A frontier model handles the chat, where I’m the one choosing what to paste in.

Secrets live in the operating system keychain.

QMD indexes roughly 2,800 markdown notes, which is what it searches when I ask something like “what did I decide about this last spring”.

My rules are short. Read-only by default. Every alert must quote its source. No news means no message. Anything that writes needs an approval.

What I got wrong

The first version was far too chatty. I turned most of that off fast. Now it only speaks when there’s something to say.

False urgency is worse than silence. One wrong “urgent” and I stopped trusting all of them. The quoting rule came out of that: if it can’t show me the line, it can’t call it urgent.

Local models are good enough for sorting. I assumed I’d need a big model to classify mail properly. I didn’t. A small one on the server does fine, and it’s the reason my mail can stay home.

The migration was mostly about memory. Moving the tools from OpenClaw to Hermes was the easy part. Moving what the assistant knew about me, and getting it to keep learning without drifting, took much longer. That’s why the memory review job exists and why it only proposes.

What’s still missing

My calendar isn’t connected, so it can’t tell me I’ve double-booked myself. Sending anything is still manual: it drafts, I paste. Both are on the list, and both are exactly the kind of “writes” I’m slow to hand over.

Most mornings it keeps quiet. That is still my favourite thing about it.