Jumping on the Hermes hype

I built myself an AI assistant. The marketing says it should replace my work. It does not. What it actually does is more interesting than the hype, but far less than the promise.

The AI agent hype is everywhere. Autonomous workers. Digital employees. Your second brain that never sleeps. I bought into it. I spent weeks building one using Hermes Agent from Nous Research. Now that it is running, I have an honest take: it is a tool that takes significant effort to get useful outcomes from, and the ceiling is lower than the marketing suggests.

Here is what actually happened.

How it started

I run my digital life on a remote NAS. Files, projects, notes. Everything lives there, shared across all my devices. The idea was simple: give an AI agent direct read/write access to that vault. It would see my tasks, update my projects, and I could message it like any other contact.

Simple in concept. The reality is a patchwork of VLAN rules, SMB mount points, API quirks, and constant little failures you have to chase down one by one.

The storage that would not stay up

The first approach used an MCP fileserver in a Docker container on the NAS. It worked until it crashed silently after a few hours of inactivity. The container health check was buggy. Restarting meant SSHing into the NAS.

The solution was embarrassingly simple: switch to SMB. It is built into macOS, requires zero additional software, and is rock-solid. Sometimes the boring standard protocol is the right answer.¹

A brief aside about VLAN rules

The agent runs on a Mac Mini in a different VLAN to my laptop. SSH kept timing out. Ping showed 100% packet loss. The machines could not see each other.

The culprit was a block-all inter-VLAN rule sitting above the SSH allow rule. Firewall rules are top-down and first-match wins. A broad accept rule above the deny fixed it.

The messaging gateway did not trust itself either. The auth token for webhook URLs was not accepted in the standard header location. The fix was a documented workaround: embed the password as a query parameter. Ugly, but functional.

The architecture pivot (and the debugging marathon)

The original plan used a proprietary coding assistant as the runtime harness. It worked, but locked me into one inference provider. Switching models meant rebuilding integrations. Adding channels meant fighting a closed platform.

So I ripped it out and replaced the entire runtime with Hermes Agent, an open-source agent framework from Nous Research. New personality definition. 18 specialist agents migrated. All skills ported. It took days.

It was the right call. It was also days of debugging serialisation bugs and chasing regressions.

What actually works

My agent has its own Apple ID. I message it like any contact. Calendar invites, email CCs, task updates. All native. No app to open, no web UI to load. This is genuinely the best part of the setup. It can read a calendar invite, but try asking it to negotiate a meeting time with three busy people. It falls apart.

The infrastructure survives a power cycle. I tested it. Unplugged the Mac Mini, waited, plugged it back in. Within three minutes the agent was online and responding. Zero manual intervention. This is table stakes for a server, but it was oddly satisfying to see it work.²

The storage stack is resilient. SMB is reliable and OS-native. The old MCP server is still installed as a backup. When something fails, there is a fallback path. Zero data loss. This is boring infrastructure engineering, not magic.

There are now 12+ specialist skills live. Financial planning, portfolio analysis, morning briefings, video processing, diagram building, CV optimisation. Each one is a reusable capability. Some of them genuinely save me time. None of them are autonomous. Every single one required hand-holding and prompt engineering to get working, and they still drift over time as the underlying model changes.

The shared vault is the single source of truth. Tasks, projects, knowledge, notes. All in markdown files on the shared storage. My agent reads and writes them. I edit them in Obsidian. The same files, no sync, no API, no lock-in.

The flyer photo moment

The biggest surprise was something small. I snapped a photo of a flyer on a community board, sent it to my agent, and asked it to add the event to my calendar. It read the text from the image, extracted the date and time, found the venue, created the calendar entry, and replied with a confirmation.

It worked. It was ridiculous. And that one moment captures the whole experience perfectly.

Because here is the thing: that is not an autonomous executive assistant doing my job for me. It reads flyers and adds events to a calendar. It surfaces my tasks for the day. It tells me the weather. It does not draft quarterly reports or have nuanced conversations with clients.

The mistake is conflating “does a few things well” with “replaces a human assistant.” A flyer photo trick is genuinely useful. It is not a revolution.

Three things I would do differently

1. Start with an open framework. The pivot to Hermes Agent was the inflection point. Open source means model-agnostic, flexible channels, zero vendor lock-in.
2. Use boring infrastructure. SMB over HTTP MCP. Standard firewall rules tested against the hardware. The exotic solution is rarely the durable one.
3. Test the boot sequence. An agent that does not survive a reboot is not an agent. It is a hobby.

The honest truth

Building a personal AI assistant is an interesting technical project. It will teach you a lot about networking, storage, and prompt engineering. It will frustrate you repeatedly. And at the end, you will have something that can do a handful of useful things, inconsistently, as long as the model has not drifted and your infrastructure has not changed.

My agent handles some of my mornings. It occasionally helps with my inbox. It tracks projects in a markdown file. It is not a second brain. It is a note-taking system with a language model attached. That language model hallucinates about 5% of the time.

It runs on a £600 Mac Mini in my house, costs £20 a month in cloud inference, and keeps my data private. That part I genuinely like.

Would I recommend building one? If you enjoy the tinkering, yes. If you are looking for something that meaningfully reduces your workload, set your expectations low. The agent hype is real, but the actual capability is still modest. It is a tool that can occasionally do real work. That is the truth of it.

That flyer photo trick was worth every hour of VLAN debugging. But I am not pretending I hired an employee.