I Have a Chief of Staff Now - and It's an AI Agent That Runs My Entire Day
I handed the mental overhead of my life to an open-source agent named Neo. The hard part wasn't teaching it what to do. It was deciding what it's never allowed to touch.
For a while there, I was spending more energy remembering my life than actually living it.
The day job. The side projects I kept swearing I'd ship. Research papers piling up unread in a tab graveyard. Investments I kept meaning to look at. A gym streak that died, reliably, every couple of weeks. None of it was hard on its own. It never is. The exhausting part was holding all of it in my head at once, and noticing when one thing quietly started eating the others.
At some point it tipped over. I was tired of being the thing it all depended on, the only reason the plates kept spinning. So I stopped trying to hold it, and set up an agent to hold it for me. I named it Neo.
It lives on a server that's always on and reaches me through Discord, so it fits into a place I'm already in all day, with no new dashboard to babysit. The relationship is simple: it reports to me, and it holds me accountable. It checks in on a schedule, tells me what needs attention, and asks the questions I'd otherwise conveniently forget: did that get done, did you train, what's slipping. I fire tasks at it as they come up and it files them without friction. Less "someone I chat with," more a chief of staff that keeps the books on my day and won't let me quietly off the hook.
First, the honest part: getting it dialed in took work
I want to be upfront, because the polished version you're about to see is not how it started.
The setup wasn't brutal, but it wasn't plug-and-play either. The first real surprise was cost. I'd picked the minimal plan, but then pointed it at the most capable model, and in a single setup step I watched it chew through about 5% of my entire monthly token allowance. That got my attention fast.
So the next chunk of the project became model routing. Not every task deserves the expensive model, and different kinds of tool actions have different needs. A quick routine nudge needs nothing like the horsepower of a research roundup or a piece of writing. So I matched models to jobs: cheap and fast where that's fine, heavier only where it earns it. That alone changed the economics of the whole thing.
Then came the experimenting, trying out various MCPs to see what was actually worth wiring in. The email side took the most trial and error: I started with Himalaya, hit walls, and settled on Resend as the thing that just worked for sending the digest.
A lot of re-reading docs, a lot of tearing the config apart and rebuilding it. It did not click on the first try, and it did not click on the fifth.
And that turned out to be the whole point, in a way I didn't expect going in.
There's a line in Atomic Habits that stuck with me through all of this: you don't rise to the level of your goals, you fall to the level of your systems. I didn't need another goal to "be more productive." I'd had plenty of those. I needed a system that runs whether I feel like it or not. And systems, it turns out, you have to actually build. Badly first. Then a little less badly. Then one day it just... works, and quietly keeps working.
Wait - did I actually build an AI?
No. And this distinction matters, so let me be clear about it.
Neo runs on Hermes Agent, an open-source agent from Nous Research. Hermes is the part that's genuinely hard to build: the runtime, the persistent memory, the tool-calling, the ability to live across Discord, email, and a dozen other surfaces at once. That already existed. I didn't write any of it.
What I built was everything that turns a general-purpose agent into my chief of staff. The routines and the exact wording of each one. The schedule. The model routing. The integrations with my brokerage and my email. The permission boundaries, what Neo is and isn't allowed to touch. And the prompts that define how it behaves, what it nudges me about and what it leaves alone.
Hermes is the engine. The system wrapped around it is mine, and that's the more interesting half. Anyone can install an agent; the work is in shaping it into something that fits your actual life.
A day with Neo
Here's what it eventually became.
Every morning, a briefing lands before I've opened anything else. My top priorities for the day, what's actually moving in AI that morning, and a single CS flashcard that rotates through the fundamentals on a spaced-repetition schedule: system design on Mondays, databases on Tuesdays, operating systems Wednesdays, and so on. By the time I've finished my coffee, I already know what the day is supposed to be about, and I've revised one concept without even trying.
During the day, it keeps an eye on my portfolio - and this is the part I'm quietly proud of, for reasons that are more about restraint than capability.
My brokerage is connected over MCP, the protocol that lets agents talk to external tools. But it's connected read-only. Here's the part most people won't see: when I was wiring it in, the integration offered me the full set of tools, including the ones that place, modify, and cancel real orders. Live trades, real money. I went through that list and switched off every trade-capable tool, keeping only the ones that read.
An agent that can glance at your holdings and tell you how you're doing is genuinely useful. An agent that's reachable over a chat app and can move your money is a disaster waiting for the wrong prompt: a bad instruction, a confused context, a clever injection buried in some data it reads. That boundary wasn't a default. I drew it on purpose, and it's the design decision I'd most want another engineer to notice.
And at night, it closes the loop. What got done? Did I train? What's quietly sliding into tomorrow? No pep talk, no guilt-tripping me over a planned rest day, just an honest tally before the day shuts so nothing falls through the cracks.
The gym nudging is smarter than a blanket reminder, too. It knows which days I work from home, and those plus weekends are when it pushes me to actually train, when I've got no excuse not to, rather than firing a generic "go to the gym" at the same time every day. Consistency is the whole game there, and it's quietly good at keeping the streak alive.
The one thing that earns my inbox
Almost everything Neo does happens in Discord, because for a quick nudge a chat message is exactly the right size. There's one exception.
Every Friday evening, Neo pulls together a research digest: what actually mattered that week in voice AI, LLMs, agents, and the tooling around them, plus a short roundup of papers worth knowing about. Instead of dumping that into a chat message, it sends a proper email, formatted and clean, the kind of thing you'd want to open on a Saturday morning rather than scroll past. That one earns the inbox.
How it's actually wired, for the curious
Under the hood it's almost boringly simple, once it works.
Hermes Agent does the heavy lifting: the brain that holds context, remembers across days, and calls tools. On top of that, the wiring is: Discord as the interface, locked down to just my account so nobody else can talk to it. Plain cron for the schedule, which means every briefing and check-in is, underneath, just a scheduled prompt firing at the right time. Resend's free tier for the weekly email. And MCP to connect the brokerage, read-only.
Beyond the server it runs on and the LLM tokens it burns to think, it costs basically nothing to keep alive.
What I actually took away from this
If there's a lesson in any of this, it isn't the obvious one. It's not "look, AI can automate your life."
It's almost the reverse. These agents are powerful precisely because they can act in the real world: send the email, read the portfolio, run the job. Which is exactly why the bulk of the real work isn't teaching them what to do. It's deciding, deliberately, what they are never allowed to touch. The capability is the easy part now. The judgment about where to put the walls is the part that's on you.
Neo doesn't make me smarter or freer in some grand way. It just holds the things I used to drop, and asks the questions I used to dodge. That's enough. The plates keep spinning, and for once I'm not the only reason they do.
Referenced in this piece
- Hermes Agent by Nous Research - the open-source agent Neo runs on
- Model Context Protocol - how the brokerage connects, read-only
- Resend - what sends the weekly digest email
- Atomic Habits by James Clear - systems over goals
Filed, scheduled, and quietly nagged about by Neo.