Before We Begin

Welcome back. This is a follow-up from our part 1 agent poast I did a few months ago. A lot has changed since the first agentic post that I published. I built a new AI cluster, my v2 machine, and it’s juiced up big time compared to the previous machine, when I did my first agentic research. On top of that, the Hermes Agent from Nous Research has evolved extremely fast. Major shoutout to Nous Research. While I was doing this project: GLM 5.3 Flash, Qwen 3.8 Flash and FreeToken all dropped. The AI industry is insane – it feels like we’re approaching a point where one needs to be unemployed and have a Neuralink installed in their brain just to keep up.

This project is broken up into two parts:

1. Running the newly released Qwen 3.8 27B model and how I tested it
2. Running the tested Qwen model with Hermes Agent to create a local agent that monitors my WHOOP data.

This two-part structure matters because the first half shows what inference engineering entails, while the second shows what happens when the optimized model leaves the benchmark and receives a real-world job. The feature image comes from Where the Wild Things Are by Maurice Sendak, and its meaning is deliberate. In the story, Max is sent to his room, sails to the land of the Wild Things, tames them, becomes their king, and eventually realizes that he wants to return home. I chose that image because we are attempting to tame our own Wild Things—AI agents—without losing sight of why we built them or when we need to bring them home. It was also one of my favorite books that my parents read to me as a child.

The Cost of Intelligence Is Going to Zero

With the release of GLM 5.2/5.3 , Kimi K3, GLM 5.3 Flash, Qwen 3.8 27B, Qwen 3.8 Flash and DeepSeek V4, we have approached a point where we are all Max in the book. The cost of intelligence is going to zero, and my prediction is that by the end of this year (2026) or next (2027), we will have Opus 4.8 intelligence running on local consumer hardware. We have hit the critical mass point where we need to decide if we are to return back to our old ways or live amongst the wild things forever. I mean this literally. WE are likely to choose the latter – live amongst the wild things forever. Once we make this decision – there is no going back – ever. Agents have exploded into our daily lives to the extent that they are writing code, automating tasks, buying and selling cryptocurrency and being implemented into the real world.

I wanted to create an agent project that was simple, showcased my hardware, and have a real-world use case.

This is not what this poast will be:

• “I recreated Palantir’s intelligence dashboard with a 27B parameter model.”
• “I gave an agent $69, threatened to shut it down if it failed, and somehow made $5,000.”
• “this dude ran the full-precision qwen 27B model entirely on a single RTX 3090” (btw this is impossible for now)

You will find how simple and yet effective/powerful this project is, because it’s easy to do. It shows the power of agents and the world that we have decided to exist in. There is no going back.

Let Us Begin

My WHOOP knows more about my bad decisions than almost anyone. The data is useful, but the UI – not so much (sorry Whoop).

WHOOP has built an impressive product, and I have worn one for almost three years. I originally bought it for sleep. Over time, the app became a dense wall of recovery, sleep, strain, stages, trends, coaching, and hidden submenus. The information I cared about most every morning was scattered across several screens. Maybe it’s because I’m lazy, or the novelty of the product wore off etc. I did not need another dashboard. I needed intelligence from the data.

The obvious 2026 answer is to hand the data to a frontier model and ask it to summarize everything. That would work. It would also send some of my most personal data to an agent running in somebody else’s data center.

This was the exact problem I built a local agent to solve. I am aware that WHOOP has an “AI agent” feature, but I wanted to see if I could create one locally with my hardware.

In my part 1 agent research, I showed that Hermes could run extremely well with open models on two RTX 3090s and do actual agentic work. It called tools, wrote and debugged code, searched the internet, and followed me onto my phone through Telegram.

Part 2 asks the harder question: letting an agent loose into the wild, a place we call human reality.

I chose the newly released Qwen 3.8 27B dense model for the agent intelligence, paired with the Hermes Agent, an incredible harness with an amazing team of people building it.

The Selected Stack

Inference engineering is the work of matching a model, serving configuration, and workload to the hardware and users you actually have. In this case, I am the only user, and the system is a dual-RTX 3090 Threadripper Pro workstation. If you want the complete hardware build, I documented it in my new AI cluster post.

Most controlled benchmark and Hermes-campaign measurements in this article were recorded on stable vLLM 0.27.1. After those campaigns, I upgraded the dedicated environment to official vLLM 0.28.0 and revalidated model discovery, text, reasoning parsing, tool calls, and vision without changing the selected configuration. The table above describes the final deployed service; I keep 0.27.1 attached to the historical measurements rather than retroactively relabelling them.

There is an important caveat behind the letters “FP8.” RTX 3090s are Ampere cards. They do not have native FP8 compute. vLLM stores the checkpoint in FP8 and uses Marlin kernels for weight-only FP8 compression, while much of the actual compute remains in BF16-class paths. That still matters because memory capacity is the name of the game when a 27B multimodal model, a long KV cache, and a speculative head need to share two 24 GB cards.

The goal was not “make the benchmark number as large as possible.” The goal was to find the fastest configuration I would trust for a long Hermes session.

MTP, DFlash, and Why the Fastest Isn’t Always the Best

Inference engineering is like a puzzle. Any time a new model gets released, there is a race for zero-day inference support. It depends on the hardware you have, the volume traffic/customers, what quality you want to serve etc. It’s a unique puzzle for every model, but they all follow a similar guideline. This is why it’s really important you get good at solving this puzzle – otherwise, you will get left behind.

I used the official FP8 quant from Alibaba when they released Qwen 3.8 27B. Spoiler – it’s a slight letdown, but I am still grateful they released it. Qwen 3.8 is essentially an updated version of Qwen 3.6: Qwen 3.8’s attention structure is basically identical to Qwen 3.6, which means the improvements likely came from its training set or distillation patterns. I suspect this is why we didn’t see a major jump like we did with Qwen 3.6 – just my theory. That being said, Alibaba released Qwen 3.8 Flash very quickly after Qwen 3.8 – so they probably did not want to cannibalize their models – understandable.

The checkpoint includes one MTP module for speculative decoding. MTP-1, MTP-2, and MTP-3 do not load three independent draft models; vLLM reuses the same MTP layer to propose additional token positions.

Those are the three exact changes needed in the current live post (https://blog.zacharycangemi.com/2026/08/31/owning-your-agent-part-2-local-agents-in-the-wild-using-hermes-to-upgrade-my-whoop/).

Inference engineering is fun when you don’t have a gun pointed at your head. What I mean by this is that if you are Nous Research, OpenRouter, vLLM, llama.cpp etc. – you are in a race against time to have the fastest zero-day support for a new model release. In the example of Qwen 3.8 27B – supporting a model first is not the same as making it usable first. When you’re GPU poor like me, and you are working with scraps, you’re your own customer. The positive of this, is that there is no rush and you can enjoy the process of testing out different methods to optimize inference – and that is what I did here.

I ran five scored cold-start configs through the same text, repeated prefix and vision workload.

We have the parent variant (parent model without speculative decoding) MTP-1, MTP-2, MTP-3 and DFlash2-7 decode variants.

DFlash won the throughput benchmark at 92.35 t/s, but it lost the deployment decision. It required a separate experimental vLLM environment, left only about 0.04 GB—40 MiB—of VRAM headroom at peak, and produced zero prefix-cache hits in the repeated-prefix test. Its seventh speculative position was accepted only 11.43% of the time. The target model could process images, but the DFlash draft path could not use the multimodal embeddings.

Translation – DFlash was unusable for this Hermes agent deployment. For pure text inference – decode speed wins in this regard, but once you break past the 4th wall, and leave normie land, decode is not eveything.

For Hermes, it repeatedly sends a large, mostly stable system and tool prefix. Avoiding that prefill work can matter more than winning another ten tokens per second on a short decode. In the isolated repeat, the parent reduced time-to-first-token from 3.785 seconds to 0.123 seconds. MTP-2 reduced it from 3.835 seconds to 0.823 seconds and reused 6,400 prefix tokens. DFlash2 barely moved and logged no prefix reuse. This is what I mean by breaking through the 4th wall of normie land for inference engineering.

MTP-2 was not the fastest benchmark. It was the best system.

What Happens When the Context Is Actually Full?

Short prompts hide long-agent problems.

With DFlash right off the bat eliminated as an option, I ran four cold-server arms- the parent, MTP-1/2/3 through 12 exact context anchors. Each anchor generated three fixed 256-token outputs. The last request contained 130,816 prompt tokens and produced 256 tokens landing exactly on the 131,072-token server ceiling.

That produced 144 measured requests and four completed decode curves.

Because I had already tested Qwen3.6-27B, I had a useful prior: Qwen3.8-27B retains the same core decoder layout—64 layers arranged as 48 Gated DeltaNet layers and 16 full-attention layers, with full attention every fourth layer. I therefore expected decode throughput to decline gradually, rather than collapse, as the context filled. The full-context test confirmed that expectation for the parent model.

The parent produced the cleanest picture of context cost. Its curve declined almost linearly, with an R² of 0.9952. The MTP curves combined that growing attention cost with changing speculative acceptance, which is why one depth could occasionally overtake another at a later anchor.

The important result was not that MTP always wins. It does not. Acceptance is prompt-dependent. The result was that every MTP configuration still finished the full context around 56–58 tokens per second—roughly 32–35% faster than the parent endpoint.

The thermals were equally uneventful: peak GPU core temperature was 64°C, peak board power was around 349 W per card, and the tests recorded no evidence of sustained hardware thermal throttling.

The frightening number was memory. MTP configurations ended with roughly 0.6 GB of observed headroom in the earlier full-context telemetry. That was tight enough that I refused to increase the context just because the model advertised a 262K native ceiling.

The server could allocate more aggregate KV tokens than the configured per-request context, but those are different promises. I stayed at 131K, kept GPU memory utilization at 0.92, and treated 0.90 as the fallback rather than changing five variables at once.

The Hermes Bug

I gave Hermes a long, multi-phase campaign: research primary sources, create a frozen evidence packet, build an offline interactive website, validate it in a browser, run a bounded audit, repair what failed, push the context through compaction pressure, and then recall hidden sentinels without tools.

The first run looked successful until I audited the wire traffic. At the time, my Hermes request path was sending max_tokens: 65536 on every request. With a 131,072-token server context, vLLM requires:

input tokens + reserved output tokens ≤ 131,072 

That 65,536-token output reservation meant the server started rejecting prompts around 65K input, making my configured 80% compaction point—roughly 105K—mathematically unreachable. I did not conclusively isolate whether the reservation came from the Hermes version I was running, my provider configuration, or the local integration path, so I treat it here as a deployment bug rather than an upstream Hermes design decision.

The agent recovered by reacting to the HTTP 400 errors, which made the run look more robust than it actually was. I thought I had tested proactive compaction. I had tested rejection followed by emergency recovery.

That result was invalid for the question I meant to ask. After correcting the local Hermes provider configuration so normal requests sent `max_tokens: null`, I reran the entire campaign.

The corrected run:

• lasted 2 hours, 52 minutes, and 48 seconds;
• captured 251 chat requests, all HTTP 200;
• reached a maximum retained prompt of 108,650 tokens;
• performed real proactive compression around the 80% threshold;
• processed 16.28 million cumulative prompt tokens;
• generated 697,258 tokens;
• achieved 77.62 t/s weighted streamed generation;
• reused 14.45 million prefix tokens, an 88.76% hit share;
• accepted 76.69% of proposed MTP-2 draft tokens; and
• recalled all six original and pressure sentinels after compaction.

The model did not retain one exact product title and correctly returned

`null` instead of guessing. That is the kind of failure I trust more than a confident hallucination.

Compaction worked, but it was expensive. Ten meaningful compression calls consumed 69 minutes and 45 seconds—40.37% of the entire campaign. Hermes used the full 27B model to summarize its own history, and Qwen can be extraordinarily verbose.

The Agent Built Its Own Evidence

The Hermes Agent + Qwen model created its own offline guide called: “When 27B Feels Bigger: A Local Qwen 3.8 Agent Field Guide.”

The website contains eight sections, six SVG charts, exact-value tables, interactive MTP comparisons, limitations, and source notes. Hermes then opened it in a browser, tested desktop and mobile layouts, traversed it by keyboard, changed interactive states, used vision to inspect the rendered result, found a doubled word, repaired contrast problems, and corrected an overclaim.

Final runtime errors: zero.

• Open the interactive Qwen Field Guide: Field Guide
• View the source on GitHub: https://github.com/zacangemi/owning-your-agent-part-2/tree/main/docs/field-guide

There is an important provenance limitation. A hardcoded workspace path split phase 1 from phases 2–6. The artifact was created during the rerun from the frozen evidence packet, but the campaign was not a perfectly clean single-workspace chain. I fixed the runner and kept the failure in the report.

I published the website because it is a better artifact than another benchmark table. The model did not just answer questions about the research. It turned the research into something another human can explore, which I think is really cool.

WHOOP, Qwen 3.8 27B and the Hermes Agent

I named this part of the project Brevity Intelligence. The product brief was intentionally small:

• When WHOOP finishes scoring my sleep and recovery, give me local Hermes Agent the data and send me one useful morning briefing through Telegram
• I can then ask the agent questions about my sleep, overall recovery and all of my WHOOP data – essentially make my WHOOP data intelligent via an agent

This is a personal, noncommercial research project built against WHOOP’s official developer API. It is not affiliated with, endorsed by, or sponsored by WHOOP. Nothing here is being monetized or sold, and the project does not provide medical advice.

The official WHOOP API exposes the pieces that I wanted:

• recovery score
• sleep performance, efficiency, and consistency
• time in bed and time asleep
• awake time and disturbances
• REM, deep, and light sleep
• sleep need and sleep debt
• resting heart rate and HRV
• respiratory rate
• blood oxygen
• skin temperature

With the agent – I can now actually have an intelligent interface that is private – running on my local GPUs. The agent’s job is not to invent medical meaning. I can compare each night’s data, as well as ask questions about the differences etc., without needing to open up the WHOOP app. Yes, I am aware that WHOOP has something like this – but does anyone know how it works? As far as I know, that code is closed source – and is it a ChatGPT wrapper?

WHOOP API Work

Getting the WHOOP API set up was not that great – and I am being generous here. I have to be very careful with what I say about WHOOP, as I do not want to lose my API access or cause problems. Long story short, getting WHOOP API access was easy, but setting it up was a pain. I had to use Postman to grant a one-time authorization access, and then have Codex build out an API access system to get the data, which then feeds the local LLM on my GPUs.

The process summarized:

1. Postman on my personal machine: one-time browser content, initial refresh token
2. My AI server: owns every later refresh, atomically stores each replacement token pair, retrieves read-only WHOOP data

The Python client:

1. refreshes only near expiration;
2. saves the new Access Token, Refresh Token, and expiry;
3. uses a local lock so two processes cannot rotate the same token;
4. writes replacements atomically;
5. retries one WHOOP request after an unexpected 401;
6. rejects loose file permissions and symlinks; and
7. never prints a credential.

Credentials live outside of the project directory. Health snapshots do as well. This is the privacy boundary that matters: the open-source code knows how to retrieve the data, but the repo never contains values that authorize it. This is what local AI is all about – keeping things like health data private.

Give the agent a keyhole, not the vault

Hermes does not read the OAuth files.

I created a small deterministic bridge called whoop.current. It asks the
private client for a fresh snapshot and returns only the normalized fields the agent needs:

• explicit units
• current recovery and sleep
• derived totals
• recent personal baselines excluding the current night
• interpretation guardrails

It strips WHOOP user IDs and every OAuth field.

Then I created a constrained Hermes skill named brevity-whoop. The skill permits one read-only command, tells the model how to interpret the returned JSON, and explicitly prohibits inspecting the private storage directories.

That produced the first useful conversation:

Me: What is my sleep debt?
Hermes: You're carrying 74.3 minutes—about 1 hour 14 minutes.
Me: What time did I go to sleep?
Hermes: 2:47 AM local time.
Me: What about my REM data?
Hermes: 70.1 minutes, about 17.2% of total sleep.

The first briefing also revealed a normal local-model mistake. Qwen converted the wake time from UTC to Eastern correctly but initially printed the sleep start in UTC while labeling it EDT. I challenged it, it inspected the source timestamps, and corrected the interval. I still don’t understand why the concept of time is an issue for LLMs – maybe this is a separate post for later?

From the terminal to Telegram

This is the part of the project where it all came together – the best part. Hermes already had the feature I needed: a Telegram gateway.

I created a bot with BotFather on Telegram and made that private chat the Hermes home channel. The same local agent became available from my phone without exposing the vLLM endpoint beyond loopback.

The end-to-end path became:

WHOOP API v2
  -> private token-owning Python client
  -> deterministic normalized bridge
  -> constrained Hermes skill
  -> local Qwen3.8-27B-FP8 MTP-2
  -> Telegram

Text worked. Follow-up questions worked. I uploaded an image through Telegram, and vision worked while MTP-2 remained enabled.

Then I removed the last manual step.

Why I used cron instead of a webhook

WHOOP supports webhooks. A production-style event flow would let WHOOP notify a public HTTPS endpoint when a sleep or recovery record changes.

That would be faster. It would also require:

• a publicly reachable HTTPS receiver
• HMAC signature verification
• replay protection
• coordination between separate sleep and recovery updates
• recovery when the server misses an event
• another internet-facing service to secure

WHOOP does not push the completed health summary directly into Hermes. The webhook is a notification; the server still retrieves the record afterward. For a private single-user project, ten-minute polling was the better trade.

Every ten minutes, a deterministic gate asks:

Is there a new fully scored primary sleep and matching recovery that I have
not already briefed?

If the answer is no, it returns wakeAgent=false. Hermes skips Qwen and sends nothing. If the answer is yes, Hermes wakes once, retrieves the data, prepares the briefing, acknowledges that sleep ID, and delivers it through Telegram.

Naps, workouts, strain changes, and updates to an already acknowledged sleep do not trigger the morning briefing.

The polling path makes roughly 432 ordinary WHOOP reads per day, comfortably under WHOOP’s documented default limit of 10,000. It introduces at most ten minutes of latency and requires no public inbound endpoint.

The mature architecture would be hybrid: webhook for immediate delivery, cron for reconciliation. This project did not need the extra surface area.

The first scheduled message

The manual scheduled validation delivered:

Recovery: 44% — below the recent 53.3% baseline.
Sleep: 6h 40m, 81% performance, 97.4% efficiency.
HRV: 57.9 ms versus a 67 ms baseline.
Resting heart rate: 64 bpm versus 61.2 bpm.
Sleep debt: 82 minutes.

It correctly converted the final sleep interval to 1:55–8:46 AM EDT, broke out REM, deep, and light sleep, and stayed under 250 words. The test found one more integration defect. Hermes launched the pre-run wrapper
from its own scripts directory, so Python could not initially import the
project’s sibling WHOOP module. The attached skill still produced the briefing, but the cron execution report retained the traceback. I corrected the wrapper, ran it directly, and received wakeAgent=false for the acknowledged sleep. The next scheduled run recorded “agent skipped” and sent no duplicate message.

Unattended result

On August 31, I ran the first true unattended overnight test. I left vLLM, Qwen, the Hermes gateway, and the ten-minute WHOOP polling job running while I slept. I did not manually request a briefing. I woke up, didn’t even check my whoop, and the magic happened – I got a message on Telegram from my local agent!

At 7:38 AM EDT, the system detected my newly scored WHOOP sleep and recovery data and automatically delivered the morning briefing through Telegram. It reported a 26% recovery score, 5 hours and 26 minutes of sleep, a 47.1 ms HRV, and 96 minutes of sleep debt. It also compared those results with my recent personal baselines and gave me one cautious takeaway.

The complete path worked without my involvement: WHOOP provided the data, the deterministic gate detected the new scored sleep, Hermes woke Qwen when needed, and Telegram delivered the result to my phone.

Making my WHOOP Intelligent and More Private

Part 1 was about sovereignty as capability. I could run the model, connect the agent, call tools, and use it from my phone.

Part 2 is about sovereignty as behavior.

The agent now waits on my machine. It checks a data source I authorized, keeps the credential chain outside the repository, and wakes only when a narrowly defined event occurs. It interprets the selected data locally, sends me a morning briefing, and remains available for follow-up questions through Telegram.

The screenshots show that this became more than an automated notification. I could request night-over-night comparisons, explore changes in sleep debt, and revise a plan after adding real constraints such as my weekday work schedule. The model was not always correct, but the entire interaction remained observable and under my control.

That sounds simple because the final product is simple.

Getting there required speculative-decoding tests, full-context curves, GPU
telemetry, a broken compaction campaign, a corrected three-hour rerun, 251 captured requests, rotating OAuth tokens, private file boundaries, 19 tests, a Hermes skill, Telegram, systemd, and a cron gate.

The complexity belongs underneath. The experience should be:

I wake up. My recovery is ready. My agent tells me what matters.

That is the difference between running a local model and owning an agent.

The point was never to replace the WHOOP app. The point was to make one part of my life quieter, more private, and more useful with an agent I own. Intelligence that I wanted. Not the brightest or best model – one that works for real world things – the China method.

Conclusion

The agents are here in full force and we are never going back. I own my own hardware and this allows me to set local agents upon the wild, like a pack of intelligent dogs. The final experience is simple. The engineering underneath was not. I could have expanded upon it a lot more, but I wanted to showcase the power of local agents. My health data is mine. No one else should have access to it.

Using my hardware allows my own local personal agents to see my sensitive health data, and interface with it. I deliberately picked health + AI as a combo package for this project because my local machine matches perfectly with health data – something private.

I had a lot more ideas on how to expand this project, but I felt that simple was the best way to display the world we are beginning to perma-exist in.

The formula:

• My hardware: two RTX 3090 GPUs
• The model: Qwen 3.8-27B-FP
• The Harness: The Hermes Agent
• My WHOOP data

Summed up, this is it. As long as you have the hardware, everything else is interchangeable. You can try a different model, a different harness, and use any data that you want to unleash agents locally into the world. Fundamentally, this process is eventually going to get streamlined either into one single pipeline/utility or the agents will get so good, you will just need hardware – and that’s it.

UPDATE:

I am getting married in a few weeks, so I may drop off the face of the Earth with my amazing soon-to-be wife until I return. I will be making a separate post about expectations for what I see possibly playing out for 2027, as well as some different angles, non-AI etc. Here are some of the thoughts I have been thinking about lately

• What is consciousness?
• Biology and AI – it’s coming, and it will be bigger than anything ever seen before, let alone just for AI.
• If all software is AI generated, therefore all malware is AI generated, thereby, extending the biological war relationship between antibiotics and bacteria.
• When will the AI bubble pop, so I can afford a crumb of RAM?
• With Tim Cook out as CEO of Apple (yes, the Apple), we should all be paying massive attention to see what Apple does – this is a big one.
• OpenAI making chips? Will they succeed?
• The Ship of Theseus – what does it mean to be human, in an age of intelligent machines, especially when parts of our minds and physical bodies will be replaced with hardware or enhanced?

Follow along at blog.zacharycangemi.com or explore this agent project in the Owning Your Agent Part 2 repository.