I Built An Event Bus For My Monitoring System. Then I Tore It Out.

Last year, I spent three weeks building what I thought was the "right" architecture for monitoring a fleet of microservices. The design was clean—everything emitted events to a central bus, a single source of truth. It felt sophisticated. It felt correct. Six months later, I was debugging why a critical dependency failure went completely unnoticed, and the answer made me sick: the third-party service that was supposed to emit the failure event had crashed before it could send anything. My "single source of truth" had a massive blind spot, and I didn't even know it was there.

I ran into a detailed post recently about rejecting an event bus entirely for a similar problem, and it hit different because I'm still nursing the scars from my own version of that mistake. The author built a pull-based monitoring system instead—querying state directly rather than waiting for things to report themselves. Reading it forced me to confront something I'd been avoiding: maybe the event bus wasn't the problem I actually needed to solve.

The Trap of the "Clean" Architecture

Event buses feel like the right answer because they look right. You get append-only logs, eventual consistency, decoupling—all the buzzwords that make you feel like you're building something mature and scalable. The architecture diagram is beautiful: components emit events, a handler reads them, work gets done. It's the pattern everyone reaches for.

But here's what nobody tells you when you're drawing that diagram: the moment a component doesn't emit an event, your system stops seeing reality. It's not a graceful failure. It's a silent lie. A third-party tool crashes? Your event bus never knew it happened. A cron job exits with an error before it can log anything? Still invisible. The components you least control are exactly the ones an event bus can't monitor properly.

I learned this the hard way. My production issue wasn't about architecture elegance—it was about a dependency that failed silently and my "single source of truth" had no idea.

State Doesn't Care About Your Event Bus

The key insight that changed my thinking: state is evidence. Events are claims. An event only exists if the thing that made it survived long enough to send it. State exists whether or not anyone bothered to report it. A crashed process leaves behind stale files. A failed health check is a fact regardless of whether something emit() it into your bus.

Pull-based monitoring—reading file modification times, checking process liveness, polling actual endpoints—is what I should have built from the start. A new component shows up the moment it writes a file. A dependency breaks and the health check fails immediately, visible without any instrumentation on my part. I don't need cooperation from third-party tools to see what's actually happening.

// What I should have done instead of an event bus
const checkStatus = (job) => {
  return {
    alive: isProcessRunning(job),
    fresh: getNewestOutputMtime(job) > lastReadTime,
    healthy: checkDependencyEndpoint(job.dependency)
  };
};

// No emit() calls. No instrumentation tax.
// Reality is visible whether components cooperate or not.

What Actually Happened to My Event Bus

I'd been running my bus for months before the failure. The problem wasn't that the concept was bad—it's that I'd created a system that looked authoritative while quietly lying about edge cases. When something went wrong with a third-party dependency, my inbox didn't show it because the service that was supposed to report the failure had already crashed.

The real cost wasn't in the architecture—it was in false confidence. I thought I had full visibility. I didn't.

My Take

I'm not saying event buses are always wrong. For systems you completely own and control, where you can mandate that every component emits properly, the pattern has real benefits. But for monitoring systems, for observability, for anything that needs to see reality as it actually exists? Pull beats push every single time.

The other thing that matters: I was adding unnecessary infrastructure. An event bus needs its own monitoring. It accrues schema drift. It grows unbounded. I traded simplicity for what felt like sophistication, and I paid the price in complexity I didn't actually need.

If you're building a monitoring system, pull state directly. Check what's actually there. Let reality be your single source of truth, not a log of claims about reality.

What's Your Experience?

Have you built something with an event bus that you later regretted? Or do you have a monitoring architecture that's working well? I'm genuinely curious how this plays out at different scales—pull-based scanning obviously works for small fleets, but I wonder where it breaks down.

Source: This post was inspired by "Why I Rejected an Event Bus for My Solo Agent Fleet: State Is Truth, Events Are Rumors" by Dev.to. Read the original article