monitoring
monitoring
Building a Single-Pane RED Dashboard for Microservices with OpenTelemetry and Semantic Conventions
Three months ago we had 34 microservices and 34 Grafana dashboards, each one hand-built by whoever owned that service at the time. Some had rate/error/duration panels. Some had CPU and memory but no latency. One had a pie chart of HTTP status codes that nobody had looked at in a year. When we had an incident that touched five services at 2am, the on-call engineer had to open five different dashboards, each with different label names (route vs path vs endpoint), different histogram bucket boundaries, and different naming for the same metric.
How We Cut Our Prometheus / VictoriaMetrics Storage Bill by 60% (Cardinality Audit Walkthrough)
The bill that made me open a support ticket with myself Last October our VictoriaMetrics cluster crossed 340 million active time series and our monthly infra cost for the storage tier hit $4,900. Nobody had approved that number. It just… grew, the way disk usage always grows, one Helm chart install at a time until finance asks why the “monitoring” line item is bigger than the “database” line item. I spent the better part of a week doing a cardinality audit on that cluster.
Fixing alert fatigue: what actually works
Every noisy alert your team learns to dismiss makes the real one easier to miss too. Here's how alert fatigue actually happens, and how to fix it.
How to Build Your Infrastructure Monitoring in 2026
Every year I get asked the same question by teams starting from scratch: “we have Grafana, we have some dashboards, why do we still get paged for things we didn’t see coming?” Most of the time, the answer isn’t a missing tool. It’s a missing method. Teams jump straight to “let’s install Prometheus” or “let’s buy a SaaS observability platform” before answering a much simpler question: what does “healthy” actually mean for this business?
Set Up Grafana Alerts and Notifications via Slack, Email, or Webhook
I’ll admit it: I used to be the person who constantly refreshed my Grafana dashboards, nervously watching a CPU usage graph creep upwards. It was like babysitting a pot of water, waiting for it to boil. This, of course, is not a scalable strategy for managing a smart home system or any observability stack. The whole point of having these beautiful dashboards is to be proactive, not reactive. That’s where Grafana alerts come in.
Using Labels and Relabeling in Prometheus for Clean, Scalable Metrics
I’ll admit it: the first time I looked at a raw Prometheus metrics endpoint, I felt a bit overwhelmed. It was like staring at a tangled ball of yarn, with metric names and labels sprawling in every direction. I was collecting thousands of time series, but finding the specific signal in the noise felt nearly impossible. That’s when I truly understood the power of labels and relabeling in Prometheus. They aren’t just decorative tags; they’re the fundamental tools for organizing your metrics universe.
How to Set Up Prometheus Remote Write to Push Metrics to VictoriaMetrics
I’ve been running my own monitoring stack for years, and if there’s one thing I’ve learned, it’s that Prometheus alone can feel like trying to store all your holiday decorations in a single shoebox. It works fine at first, but soon you’re dealing with capacity issues and wondering where that one crucial metric from three months ago disappeared to. That’s why I recently set up Prometheus remote write to push metrics to VictoriaMetrics for centralized, long-retention storage.
Monitor Home Internet Speed Automatically with Prometheus and Speedtest Exporter
I've been battling my internet service provider for months over sporadic slowdowns that always seem to happen right before my important video calls. Sound familiar? After one too many "everything looks fine on our end" responses, I decided to take matters into my own hands and build a system that automatically tracks my internet speed 24/7. In this tutorial, I'll show you how to set up your own internet speed monitoring system using Prometheus and a Speedtest Exporter.
Using PromQL to Analyze CPU, Memory, and Network Metrics Effectively
If you’ve ever stared at a Grafana dashboard wondering why your server’s CPU is spiking like a caffeinated squirrel, you’re not alone. Prometheus and PromQL are my go-to tools for making sense of infrastructure metrics—once you get the hang of them, they’re like having X-ray vision for your systems. In this guide, I’ll walk you through writing effective PromQL queries to monitor CPU, memory, and network performance. Whether you’re debugging a mysterious latency issue or just keeping an eye on resource usage, these tips will save you hours of head-scratching.
Prometheus vs VictoriaMetrics: Performance, Storage, and Query Language Compared
I’ve spent years tinkering with monitoring tools—first as a hobbyist automating my smart home, then professionally while scaling observability pipelines. Two names kept popping up: Prometheus (the de facto standard) and VictoriaMetrics (the “faster, cheaper” alternative). But which one should you use? Let’s break it down with real-world benchmarks, storage deep-dives, and query language quirks. Why Compare Prometheus and VictoriaMetrics? Prometheus is the Kubernetes-monitoring darling, but VictoriaMetrics promises better performance with lower resource usage.