Monitor Home Internet Speed Automatically with Prometheus and Speedtest Exporter

Why Monitor Your Internet Speed?

Before we dive into the technical details, let me share why this project has been so valuable for me. Beyond just satisfying my data-obsessed brain, having historical speed data has:

• Provided concrete evidence during ISP support calls
• Helped me identify peak usage times in my neighborhood
• Allowed me to verify I’m getting what I’m paying for
• Detected outages even when I wasn’t home

The best part? Once set up, this system runs completely automatically in the background, collecting data and sending alerts when things go sideways.

What You’ll Need

Before we start, here’s what you’ll need to follow along:

• A always-on computer (I’m using an Intel NUC running Ubuntu Server)
• Docker and Docker Compose installed
• Basic comfort with command line operations
• About 30-60 minutes of setup time

Step 1: Setting Up the Speedtest Exporter

The heart of our monitoring system is the Speedtest Exporter, which automatically runs speed tests and exposes the results in a format that Prometheus can scrape.

First, let’s create a directory structure for our project:

mkdir -p internet-monitoring/{config,data}
cd internet-monitoring

Now, let’s create a docker-compose.yml file:

version: '3.8'

services:
  speedtest-exporter:
    image: miguelndecarvalho/speedtest-exporter
    container_name: speedtest-exporter
    restart: unless-stopped
    environment:
      - MODE=on-demand
      - INTERVAL=30m
      - WEB_PORT=9798
    ports:
      - "9798:9798"
    volumes:
      - ./data/speedtest-exporter:/tmp

Let me explain what’s happening here:

• MODE=on-demand: The exporter runs speed tests on a schedule rather than on every scrape
• INTERVAL=30m: Runs a speed test every 30 minutes (adjust this based on your needs)
• WEB_PORT=9798: The port where metrics will be exposed

Start the Speedtest Exporter with:

docker-compose up -d speedtest-exporter

Verify it’s working by visiting http://your-server-ip:9798/metrics. You should see Prometheus metrics after the first test completes (which might take a few minutes).

Step 2: Installing and Configuring Prometheus

Prometheus will be responsible for scraping metrics from our Speedtest Exporter and storing them as time-series data.

Create a prometheus.yml file in your config directory:

global:
  scrape_interval: 60s
  evaluation_interval: 60s

scrape_configs:
  - job_name: 'speedtest-exporter'
    static_configs:
      - targets: ['speedtest-exporter:9798']
    scrape_interval: 60s

Now, add Prometheus to your docker-compose.yml:

services:
  # Previous speedtest-exporter service here
  
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - ./config/prometheus.yml:/etc/prometheus/prometheus.yml
      - ./data/prometheus:/prometheus
    ports:
      - "9090:9090"
    depends_on:
      - speedtest-exporter

Update your services with:

docker-compose up -d

Check that Prometheus is scraping metrics correctly by visiting http://your-server-ip:9090/targets. You should see the speedtest-exporter target as “UP.”

Step 3: Visualizing Data with Grafana

Now for the fun part—making all that data pretty! Let’s add Grafana to our docker-compose.yml:

services:
  # Previous services here
  
  grafana:
    image: grafana/grafana-enterprise
    container_name: grafana
    restart: unless-stopped
    volumes:
      - ./data/grafana:/var/lib/grafana
    ports:
      - "3000:3000"
    depends_on:
      - prometheus

Update your services again:

docker-compose up -d

Now, access Grafana at http://your-server-ip:3000 (default login: admin/admin). You’ll need to:

1. Add Prometheus as a data source (URL: http://prometheus:9090)
2. Import a dashboard for visualizing speed test data

I’ve created a dashboard that you can import using ID 18691 (or find it at Grafana Labs Dashboards). Alternatively, create your own with these key panels:

• Download/Upload speed over time (using speedtest_download_bits_per_second and speedtest_upload_bits_per_second)
• Latency measurement (using speedtest_ping_latency_milliseconds)
• Current status and last test time

Step 4: Setting Up Alerts

What good is monitoring without alerts? Let’s set up Prometheus alerting to notify us when our internet connection has issues.

Create an alert.rules.yml file in your config directory:

groups:
- name: internet-monitoring
  rules:
  - alert: InternetDown
    expr: up{job="speedtest-exporter"} == 0
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "Internet connection is down"
      description: "Speedtest exporter has been unreachable for 5 minutes, indicating possible internet outage."
  
  - alert: DownloadSpeedLow
    expr: speedtest_download_bits_per_second / 1000000 < 50
    for: 15m
    labels:
      severity: warning
    annotations:
      summary: "Download speed is consistently low"
      description: "Download speed has been below 50 Mbps for 15 minutes. Current value: {{ $value }} Mbps"
  
  - alert: LatencyHigh
    expr: speedtest_ping_latency_milliseconds > 100
    for: 10m
    labels:
      severity: warning
    annotations:
      summary: "Network latency is high"
      description: "Ping latency has been above 100ms for 10 minutes. Current value: {{ $value }} ms"

Update your prometheus.yml to include the alerting rules:

# Add to your existing prometheus.yml
rule_files:
  - '/etc/prometheus/alert.rules.yml'

alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # We'll set up Alertmanager in the next step

For alert notifications, let’s add Alertmanager to our docker-compose.yml:

services:
  # Previous services here
  
  alertmanager:
    image: prom/alertmanager
    container_name: alertmanager
    restart: unless-stopped
    volumes:
      - ./config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
    ports:
      - "9093:9093"

Create an alertmanager.yml configuration for email alerts (or other notification channels):

route:
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 3h
  receiver: 'email-notifications'

receivers:
- name: 'email-notifications'
  email_configs:
  - to: 'your-email@example.com'
    from: 'alertmanager@your-domain.com'
    smarthost: 'smtp.gmail.com:587'
    auth_username: 'your-email@gmail.com'
    auth_password: 'your-app-password'
    send_resolved: true

Finally, update your Prometheus configuration to point to Alertmanager:

alerting:
  alertmanagers:
    - static_configs:
        - targets: ['alertmanager:9093']

Restart all services:

docker-compose up -d

Step 5: Optimizing and Fine-Tuning

Now that everything is running, let’s optimize our setup:

Adjust test frequency based on your needs:

• More frequent tests (15m) = more data but higher data usage
• Less frequent tests (60m) = less granularity but lower impact

Reduce storage requirements by adding retention settings to Prometheus:

# In your prometheus.yml under global:
global:
  scrape_interval: 60s
  evaluation_interval: 60s
  retention: 30d  # Keep data for 30 days

Add resource limits to your Docker Compose to prevent excessive resource usage:

services:
  speedtest-exporter:
    # ... existing config
    deploy:
      resources:
        limits:
          memory: 256M

Troubleshooting Common Issues

Here are some problems I encountered and how to fix them:

Speedtest fails with “No servers defined” error:

# Check logs
docker logs speedtest-exporter

# Sometimes restarting helps
docker restart speedtest-exporter