Bringing the QNAP NAS into Prometheus Monitoring

This article was last updated on: November 9, 2026 pm

Introduction

The National Day holiday is finally here! There’s nothing quite like staying home and tinkering with my HomeLab, K8s, and NAS server cluster. I really got a lot done this holiday — if you’re interested, stay tuned for the coming flood of posts; if you can’t wait, you can go straight to the PRs and commits in my repo, east4ming/homelab2.

In homelab2 there’s a QNAP NAS that has been running for several years, and it’s always been a nagging worry: it isn’t monitored.

The cluster already has full monitoring for nodes, pods, middleware, and URLs, yet this NAS is a black box. It holds my photos, backups, TV shows, and movies, and whether a disk has started reporting errors or a volume is about to fill up is something I only know if I occasionally log into the web console and take a peek. That’s classic “defensive ops” — you only find out after something breaks.

This had been on my to-do list for at least two years, so I used the National Day holiday to finally get it done. Once I’d finished, I found there were far more pitfalls than I expected — especially that “the dashboard has curves but the data is fake” kind of silent failure. Honestly, it was a bit of a slog.

First, the subject of this post:

Item Value
Model QNAP TS-453Bmini (NAS33657A)
OS QTS 5.2.10 / kernel 5.10.60-qnap / x86_64
Memory 16GB
LAN 192.168.3.216
Bays 4 internal + 1 external USB drive
Volumes 5

Final Result

Here are a few screenshots of the final result:

NAS Dashboard 1

NAS Dashboard 2

NAS Prometheus Alertrules

NAS alert received on phone

The Approach: Run Two Read-Only Exporters in Container Station

You can’t install a K8s agent on the NAS, but QNAP ships Container Station (essentially Docker). So the approach is straightforward:

  1. Run two exporters on the NAS over the host network, so the cluster can scrape them directly by IP:Port;
  2. Add two static jobs in Prometheus on the cluster side;
  3. Manage alerts and dashboards uniformly on the cluster side.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# metal/nas/docker-compose.yml
services:
  nas-node-exporter:
    image: quay.io/prometheus/node-exporter:v1.8.2
    container_name: nas-node-exporter
    network_mode: host
    restart: unless-stopped
    pid: host
    privileged: true
    command:
      - --path.rootfs=/host
      - --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|mnt/snapshot|share/NFSv=4)($|/)
    volumes:
      - /:/host:ro,rslave

  nas-smartctl-exporter:
    image: docker.io/prometheuscommunity/smartctl-exporter:v0.14.0
    container_name: nas-smartctl-exporter
    network_mode: host
    restart: unless-stopped
    privileged: true
    user: "0:0"
    command:
      - --smartctl.device=/dev/sda;sat
      - --smartctl.device=/dev/sdb;sat
      - --smartctl.device=/dev/sdc;sat
      - --smartctl.device=/dev/sdd;sat
      - --smartctl.device=/dev/sde;sat

📝Notes: restart: unless-stopped is mandatory — after the NAS loses power and reboots, collection has to resume automatically, otherwise you’ve got another silent blind spot.

Measured overhead: node-exporter uses 3101 series / 19.89MiB, with a 23.3MB image; smartctl-exporter uses 496 series / 6.449MiB, with a 32.2MB image. On a 16GB NAS that’s essentially imperceptible.

🐾 Note: Pick your image registry carefully. In testing, quay.io/prometheus-community/smartctl-exporter returned unauthorized; docker.io/prometheuscommunity/smartctl-exporter is the one that works. node-exporter is fine from quay.io.

Pitfall 1: Temperature Stuck at 0 — Silent Failure 😱

This one is the sneakiest.

In the first version of the compose file, I was lazy and didn’t specify devices, letting the exporter scan on its own:

1
2
3
4
5
smartctl --scan-open
# /dev/sda -d scsi
# /dev/sdb -d scsi
# /dev/sdc -d scsi
# /dev/sdd -d scsi

All four SATA mechanical drives were detected as SCSI. What’s the consequence?

  • smartctl_device_temperature is always 0;
  • ATA attributes (power-on hours, reallocated sectors, all SMART raw values) are completely missing;
  • Yet the exporter keeps exporting and Grafana keeps drawing curves as usual. 😑

This is what’s called a silent failure, and it’s far more dangerous than having no data at all. With no data you’d go investigate; with data you just believe it. If a disk ever really overheats, my alert rule would never fire.

The fix: force the SAT type.

1
--smartctl.device=/dev/sda;sat

📝Notes: The <device>;<type> syntax requires image ≥ v0.14.0. I tried v0.12.0 first and it just reported No such device, wasting half an hour.

Only after the change did temperatures look right: sda 46°C, sdb 49°C, sdc 45°C, sdd 44°C, sde 44°C, with smart_status equal to 1 for all five drives.

Pitfall 2: Devices Not Visible + Container Running as nobody

After forcing /dev/sdX, the second pitfall hit immediately: /dev/sd* simply doesn’t exist inside a normal container.

It only appears once you add privileged: true. But even then it can’t be opened — the container’s default UID is nobody, while /dev/sda has permissions brw------- root:root.

So these two lines must both be present:

1
2
privileged: true
user: "0:0"

📝Notes: This is the kind of combination I personally dislike: privileged + root. But reading SMART is inherently a privileged operation, so there’s no way around it. The mitigations are: the container mounts the host / read-only, doesn’t mount docker.sock, doesn’t listen on the public internet, and only exposes ports on the LAN IP.

Pitfall 3: Mount-Point Noise

Mounting the host root into node-exporter drags in a pile of junk:

  • 25 /mnt/snapshot/<pool>/<snap> snapshot views
  • /share/NFSv=4/* re-exports
  • /share/CACHEDEV*_DATA/.qpkg/* QPKG sandboxes
  • the host’s own /sys and /proc

Without exclusions, the capacity dashboard will render dozens of rows, all of them bogus.

After excluding them, node_filesystem_size_bytes is down to exactly 10 mount points, of which 6 are real storage volumes: /share/CACHEDEV1..5_DATA plus /share/external/DEV3303_1.

1
2
- --collector.filesystem.mount-points-exclude=^/(dev|proc|sys|run|mnt/snapshot|share/NFSv=4)($|/)
- --collector.filesystem.fs-types-exclude=^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$

Bay Labels: Why Hard-Coding Is Necessary

A disk alert that just says /dev/sdc is useless; what I want is “the drive in bay1”. The bay mapping is based on actual measurements:

1
2
3
4
5
qcli_storage -d
# sdc → bay1
# sdd → bay2
# sda → bay3
# sdb → bay4

Static mappings, one by one, in metric_relabel_configs:

1
2
3
4
5
6
metric_relabel_configs:
  - source_labels: [device]
    regex: /dev/sdc
    target_label: disk
    replacement: bay1
  # ... and the same for the other three

🤔 Some of you may ask: why not use the labelmap action in relabel_configs to map them in bulk?

Because separator is a global parameter, bulk mapping produces things like /dev/sdc;bay1, and you can’t get a clean disk=bay3. For a small case like this, honestly writing four lines is the least hassle.

Alerts: Two Cases of “Permanently False the Moment They Go Live”

Six rules: two-level volume capacity thresholds, volume remounted read-only, RAID loss of redundancy, SMART health failure, and high disk temperature. For scrape interruptions I reuse the existing TargetDown rather than defining a duplicate.

But passing promtool check rules after writing them doesn’t mean the rules are ready to go live. Evaluating them against real-environment samples revealed that two of the rules are permanent false positives:

False positive 1: /mnt/ext

It’s a 436MB firmware DOM (/dev/md13), and in testing only 7.69% was free. This is QNAP’s fixed layout and isn’t configurable. The volume capacity rule must scope itself to exclude the firmware DOM:

1
2
3
expr: |
  node_filesystem_avail_bytes{mountpoint=~"/share/.+"}
    / node_filesystem_size_bytes{mountpoint=~"/share/.+"} < 0.15

False positive 2: md9 / md13

These two are firmware DOM mirrors with 32 slots, of which only 4 are actually used, so node_md_disks{state="active"} < node_md_disks_required is always true. The RAID rule must hard-code the range and use on (device) for vector matching:

1
2
3
expr: |
  node_md_disks{state="active", device=~"md1|md2|md256|md322"}
    < on (device) node_md_disks_required

After the fix, all six rules are inactive in the current environment (data volumes 29%–90% free, SMART all 1, highest disk temperature 49°C).

📝Notes: These two are domain constraints, not bugs. You can’t change QNAP’s firmware DOM design, so you can only hard-code the scoping into the rules. When writing alerts, checking whether the syntax passes is nowhere near enough — you have to evaluate each rule against real samples.

The Boundary of GitOps: ArgoCD Can’t Manage Containers Outside the Cluster

All cluster-side resources (scrape config, rules, dashboard) go through ArgoCD — no argument there. The debate is on the NAS side.

I rejected several options one by one:

Option Why It Was Rejected
Install k3s on the NAS and join it as a node A 4-bay home NAS — installing Kubernetes just to run two exporters is using a sledgehammer to crack a nut
ArgoCD ConfigManagementPlugin Still requires landing a container on the NAS; the problem isn’t solved
Build a custom webhook controller Maintenance cost far outweighs the benefit
Non-container install via Entware No version pinning, no rollback, pollutes the QTS system

The final approach: commit the compose file to the repo + import it manually in Container Station.

The upside is that the configuration is versioned, traceable, and reviewable; the price is that the import step is manual — change the compose file and you have to go rebuild the application in Container Station.

Honestly, I accept that price. ArgoCD’s jurisdiction ends at “inside the cluster”; forcing it out onto devices outside the cluster only produces a pile of more brittle glue. The NAS is not a cluster node and shouldn’t be stuffed into metal/inventories/prod.yml. So the NAS scrape definitions live separately under metal/nas/, each minding its own business.

Metrics I Didn’t Collect, for the Record

  • Chassis temperature, fan speed: the QNAP controller doesn’t expose these via hwmon; under /sys/class/hwmon there’s only the CPU’s coretemp. So temperature monitoring covers only the drives.
  • qcli_hdd: requires an interactive QCLI login and would introduce admin credentials — rejected.
  • get_hd_smartinfo: produced no output in testing and depends on a proprietary library — rejected.
  • There is no smartctl or snmpd on the NAS.

I looked at two repos (zottelbeyer/QNAP-collectd-influxdb-grafana, which pushes to InfluxDB via collectd, and sandrotosi/qnap-dashboards, which relies on SNMP); neither is compatible with this repo’s Prometheus pull model, so I used them only as references for metric dimensions and didn’t adopt them.

Operational Notes

  1. Adding or removing a drive: update the --smartctl.device list in the compose file and rebuild the application in Container Station. Once devices are specified explicitly, the exporter no longer rescans automatically.
  2. Device names changing after a QTS upgrade: re-verify bay assignments with qcli_storage -d.
  3. The NAS gets a new IP: update the two static targets in values.yaml.
  4. Drive spin-down: SMART reads may wake the drives; if that bothers you, weigh it against the scrape interval.

Summary

Getting the NAS into monitoring is only skin deep; what really taught me a lesson were those silent failures.

Curves on a dashboard and metrics being exported don’t mean the data is correct. SCSI misdetection, a UID that can’t open the device, mount-point noise, the permanent false positives from the firmware DOM — none of them throw an error; they just quietly lie to you.

What you get from books is ultimately shallow; only by doing it yourself do you truly understand. Rule syntax passing and promtool passing are only the first step; what actually makes rules trustworthy is running them one by one against real-world samples, checking them, and trying to falsify them.

This setup now runs on homelab2: a 13-panel nas.json, six alert rules all inactive, five drives at 44–49°C, and data volumes 29%–90% free. No more logging into the web console. 🎉

That’s all.

📚️ References