Cilium Series Part 12: Enabling BBR Congestion Control for Pods

This article was last updated on: May 17, 2026 am

Series Articles

Introduction

Switching the Kubernetes CNI from other options to Cilium already provides a significant boost in network performance. However, by toggling different Cilium modes and enabling additional features, you can further improve Cilium’s network performance. Tuning options include but are not limited to:

  • Enable Native Routing
  • Fully replace KubeProxy
  • Switch IP Address Masquerading to eBPF-based mode
  • Run Kubernetes NodePort in DSR (Direct Server Return) mode
  • Bypass iptables Connection Tracking
  • Switch Host Routing to BPF-based mode (requires Linux Kernel >= 5.10)
  • Enable IPv6 BIG TCP (requires Linux Kernel >= 5.19)
  • Disable Hubble (not recommended — observability is more important than a marginal performance gain)
  • Set MTU to jumbo frames (requires network conditions to allow it)
  • Enable Bandwidth Manager (requires Kernel >= 5.1)
  • Enable BBR congestion control for Pods (requires Kernel >= 5.18)
  • Enable XDP acceleration (requires native XDP driver support)
  • (Optional for advanced users) Adjust eBPF Map Size
  • Linux Kernel optimization and upgrade
    • CONFIG_PREEMPT_NONE=y
  • Other:
    • tuned network-* profiles, e.g.: tuned-adm profile network-latency or network-throughput
    • Set CPU to performance mode
    • Stop irqbalance and pin NIC interrupts to specific CPUs

When network/NIC/OS conditions permit, we enable as many of these tuning options as possible. Related optimizations will be covered one by one in subsequent articles. Stay tuned.

Today we’ll tune Cilium by enabling BBR congestion control for Pods, delivering higher bandwidth and lower latency for internet traffic.

Test Environment

  • Cilium 1.13.4
  • K3s v1.26.6+k3s1
  • OS
    • 3 Ubuntu 23.04 VMs, Kernel 6.2, x86

BBR Congestion Control

The infrastructure around MQ/FQ setup provided by Cilium’s Bandwidth Manager also enables the use of TCP BBR congestion control for Pods. BBR is particularly useful when Pods are exposed behind Kubernetes Services to external clients from the internet. BBR delivers higher bandwidth and lower latency for internet traffic — for example, BBR has been shown to achieve up to 2,700x higher throughput than the best existing loss-based congestion control, and reduce queuing delay by 25x.

For BBR to work reliably on Pods, it requires kernel version 5.18 or later.

BBR also requires eBPF Host-Routing to preserve the socket association of network packets until they enter the FQ queuing discipline on the physical device in the host namespace.

Requirements

  • Kernel >= 5.18
  • Bandwidth Manager
  • eBPF Host Routing

Implementation

Enable the Bandwidth Manager and BBR for Pods:

1
2
3
4
helm upgrade cilium cilium/cilium --version 1.13.4 \
  --namespace kube-system \
  --reuse-values \
  --set bandwidthManager.bbr=true

Verification

To verify that your installation is running with BBR for Pods, run cilium status in any Cilium pod and look for the line reporting “BandwidthManager” status, which should show EDT with BPF along with [BBR]. Example:

1
2
$ kubectl -n kube-system exec ds/cilium -- cilium status | grep BandwidthManager
BandwidthManager:           EDT with BPF [BBR] [eth0]

Summary

This article continues tuning Cilium by enabling BBR congestion control for Pods, delivering higher bandwidth and lower latency for internet traffic. The prerequisites are Kernel >= 5.18, with the Bandwidth Manager and eBPF-based Host Routing enabled.

At this point, the following performance tuning items have been validated in practice:

  • ✔️ Enable Native Routing
  • ✔️ Fully replace KubeProxy
  • ✔️ Switch IP Address Masquerading to eBPF-based mode
  • ✔️ Run Kubernetes NodePort in DSR (Direct Server Return) mode
  • ✔️ Bypass iptables Connection Tracking
  • ✔️ Switch Host Routing to BPF-based mode (requires Linux Kernel >= 5.10)
  • ❌ Enable IPv6 BIG TCP (requires Linux Kernel >= 5.19, supported NICs: mlx4, mlx5)
    • Could not be validated due to lack of supported NICs
  • ❌ Set MTU to jumbo frames (requires network conditions to allow it)
  • ✔️ Enable Bandwidth Manager (requires Kernel >= 5.1)
  • ✔️ Enable BBR congestion control for Pods (requires Kernel >= 5.18)
  • Enable XDP acceleration (requires native XDP driver support)

📚️ References