Promises and Potential of BBRv3 (Best paper award award)

This paper was presented at the 2024 Passive and Active Measurement Conference (PAM '24)

Overview

The Bottleneck-Bandwidth and Round-trip (BBR) congestion control algorithm was introduced by Google in 2016. Unlike prior congestion-control algorithms (CCAs), BBR does not rely on signals that are weakly correlated with congestion (e.g., packet loss and transient queue delay). Instead, it characterizes a path using two parameters, bottleneck bandwidth and round-trip propagation time, and is designed to converge with a high probability to Kleinrock’s optimal operating point.

This paper focuses on characterizing the promises and potential of the third, and most recent, revision of BBR—introduced to the public in July 2023. We empirically evaluate BBRv3’s performance across a range of network scenarios, e.g., considering different buffer sizes, round-trip times, packet losses, and flow-size distributions.

In our evaluations, we find that BBRv3 enables competing flows to converge to their fair shares quicker than prior versions. If these flows, however, do not start (or arrive at the bottleneck) at the same time, we also observe that its convergence time deteriorates substantially. We note that BBRv3 (similar to BBRv2) is biased towards long-RTT flows; they receive a higher bandwidth than a competing short-RTT flow, regardless of which flow starts first. We observe that BBRv3 struggles to co-exist with loss-based CCAs such as Cubic; it offers little to no room for Cubic flows to co-exist even if flow RTTs are similar.

Results

In the following, we highlight some of the key results. For more details please refer to the paper.

I. Similar and synchronous flows

In this simple scenario we evaluate two similar flows, i.e., using the same CCA, experiencing the same RTT (of 100 ms), starting at the same time and contending for bandwidth on a bottleneck. BBRv3 significantly reduces the throughput oscillations and allows the flows to converge much quicker than either of the earlier two versions.

Fig. 1: Throughput achieved by two similar, synchronous (i.e., starting at the same time) flows when competing for bandwidth on a bottleneck with a deep buffer of size 16×BDP.

II. Similar but staggered flows

In this scenario we repeat the previous evaluation, but we stagger the flows so that the second flow starts 15 seconds after the first.
It seems that BBR implementations may face challenges when it comes to quickly reaching an equitable share of bandwidth for flows that don't start at the same time. As an example, a BBRv3 flow joining a bottleneck link 15 seconds later with a 16 × BDP buffer, it can take over 4 minutes for it to achieve equitable bandwidth sharing.

Fig. 2: Throughput achieved by two similar but staggered flows (i.e., second flow starts 15 s after the first) when competing for bandwidth on a bottleneck with a deep buffer of size 16×BDP.

III. Dissimilar flows: RTT fairness

This experiment evaluates the bandwidth share when flows using the same CCA traverse different routing paths, thus experiencing different RTTs, and sharing a common bottleneck along the way. We run two flows using the same CCA, where the first flow is our base flow with a fixed RTT of 160 ms and the second flows joins after 15 second and experiences a different RTT—one of 10, 20, 40ms, each constituting a different experiment.
Our inferences are in agreement with prior work that demonstrated BBR favoring long-RTT flows; they receive much higher bandwidth than the second flow regardless of which flow starts first. Our results confirm that BBRv3’s behavior does not vary substantially from BBRv2.

Fig. 3: Average throughput difference (TD); 160 ms flow starts first, then 15 s later the (10 ms, 20 ms, 40 ms) flow starts.

IV. Dissimilar flows: Inter-CCA fairness

To characterize BBRv3 flows’ ability to co-exist with loss-based CCAs such as Cubic, we run two (co-existing) long-lived flows, each using a different CCA.
BBRv3, which Google claims will quickly converge to fair share, performs even slightly worse than BBRv1. Even if flows experience same RTTs, BBRv3, in its current form, offers no room for Cubic flows to co-exist; any RTT differences between the flows may only exacerbate this situation.

Fig.4: Throughput of a BBR flow when competing for bandwidth with a Cubic flow on a bottleneck link with a 1×BDP buffer.

Code

This repository includes the testbed configuration and evaluation artifacts.

Contact

Danesh Zeynali
Emilia Ndilokelwa Weyulu
Seifeddine Fathalli