Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions etc/kmod/whitelist.conf
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ whitelist realtek # Realtek PHY driver
whitelist rng_core # H/W Random Number Generator (RNG) driver
whitelist rpcsec_gss_krb5 # Sun RPC Kerberos 5 module
whitelist sb_edac # MC Driver for Intel Sandy Bridge and Ivy Bridge memory controllers
whitelist sch_fq # Fair Queue Packet Scheduler
whitelist sch_fq_codel # Fair Queue CoDel discipline
whitelist sch_prio # Simple 3-band priority qdisc
whitelist sch_sfq # Stochastic Fairness qdisc
whitelist sha3_generic # SHA-3 Secure Hash Algorithm
Expand Down
19 changes: 19 additions & 0 deletions etc/sysctl.d/90-default-qdisc.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Use fq instead of pfifo_fast as the default qdisc.
#
# fq paces locally generated TCP traffic at the rate the TCP stack
# computed for the socket. Data then leaves in an even stream instead of
# line-rate bursts, which are likely to overflow the small buffers of
# access points and switch ports in front of slow links (for example a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“likely to overflow the small buffers of access points” contradicts our UDP measurements, where bursts of about 600 kB passed without loss. claude-opus-5.5 suggests: “which some access points (seen with Cisco) handle badly”

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure, UDP bursts will also overrun the buffers sooner or later and we can't help it. However with tcp we can because tcp can pace the transmit data (even CUBIC) but this is not honored by pfifo_fast. I don't agree that Cisco devices handle this badly. It does make sense to bundle packets to save air time and this is not a bug per se although overruns by badly throttled senders are more likely to happen in that case.

In any case, fq seems to be the better choice.

# laptop on wifi).
#
# fq also keeps a separate queue per flow and serves them in turn. When
# the local link is saturated, a single bulk transfer can no longer delay
# all other flows on the interface, as it can with the shared FIFO of
# pfifo_fast.
#
# Note: fq is meant for hosts that terminate TCP connections. On a router,
# fq_codel may be the better choice: forwarded packets have no local
# socket, so fq cannot pace them, while the CoDel part of fq_codel actively
# keeps queueing delay low.

net.core.default_qdisc = fq
19 changes: 18 additions & 1 deletion mxrouter/mxrouterctl
Original file line number Diff line number Diff line change
Expand Up @@ -423,10 +423,27 @@ sub radvd {
our $want_if = {'lo' => 1}; # includes lo, veth and physical devices but not vlan devices
our $want_veth;

# Use fq_codel, which suits forwarded traffic, instead of the global default
# qdisc. Keep mq on multiqueue devices and use fq_codel per tx queue.
sub set_qdisc_fq_codel {
my ($dev) = @_;
my $tx_queues = grep /^tx-/, scandir("/sys/class/net/$dev/queues");
if ($tx_queues > 1) {
sys('tc', 'qdisc', 'replace', 'dev', $dev, 'root', 'handle', '1:', 'mq');
for my $i (1 .. $tx_queues) {
sys('tc', 'qdisc', 'replace', 'dev', $dev, 'parent', sprintf('1:%x', $i), 'fq_codel');
}
} else {
sys('tc', 'qdisc', 'replace', 'dev', $dev, 'root', 'fq_codel');
}
}

sub configure_if {
my ($want_if) = @_;
for my $if (keys %$want_if) {
netif_is_up($if) or sys('ip', 'link', 'set', $if, "up");
netif_is_up($if) and next;
sys('ip', 'link', 'set', $if, "up");
-d "/sys/class/net/$if/device" and set_qdisc_fq_codel($if);
}
}

Expand Down