Early native ipv6 latency under load tests of comcast ipv6

I got a new cablemodem for christmas that was IPv6 capable. Installing it was a breeze. Getting ipv6 running was still kind of hard, there are still multiple things busted in CeroWrt or upstream that have to get fixed (notably we're not seeing RAs internally). But dhcpv6-pd works, we have some fixes to do to deal with exceedingly large numbers of RAs from the CMTS, and with SOME twiddling you get ipv6.

So I did a bit of testing of performance relative to ipv4 over wired and wireless. Note that I'd changed two variables - the cable modem, and was also fiddling with the protocols (ipv4 and ipv6). It turned out the cable modem result was pretty interesting.

Defeating overbuffering at the CMTS

For starters, there is simply a "heroic" amount of downstream buffering configured on my CMTS in FT Myers, FL. Having well over a second of downstream buffering is good for some benchmarks and lousy for general use. So, on this first test without shaping, uploads are completely drowned out by downloads.

But once you add in cerowrts's shaper to get both up and downloads under control, you get much more reasonable performance overall:

But in seeing the download speed achieved on the first graph, I realized that this modem was capable of running at least twice as fast as the prior one. Speedtest was inconclusive, giving download rates varying between 8mbit and 74mbit. So after some twiddling I settled on a download rate of 60mbits and an upload of 10mbit...

I turned off download shaping for a test at these speeds, but kept upload shaping the same. This test shows that fixing download buffering is still massively needed, with over 1.6 seconds of buffering in the CMTS (or cable modem) It really is the biggest problem....

Further testing showed that the current cerowrt HTB shaper tops out at about 60mbit down and 10mbit up - eating 100% of cpu. So some twiddling later, I got 15% of cpu back, and got this result:

Note that due to a bug in netperf with ipv6 vs ipv4, the ipv4 portion of the test starts 20 seconds after it should. So your effective early bandwidth is twice the early average, and 4x the average in the middle of the test. Note that roughly the same download speed was achieved and roughly double the upload, and the latency induced was nearly immeasurable. Your voip calls, gaming, etc, will rock, always.

In conclusion, ipv6 WORKS! YEA! Also in conclusion, we CAN have high bandwidth, in both directions, with low latency, with cerowrt, up to about 60mbit down/10 up (with no other traffic on the router). It's still hard, though, and the last generation chipset in cerowrt simply cannot run fast enough to run the htb code at the speeds needed, presently. A replacement for HTB might help.

It is my hope that overbuffered CMTS's will become a thing of the past, and that cable modems will gain AQM technologies that work halfway decently so we don't have to shape stuff, but I'm not holding my breath.

Until then, so long as you are running at less than 70mbit or so, cerowrt's SQM system can help enormously.

A few other benchmarks

This is just a few other benchmarks. We can't compare native ipv6 vs ipv4 performance without the shaper in place,so we end up wtih ipv4 and ipv6 looking alike, and for reference I stuck at the earlier 25mbit/4mbit values

Two interesting things about the above test are that there is a bug in starting netperf to do both ipv6 and ipv4 that causes nearly 20sec of delay in the ipv4 startup (I don't know what causes it). And: we did ECN in both directions, and the capture has nearly 0 loss. You can also see that the diffserv values are stomped on, nearly all the traffic is marked with CS1 (background) on it's return from the other side, instead of CS5. (this is valid diffserv behavior)

We can test wifi over ipv6 too, but as wifi is the bottleneck here we shouldn't see much difference:

But wait! This is native ipv6 unshaped over wifi. Note that still not having the shaper doesn't hurt as much as we are unable to drive the wifi hard enough to hit the bandwidth limit in this case.

As well as over a longer haul

Over a longer haul shaping does a bit of good, still.