Veserv 3 days ago

They claim that at that speed they are spending most of their time in the kernel UDP stack. Is the Mac UDP stack really that abysmal?

Encapsulating byte blobs into UDP packet payloads can easily be designed to go at nearly memory bandwidth speeds which, on a base M1 Mac, is ~500 Gb/s, nearly 100x faster than the speed in the article. Saturating their 40 Gb/s link should be trivial on a single core unless the UDP stack is incompetently designed.

  • jiehong 3 days ago

    With Http3 and quic, the Linux kernel was found to also have pretty inefficient UDP path not long ago, and I believe still not as good as TCP for throughput.

    Consequently, it’s not too surprising it’s the case in other kernels.

  • wh33zle 3 days ago

    > Is the Mac UDP stack really that abysmal?

    Unfortunately yes.

    It is quite easy to replicate if you have a Mac and another machine with a Thunderbolt port.

    With everything else stripped out, i.e. bare UDP blasting out the interface, you can get the number up to ~675k packets per second which is about 1.25µs of kernel CPU per packet. For comparison, on Linux with segmentation offloading, you end up with around 0.2µs per packet.

gkbrk 3 days ago

A good coincidence for me. I recently improved the performance of my mesh network using the same sendmsg_x / recvmsg_x API they ended up using in the article too.

Unlike Linux, it's a private API and not something that has support guarantees, but so many projects use it because there's no alternative that can send multiple UDP packets with one syscall.

https://github.com/gkbrk/smolmesh2/commit/10b0c4b9ae898e0fe7...