systemd'oh! DNS lib underscore bug bites everyone's favorite init tool, blanks Netflix
Repeat after me: _ is allowed in domain names
A few Penguinistas spent a weekend working out why they can't get through to Netflix from their Linux machines, because when they tried, their DNS lookups failed.
The issue emerged July 22, when Gentoo user Dennis Schridde submitted this bug report to the Systemd project. Essentially, he described a failure within systemd-resolve, a Systemd component that turns human-readable domain names into IP addresses for software, like web browsers, to connect to. It's the thing that converts, say, theregister.co.uk into 184.108.40.206.
The Systemd resolver couldn't look up Netflix's servers for Schridde's web browser, according to the report. In his detailed post, Schridde said he expected this to happen:
ipv6_1-cxl0-c088.1.lhr004.ix.nflxvideo.net gets resolved to 220.127.116.11 or 2a00:86c0:5:5::142.
When in reality, that wasn't happening, so Netflix couldn't be reached on his box. His speculation that libidn2, which adds internationalised domain names support to the resolver, was at fault turned out to be accurate. Rebuilding Systemd without that library cleared the problem.
“I just rebuilt Systemd without libidn2 support and am now certain that the wrong behaviour is directly related to the
-Dlibidn=false -Dlibidn2=true Meson flags,” wrote Schridde.
The library was stripping underscores from some domain names – such as Netflix's ipv6_1-cxl0-c088 node – and that caused everything relying on the resolver to fail, Schridde reported. This problem affects Systemd version 234, we're told.
If you're affected by this DNS problem, rebuild Systemd without libidn2, stop using Systemd as your resolver if possible, apply this temporary patch – or better yet, wait for libidn2 to be fixed to cope with underscores, which are, in special circumstances, allowed in domain names. What a mess. ®