Fix: Systemd-resolved Stops Resolving DNS Entries
Hey guys! Having trouble with systemd-resolved suddenly stopping DNS resolution? It's super frustrating, I know! This guide will walk you through troubleshooting and hopefully fixing this annoying problem. We'll cover everything from checking your systemd version to digging into those cryptic logs.
Understanding the Issue
So, the main problem is that systemd-resolved, which is supposed to be handling your DNS queries, just...stops. After a couple of minutes, you're left with no internet access because your system can't translate domain names into IP addresses. Restarting the service is a temporary fix, but who wants to do that every few minutes?
Key Symptoms
- DNS resolution failure: The most obvious symptom is that your computer can't resolve domain names. Web pages won't load, and other network applications might fail.
- Temporary fix: Restarting
systemd-resolvedtemporarily solves the problem, but it returns after a short period. - Log messages: The system logs show a "degraded" feature set, with
systemd-resolvedswitching from UDP+EDNS0 to DNS and then to TCP. This indicates it's having trouble with the initial, preferred methods of communication.
Investigating the Problem
Alright, let's get our hands dirty and start figuring out what's going on. Here's a breakdown of the areas we'll be looking at:
1. Check Your systemd Version
First, confirm the version of systemd you're running. The reported issue was seen with systemd 258 (258.3-2.fc43). Knowing your version helps determine if you're running into a known bug. Run this command in your terminal:
systemd --version
Make sure that the systemd version matches the one you are debugging, in this case it's systemd 258 . If not, then your issue may be different.
2. Identify Your Distribution and Kernel
Note down the Linux distribution you're using (e.g., Fedora 43) and the kernel version. This information can be crucial when searching for distribution-specific bugs or compatibility issues. Use these commands:
cat /etc/os-release
uname -r
If your distribution is Fedora 43 and the kernel version is 6.18.5-200.fc43.x86_64 then proceed to the next steps, otherwise you may be facing a different error.
3. Examine the Logs
The logs are your best friend here. They'll give you clues about what's going wrong. Use journalctl to view the logs for systemd-resolved:
journalctl -u systemd-resolved
Look for any error messages or warnings that might indicate the cause of the problem. Pay close attention to messages related to the "degraded" feature set.
Analyzing the Logs
The logs provided show a recurring pattern: systemd-resolved starts off using UDP with EDNS0, then downgrades to plain DNS over UDP, and finally falls back to TCP. This usually indicates a problem with the network path to your DNS server or the server itself. Common causes include:
- MTU issues: The Maximum Transmission Unit (MTU) might be too large for your network, causing UDP packets to be fragmented and dropped.
- Firewall interference: A firewall might be blocking UDP traffic on port 53 (the standard DNS port).
- DNS server issues: The DNS server you're using might be unreliable or misconfigured.
4. Check DNS Server Configuration
Make sure your DNS servers are correctly configured in /etc/resolv.conf or through your network manager. You can try using public DNS servers like Google's (8.8.8.8 and 8.8.4.4) or Cloudflare's (1.1.1.1 and 1.0.0.1) to see if the issue is with your current DNS server.
To edit /etc/resolv.conf, you might need to disable systemd-resolved temporarily:
systemctl stop systemd-resolved
systemctl disable systemd-resolved
Then, edit the file:
vi /etc/resolv.conf
Add the following lines:
nameserver 8.8.8.8
nameserver 8.8.4.4
After editing, re-enable systemd-resolved:
systemctl enable systemd-resolved
systemctl start systemd-resolved
5. MTU Discovery Issues
MTU (Maximum Transmission Unit) issues can lead to DNS resolution problems, especially when UDP packets are involved. If the MTU is too large, packets might be fragmented, leading to failures. Let's explore how to diagnose and address MTU-related issues.
Diagnosing MTU Issues
-
Determine the Current MTU: Use the
ipcommand to check the MTU of your network interfaces:
ip link show ```
Look for the `mtu` value in the output for each interface (e.g., `eth0`, `wlan0`).
-
Ping with the "Don't Fragment" (DF) Flag: Use the
pingcommand with the-M dooption to test the path MTU to a known host (like Google's DNS server):
ping -c 3 -M do -s
Start with a size close to your interface MTU (e.g., 1472 for a 1500 MTU) and decrease it until the pings succeed. The `-s` option specifies the packet size, and the `-M do` option sets the "Don't Fragment" flag. The size is the data portion of the IP packet, to calculate the full packet size you need to add the IP and ICMP headers, which are typically 28 bytes (20 bytes for IP header + 8 bytes for ICMP header). So, for an MTU of 1500, the data size would be 1472.
- Interpret the Results: If you receive "Fragmentation required" errors, it indicates that the path MTU is smaller than the packet size you specified. Reduce the packet size until you find the largest size that works without fragmentation.
Addressing MTU Issues
-
Adjust the MTU on Your Interface: If you've determined that your MTU is too high, you can adjust it using the
ipcommand. For example, to set the MTU ofeth0to 1400:
sudo ip link set eth0 mtu 1400 ``` 2. Make the Change Persistent: The above command only changes the MTU temporarily. To make it permanent, you'll need to configure it through your network configuration files. The method varies depending on your distribution and network management tool.
* **NetworkManager:** Edit the connection settings in NetworkManager and set the MTU in the IPv4 or IPv6 settings.
* **systemd-networkd:** Edit the `.network` file for your interface and add the `MTUBytes` option in the `[Network]` section.
* **ifupdown (Debian/Ubuntu):** Edit the `/etc/network/interfaces` file and add the `mtu` option to your interface configuration.
Specific Solutions Based on Logs
Given that the logs indicate a "degraded" feature set, it's likely the issue lies in the negotiation of DNS features. Here's what you can try:
1. Disable UDP Checksums Offloading
Some network cards have issues with UDP checksum offloading. Disabling this feature might help:
sudo ethtool -K <interface> rx off tx off
Replace <interface> with your network interface name (e.g., eth0, wlan0). To make this permanent, you'll need to find the appropriate configuration file for your distribution.
2. Adjust systemd-resolved Configuration
You can modify the systemd-resolved configuration file (/etc/systemd/resolved.conf) to explicitly set the DNS protocols. For example, you can force it to use TCP only:
[Resolve]
DNS=8.8.8.8 8.8.4.4
DNSOverTLS=no
DNSSEC=no
FallbackDNS=8.8.8.8 8.8.4.4
TCP=yes
After editing, restart systemd-resolved:
systemctl restart systemd-resolved
3. Check Firewall Settings
Ensure that your firewall isn't blocking UDP or TCP traffic on port 53. If you're using firewalld, you can check the rules:
sudo firewall-cmd --list-all
Make sure that DNS traffic is allowed. If not, add the necessary rules:
sudo firewall-cmd --add-service=dns --permanent
sudo firewall-cmd --reload
Conclusion
Troubleshooting DNS issues can be tricky, but by systematically checking your configuration, examining the logs, and trying different solutions, you should be able to get systemd-resolved working reliably. Remember to take it one step at a time and test after each change to isolate the problem. Good luck, and happy debugging!