Before you read. From the last lesson, the two-command test:
ping -c 2 1.1.1.1 # works ping -c 2 example.com # does notThe network is proven good. Packets go out and come back. So the failure is in the step before any packet is sent: turning
example.cominto a number.Here is the question worth holding.
dig example.comreturns a perfectly good answer, and the application on the same machine still cannot connect. How can two programs on one machine disagree about what a name means?
Because they do not ask the same thing. dig talks to a DNS server directly.
Almost everything else goes through a system library that consults several
sources, of which DNS is only the last.
That gap is where most name resolution problems live, and it is the reason this lesson spends more time on files than on DNS itself.
Some words you will need
- resolver
- The code that turns a name into an address. Part of the C library, so nearly every program uses the same one.
- nameserver
- A DNS server the machine asks. Listed in
/etc/resolv.conf. - FQDN
- Fully qualified domain name: the whole thing,
web01.example.com, not justweb01. - search domain
- A suffix automatically appended to short names, so
web01is tried asweb01.example.com. - TTL
- How long an answer may be cached, in seconds. Why a DNS change does not take effect immediately.
- NXDOMAIN
- The authoritative answer "that name does not exist". Different from no answer at all.
What breaks without this
Everything, intermittently. Name resolution sits under every connection a machine makes. When it is slow, everything is slow; when it is partial, failures look random.
You blame the wrong layer. “The application cannot reach the database” is usually networking, sometimes the database, and quite often a name.
You edit a file that gets overwritten. /etc/resolv.conf is generated on most
modern systems, so a hand edit works until the next network event and then
silently reverts.
The order things are asked in
The single most important file here is not the DNS one:
# Fedora CoreOS 44.20260707.3.1 on a virtual machine, aarch64
$ cat /etc/resolv.conf; echo "--- nsswitch hosts line ---"; grep "^hosts" /etc/nsswitch.conf
# Generated by NetworkManager
search home.arpa
nameserver 192.168.127.1
--- nsswitch hosts line ---
hosts: files myhostname resolve [!UNAVAIL=return] dnsRead the hosts: line left to right. That is the order.
files is at the front of it. That is why an entry in /etc/hosts beats DNS every time, on every Linux machine, and why a stale line in that file produces a name that resolves to the wrong address no matter what the DNS server says. /etc/resolv.conf only matters once the search reaches the last box.| Source | Means |
|---|---|
files |
/etc/hosts, checked first |
myhostname |
This machine’s own name, answered locally |
resolve |
Ask systemd-resolved |
[!UNAVAIL=return] |
If the previous source answered anything other than “I am unavailable”, stop here |
dns |
Fall back to querying /etc/resolv.conf’s nameservers directly |
/etc/hosts wins over DNS. Always, on every Linux system, because files
comes first. That one fact explains a large fraction of the confusing cases in
this lesson.
And /etc/resolv.conf is the DNS configuration, consulted only when the search
reaches dns:
nameserver 192.168.127.1, who to ask. Up to three; they are tried in order, and the second is only used when the first does not answer at all.search home.arpa, the suffix to try on short names.# Generated by NetworkManager, the warning. This file is written by something else, and your edits will not last.
If you already administer Linux: search domains, and the query you did not know you were making
The search line in resolv.conf is the most quietly destructive piece of
configuration on a Linux host, because it changes queries you thought were
absolute.
Anything without a trailing dot is a relative name. With search corp.example.com example.com, looking up db01 produces up to three queries
(db01.corp.example.com, db01.example.com, and finally db01) in that
order, stopping at the first that answers. ndots:1 is the threshold: a name
with fewer than that many dots gets the search list appended before being
tried as-is.
Three consequences worth having met:
A name that resolves for you and not for a colleague is usually a search
domain doing the work, not DNS. api resolving to api.corp.example.com on
your machine and nothing on a server with a different search list is the same
query producing two answers, correctly.
Latency, in containers especially. Kubernetes sets ndots:5 by default, so
api.example.com (three dots, fewer than five) is tried as
api.example.com.namespace.svc.cluster.local first, then two more suffixes,
before the real query. That is four round trips for every external lookup, and
it shows up as unexplained per-request latency. A trailing dot,
api.example.com., makes the name absolute and skips all of it.
A wildcard record turns a typo into a success. With a search domain whose zone
has *.corp.example.com, a misspelled hostname resolves to something rather
than failing, and the failure moves from DNS to whatever answers.
dig +search makes dig use the search list, which it otherwise ignores,
that alone explains a good share of “dig works and the application does not”.
The reverse check is dig +nosearch or a trailing dot.
cat /etc/resolv.conf
dig +search db01
dig db01.
resolvectl query db01On a machine using systemd-resolved, /etc/resolv.conf is a symlink and
frequently a lie. It points at a stub listing 127.0.0.53, and the real
per-interface configuration, including per-link search domains and
split-horizon routing, is only visible through resolvectl status. Editing
the file directly on such a machine is overwritten, and the edit appears to
work until the next reconnect.
The tools, and why they disagree
# Fedora CoreOS 44.20260707.3.1 on a virtual machine, aarch64
$ dig +noall +answer example.com; echo "--- and the short form ---"; dig +short example.com
example.com. 0 IN A 104.20.23.154
example.com. 0 IN A 172.66.147.243
--- and the short form ---
104.20.23.154
172.66.147.243dig is the detailed tool. +noall +answer strips everything except the answer
section; +short gives just the addresses.
The columns are: name, TTL, class, record type, value. Two A records means
the name resolves to two addresses and clients pick one, which is the simplest
form of load balancing there is.
host queries DNS directly. getent hosts goes through the name service switch,
which consults /etc/hosts first and only then asks DNS.
For a name with no entry in /etc/hosts, both end up asking the same DNS server. Should their answers match, and does that mean the two commands are interchangeable?
# Fedora CoreOS 44.20260707.3.1 on a virtual machine, aarch64
$ host example.com; echo "--- getent goes through nsswitch ---"; getent hosts example.com
example.com has address 104.20.23.154
example.com has address 172.66.147.243
example.com mail is handled by 0 .
--- getent goes through nsswitch ---
104.20.23.154 example.com
172.66.147.243 example.comThey agree here, and they are not interchangeable. The answers match only
because nothing local overrides this name. getent hosts is the one that
tells you what an application will actually get, because applications call
the same resolver library it does, so a wrong /etc/hosts entry, an
nsswitch.conf ordering, or systemd-resolved’s cache all show up in
getent and are invisible to dig.
That is the whole diagnostic value: when dig and getent disagree, the problem
is not DNS. It is something between the application and DNS, and the four lessons
worth of configuration in this topic are where to look.
| Tool | Asks | Use it to find out |
|---|---|---|
dig |
DNS directly | What DNS actually says |
host |
DNS directly | The same, more briefly |
nslookup |
DNS directly | The same; familiar from Windows |
getent hosts |
The full nsswitch chain | What your applications will get |
getent hosts is the one people do not know and should. It goes through
exactly the path a normal program takes (/etc/hosts first, then the rest) so
its answer is the answer your application will see.
dig, host, and nslookup all bypass /etc/hosts entirely.
Which produces this:
Somebody adds 203.0.113.10 example.com to /etc/hosts. What does getent hosts example.com return, and what does dig +short example.com return?
# Debian 13 (trixie), x86_64
$ echo "203.0.113.10 example.com" >> /etc/hosts; echo "--- what the system thinks ---"; getent hosts example.com; echo "--- what DNS actually says ---"; dig +short example.com
--- what the system thinks ---
203.0.113.10 example.com
--- what DNS actually says ---
104.20.23.154
172.66.147.243They disagree completely, and both are correct.
getent follows nsswitch, which checks files first, finds the entry in
/etc/hosts, and stops. It never asks DNS at all.
dig queries the DNS server directly and gets the real, public answer.
Every application on this machine will connect to 203.0.113.10, because
applications use the resolver library and the resolver library reads
/etc/hosts.
This is the shape of one of the most frustrating faults you will meet.
“DNS is broken”, except dig proves DNS is perfect. The person diagnosing it
runs the DNS tools, sees correct answers, and concludes the problem is
elsewhere, when a single line in a file is overriding everything.
getent hosts finds it in one command. So does grep <name> /etc/hosts, once it
occurs to you to look.
For what it is worth, the mechanism is genuinely useful: overriding a name in
/etc/hosts is how you test a migration before changing DNS. The trouble comes
entirely from the line nobody removed afterwards.
What “not found” actually means
# Fedora CoreOS 44.20260707.3.1 on a virtual machine, aarch64
$ ping -c 3 192.168.127.1; echo "--- a name that does not exist ---"; dig +short nosuchname.example.com; echo "exit status $?"; host nosuchname.example.com
PING 192.168.127.1 (192.168.127.1) 56(84) bytes of data.
64 bytes from 192.168.127.1: icmp_seq=1 ttl=64 time=0.711 ms
64 bytes from 192.168.127.1: icmp_seq=2 ttl=64 time=0.249 ms
64 bytes from 192.168.127.1: icmp_seq=3 ttl=64 time=0.230 ms
--- 192.168.127.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2063ms
rtt min/avg/max/mdev = 0.230/0.396/0.711/0.222 ms
--- a name that does not exist ---
exit status 0
Host nosuchname.example.com not found: 3(NXDOMAIN)dig +short printed nothing and exited 0. No answer, and no indication that
anything was wrong. host said NXDOMAIN plainly.
So do not test whether a name exists with dig +short’s exit status. Use host,
or dig without +short and read the status: field in the header.
Three failures worth telling apart, because they point at different places:
| Result | Means | Look at |
|---|---|---|
NXDOMAIN |
The server authoritatively says the name does not exist | Spelling, the zone, the search domain |
SERVFAIL |
The server tried and failed | The DNS server, or DNSSEC validation |
connection timed out; no servers could be reached |
Nothing answered | Reachability of the nameserver, firewall on port 53 |
That third one is not a DNS problem at all. It is a networking problem wearing
a DNS costume, and ping to the nameserver address distinguishes it
immediately.
If you already administer Linux: systemd-resolved, and where the real nameservers hide
/etc/resolv.conf frequently does not contain the nameservers. On a
systemd-resolved machine it is a symlink to
/run/systemd/resolve/stub-resolv.conf and holds a single entry, nameserver 127.0.0.53, a local stub listener. The actual upstream servers are held by
resolved and appear nowhere in that file.
So on any machine running resolved, cat /etc/resolv.conf answers the wrong
question and resolvectl status answers the right one. It lists, per link,
the current DNS servers, the search domains, and whether DNSSEC and DNS-over-TLS
are in play.
Per-interface DNS is resolved’s genuinely useful feature and its most
confusing one. Each link can have its own servers and its own routing domains,
so a VPN can resolve *.corp.internal while everything else goes to the public
resolver. It also means “which server answered this” has a per-name answer, and
two lookups from one machine can legitimately go to different places.
Three more commands worth having: resolvectl query name is the diagnostic
equivalent of getent, and reports which link and which server produced the
answer. resolvectl flush-caches clears the cache without restarting
anything, which matters after a DNS change. And resolvectl statistics shows
the cache hit rate, which is how you notice a resolver that is not caching at
all.
Configuration goes in /etc/systemd/resolved.conf or a drop-in under
resolved.conf.d/, never in resolv.conf.
Search domains
search home.arpa in resolv.conf means short names get that suffix appended.
So ping web01 tries web01.home.arpa before giving up.
Convenient, and the source of two surprises.
A short name can resolve to something you did not expect. With search example.com set, ping mail may reach mail.example.com, including on a
machine where you meant an entirely different mail.
A trailing dot means “exactly this”. example.com. with the dot is fully
qualified and the search list is not consulted. Without it, some resolvers try
the search domains first.
Use FQDNs in configuration files. Short names depend on the search list, which is set by DHCP on many machines and therefore is not something you control. A configuration file that works in one network and not another is very often this.
If you already administer Linux: systemd-resolved, split horizon, caching, and TTL
/etc/resolv.conf is frequently a symlink. On a systemd-resolved machine
it points at /run/systemd/resolve/stub-resolv.conf and contains nameserver 127.0.0.53, a local stub listener, not a real server. The actual upstream
servers are held by resolved and are invisible in that file, which is why
resolvectl status is the command that answers “what is this machine really
using”. resolvectl query name is the diagnostic equivalent of getent, and
resolvectl flush-caches clears the cache without restarting anything.
Per-interface DNS is resolved’s genuinely useful feature and its most
confusing one. Each link can have its own servers and its own routing domains, so
a VPN can resolve *.corp.internal while everything else goes to the public
resolver. It also means “which server answered” has a per-name answer, and
resolvectl status showing different servers per link is the only way to see it.
Split-horizon DNS returns different answers depending on who is asking,
internal clients get a private address for app.example.com, external clients
get the public one. Correct and common, and it makes “it works from my laptop
and not from the server” an expected result rather than a mystery. Always ask
where a lookup was performed before comparing two answers.
Caching and TTL. The TTL in a record is how long a resolver may keep it. Lower it to 300 seconds a day or two before a planned migration, then restore it afterwards; leaving it low permanently costs you a query per user per five minutes. Negative answers are cached too, governed by the zone’s SOA minimum, which is why a name can stay “not found” for a while after you create it, and why creating a record and immediately testing it teaches you nothing useful.
dig beyond the basics. dig @8.8.8.8 name asks a specific server, which is
how you prove whether the problem is your resolver or the zone. dig +trace name
walks from the root down, showing every delegation, and is the tool for
“resolution works from one place and not another”. dig -x 1.2.3.4 does the
reverse lookup, which mail servers care about a great deal more than anything else
does.
nsswitch sources beyond files and dns. mdns4_minimal handles .local
names on desktops and should come with [NOTFOUND=return] so a missing .local
name does not fall through to a slow public query. sss appears on domain-joined
machines. The order in that line is a policy decision somebody made, and it is
worth reading rather than assuming.
Across distributions
| RHEL family | Debian | Ubuntu | |
|---|---|---|---|
Writes /etc/resolv.conf |
NetworkManager | resolvconf or nothing | systemd-resolved |
/etc/resolv.conf is |
A real file | A real file | A symlink to the stub |
| Caching resolver by default | No | No | Yes, systemd-resolved |
| Tools package | bind-utils |
dnsutils |
dnsutils |
| Query tool | dig, host |
dig, host |
dig, host, resolvectl |
The package name difference bites often enough to be worth memorising. dig
is in bind-utils on the RHEL family and dnsutils on Debian and Ubuntu, and
searching for a package called dig finds nothing on either. That is the
“searching for a command name instead of a package name” trap from lesson 08,
and this is its most common instance.
If you already administer Linux: dig beyond +short, and reading a delegation
dig @server name is the single most useful variant. Asking a specific
server bypasses whatever your machine is configured with and settles the
“is it the zone or my resolver” question in one command. dig @8.8.8.8 name against dig @<internal server> name is the standard pair.
dig +trace name walks the delegation from the root down, showing each
referral. It is the tool for “resolution works from one network and not
another”, because it shows exactly where the two paths diverge, and it
bypasses caches entirely, so it reports what is authoritative rather than what
is remembered.
Record types worth asking for by name. dig NS example.com shows who is
authoritative, which is the first question when a zone misbehaves. dig SOA example.com gives the serial, comparing serials across a zone’s nameservers
is how you catch a secondary that has stopped transferring, and the minimum
TTL, which governs how long negative answers are cached. dig -x 1.2.3.4
does the reverse lookup, which mail servers care about more than anything else
does.
Negative caching is the one that catches people. Create a record, test it
immediately, get NXDOMAIN, and conclude the record did not save. It did; the
resolver cached the previous “does not exist” answer and will hold it for the
zone’s SOA minimum. Test with dig @<authoritative server> to bypass the cache,
and lower the TTL before a planned change rather than after.
Prove it
Work down the list. Each step assumes the one above passed:
# 1. Is it a name problem at all
ping -c 2 1.1.1.1
# 2. What will my applications actually get
getent hosts theproblemname
# 3. What does DNS itself say
dig +short theproblemname
# 4. If 2 and 3 disagree, look here
grep theproblemname /etc/hosts
grep '^hosts' /etc/nsswitch.conf
# 5. Which server am I even asking
cat /etc/resolv.conf
resolvectl status | head -20 # if systemd-resolved is running
# 6. Is that server reachable and answering
ping -c 2 <nameserver>
dig @<nameserver> theproblemnameSteps 2 and 3 together are the whole lesson. Same answer: DNS is doing what you think, and the fault is elsewhere. Different answers: something in the nsswitch chain is intercepting, and step 4 finds it.
What trips people up
1. dig works and the application does not
The subject of the prediction. dig bypasses /etc/hosts; applications do not.
getent hosts <name> is the command that reproduces what the application sees.
If it differs from dig, check /etc/hosts first and the hosts: line in
/etc/nsswitch.conf second.
2. A short name resolves to the wrong thing
Search domains. ping web01 becomes web01.whatever-the-search-domain-is, and
that suffix may have come from DHCP without anyone choosing it.
cat /etc/resolv.conf to see the search list. Use FQDNs in anything written
down.
3. /etc/resolv.conf keeps being overwritten
Because it is generated. NetworkManager, systemd-resolved, or the DHCP client rewrites it whenever the network changes, and your edit goes with it.
Configure DNS through whichever system owns networking. That is lesson 17’s question again:
| System | How to set DNS |
|---|---|
| NetworkManager | nmcli connection modify "name" ipv4.dns "1.1.1.1 9.9.9.9" |
| netplan | nameservers: block in the YAML |
| systemd-resolved | DNS= in /etc/systemd/resolved.conf |
| Debian ifupdown | dns-nameservers in /etc/network/interfaces |
4. Expecting a DNS change to take effect at once
TTL. Resolvers between you and the record keep the old answer until it expires, and there is no way to make somebody else’s cache forget.
Lower the TTL a day or two before a planned change. Locally, resolvectl flush-caches clears your own.
5. Reading a timeout as a DNS fault
connection timed out; no servers could be reached means nothing answered.
That is a reachability problem: the nameserver is down, unreachable, or port 53 is
blocked.
ping the nameserver. If it responds, dig @<that server> directly. Two commands
and you know whether to call the DNS team or the network team.
Work it through
An application server cannot reach the database at db01.internal.example.com.
The application logs Name or service not known.
You are told “DNS is broken”. Reason it out before reading on.
Is it a network problem in disguise? ping -c 2 1.1.1.1. If that fails,
this is lesson 16 and nothing here applies. Assume it works.
What does the application actually see?
getent hosts db01.internal.example.comThis is the important command, because it uses the same path the application does. Two outcomes and they lead in opposite directions.
If getent returns an address, then resolution works and the
application’s failure is something else, the wrong name in its config, a stale
cached negative answer inside a long-running process, or a container with a
different /etc/hosts from the host it runs on. The message said Name or service not known, so the name it failed on may not be the name you just
tested. Check the configuration for a typo before anything else.
If getent returns nothing, keep going.
Third, ask DNS directly:
dig +short db01.internal.example.comIf dig answers and getent does not, something in the nsswitch chain is
interfering. Read /etc/nsswitch.conf’s hosts: line and check for an
[NOTFOUND=return] earlier in it stopping the search before dns is reached.
If neither answers, the question becomes which failure it is. Run dig
without +short and read the status: field:
NXDOMAIN. The server is working and says the name does not exist. Either the record was never created, or you are asking a server that does not host that zone. An internal name asked of a public resolver gives exactly this.SERVFAIL, the server tried and failed. Its problem, not yours.timed out, nothing answered. Checkcat /etc/resolv.conffor the server’s address, thenpingit.
The likeliest answer for a name ending .internal.example.com: the machine is
using a public resolver (often 8.8.8.8, put there by somebody debugging
something else and never removed) which cannot possibly know about an internal
zone. cat /etc/resolv.conf shows it, and dig @<the internal server> <name>
answering correctly confirms it in one more command.
The fix is not to edit /etc/resolv.conf, because it will be overwritten. Set
DNS through whichever system owns networking, exactly as in trip-up 3, and then
reboot to prove it holds.
The habit worth taking: getent before dig. dig answers what DNS says;
getent answers what your software will get. When somebody says “DNS is
broken”, the second question is more useful and almost nobody asks it first.
Try it
Optional, on any machine.
cat /etc/resolv.conf. Is it a real file or a symlink? Which servers, and is there a search domain?grep '^hosts' /etc/nsswitch.conf. Read the sources in order and say which wins.getent hosts example.comanddig +short example.com. Confirm they agree.- Add
203.0.113.99 test.example.comto/etc/hosts, then run both commands against that name. Watch them disagree. Remove the line afterwards. dig +short nosuchname.example.com; echo $?and thenhost nosuchname.example.com. Note which one tells you the truth.dig @1.1.1.1 example.comand compare with your default server.- If systemd-resolved is running:
resolvectl statusand find the actual upstream servers.
Verification step. You have it when you can be told “DNS is broken” and, in
three commands, say whether it is /etc/hosts, the resolver configuration, the
nameserver, or not DNS at all.
Check yourself
Why can dig return a correct answer while an application on the same machine cannot resolve the name?
They use different paths. dig sends a DNS query straight to a
nameserver. Applications call the C library resolver, which follows the
hosts: line in /etc/nsswitch.conf, normally files first, meaning
/etc/hosts, then other sources, with DNS last.
So an entry in /etc/hosts overrides DNS completely for every normal program,
and dig never sees it.
getent hosts <name> is the tool that reproduces what an application gets,
because it goes through the same nsswitch chain. Comparing getent with dig is
the fastest way to find this class of problem.
What does search example.com in /etc/resolv.conf do, and how does it cause surprises?
It appends that suffix to short names, so ping web01 also tries
web01.example.com.
Two surprises. A short name can resolve to something unintended, with a
search domain set, mail may become mail.example.com, which might be a
completely different machine from the one you meant.
Behaviour changes with the network. The search list is frequently supplied by DHCP, so the same configuration file works on one network and fails on another, with nothing on the machine having changed.
Use fully qualified names in anything written down. A trailing dot,
example.com., makes a name explicitly absolute and skips the search list
entirely.
Distinguish NXDOMAIN, SERVFAIL, and a timeout. Where does each point?
NXDOMAIN, the server answered authoritatively that the name does not exist. DNS is working. Look at the spelling, at whether the record was created, or at whether you are asking a server that hosts that zone at all. Asking a public resolver for an internal name gives exactly this.
SERVFAIL, the server tried and could not produce an answer. The problem is at the server or upstream of it: a broken zone, a failed DNSSEC validation, an unreachable authoritative server. Not your machine.
connection timed out; no servers could be reached, nothing answered at all.
This is not a DNS problem; it is reachability. The nameserver is down,
unreachable, or port 53 is blocked. ping the nameserver address to confirm.
Three different messages pointing at three different teams, which is why reading the actual status is worth more than “DNS is broken”.
You edit /etc/resolv.conf and it reverts within minutes. Why, and what should you do instead?
The file is generated. NetworkManager, systemd-resolved, or the DHCP client rewrites it whenever the network changes, and on Ubuntu it is not even a real file. It is a symlink to a stub configuration.
The # Generated by ... comment at the top is the warning.
Set DNS through whichever system owns networking:
- NetworkManager:
nmcli connection modify "name" ipv4.dns "1.1.1.1 9.9.9.9" - netplan: the
nameservers:block in the YAML - systemd-resolved:
DNS=in/etc/systemd/resolved.conf - Debian ifupdown:
dns-nameserversin/etc/network/interfaces
Then reboot to prove it holds, the same reboot test as the previous lesson, for the same reason.
Why does dig +short somename returning nothing not prove the name does not exist?
Because +short suppresses the header, and the exit status stays 0 regardless.
An empty result and a successful exit look the same whether the name genuinely
does not exist, the query timed out, or the server returned an error.
host somename states NXDOMAIN explicitly. dig without +short shows a
status: field in the header saying NXDOMAIN, SERVFAIL, or NOERROR.
This matters most in scripts, where testing dig +short’s exit status
produces a check that passes when DNS is broken, which is precisely backwards
from what the check was written to do.
References
- resolv.conf(5) - Linux man-pages project. Accessed 2026-08-07.
- nsswitch.conf(5) - Linux man-pages project. Accessed 2026-08-07.
- hosts(5) - Linux man-pages project. Accessed 2026-08-07.
- getent(1) - Linux man-pages project. Accessed 2026-08-07.
- systemd-resolved.service(8) - Linux man-pages project. Accessed 2026-08-07.
- dig(1) - Debian Project. Accessed 2026-08-07.
- host(1) - Debian Project. Accessed 2026-08-07.
Command output was captured on the podman machine and on the pinned container images. Blocks without a distribution and architecture header are illustrative.