Threat Hunting via DNS

Threat Hunting via DNS

Eric Conrad (econrad@backshore.net)

https://ericconrad.com
Twitter: @eric_conrad
Welcome!

• Welcome to my talk!
• A copy of these slides are available on https://ericconrad.com

Threat Hunting via DNS 2

CIS 8.7: Malware Defenses

• Enable Domain Name System (DNS) query logging to detect

hostname lookups for known malicious domains.1
• DNS logs are one of the most actionable threat
hunting/SOC/SIEM data sources
• In addition to logging, viewing/dumping and inspecting the
DNS cache is a good short-term investigative tool
• Note that DNS may be logged on the DNS server or endpoints,
or sniffed on the network using tools like Zeek
o Encrypted DNS is impacting both, as we will discuss shortly
Threat Hunting via DNS 3
Methods for Collecting DNS logs

• Sniff on the wire, analyze with Zeek

o A great approach, now heavily impacted by DNS encryption (discussed next)
• Have clients resolve via local recursive DNS servers and log there
• All major DNS server software supports query logging (responses can
be tricky):
o Bind (syslog or local text file)
o DNS Query Logging on Windows 2008/2012 (local text file)
o DNS Analytical Logging on Windows 2012R2+ (logs in event log format to
(Logs\Microsoft\Windows\DNS-Server)
• Sysmon supports Windows client logging
Threat Hunting via DNS 4
DNS Encryption

A big trend on the encryption front that is impacting a vital

analytics source: DNS queries
DNS query encryption concerns itself primarily with increasing
the privacy of users' communications
• This dovetails nicely with the push toward ubiquitous HTTPS from a
traffic privacy perspective
Inscrutable DNS queries can pose secops challenges:
• Blindness to adversaries' intentional use of DNS
• Diminished user monitoring/analytic capabilities
Threat Hunting via DNS 5
Facing Reality

• This talk will not debate the merits of encrypted DNS vs.
traditional DNS via UDP/TCP port 53 (sometimes called Do53)
o Encrypted DNS provides privacy to the end user
o Do53 provides easy centralized monitoring for companies, ISPs, etc.
• And easy monetization for ISPs
• Years of network defense have taught me to be a realist, and
not fight the incoming tide
• DNS over HTTPS (DoH) is coming on like a freight train
o Network defenders need to prepare accordingly
Threat Hunting via DNS 6
DNS over HTTPS (DoH) and DNS over TLS (DoT)

• DNS over HTTPS (DoH) and DNS over TLS (DoT) are impacting the ability to monitor
DNS queries
o This is true for Intrusion Detection Systems such as Zeek, as well as logging requests on the local
DNS resolver/forwarder
• DNS over HTTPS uses TCP port 443 and looks like normal HTTPS traffic from a
network perspective
• DNS over TLS uses TCP port 853, so network operators/defenders know that it’s
(encrypted) DNS traffic
o DoT can be easily blocked by a firewall, forcing resolution back to DNS
• In both cases: analyzing the content on the wire requires SSL/TLS
interception/decryption

Threat Hunting via DNS 7

The Only Constant is Change

• This talk with track DoH in Firefox most closely

o Firefox is the currently the most aggressive browser in regard to DNS
encryption
• DoH/DoT adoption is evolving very rapidly
• I will track updates on https://ericconrad.com
• Jim Troutman’s 2020 Shmoocon Firetalk is fantastic:
o http://www.nepeeringforum.org/troutman/troutman-DoH-DoT-
QuadX-Da-Faq.pdf

Threat Hunting via DNS 8

DoH and DoT

• The early trend: browsers tend to support DNS over HTTPS (for
resolution within the browser), while Linux operating systems tend to
support DNS over TLS for default operating system resolution
o DNS over TLS is now used by default by Android (called “Private DNS Mode”)

• Firefox and Chrome now support DNS over HTTPS

• Microsoft recently announced plans to support DoH
in Windows 10
o Windows 10 Insider Preview currently supports DoH (not
enabled by default)
• In the short-term: DoH is “winning”
Threat Hunting via DNS 9
Paul Vixie on DoH

Threat Hunting via DNS 10

DoH Status update

• Chrome 83 (released May 19th 2020) enables DoH: "We've

enabled an experiment in Chrome 83 for a fraction of our
users with the following scope: platforms: Windows, Mac,
Chrome OS."
• Firefox now enables DoH by default in the US (it prompts first)

Threat Hunting via DNS 11

Firefox/DoH Status Check (June 24th 2020)

Threat Hunting via DNS 12

DoH in Firefox and Chrome

• Firefox bypasses the local system DNS settings when using DoH, and
sets the DNS provider to Cloudflare by default
o Other options include NextDNS and Custom
o This bypass policy has proven to be controversial
• Chrome uses a different approach: If the system is using a provider on
this list for DNS resolution, Chrome will “auto-upgrade” the DNS
setting from DNS to DoH, and keep the same provider:
o Cleanbrowsing, Cloudflare, Comcast, DNS.SB, Google, OpenDNS, Quad9
o Otherwise: Chrome will continue using regular DNS, and the existing provider

Threat Hunting via DNS 13

What is your Organization's Encrypted DNS Policy?

Some options to consider:

• Embrace the privacy, and use it
o Easy decision for organizations that don't currently log/analyze DNS
o Great personal choice for home/travel/etc.
• Disable DoH and DoT (when possible), force resolution via Do53, and
log via traditional methods
• Allow both DoH and DoT to local servers, and log there
• Worth noting: much like VPN traffic: most encrypted DNS will
eventually resolve via Do53 upstream
o One exception DoH/DoT traffic to an authoritative name server
Threat Hunting via DNS 14
Traditional Do53 Architectural Diagram Sniff here,
analyze
with Zeek

Do53 Do53

Log locally
here
Threat Hunting via DNS 15
Third-Party DoH Architectural Diagram

No network
analysis without
TLS interception

DoH DoH Do53

Google,
Cloudflare,
Quad9, etc.
Threat Hunting via DNS 16
Custom DoH Architectural Diagram

Sniff here,
analyze
with Zeek

DoH Do53

Log locally
here
Threat Hunting via DNS 17
Disabling DoH in Firefox and Chrome

Firefox: Chrome:
• To disable Firefox DoH for the • There is no canary domain
enterprise: do not allow this support
canary domain to resolve: use- • If using a supported DNS
application-dns.net provider, Chrome will auto-
• To disable DoH in a browser, go upgrade any Do53 connection to
to Settings -> Network Settings - DoH
> Connection settings, and • Workaround: if you don't use a
uncheck “Enable DNS over supported DNS provider, Chrome
HTTPS” will use Do53

Threat Hunting via DNS 18

Setting up your own DoH server

• This guide is fantastic

• Instructions for Ubuntu
18.04
o Also has sections on
setting up PiHole and DoT
• I was able to set up a DoH
server in Digital Ocean’s
cloud in <10 minutes
• https://www.aaflalo.me/2018/10/tutorial-setup-dns-over-https-server/

Threat Hunting via DNS 19

Logging on a local doh-server

• Configure Firefox to use a custom DoH server

• Set verbose to “true” in doh-server.conf

o Logs queries only. Does not appear to have an option for logging responses, but
it’s open source, and can be modified to do so

Threat Hunting via DNS 20

Detection: DoH is HTTPS

• DoH is HTTPS
o Uses web servers such as Nginx and Apache, leverages x.509 certs, etc.
• For example:
o https://dns.zez.me – regular HTTPS site
o https://dns.zez.me/dns-query - resolves DoH requests via a POST

Threat Hunting via DNS 21

 DoH DNS Request

Firefox DoH request via

dns.zez.me

The DoH virtual server

name is shown in the
Server Name Indication
(SNI) field
The actual DNS query
carried via DoH is
encrypted
Threat Hunting via DNS 22
Network-based DoH prevention

• If you can't configure each client or use canary domains to

disable DoH: Network-based DoH prevention (such as
firewalling) isn’t practically possible, short of SSL/TLS proxying
and inspection
• HTTPS access to known DoH resolvers can be blocked
o 1.1.1.1:443, 8.8.8.8:443, etc.
• HTTPS access to unknown DoH resolvers cannot be easily
blocked
o 206.189.185.210:443 (my custom DoH server)
Threat Hunting via DNS 23
Network-based DoH Detection

• Known DoH resolvers can be detected via simple IP/port-based IDS

rules (1.1.1.1:443, etc.)
• Beaconing detection can detect DoH to any site, including unknown
resolvers
o Browsers usually resolve via the same DoH server (HTTPS site) 1000+ times/day
• RITA is a great tool for detecting beaconing
o https://www.blackhillsinfosec.com/projects/rita/
• Check out SANS STI student Drew Hjelm’s amazing paper: A New
Needle and Haystack: Detecting DNS over HTTPS Usage
o https://www.sans.org/reading-room/whitepapers/dns/paper/39160
Threat Hunting via DNS 24
DNS Logging via Sysmon

• Microsoft’s Sysmon can now log local DNS queries

• Plays nicely with centralized event collection via Windows
Event Forwarding
• Killer threat hunting feature: it shows the client application
that made the DNS request
• Note that Firefox’ DoH implementation bypasses local
resolving entirely
• Sysmon does not log Firefox’s DoH DNS requests
Threat Hunting via DNS 25
Sysmon DNS Logging Example

adf
Image:
C:\WINDOWS\SYSTEM32\PING.EXE

Threat Hunting via DNS 26

Now That We're Logging: Check Your DNS

• Malware, like most network • The following should be

software, uses DNS for resolving monitored:
names to IP addresses (and so o Requests to thousands of hosts or
on) subdomains in one domain
• It also uses DNS for command o Large DNS queries with high
and control (C2) traffic entropy
o It’s usually allowed outbound o Large TXT record responses
o It’s usually ignored o Attempts to resolve NULL records
o High volumes of DNS resolution
failures
o Requests to "baby" domains
(registered very recently)
Threat Hunting via DNS 27
Zeus Botnet C2 via DNS

Note the large DNS TXT records used by the Zeus botnet for
Command and Control (C2):

Threat Hunting via DNS 28

DNS: the Ideal C2 Channel

• DNS tunnels are the ideal C2 channel, IMO

o DNS is usually allowed outbound
o It’s usually ignored
o Works via multiple forwarders (i.e. DNS proxies)
o Locked down internal subnets with 'no internet access' often allow public DNS
resolution
• An internal system has direct bidirectional internet access if it can
resolve 'google.com' and receive the answer
• DNS tunnels are much more difficult to mitigate via preventive controls

Threat Hunting via DNS 29

Iodine: Advanced DNS Tunneling

• Iodine offers a true routable tunnel via DNS

o Can tunnel any IPv4 protocol
o Quite easy to set up, and NIDS detection is poor
• Available at: http://code.kryo.se/iodine/
• Can forward via a local DNS server, or…
o it may also happen that _any_ traffic is allowed to the DNS port (53 UDP) of any
computer. Iodine will detect this, and switch to raw UDP tunneling if possible. 1
o [1] http://code.kryo.se/iodine/README.html

Threat Hunting via DNS 30

Iodine Wireshark View – DNS Tunnel (Forwarded via Local Resolver)

Threat Hunting via DNS 31

Zeek View – sort, sed, etc…

$ cat dns.log |zeek-cut query | sort -u | sed

"s/^[a-zA-Z0-9-]*\.//g"| sort | uniq -c | sort –n

Threat Hunting via DNS 32

Programmatic Entropy Analysis

• Without trying, the human brain often can detect something as

potentially random generated
o Programmatically achieving this proves more difficult than expected
• Many tools exist for calculating entropy, the often built-in Linux tool,
ent being a simple example
• Classic entropy analysis using tools like ent can be leveraged to
determine the degree of randomness of provided input…
o …but ASCII has 256 characters
o A DNS name containing letters (26 characters) and numbers (10 characters) uses
a maximum of 36 of 256 total ASCII values (14%)
o Any cryptologist will tell you: that equals low entropy

Threat Hunting via DNS 33

Bring Out the Baggett

• Solving problems like detecting random (before morning break) is why

you always have @MarkBaggett (GSE #15) take your classes
o freq.py tool is a huge boon to finding random generated strings where they
perhaps shouldn't be
o https://github.com/sans-blue-team/freq.py
• The approach looks at the likelihood of character pairings
occurrence based on frequency analysis
o Simple example: in English text, “q” is pretty much always
followed by a “u,” so seeing a “q” followed by something else
would be rather unlikely to occur

Threat Hunting via DNS 34

Domain Generation Algorithms DGAs

• One of the most obvious,

and incredibly useful, ways
to employ freq.py is
looking at DNS names for
signs of randomness
• You will necessarily need to
do whitelisting
o Public CDNs (Content Delivery
Networks)
o Major cloud services
(Microsoft, Amazon, Google)
often have their own CDN

Threat Hunting via DNS 35

DGA++ - Beyond Domain Generation Algorithms

Though DGA detection can be very effective, think more broadly about
places where adversaries might programmatically generate large volumes
Detecting randomness can be a tremendous indicator of otherwise
unknown malice
• Thread/Process names • Subdomains (Domain Shadowing1)
• File names (binaries, scripts, etc.) • Certificate subject names and issuers
• Workstation names • Usernames
• Service names • Many additional possibilities

Threat Hunting via DNS 36

freq_server.py - freq-ing At Scale
As additional use cases are discovered, you will soon feel the need to wield freq.py
at scale
Although the initial script is, without question, a work of art, it was not intended to
have a system perform 100,000+ freq.py/sec
Have no fear, @MarkBaggett worked with SANS SIEM course author and 511
instructor Justin Henderson (@SecurityMapper, GSE #108, SANS SIEM
Author) and developed a new feature/deployment model
• freq_server.py - https://github.org/sans-blue-
team/freq.py/
• freq_server.py designed to allow for remote calls from tools such
as LogStash
• Implementation and analysis techniques discussed in SANS SIEM class
Threat Hunting via DNS 37
dnstwist

• Use dnstwist to protect against cousin domains (sec530.com

vs.
sec53O.com) and Internationalized Domain Name (IDN) homoglyph
attacks
o pɑypɑl.com vs. paypal.com
o Block with firewall/proxy, or detect via DNS and other sources
o dnstwist calculates permutations against a given domain
o Also checks to see if any domains have been registered
o And provides additional information about the domain
• Use dnstwist with scripting to handle evil cousins and homographs

Threat Hunting via DNS 38

Baby Domain Detection: domain_stats

• Domain_stats is another great tool by Mark Baggett

o https://github.com/MarkBaggett/domain_stats
• Can query the Alexa or Cisco Umbrella top million
• Can also query RDAP data to discover domain creation time (to
discover newly-registered "baby domains")
o And much more
• RDAP (Registration Data Access Protocol) is the (eventual) replacement
for WHOIS
o WHOIS: blobs of inconsistent and poorly-formatted data
o RDAP: can output in JSON
Threat Hunting via DNS 39
domain_stats in action

Threat Hunting via DNS 40

Thank you! – econrad@backshore.net

• Thank you for attending my talk!

• A copy of these slides are available on https://ericconrad.com

Threat Hunting via DNS 41