1. Introduction
One of the most common questions DNSDB users have is, “What’s a ‘bailiwick’?”
It’s easy to see what motivates this question. If you use the web interface to DNSDB, the bailiwick field is front and center on that web form (see the red boxed area in Figure 1):
Figure 1. Sample DNSDB Search Web Interface
Alternatively, if you use the DNSDB API (either via your own code or through one of the command line API demonstration clients such as dnsdb_query.py), you’ll have seen references to “bailiwicks” there, too. See Figures 2-5:
Figure 2. DNSDB API Information from https://api.dnsdb.info
Figure 3. More DNSDB API Information from https://api.dnsdb.info
Figure 4. Bailiwick Reference in dnsdb_query.py Command Synopsis
Figure 5. An Example of How “bailiwicks” Get Mentioned in the Output From dnsdb_query.py
Bailiwick, bailiwick, BAILIWICK! Clearly, “bailiwick” is a term you’ll routinely bump into when playing around with DNSDB.
2. “Bailiwick” is Commonly Seen — But Often Not Understood
Our impression is that while the term “bailiwick” is quite commonly seen in conjunction with DNSDB, few users understand what that term actually means. Because most users don’t understand what “bailiwick” means, only a few users take advantage of bailiwick filtering when working with DNSDB.
In fact, we believe that most users simply “tune out” the whole bailiwick “thing” entirely. Fortunately, simply ignoring bailiwicks (while suboptimal), isn’t catastrophic. DNSDB will deal with any silence about bailiwicks by politely giving you results for ALL potentially-relevant bailiwicks by default.
For example, consider someone hypothetically interested in the name servers used by the domain ieee.org over the last week. They could use the dnsdb_query.py API demonstration client to make the following query:
The output from that request, as shown above, returned three blank-line-separated “chunks” of data:
Nameserver information from the Zone File Access (ZFA) program. This will always be for a bailiwick equal to the top level domain in use (in this case, “org”).
Nameserver information for a bailiwick equal to the top level domain (as above), BUT based on observed data contributed by Farsight’s sensor operators (rather than being based on ZFA data).
Nameserver information for a bailiwick equal to the second-level domain (in this case
ieee.org), again based on observed data contributed by Farsight’s sensor operators.
In this simplistic example, the data was entirely consistent across bailiwicks, and there might be a temptation to scoff and say, “Pshaw! This whole bailiwick thing is just a boondoggle and a waste of time! There’s no difference in the results for the different bailiwicks!”
It’s true that there was no difference this time, but that will NOT always be true, as we’ll show below.
In fact, if you DON’T learn about bailiwicks, how they’re derived, and what they can potentially do for you, you may find yourself missing out on some potentially valuable filtering capabilities. For instance, if your DNSDB query found more total results than you can normally display, specifying a bailiwick is one potentially easy way to get the volume of results down under your cap.
And if you don’t know about bailiwicks, you run the risk of accientally misinterpreting the data you’re shown, or always finding yourself wondering in the back of your mind, “So what’s that dang bailiwick thing?”
Finally, if you don’t know about bailiwicks, you may find yourself expecting to see some data in DNSDB (such as evidence of DNS cache poisoning) that you’ll actually never see, because Farsight’s gone to a fair amount of effort to intentionally filter cache poisoning traffic out of DNSDB.
If you stick with this blog post a little more, we’ll help you see how bailiwicks are actually quite important and potentially helpful to your work. Let’s dive in.
2. Bailiwicks in Real Life and Bailiwicks in the Domain Name System
If we set aside the DNS world for the moment, and just focus on the everyday world, you may have previously encountered the term “bailiwick” in reference to one’s areas of talent or expertise.
For instance, if you’ve ever heard me attempt to play a musical instrument, you’ll know that playing music is “not my bailiwick:” I’ve never had music lessons, I can’t read music, and I’m tone deaf. That lack of expertise and talent is definitively noticable. You’d never mistake me for Billy Joel’s “Piano-Man”, and in fact I can only wish that I might someday be as much of a musician as the most excellent “Kazoo Men”.
In the DNS world, “in-bailiwick” has two formal and not particularly approachable definitions in RFC 7719, “DNS Terminology”. The first of those reads in part:
In-bailiwick:
(a) An adjective to describe a name server whose name is
either subordinate to or (rarely) the same as the zone
origin. In-bailiwick name servers require glue records
in their parent zone [...]
For example, under definition a, ns1.example.com might be an
“in-bailiwick” name server for example.com, while ns2.whatever.net
would be “out-of-bailiwick” for example.com.
This definition is the way the term “in-bailiwick” is most commonly used by people who are part of the technical DNS community. [We recognize that that definition likely may not mean much to you if you’re not a DNS geek, and that’s okay]
There’s a second definition for in-bailiwick mentioned in RFC 7719, and while it is equally arcane, this is the one that’s actually relevant to DNSDB (and this post):
(b) Data for which the server is either authoritative, or
else authoritative for an ancestor of the owner name. [...]
By paying attention to a name server’s “bailiwick” as defined in definition b, DNSDB can avoid mis-accepting DNS results from untrustworthy sources. Thus, bailiwick checking is an important part of how Farsight actively works to keep garbage out of DNSDB. We’ll come back to this data quality assurance function in section 4.
For now, let’s see how an analyst might use the bailiwick field to filter DNSDB traffic.
3. Selecting Data By Bailiwick
As an analyst, you’ve got a choice about what sort of data you retrieve from DNSDB for review:
Do you want to focus on name server information that’s officially registered with a domain TLD registry via a registrar?
Or, recognizing, that a domain owner may elect to “call an audible” and change the name servers his domain is using “on the fly,” do you want to focus on the data returned by the domain’s actual name servers?
To see how these can be different, consider the name servers for the domain handwashmaterials[dot]com (since that domain is listed on the Spamhaus Domain Block List at the time we wrote this article, we’ve “defanged” that domain by replacing the dot in that name with the string literal “[dot]” in an effort to keep you from accidentally visiting what may be a potentially problematic site).
Checking domain whois for that site, we can find its officially registered name servers. At the time this was written, the regisitry whois reported:
$ whois handwashmaterials[dot]com Domain Name: HANDWASHMATERIALS[dot]COM Registrar: MONIKER ONLINE SERVICES LLC Sponsoring Registrar IANA ID: 228 Whois Server: whois.moniker.com Referral URL: http://www.moniker.com Name Server: NS32.HANDWASHMATERIALS[dot]COM Name Server: NS33.HANDWASHMATERIALS[dot]COM [etc]
The registrar whois reported the same name servers. If you were looking just at whois, you’d assume that the name servers shown were the be-all, end-all, answer to the question “What nameservers are being used by this domain.”
What do we see if we look in DNSDB?
To keep the output manageable, let’s just specify that we only want to see the domains’s NS records, only for the top-level “com” bailiwick, and only for the last week. The required command and resulting output look like:
$ dnsdb_query.py -r handwashmaterials[dot]com/NS/com --after=7d ;; bailiwick: com. ;; count: 152 ;; first seen in zone file: 2016-09-14 16:01:48 -0000 ;; last seen in zone file: 2017-02-13 17:02:32 -0000 handwashmaterials[dot]com. IN NS ns32.handwashmaterials[dot]com. handwashmaterials[dot]com. IN NS ns33.handwashmaterials[dot]com. ;; bailiwick: com. ;; count: 9,373 ;; first seen: 2016-09-15 02:34:41 -0000 ;; last seen: 2017-02-13 22:13:26 -0000 handwashmaterials[dot]com. IN NS ns32.handwashmaterials[dot]com. handwashmaterials[dot]com. IN NS ns33.handwashmaterials[dot]com.
Decoding the command we just executed:
dnsdb_query.py This is one of Farsight's sample command line clients
-r Run a "left hand side" ("rname") query
handwashmaterials[dot]com This is the domain we're interested in
/NS Return Name Server records only
/com Filter out any records not from the top-level "COM" bailiwick
--after=7d Exclude any records more than a week old
Decoding the output from the command we just executed, note that the first chunk of those results was pulled from the dot com zone file. We can tell that data comes from the zone files by noting the “first seen in ZONE FILE…” comment. [more on zone files below]
The second chunk of results are largely identical except for timestamps, and except for the source of the results. The second chunk of results came from data contributed by Farsight sensor operators (note the bare “first seen” comment, with no referrence to “zone files”).
All of the above is good so far.
However, if we were to now make a new query, asking for the name servers used by that domain this past week AND specify that we want to use a more specific bailiwick (“handwashmaterials[dot]com” intead of just “com”), we get a totally different answer:
$ dnsdb_query.py -r handwashmaterials[dot]com/NS/handwashmaterials[dot]com --after=7d ;; bailiwick: handwashmaterials[dot]com. ;; count: 8,073 ;; first seen: 2016-07-12 07:52:02 -0000 ;; last seen: 2017-02-14 05:04:09 -0000 handwashmaterials[dot]com. IN NS ns1.handwashmaterials[dot]com. handwashmaterials[dot]com. IN NS ns2.handwashmaterials[dot]com.
Note the “ns1” and “ns2” name servers rather than the “ns32” and “ns33” name servers seen previously!
This domain owner has “called an audible” and effectively changed the name servers he wants to have used for his domain, and he’s done so without updating the name servers officially listed for his domain!
Note that the two sets of name servers,
(
ns32.handwashmaterials[dot]com.andns33.handwashmaterials[dot]com.), and(
ns1.handwashmaterials[dot]com.andns2.handwashmaterials[dot]com.)
could be on totally different sets of IP addresses, and could potentially deliver totally different responses when queried. Moreover, many users, perhaps accustomed to just checking whois or looking at copies of zone files for bulk name server information, might not even know that ns1.handwashmaterials[dot]com. and ns2.handwashmaterials[dot]com. even existed or were in use.
YOU, on the other hand, with the “power of bailiwicks” and DNSDB in your professional repetoire of techniques, understand what you’re seeing if you see different results for a TLD-level bailiwick (such as “com”) vs. a 2nd-level domain bailiwick (such as “handwashmaterials[dot]com.“). You can dig in and follow an analysis target even if they attempt to jink and dive away from your investigation.
We have one more subtopic we need to tackle before we wrap up this post. Let’s take a minute or two to explain how in-bailiwick/authoritative name servers can be found. We’ll describe that process for:
The root zone,
A top-level domain, and
A 2nd-level domain.
This process of paying attention to bailiwick filtering is operationally critical to keeping potentially misleading garbage out of DNSDB.
4. So How DOES Farsight Know What Nameservers Are Genuinely IN-Bailiwick/Authoritative For The Root Zone?
When it comes to deciding what name servers are in-bailiwick or out-of-bailiwick, it all starts with the apex or “root” of the DNS hierarchy (normally shown as a bare “dot”).
The name servers for that root zone cannot be retrieved via the DNS because until you know how to reach the root zone, DNS won’t work. (nice circular dependency, eh?)
To overcome this circular dependency, DNS gets “bootstrapped” from a relatively small static root zone “hints” file, shipped as part of each DNS server’s software.
The “hints” file specifies the name servers that are to be relied upon for the root zone, and the IPv4 and IPv6 addresses where those root name server instances live. For example, for the first and last root name servers defined in that file:
. 3600000 NS A.ROOT-SERVERS.NET. A.ROOT-SERVERS.NET. 3600000 A 198.41.0.4 A.ROOT-SERVERS.NET. 3600000 AAAA 2001:503:ba3e::2:30 [...] . 3600000 NS M.ROOT-SERVERS.NET. M.ROOT-SERVERS.NET. 3600000 A 202.12.27.33 M.ROOT-SERVERS.NET. 3600000 AAAA 2001:dc3::35
Given a true copy of that file, we now explicitly know the full set of name servers we SHOULD trust for the root zone.
Because we’ve been given an exhaustive list of nameservers we SHOULD trust to act as name servers for the root zone, that means that we ALSO know that we SHOULDN’T trust ANY OTHER name servers that may spontaneously “volunteer” to be root zone name servers. Any other name server would be an “out-of-bailiwick” (untrustworthy) name server for the root zone, and must be disregarded
Let’s check DNSDB for the root name servers which we’ve seen over the last month:
$ dnsdb_query.py -r ./NS --after=30d ;; bailiwick: . ;; count: 2,484 ;; first seen in zone file: 2010-04-13 18:39:17 -0000 ;; last seen in zone file: 2017-02-12 20:00:04 -0000 . IN NS a.root-servers.net. . IN NS b.root-servers.net. . IN NS c.root-servers.net. . IN NS d.root-servers.net. . IN NS e.root-servers.net. . IN NS f.root-servers.net. . IN NS g.root-servers.net. . IN NS h.root-servers.net. . IN NS i.root-servers.net. . IN NS j.root-servers.net. . IN NS k.root-servers.net. . IN NS l.root-servers.net. . IN NS m.root-servers.net. ;; bailiwick: . ;; count: 6,596,511,613 ;; first seen: 2010-06-24 03:10:38 -0000 ;; last seen: 2017-02-13 16:10:48 -0000 . IN NS a.root-servers.net. . IN NS b.root-servers.net. . IN NS c.root-servers.net. . IN NS d.root-servers.net. . IN NS e.root-servers.net. . IN NS f.root-servers.net. . IN NS g.root-servers.net. . IN NS h.root-servers.net. . IN NS i.root-servers.net. . IN NS j.root-servers.net. . IN NS k.root-servers.net. . IN NS l.root-servers.net. . IN NS m.root-servers.net.
You’ll note that this list of servers (e.g., a.root-servers.net through m.root-servers.net) agrees with the contents of the root zone hints file, and the now-familiar presence of data from the Zone File Access Program as well as data from actual sensor contributions.
All of the above is “as it should be.” However, you may wonder if Farsight has ever seen out-of-bailiwick name servers for the root zone. As a matter of fact, we have. You can see this historically if you query DNSDB without time fencing your results:
$ dnsdb_query.py -r ./NS [selected output only shown below] ;; bailiwick: . ;; count: 2 ;; first seen: 2013-01-15 18:07:19 -0000 ;; last seen: 2013-01-15 18:07:19 -0000 . IN NS . ;; bailiwick: . ;; count: 2 ;; first seen: 2015-10-16 09:38:23 -0000 ;; last seen: 2015-10-16 09:38:23 -0000 . IN NS b.nic.dk. ;; bailiwick: . ;; count: 2 ;; first seen: 2015-10-16 08:22:22 -0000 ;; last seen: 2015-10-16 08:22:22 -0000 . IN NS c-dns.pl. ;; bailiwick: . ;; count: 1 ;; first seen: 2013-01-26 17:16:50 -0000 ;; last seen: 2013-01-26 17:16:50 -0000 . IN NS 127.53.0.1. ;; bailiwick: . ;; count: 2 ;; first seen: 2015-10-16 10:53:01 -0000 ;; last seen: 2015-10-16 10:53:01 -0000 . IN NS ns-nl.nic.fr. ;; bailiwick: . ;; count: 5,403,834 ;; first seen: 2013-01-05 20:58:00 -0000 ;; last seen: 2013-01-28 21:09:59 -0000 . IN NS ns1.trafficz.com. . IN NS ns2.trafficz.com. ;; bailiwick: . ;; count: 1 ;; first seen: 2014-04-15 19:15:07 -0000 ;; last seen: 2014-04-15 19:15:07 -0000 . IN NS a.gov-servers.net. ;; bailiwick: . ;; count: 2 ;; first seen: 2014-09-25 03:28:20 -0000 ;; last seen: 2014-09-25 03:28:20 -0000 . IN NS a.rokt-servers.net. . IN NS b.rokt-servers.net. . IN NS c.rokt-servers.net. . IN NS d.rokt-servers.net. . IN NS f.rokt-servers.net. . IN NS g.rokt-servers.net. . IN NS h.rokt-servers.net. . IN NS i.rokt-servers.net. . IN NS j.rokt-servers.net. . IN NS k.rokt-servers.net. . IN NS l.rokt-servers.net. . IN NS m.rokt-servers.net. ;; bailiwick: . ;; count: 235,800 ;; first seen: 2013-01-09 18:08:12 -0000 ;; last seen: 2013-01-28 20:06:50 -0000 . IN NS ns0.dnsmadeeasy.com. . IN NS ns1.dnsmadeeasy.com. . IN NS ns2.dnsmadeeasy.com. . IN NS ns3.dnsmadeeasy.com. . IN NS ns4.dnsmadeeasy.com. ;; bailiwick: . ;; count: 1,833,521 ;; first seen: 2013-01-05 22:14:59 -0000 ;; last seen: 2013-01-28 21:10:56 -0000 . IN NS ns1.ndoverdrive.com. . IN NS ns2.ndoverdrive.com. ;; bailiwick: . ;; count: 2,315 ;; first seen: 2013-01-20 17:06:01 -0000 ;; last seen: 2013-01-28 20:31:41 -0000 . IN NS ns1.devnameserver.com. . IN NS ns2.devnameserver.com.
Why do we need to look at historical data to find out-of-bailiwick entries for the root domain? Well, these days Farsight’s filtering of out-of-bailiwick data is much improved, and there no recent out-of-bailiwick root zone data to look at.
So what would have happened if you were to have TRUSTED a non-authoritative/out-of-bailiwick name server, like the historical ones mentioned above?
Let’s consider a simple example using the name www.uoregon.edu.
Resolving that name normally we see:
$ dig www.uoregon.edu +short drupal-cluster5.uoregon.edu. 128.223.142.244
128.223.142.244 is the known/expected IP address for that host… no problem so far.
Now let’s try one of the servers we saw that was formerly willing to answer for domains outside it’s bailiwick. Is it possible that it could still be doing so?
$ dig +aaonly +norecurse www.uoregon.edu @ns1.ndoverdrive.com +short 185.53.179.6
YES! ns1.ndoverdrive.com is still answering queries it shouldn’t be.
185.53.179.6 is NOT what we expected to have returned for www.uoregon.edu! The importance of bailiwick filtering quickly becomes evident when you consider concrete examples like this one of how a user could be lead astray if you were to trust an out-of-bailiwick root name server!
5. What About Name Servers for Top Level Domains Like .com, .net, etc.? How Do We Know What Name Servers Are “In-Bailiwick” For Those TLDs?
Just as a root hints file was provided for the root zone, we can also retrieve a copy of the root zone file for the TLD zones here.
If you look at that file, you’ll see that it has the authoritative name servers for each TLDs. Any name servers OTHER than the name servers listed for each Top Level Domain would be “out-of-bailiwick” for that TLD.
MORE GENERALLY, however, we don’t need to rely on downloading the root.zone zone file to learn what name servers are in (or out) of bailiwick for TLDs. Since we’ve bootstrapped the root zone, we can just ask the trusted root name servers (or passively watch existing traffic to those name servers), and let those trusted servers tell us what name servers we should, in-turn, trust for any specific TLD. For example, actively querying for the biz TLD:
$ dig +aaonly +norecurse biz @h.root-servers.net [...] ;; QUESTION SECTION: ;biz. IN A ;; AUTHORITY SECTION: biz. 172800 IN NS a.gtld.biz. biz. 172800 IN NS b.gtld.biz. biz. 172800 IN NS c.gtld.biz. biz. 172800 IN NS e.gtld.biz. biz. 172800 IN NS f.gtld.biz. biz. 172800 IN NS k.gtld.biz. ;; ADDITIONAL SECTION: a.gtld.biz. 172800 IN AAAA 2001:502:ad09::30 f.gtld.biz. 172800 IN AAAA 2001:500:3682::12 k.gtld.biz. 172800 IN AAAA 2001:503:e239::3:2 a.gtld.biz. 172800 IN A 156.154.124.65 b.gtld.biz. 172800 IN A 156.154.125.65 c.gtld.biz. 172800 IN A 156.154.127.65 e.gtld.biz. 172800 IN A 156.154.126.65 f.gtld.biz. 172800 IN A 209.173.58.66 k.gtld.biz. 172800 IN A 156.154.128.65 [...]
6. What About Finding In-Bailiwick Name Servers for 2nd-Level Domains?
In-bailiwick name servers for the hundreds of millions of 2nd-level domains are too numerous and too dynamic to efficiently load via a static file (although TLD Zone File Access Program files do exist and do have the data that’s needed if you wanted to try to do that).
In reality, however, the information that Farsight needs for bailiwick filtering 2nd-level domains will typically get bootstrapped from observed sensor traffic.
As an exercise, however, we could directly query the appopriate TLD server for a 2nd-level domain of interest. For example, if we wanted to find what name servers are in-bailiwick for the arbitrarily-selected domain panasonic.biz, we could ask one of the biz nameservers we found for the biz TLD in part 5 of this article:
$ dig +aaonly +norecurse panasonic.biz NS @a.gtld.biz [...] ;; QUESTION SECTION: ;panasonic.biz. IN NS ;; AUTHORITY SECTION: PANASONIC.biz. 7200 IN NS A1-35.AKAM.NET. PANASONIC.biz. 7200 IN NS A10-64.AKAM.NET. PANASONIC.biz. 7200 IN NS A12-66.AKAM.NET. PANASONIC.biz. 7200 IN NS A16-64.AKAM.NET. PANASONIC.biz. 7200 IN NS A11-65.AKAM.NET. PANASONIC.biz. 7200 IN NS A13-67.AKAM.NET.
Fortunately we don’t need to do that sort of thing by hand; it gets handled automatically for us by DNSDB.
7. Conclusion
We know that this has been a rather long answer to what sounded like a short and simple question, but we hope that you’ve come away from this post with a better understanding of what bailiwicks are, why they’re important to the accuracy and usability of Farsight Security’s DNSDB passive DNS data, and how they can help your DNSDB investigations.
For more information about obtaining access to DNSDB, please contact Farsight Security, Inc., Sales at https://www.farsightsecurity.com/order-services/.
—
Joe St Sauver, Ph.D. is a Scientist with Farsight Security, Inc.
