Help The Internet -- And Yourself -- By Running a Passive DNS Sensor Node and Contributing Data To The Farsight Security DNS Database (DNSDB)(tm)
Introduction
Farsight Security operates a Passive DNS database called the DNSDB ™. It contains DNS data contributed by numerous volunteer Passive DNS sensor node operators located all over the world.
While we greatly appreciate the data we receive from each and every one of our contributors, we’re always interested in adding more.
Operating a Passive DNS sensor and contributing data is technically straightforward. Since the data contributions focus solely on upstream cache miss traffic, there is no privacy impact for local users.
Each Passive DNS data contributor helps the community to better understand what’s happening on the Internet — including providing data that will help the community win the fight against cybercrime, botnets, malware, and other sorts of online abuse. Although Farsight Security is a for-profit company and we finance our operations through subscription fees, we are committed to supporting law enforcement agents, academic researchers, and non-profit organizations with full or partial grants of our services. To learn more about ordering our services, please visit our Order page.
Why Focus on DNS?
In an increasingly-opaque network world, Passive DNS data remains uniquely accessible as a “proxy measure” for virtually everything that’s happening on the Internet. That is, whenever you visit a web page, send an email message, or do pretty much anything else online, you rely on DNS. If that DNS activity is able to be sampled, you’ve got a terrific indirect measure for the substantive underlying behaviors that we may NOT be able to be directly measure.
Thus, Passive DNS data, broadly collected and properly analyzed, can objectively inform researchers about what’s new, what’s suddenly popular, and what may be going badly awry on the Internet — all more or less in real time. Passive DNS data can also be used historically to provide insights on past Internet activity.
I’m Interested, But What About My Own Users’ Privacy?
Farsight Security has no interest in personally identifiable information (PII). We intentionally only collect DNS data upstream from caching recursive resolvers. That means we only see “cache miss” traffic, e.g., requests for domain names that aren’t already in the local recursive resolver’s cache, and the apparent source of those queries will always be your caching recursive resolver, not your end user. Farsight Security can also arrange to completely suppress collection of data from your sensor for any queries pertaining to your own domains’ names, if that provides additional reassurance that the privacy of local users will be totally respected.
Why Is Farsight Security Seeking Additional Passive DNS Sensor Operators?
When it comes to DNS, Farsight Security knows that there can be substantial variation in traffic patterns from region to region. If we were to hypothetically only get Passive DNS data from American service providers, we’d often end up missing Asian-, European- or southern hemisphere-only DNS phenomena. For example, while .com domains are very popular worldwide, in Germany, .de domains are prevalent. If we didn’t have Passive DNS sensors providing DNS data from Germany (and other Germanic countries), we might miss (or at least substantially underestimate) the prevalence and importance of .de traffic.
This is no different than collecting climate data. If you only observe weather phenomena in warm locales (such as San Diego, Miami, or Honolulu), you’re going to be poorly positioned to understand what people in cold locales (such as Buffalo, Fargo, or Fairbanks), are experiencing, particularly during the winter! You need a wide range of measurement points in order to have a broad sense of what’s happening across the country (or around the world).
Redundant Passive DNS sensor node coverage also ensures that the Farsight Security data will remain robust if we lose an individual node or data contributor. Redundancy provides excellent insurance against unexpected and otherwise unavoidable interruptions.
Who Are Some Of The Current Passive DNS Sensor Operators?
Farsight Security does not publish this information. We protect the identities of those who share data with us because we don’t want to accidentally perturb the data that is contributed. That is, if the bad guys were to hypothetically learn that a particular service provider is working with us, they might strive to avoid that site so as not to be noticed, just as many bad guys work hard to avoid spam traps, honey pot networks, dark space telescopes, and other Internet data collection infrastructure.
Joining The Farsight Security Passive DNS Sensor Network
In thinking about whether or not your company or organization should join our sensor collective, you may wonder, “Do we really have DNS data that would be useful to contribute?” In many cases, yes, you really do. The sort of partners that we believe would likely be particularly interesting to add are listed below.
Select “Eyeball” Networks (Networks Where Traffic To Local Customers Dominates Traffic From Local Customers)
- Large wireline broadband residential service providers (both here in the United States and abroad)
- Cellular providers offering cellular data/4G services for the ever-growing population of smart phones, tablets, and similar devices
- Providers working from remote regions of the Internet, including service providers in the southern hemisphere
- K-12 and higher education networks (particularly data from state/regional/national education networks)
- Federal, state, local and tribal government networks (again, particularly including state/regional/national networks), and the international government counterparts thereof
Select “Content Provider” Networks (Where Outbound Traffic to the Internet Tends to Dominate Inbound Traffic)
- Cloud-based application providers, including high-density web hosting companies, popular web email providers, and hosted desktop providers
- Search engine operators
- Outsourced/third-party recursive resolver operators
- Web URL shortener/redirector operators
- Blocklist operators and DNS reputation service providers
- Content distribution networks and DDoS protection/web reverse-proxy service providers
Appliance Vendors and Network Software Package Authors
- DNS appliance/network appliance companies (we’d love to have Farsight Security software integrated and “ready to go” with just minor configuration required in the management console, so that if customers want to contribute data, it can be easily shared)
- DNS software and other network software package authors (as for the appliance case above, we’d love to see software package authors “baking in” Farsight Security’s data sharing technology, thereby making it easy for users of that package to contribute their data, should they and Farsight Security decide that doing so makes sense)
Is The View Worth The Climb?
In other words, “Why should I bother sharing my DNS data with you guys?” This is a terrific question, and one where motivations may vary widely from participant to participant:
- Many of you understand the importance of data-driven decision making, and know that some data may be seen from only a single source — you. The data you share can be vitally important. If you don’t contribute your data, we may miss potentially important events entirely. Thank you for sharing what only you can share!
- Others contribute data because they explicitly want to help fight online crime and cyber abuse. If that’s you, thank you for helping to create a safer Internet! When you contribute data, it potentially helps the victims of cyber crime, incident handling teams, law enforcement agencies, researchers and many others to understand and address the attacks and other phenomena we all confront online. The online threats you help thwart and forensically solve may be the ones that targeting your customers — or your friends and family.
- Some of you may have terrific data to share, but no desire to be publicly known as the source of that shared data. Sharing data through Farsight Security can help data providers to anonymize their data and thereby expand its potential availability.
- Contributing data to Farsight Security also ensures that if/when you use Farsight Security’s historical Passive DNS database, DNSDB ™, to research your own incidents, relevant data (your own data!) will already be part of Farsight Security’s database, thereby simplifying your analysis and increasing the likelihood that the data you need will have been observed and archived.
- Other contributors may simply be interested in ways to potentially reduce the cost of access to Farsight Security’s products and services. We definitely recognize the value of the data our partners share, and we’re happy to negotiate discounted rates for access to Farsight Security products and services for our data sharing partners (in general, the more data you have to share, the greater the discount level we can offer).
- When you use Farsight Security’s products and services, or contribute data to Farsight Security, we’d also be happy to discuss potentially featuring your efforts as a case study on our web site, highlighting the benefits of our work together, IF you’d like that sort of public acknowledgement.
- Finally, we know that some researchers may have unique data to share, but no infrastructure from which to share it. Rather than “reinventing the wheel” or deploying duplicative (and expensive!) infrastructure of your own, why not work with Farsight Security to broker the distribution of your commercial data products? Outsourcing distribution of your company’s unique data driven security products through Farsight Security will leave you free to focus on what you do best, and can be a surprisingly affordable option.
OK, How’s It Work?
As discussed in the official documentation, the Farsight Passive DNS sensor works by capturing raw packets from a network interface and reconstructing the DNS transactions that occurred between recursive and authoritative nameservers. It can be deployed either directly on the recursive DNS server or on a monitoring server with access to a network tap or port mirror. In the latter case, multiple DNS servers may, of course, be monitored, but both the RX and TX network directions must be monitored since the sensor tracks query/response state.
How About Resource Consumption?
By default, the total number of entries in the query table is limited and a hard memory limit of 512 MB is enforced on the Passive DNS sensor process. CPU utilization by the DNS sensor tends to be fairly low, even on heavily loaded recursive servers.
I’m In, How Do I Get Started?
It’s easy! If you would like to contribute data to the Farsight Passive DNS project, or if you have any questions, please send an email to [email protected]
Please also check out our Passive DNS Sensor FAQ.
Joe St Sauver, Ph.D. is a Distributed System Scientist for Farsight Security, Inc.
