The Registration Data Access Program (RDAP), Whois And You
I. Introduction
If you engage in any sort of online anti-abuse work, you’ll routinely use Whois for information about the domains and IPs you discover. You may do your Whois lookups from the command line, use a web Whois gateway, or perhaps even use a commercial Whois service such as DomainTools or CyberTOOLBELT, but one way or the other, Whois will normally be a big part of your daily work. As useful as Whois can be, Whois isn’t perfect:
Because Whois display output is formatted for presentation to human beings (rather than for bulk ingestion and automated processing by applications), it can be tricky to programmatically extract a particular field of interest from presentation-format Whois output — seemingly every registry or registrar uses a schema that’s formatted just a little bit differently.
Plain text (“ASCII”) Whois output is poorly suited to rendering non-Latin/non-Roman international character sets such as Cyrillic, Greek, Hangul, Kanji, Thai, etc.
Whois queries/responses are unencrypted, rendering them vulnerable to eavesdropping (then again, Whois data is normally publicly available and non-sensitive)
Whois referrals get handled in a somewhat ad hoc manner.
Access to point of contact (POC) details in Whois data has been a point of contention for some privacy advocates, who often assert that even with privacy/proxy services available, POC information is problematic from a privacy point of view. The fact that the European Union’s GDPR (General Data Protection Regulation will soon become enforceable (as of May 25th, 2018) adds urgency to those who must address potential privacy concerns (or risk penalties).
For all those reasons and more, the community has been hard at work on the followon to Whois: the Registration Data Access Program, or “RDAP.” RDAP is defined in a series of RFCs, RFC 7480-7485, see:
Security Services for the Registration Data Access Protocol (RDAP)
JSON Responses for the Registration Data Access Protocol (RDAP)
You can also see a timeline for the design, evolution and deployment of RDAP at
Users interested in the very latest reporting on RDAP should also be sure to review Francisco Arias’ RDAP Implementation in the gTLD Space from the ICANN Abu Dhabi meeting (28 Oct-3 Nov, 2017).
II. A Domain Whois Example: farsightsecurity.com
The easiest way to see what’s changing is by looking at an example. For instance, consider the traditional domain Whois output for farsightsecurity.com. Output from the GNU Whois client at the command line is quite long; in the interest of brevity we’ll show just selected bits of that output:
$ whois farsightsecurity.com Domain Name: FARSIGHTSECURITY.COM Registry Domain ID: 1775720738_DOMAIN_COM-VRSN Registrar WHOIS Server: whois.gandi.net Registrar URL: http://www.gandi.net Updated Date: 2016-11-25T14:36:34Z Creation Date: 2013-01-24T00:03:10Z Registry Expiry Date: 2018-01-24T00:03:10Z Registrar: Gandi SAS Registrar IANA ID: 81 Registrar Abuse Contact Email: [email protected] Registrar Abuse Contact Phone: +33.170377661 Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited Name Server: NS5.DNSMADEEASY.COM Name Server: NS6.DNSMADEEASY.COM Name Server: NS7.DNSMADEEASY.COM DNSSEC: signedDelegation DNSSEC DS Data: 60454 5 2 3672C35CFA8FF14C9C223B84277BD645C0AF54BAD5790375FE797161E4801479 [snip] Domain Name: farsightsecurity.com Registry Domain ID: 1775720738_DOMAIN_COM-VRSN Registrar WHOIS Server: whois.gandi.net Registrar URL: http://www.gandi.net Updated Date: 2017-10-26T00:13:27Z Creation Date: 2013-01-24T00:03:10Z Registrar Registration Expiration Date: 2018-01-24T00:03:10Z Registrar: GANDI SAS Registrar IANA ID: 81 Registrar Abuse Contact Email: [email protected] Registrar Abuse Contact Phone: +33.170377661 Domain Status: clientTransferProhibited http://www.icann.org/epp#clientTransferProhibited Registry Registrant ID: Registrant Name: Paul Vixie Registrant Organization: Farsight Security, Inc. Registrant Street: 177 Bovet Rd., Suite 180 Registrant City: San Mateo Registrant State/Province: California Registrant Postal Code: 94402 Registrant Country: US Registrant Phone: +1.6504897919 [...] Registrant Email: [email protected] Registry Admin ID: [...] Name Server: NS5.DNSMADEEASY.COM Name Server: NS6.DNSMADEEASY.COM Name Server: NS7.DNSMADEEASY.COM [...] DNSSEC: signedDelegation [...]
Obviously, this output has lots of useful information, but manually parsing that output for ingestion and further processing would be tedious.
III. RDAP Output For The Same Domain
In contrast to Whois, RDAP output is formatted as JSON. If we want to, we can view those JSON objects in a web browser (ideally after installing a JSON viewing add-on such as jsonview).
The default representation will show you all the data for the requested object. This can be quite voluminous. If using jsonview, you can choose what to see by clicking each section’s little triangle.
In collapsed form, it is easy to focus on just the list of JSON sections returned for our example domain:
We can selectively expand sections of that data, such as just the “events” section:
In addition to looking at the output in a browser with the jsonview plugin, you can also use a command line web client to access RDAP. For example, you can try retrieving the RDAP data with curl, and then format elements of interest using jq.
For example, to see the same “events” data show above, you could enter:
$ curl --silent https://rdap-pilot.verisignlabs.com/rdap/v1/domain/farsightsecurity.com | jq '.events'
[
{
"eventAction": "registration",
"eventDate": "2013-01-23T19:03:10Z"
},
{
"eventAction": "last changed",
"eventDate": "2016-11-25T09:36:34Z"
},
{
"eventAction": "expiration",
"eventDate": "2018-01-23T19:03:10Z"
}
]
IV. RDAP Output Is Often Missing One Key Thing: POC Information
If you look at the full output for an RDAP object (at least as a public/anonymous user of some of the pilot RDAP services), you may often notice one notable omission: Point of Contact (POC) information for the registrant, administrative contact, and technical contact for domain names will typical NOT be present, and that’s a real shame.
This redaction may be reflective of the forthcoming changes in access to that data that have been triggered by the GDPR. As a result, by default, you often WON’T have access to POC information, even though that’s often EXACTLY the information you’re often after in order to deal with an ongoing incident, and even though that data may still be available in Whois, at least for now.
Some “food for thought” around GDPR and registration POC data can be seen in the legal memorandum to ICANN from Thomas Nygren and Pontus Stenbeck, Hamilton Advokatbyrå dated 16 October 2017.
You may also be interested in the experimental authentication extensions that allow a limited set of approved users to get POC data.
V. “How Do I Find RDAP Servers for the Various TLDs?”
To find a list of known RDAP servers, visit here. Currently that file (for DNS) looks like:
While that’s obviously a limited number of registries, the fact that dot com and dot net are among the TLDs participating is quite noteworthy.
You should also check the Google docs spreadsheet linked from the ICANN List of RDAP Pilot Participants.
Obviously there are a relatively small number of domains that are currently available, but that should improve over time.
VI. Conclusion
You’ve now had a whirlwind introduction to RDAP, but hopefully this will be enough background that you’ll be able to experiment with it.
Given that the RDAP project is still in its pilot phase, it’s not yet time to abandon Whois. However, as more TLDs begin to participate and offer RDAP servers, and as issues around the GDPR and point of contact information get sorted out for both Whois and RDAP, RDAP will likely become a routine part of your cybersecurity workflows.
Joe St Sauver Ph.D. is a Scientist for Farsight Security, Inc.
