Nuances in DNSDB Time Fencing
DNSDB® 2.0 introduces a host of powerful new features and improvements – the biggest of which is Flexible Search. Both the DNSDB Standard Search API and the Flexible Search API support searching within specific time constaints, which we call “time fencing”. Time-fencing parameters are used for matching results based on before or after the first or last observation times for the DNS records.
By default, DNSDB Standard Search and Flexible API clients search for results throughout all our data, from 2010 on. Sometimes, you only care about recent data. An example is “what did the DNS look like in the past day?”, or “what new records started being used in the past day?”. Or maybe you want to know about a particular historical period, i.e. “what did the DNS look like between 12/1/2015 to 1/5/2016”. Or “what records were only used in that timeframe?”
Here is how to do those searches using our command-line tools dnsdbq and dnsdbflex:
- “what did the DNS look like in the past day?” is a loose time fence. Using dnsdbq, use a relative timestamp of -86400, computed as 60 seconds * 60 minutes * 24 hours with the -A option:
dnsdbq ... -A -86400. - “what new records started being used in the past day?” is a complete (aka strict) time fence. Add the -c option:
dnsdbq ... -A -86400 -c. - “what did the DNS look like between 12/1/2015 to 1/5/2016” is a loose time fence. Using dnsdbq, use both -A and -B with full time stamps:
dnsdbq ... -A 2015-12-01 -B 2016-01-05. - “what records were only used in that timeframe?” is a complete (aka strict) time fence. Add the -c option:
dnsdbq ... -A 2015-12-01 -B 2016-01-05 -c.
The same options work for dnsdbflex.
Note that there is a slight difference in behavior for the time-fencing queries when using DNSDB Standard API versus the Flexible Search API. We will illustrate this using dnsdbq and dnsdbflex, and add some corresponding screen shots from DNSDB Scout®.
Using dnsdbq, we search for all the farsightsecurity.com A records, sorting by time_last in csv form:
$ dnsdbq -r farsightsecurity.com -p csv -t A -s -k last
time_first,time_last,zone_first,zone_last,count,bailiwick,rrname,rrtype,rdata
"2013-07-17 22:08:50","2013-09-25 15:47:47",,,628,"farsightsecurity.com.",
"farsightsecurity.com.","A","149.20.4.207"
"2013-09-25 15:37:03","2015-04-01 06:17:25",,,6350,"farsightsecurity.com.",
"farsightsecurity.com.","A","66.160.140.81"
"2015-04-01 14:17:52","2018-09-27 00:29:43",,,36770,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.13.104"
"2018-09-27 11:18:14","2020-08-11 19:47:37",,,7503,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.14.108"
"2020-08-11 21:02:15","2020-11-06 12:30:55",,,1577,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.14.95"
Converting that csv output to a table form for readability:
Let’s look around the time stamp “2020-08-11 19:47:37” in that data. (Note that DNSDB uses UTC time, as our DNS sensors are located around the entire world.) The last two rows were seen before and after that time. Only the last row was seen only after that time. To illustrate:
A Standard Search for those within or overlapping that time range (not completely in the time range, aka loosely) – this includes those also seen before that time:
$ dnsdbq -r farsightsecurity.com -p csv -t A -s -k last -A "2020-08-11 19:47:37"
time_first,time_last,zone_first,zone_last,count,bailiwick,rrname,rrtype,rdata
"2018-09-27 11:18:14","2020-08-11 19:47:37",,,7503,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.14.108"
"2020-08-11 21:02:15","2020-11-06 12:30:55",,,1577,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.14.95"
The dnsdbq and dnsdbflex -A option is expressed in DNSDB Scout as a Time Fencing “Seen After” parameter. Their -B option is expressed in DNSDB Scout as a Time Fencing “Seen Before” parameter. The -c option, for complete or strict, as the “Strict Mode” checkbox.
Using DNSDB Scout, you should enter “2020-08-11 19:47:37” in the “Seen After” box with “Strict Mode” unchecked for a similar answer.
Then searching results completely within the time range (with -c) – this excludes those rows also seen before that time:
$ dnsdbq -r farsightsecurity.com -p csv -t A -s -k last -A "2020-08-11 19:47:37" -c
time_first,time_last,zone_first,zone_last,count,bailiwick,rrname,rrtype,rdata
"2020-08-11 21:02:15","2020-11-06 12:30:55",,,1577,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.14.95"
Using DNSDB Scout, check the “Strict Mode” box for a similar answer.
A big difference between Standard Search and Flexible Search is that Flexible Search RRNames searches only the Left-Hand Side (LHS) data, shown in the green box:
The Flex data does not have the RData values, i.e. the Right-Hand Side (RHS), in the red box, nor the other values struck-out above. Flex also has the time values, which are neither LHS nor RHS, in the blue box. Effectively, then, Flex Search has the subset of the above data, in the blue and green boxes, which effectively collapses to a single record using the earliest time_first and latest time_last:
Going back to that time stamp above, if we make a loose Flex search we’ll get a result:
$ dnsdbflex --g farsightsecurity.com. -t A -A "2020-08-11 19:47:37"
{"rrname":"farsightsecurity.com.","rrtype":"A"}
Even without the -A option, we’ll get the same result:
$ dnsdbflex --g farsightsecurity.com. -t A
{"rrname":"farsightsecurity.com.","rrtype":"A"}
And with DNSDB Scout
but if we Flex search completely within the time range, we get “no results found for query”:
$ dnsdbflex --g farsightsecurity.com. -t A -A "2020-08-11 19:47:37" -c
Query status: NOERROR (no results found for query.)
Using DNSDB Scout, check the “Strict Mode” box for a similar answer.
This should make sense now, as the row existed before and after the timestamp.
In the DNSDB data available to Standard Search, there is an A record with a new IP (104.244.14.95) at 2020-08-11 21:02:15 so including the right-hand-side we see completely a new row at that time. To repeat what we showed above:
$ dnsdbq -r farsightsecurity.com -p csv -t A -s -k last -A "2020-08-11 19:47:37" -c
time_first,time_last,zone_first,zone_last,count,bailiwick,rrname,rrtype,rdata
"2020-08-11 21:02:15","2020-11-06 12:30:55",,,1523,"farsightsecurity.com.",
"farsightsecurity.com.","A","104.244.14.95"
In summary: there are cases where time fencing with Standard Search will return more rows of data than a similar Flex Search.
References
For more information about DNSDB Time Fencing please see Joe St Sauver’s Farsight’s DNSDB Time Fencing: A Post-Attack “Time Machine”
