dnsdbflex - DNSDB flexible query tool
Synopsis
dnsdbflex [-cdFjhqTUv46] [–exclude glob|regular_expression] [–force] [–glob glob] [–mode terse] [–regex regular_expression] [-A timestamp] [-B timestamp] [-l query_limit] [-O offset] [-s search_what] [-t rrtype] [-u server_sys]
DESCRIPTION
dnsdbflex constructs and issues flexible search queries to Farsight Security’s DNSDB system. The flexible searches include regular expressions and globs (i.e. full wildcarding). The results from dnsdbflex can be displayed directly as JSON or emitted in the dnsdbq batch file input format. Using the batch file format allows “pivoting” from an flexible search into complete DNSDB results.
Values to –glob, –regex, or –exclude are called search expressions.
Search expressions must match the the data indexed in DNSDB’s flexible search database. All DNS rrnames end in a dot. All rdata indexed in the database will end in a dot if it’s a host name or will end in a double quote for other data. A search expression that does not conform with that may be a wasted query. For example, dnsdbflex –glob ‘*.fsi.io’ will not match anything because globs are right-anchored and that search expression does not end in a dot. In glob search expressions, dnsdbflex normally detects such violations and disallows them. Add a trailing dot to match something, dnsdbflex –glob ‘*.fsi.io.’
dnsdbflex doesn’t detect such violations in regex search expressions. For example, dnsdbflex –regex ‘.*.fsi.io$’ will not match anything but dnsdbflex –regex ‘.*.fsi.io.$’ does (note the dot before the dollar sign).
Search expressions must contain 7 bit clean, printable ASCII characters. Use Punycode IDN encoding to search for IDN domain names. Use \DDD (where DDD is the decimal value of the character) to match non-printable characters in rdata strings.
See the Farsight Compatible Regular Expression Reference Guide (FCRE) for a description of the wildcarding syntax, as well as more examples.
You’ll need to get an API key from Farsight to use dnsdbflex with DNSDB. Note that certain types of API keys may not be allowed to use this API, in which case, dnsdbflex will fail with error message “The type of API key is not allowed to use the DNSDB Flex API”.
OPTIONS
Either –glob or –regex must be specified. Both cannot be specified at the same time.
–exclude glob|regular_expression: Filters out results selected by a glob or regular expression. If –glob was specified, then –exclude takes a glob. If –regex was specified, then –exclude takes a regular expression.
–force: Issue search queries even if rejected by dnsdbflex‘s pattern checks.
–glob glob: Specify that dnsdbflex should do a glob search. Only the * and [] glob operators are supported. Can abbreviate as –g.
–mode terse: Specify mode of information to return in results.
terse: Can abbreviate as t. This is the only value currently supported and is the default, so the –mode option need not be specified.
For rrnames queries, returns the rrname and rrtype.
For rdata queries, returns normalized rdata, rrtype, and raw_rdata.
–regex regular_expression: Specify that dnsdbflex should do a regular expression search in the FCRE syntax. Can abbreviate as –r.
-A timestamp: Specify a backward time fence. Only results seen by the passive DNS sensor network on or after this time will be selected. See also -c. See the TIME FENCING section for more information.
-B timestamp: Specify a forward time fence. Only results seen by the passive DNS sensor network on or before this time will be selected. See also -c. See the TIME FENCING section for more information.
-c: By default, -A and -B (separately or together) will select partial overlaps of database tuples and time search criteria. To match only complete overlaps, add the -c (“completeness”) command line option (this is also known as “strict” mode). See the TIME FENCING section for more information.
-d: enable debug mode. Repeat for more debug output.
-F: specify batch output mode, outputting results in the batch format that dnsdbq -f can read. -F includes the rrtype in the batch file format queries — in contrast to -T described below.
If searching for rdata, if an rdata value is not printable or contains whitespace, it will emit it using the raw_rdata hex value and output a comment line giving the non-raw format.
See dnsdbq(1) for documentation of the batch format.
-h: emit usage and quit.
-j: output in JSON format, which is the default. -j and -F are mutually exclusive.
-l query_limit: query for that limit’s number of responses. If specified as 0 then the DNSDB API server will return the maximum limit of results allowed. If -l is not specified, then the query will not specify a limit, and the DNSDB API server may use its default limit.
-O offset: to offset by #offset the results returned by the query. This gives you approximate incremental results transfers. Results can be reordered between queries, so using progressively higher offsets is not guaranteed to return all results without duplicates or gaps. Offset cannot be negative and the default is 0.
-q: (quiet) suppresses sending warning messages to stderr.
-s rrnames|rdata: what data to search.
rrnames: Search in the rrnames part of DNS records, aka the left-hand side. Can abbreviate as n. This is the default.
rdata: Search in the rdata part of DNS records, aka the right-hand side. Can abbreviate as d.
-t rrtype: specify the resource record type desired. Default is ANY.
For rrnames queries, valid rrtypes include those defined in DNS RFCs, including ANY, except DNSSEC types are not allowed (ANY-DNSSEC, CDNSKEY, CDS, DLV, DNSKEY, DS, NSEC, NSEC3, NSEC3PARAM, RRSIG, and TA resource record types). Also valid are TYPE# values.
For rdata queries, only the following rrtypes are valid: CNAME, HINFO, MX, NAPTR, NS, PTR, RP, SOA, SPF, SRV, and TXT. Also valid are their TYPE# values.
-T: Like -F but does not include the rrtype in the batch file queries. This allows pivots to match against all available rrtypes. The batch output will also include a comment for each line including the rrtype.
-u server_sys: specifies the syntax of the RESTful URL. The only system currently supported is “dnsdb2”, which is the default.
-U: turns off TLS certificate verification (unsafe).
-v: report the version of dnsdbflex and exit.
-4: force connecting to the DNSDB server via IPv4.
-6: force connecting to the DNSDB server via IPv6.
EXAMPLES
# Regular expression search of all rrnames that contain a coke label,
# for all rrtypes, limit of 10 results.
$ dnsdbflex --regex '.*.coke..*' -l 10
# Same query without using default values
$ dnsdbflex --regex '.*.coke..*' -l 10 -s rrnames --mode terse
# Glob search of all names that contain a coke label and have an 'A' RRType.
$ dnsdbflex --glob '*.coke.*' -l 10 -t A
# Pivot those results into dnsdbq for full DNSDB API results in json
# form. Note that up to 11 DNSDB query quota units will be consumed,
# 1 by dnsdbflex and 10 by dnsdbq. If we did not specify the RRType
# to dnsdbflex, then it might return more than 10 results (one for
# each RRType for each name) and we'd use more than 11 DNSDB query
# quota units.
$ dnsdbflex --glob '*.coke.*' -l 10 -t A -F | dnsdbq -f -j
# Get names containing "coke" but then exclude all those containing "diet".
$ dnsdbflex --glob '*.coke.*' --exclude '.*diet.*' -l 10
# Same query, but using regular expressions
$ dnsdbflex --regex '.*.coke..*' --exclude '.*.diet..*' -l 10
TIME FENCING
Farsight’s DNSDB flexible search provides time fencing options for searches. The -A and -B options take a timestamp as an argument. The timestamps may be one of following forms.
- positive unsigned integer : in Unix epoch format.
- negative unsigned integer : negative offset in seconds from now.
- YYYY-MM-DD [HH:MM:SS] : in absolute form, in UTC time, as DNSDB does its fencing using UTC time.
- %uw%ud%uh%um%us : the relative form with explicit labels (w=weeks, d=days, h=hours, m=minutes, s=seconds). Calculates offset from UTC time, as DNSDB does its fencing using UTC time.
A few examples of how to use time fencing options:
# Responses after Aug 22, 2015 (midnight),
# excluding records ALSO seen before that time.
$ dnsdbflex... -c -A 2015-08-22
# Responses from 2015 (midnight to midnight),
# but not excluding records ALSO seen outside that time range.
$ dnsdbflex... -B 2016-01-01 -A 2015-01-01
Certain settings for time fences may be used to accelerate queries for rrnames and rdata values which have been recently observed or which were first observed in the distant past. Time fencing may accelerate the query if either -A or -B (but not both) are supplied without -c.
A few examples of how to use time fencing options where the query may be accelerated:
# Responses after 2015-08-22 14:36:10,
# but not excluding records ALSO seen before that time.
$ dnsdbflex... -A "2015-08-22 14:36:10"
# Responses from the last 60 minutes,
# but not excluding records ALSO seen before that time.
$ dnsdbflex... -A "-3600"
# Responses after Aug 22, 2015 (midnight),
# but not excluding records ALSO seen before that time.
$ dnsdbflex... -A 2015-08-22
# Responses before Jan 22, 2013 (midnight),
# but not excluding records ALSO seen after that time.
$ dnsdbflex... -B 2013-01-22
FILES
~/.dnsdb-query.conf or /etc/dnsdb-query.conf: configuration file which can specify the API key, etc. variables. The first of these which is readable will be used, alone, in its entirety. See the DNSDBQ_CONFIG_FILE environment variable which can specify a different con- figuration file to use.
For backwards compability, ~/.isc-dnsdb-query.conf and /etc/isc-dnsdb-query.conf are also valid, but deprecated.
The variables which can be set in the configuration file are as follows:
- DNSDB_API_KEY, APIKEY: contains the user’s API key (no default).
- DNSDB_SERVER: contains the URL of the DNSDB API server (default is https://api.dnsdb.info/), and optionally the URI prefix for the database.
- DNSDBQ_SYSTEM: contains the default value for the u option described above. Can only be “dnsdb2”. If unset, dnsdbflex will probe for any configured system.
ENVIRONMENT
The following environment variables affect the execution of dnsdbflex:
- DNSDBQ_CONFIG_FILE: specifies the configuration file to use, overriding the internal search list.
- DNSDB_API_KEY: contains the user’s apikey. The older APIKEY environment variable has been retired, though it can still be used in the configuration file.
- DNSDB_SERVER contains the URL of the DNSDB API server, and optionally a URI prefix to be used. If not set, the configuration file is consulted.
- DNSDBQ_SYSTEM See DNSDBQ_SYSTEM in the FILES section above.
EXIT STATUS
Success (exit status zero) occurs if a connection could be established to the back end database server, even if no records matched the search criteria. Failure (exit status nonzero) occurs if no connection could be established, perhaps due to a network or service failure, or a configuration error such as specifying the wrong server hostname.
SEE ALSO
- dnsdbq(1), jq(1), libcurl(3)
