User Guides

Farsight Resilient Integration User Guide

12/10/2022
Share this entry

About the Farsight Integration for IBM’s Resilient System

Farsight DNSDB integration for IBM Resilient SOAR platform is a collection of functions that connect to DNSDB API. These functions are also accompanied with Resilient workflows that demonstrate the use of each function. These workflows can then be triggered by Resilient rules to enrich the Indicators of Compromise (IoC) with Passive DNS Data provided by DNSDB API. The extension package is bundled with all of these components (functions, workflows, rules).

DNSDB is an artifact enrichment solution. Queries are possible for:

  • IP Addresses
  • DNS Names

App Host Installation

All the components for running DNSDB function in a container already exist when using the App Host app.

To install,

  • Navigate to Administrative Settings and then the Apps tab.
  • Click the Install button and select the downloaded file: app-fn_dnsdb-x.x.x.zip.
  • Go to the Configuration tab and edit the app.config file and provide the DNSDB API key and Server URL.
[fn_dnsdb]
apikey =
server =

Integration Server Installation

  • Resilient platform >= v37.2.49
  • An Integration Server running resilient_circuits>=30.0.0

Installation

  • Download the fn_dnsdb.zip.
  • Copy the .zip to your Integration Server and SSH into it.
  • Unzip the package:
$ unzip fn_dnsdb-x.x.x.zip
  • Change Directory into the unzipped directory:
$ cd fn_dnsdb-x.x.x
  • Install the package:
$ pip install fn_dnsdb-x.x.x.tar.gz
  • Import the configurations into your app.config file:
$ resilient-circuits config -u -l fn-dnsdb
  • Import the fn_dnsdb customizations into the Resilient platform:
$ resilient-circuits customize -y -l fn-dnsdb
  • Open the config file, scroll to the bottom and edit your fn_dnsdb configurations:
$ nano ~/.resilient/app.config
ConfigRequiredExampleDescription
apikeyYesd41d8cd98f00b204e9800998ecf8427eDNSDB API key
serverYeshttps://api.dnsdb.infoDNSDB API Server
  • Save and Close the app.config file.
  • [Optional]: Run selftest to test the Integration you configured:
$ resilient-circuits selftest -l fn-dnsdb
  • Run resilient-circuits or restart the Service on Windows/Linux:
$ resilient-circuits run

Uninstall the Integration

  • SSH into your Integration Server.
  • Uninstall the package:
$ pip uninstall fn-dnsdb
  • Open the config file, scroll to the [fn_dnsdb] section and remove the section or prefix # to comment out the section.
  • Save and Close the app.config file.

Troubleshooting

There are several ways to verify the successful operation of a function.

Resilient Action Status

  • When viewing an incident, use the Actions menu to view Action Status.
  • By default, pending and errors are displayed.
  • Modify the filter for actions to also show Completed actions.
  • Clicking on an action displays additional information on the progress made or what error occurred.

Resilient Scripting Log

  • A separate log file is available to review scripting errors.
  • This is useful when issues occur in the pre-processing or post-processing scripts.
  • The default location for this log file is: /var/log/resilient-scripting/resilient-scripting.log.

Resilient Logs

  • By default, Resilient logs are retained at /usr/share/co3/logs.
  • The client.log may contain additional information regarding the execution of functions.

Resilient-Circuits

  • The log is controlled in the .resilient/app.config file under the section [resilient] and the property logdir.
  • The default file name is app.log.
  • Each function will create progress information.
  • Failures will show up as errors and may contain python trace statements.

Support

NameVersionAuthorSupport URL
fn_dnsdb1.0.0Farsight Security, Inc.https://farsightsecurity.com

Use Cases

DNSDB Co-Located Hosts

This use case describes the desire to easily identify Hosts that are co-located (based on Address) based on the input of a Host and a given point in time. The response would be a set of domains that also shared the same IP.

  • Example Pre-Process Script:
if incident.start_date:
  time_first_before = str(incident.start_date/1000 + 86400)
  time_last_after = str(incident.start_date/1000 - 15552000)
else:
  time_first_before = str(incident.create_date/1000 + 86400)
  time_last_after = str(incident.create_date/1000 - 15552000)


inputs.pivot = """
[
\{\{"function": "rrset", "owner_name": "{0}", "rrtype": "{1}", "limit": {2}, \
    "time_first_before": "{3}", "time_last_after": "{4}"}},
\{\{"function": "rdata", "type": "ip", "pivot": "rdata"}}
]
""".format(artifact.value, "A", 10, time_first_before, time_last_after)
  • Example Post-Process Script:
ip_records = workflow.properties.a_records["dnsdb_records"] +
    workflow.properties.aaaa_records["dnsdb_records"]

rrname_list = []

for item in ip_records:
  rrname_list.append(item["rrname"])


c_rdata_list = []

for item in workflow.properties.cname_records["dnsdb_records"]:
  for i in item["rdata"]:
    c_rdata_list.append(i)

aggregate_results = rrname_list + c_rdata_list

hosts_list_set = set(aggregate_results)
msg = ""
for item in hosts_list_set:
  msg += "{}".format(item)
msg += ""

incident.addNote(helper.createRichText("{1} Co-Located Hosts Found for \
    {0}{2}".format(artifact.value, str(len(hosts_list_set)), msg)))

DNDB Historical Address

This use case describes the desire to identify all Addresses used as DNS A and AAAA records for a given Host based on a time window from a starting and stopping point in time.

  • Example Pre-Process Script:
inputs.owner_name = artifact.value
inputs.limit = 100
inputs.rrtype = 'A'

#calculate time_first_before, time_last_after based on incident occur date.
if incident.start_date:
  inputs.time_first_before = str(incident.start_date/1000 + 86400)
  inputs.time_last_after = str(incident.start_date/1000 - 15552000)
else:
  inputs.time_first_before = str(incident.create_date/1000 + 86400)
  inputs.time_last_after = str(incident.create_date/1000 - 15552000)
  • Example Post-Process Script:
rdata_records = workflow.properties.a_records["dnsdb_records"] +
    workflow.properties.aaaa_records["dnsdb_records"]
rdata_list = []

for item in rdata_records:
  for i in item["rdata"]:
    rdata_list.append(i)

rdata_list_set = set(rdata_list)
msg = ""
for item in rdata_list_set:
  msg += "{}".format(item)
msg += ""

incident.addNote(helper.createRichText("{1} Historical Address Found for \
    {0}{2}".format(artifact.value, str(len(rdata_list_set)), msg)))

DNSDB Historical Hosts

This use case describes the desire to identify all Hosts that resolved to a given Address based on a time window from a starting and stopping point in time.

  • Example Pre-Process Script:
inputs.value=artifact.value
inputs.type = 'ip'
inputs.rrtype = 'ANY'
inputs.limit = 100

#calculate time_first_before, time_last_after based on incident occur date.
if incident.start_date:
  inputs.time_first_before = str(incident.start_date/1000 + 86400)
  inputs.time_last_after = str(incident.start_date/1000 - 15552000)
else:
  inputs.time_first_before = str(incident.create_date/1000 + 86400)
  inputs.time_last_after = str(incident.create_date/1000 - 15552000)
  • Example Post-Process Script:
dnsdb_records = results["dnsdb_records"]

host_list = []
for item in dnsdb_records:
  host_list.append(item["rrname"])

host_names_msg = ""
for item in set(host_list):
  host_names_msg += "{}".format(item)
host_names_msg += ""

incident.addNote(helper.createRichText("{1} Historical Hosts Found for: \
    {0}{2}".format(artifact.value, str(len(set(host_list))), host_names_msg)))

Function – DNSDB Flex

DNSDB Flex function allows you to search DNSDB by regular expressions and globs (aka wildcarding).

  • Inputs:
NameTypeRequiredDescription
keyselectYesThe search key, can be rrnames(supports “forward” searches based on the owner name of an RRSet) or rdata(supports “inverse” searches based on RData record values).
limitnumberNoLimit for the number of results returned via these lookup methods.
methodselectYesThe search method, it can be regex(regular expression search) or glob(full wildcarding).
offsetnumberNoHow many rows to offset (e.g. skip) in the results.
rrtypetextNoThe resource record type of the RRSet, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal rrtype number.
time_first_aftertextNoProvide results after the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_after=-31536000” will only provide results that were first observed within the last year.
time_first_beforetextNoProvide results before the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_before=1420070400” will only provide matching DNS records that were first observed before (or older than) January 1, 2015.
time_last_aftertextNoProvide results after the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_after=-2678400” will only provide results that were last observed after 31 days ago.
time_last_beforetextNoProvide results before the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_before=1356998400” will only provide results for DNS records that were last observed before 2013.
valuetextYesThe value to search DNSDB records.
  • Outputs:
results = {
        "dnsdb_records": [
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.yahoo.com.au",
                "rrtype": "CNAME"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity-2432183.starbucks.com.cn",
                "rrtype": "A"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.com.cn",
                "rrtype": "NS"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.com.cn",
                "rrtype": "SOA"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.com.cn",
                "rrtype": "CNAME"
            },
            {
                "from_zone_file": false,
                "rrname": "www.farsightsecurity.com.cn",
                "rrtype": "CNAME"
            },
            {
                "from_zone_file": false,
                "rrname": "farsightsecurity.damai.cn",
                "rrtype": "CNAME"
            }
        ]
}

Workflows

  • Example Pre-Process Script:
inputs.value = artifact.value
  • Example Post-Process Script:
dnsdb_records = results['dnsdb_records']

incident.addNote(helper.createRichText("{0} DNSDB Flex Records \
    Found".format(str(len(dnsdb_records)))))

Function – DNSDB RData

This function queries DNSDB’s RData index, which supports “inverse” lookups based on RData record values. In contrast to the RRSet lookup method, RData lookups return only individual resource records and not full resource record sets, and lack bailiwick metadata. An RRSet lookup on the owner name reported via an RData lookup must be performed to retrieve the full RRSet and bailiwick.

  • Inputs:
NameTypeRequiredTooltip
aggrbooleanNoAggregated results group identical RRSets across all time periods and is the classic behavior from querying the DNSDB.
limitnumberNoLimit for the number of results returned via these lookup methods.
offsetnumberNoHow many rows to offset (e.g. skip) in the results.
rrtypetextNoThe resource record type of the RRSet, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal rrtype number.
time_first_aftertextNoProvide results after the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_after=-31536000” will only provide results that were first observed within the last year.
time_first_beforetextNoProvide results before the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_before=1420070400” will only provide matching DNS records that were first observed before (or older than) January 1, 2015.
time_last_aftertextNoProvide results after the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_after=-2678400” will only provide results that were last observed after 31 days ago.
time_last_beforetextNoProvide results before the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_before=1356998400” will only provide results for DNS records that were last observed before 2013.
typeselectYesThe type specifies how value is interpreted. type may be name, ip or raw
valuetextYesThe value to search DNSDB records.
  • Outputs:
results = {
        "dnsdb_records": [
            {
                "count": 606,
                "from_zone_file": false,
                "rdata": [
                    "www.farsightsecurity.com."
                ],
                "rrname": "scout.dnsdb.info",
                "rrtype": "CNAME",
                "time_first": "2020-03-27T18:37:24Z",
                "time_last": "2020-10-21T18:11:47Z"
            },
            {
                "count": 121,
                "from_zone_file": false,
                "rdata": [
                    "www.farsightsecurity.com."
                ],
                "rrname": "scout-beta.dnsdb.info",
                "rrtype": "CNAME",
                "time_first": "2020-08-20T22:52:29Z",
                "time_last": "2020-10-20T17:54:58Z"
            },
            {
                "count": 546,
                "from_zone_file": false,
                "rdata": [
                    "www.farsightsecurity.com."
                ],
                "rrname": "81.64-26.140.160.66.in-addr.arpa",
                "rrtype": "PTR",
                "time_first": "2013-12-10T01:20:08Z",
                "time_last": "2020-10-10T15:47:19Z"
            }
        ]
}

Workflows

  • Example Pre-Process Script:
inputs.value = artifact.value
  • Example Post-Process Script:
dnsdb_records = results['dnsdb_records']

incident.addNote(helper.createRichText("{0} DNSDB RData Records \
    Found".format(str(len(dnsdb_records)))))

Function – DNSDB RRSet

This function queries DNSDB’s RRSet index, which supports “forward” lookups based on the owner name of an RRSet.

  • Inputs:
NameTypeRequiredTooltip
aggrbooleanNoAggregated results group identical RRSets across all time periods and is the classic behavior from querying the DNSDB.
bailiwicktextNoThe “bailiwick” of an RRSet in DNSDB observed via passive DNS replication is the closest enclosing zone delegated to a nameserver which served the RRset. The “bailiwick” of an RRSet in DNSDB observed in a zone file is simply the name of the zone containing the RRSet.
limitnumberNoLimit for the number of results returned via these lookup methods.
offsetnumberNoHow many rows to offset (e.g. skip) in the results.
owner_nametextYesDNS name specified in DNS presentation format.
rrtypetextNoThe resource record type of the RRSet, either using the standard DNS type mnemonic, or an RFC 3597 generic type, i.e. the string TYPE immediately followed by the decimal rrtype number.
time_first_aftertextNoProvide results after the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_after=-31536000” will only provide results that were first observed within the last year.
time_first_beforetextNoProvide results before the defined timestamp for when the DNS record was first observed. For example, the URL parameter “time_first_before=1420070400” will only provide matching DNS records that were first observed before (or older than) January 1, 2015.
time_last_aftertextNoProvide results after the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_after=-2678400” will only provide results that were last observed after 31 days ago.
time_last_beforetextNoProvide results before the defined timestamp for when the DNS record was last observed. For example, the URL parameter “time_last_before=1356998400” will only provide results for DNS records that were last observed before 2013.
  • Outputs:
results = {
        "dnsdb_records": [
            {
                "bailiwick": "com",
                "count": 19,
                "from_zone_file": true,
                "rdata": [
                    "ns.lah1.vix.com.",
                    "ns1.isc-sns.net.",
                    "ns2.isc-sns.com.",
                    "ns3.isc-sns.info."
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "NS",
                "time_first": "2013-06-30T16:21:41Z",
                "time_last": "2013-07-18T16:22:47Z"
            },
            {
                "bailiwick": "com",
                "count": 157,
                "from_zone_file": true,
                "rdata": [
                    "ns.sjc1.vix.com.",
                    "ns.sql1.vix.com."
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "NS",
                "time_first": "2013-01-24T17:18:05Z",
                "time_last": "2013-06-29T16:19:01Z"
            },
            {
                "bailiwick": "com",
                "count": 1890,
                "from_zone_file": true,
                "rdata": [
                    "ns5.dnsmadeeasy.com.",
                    "ns6.dnsmadeeasy.com.",
                    "ns7.dnsmadeeasy.com."
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "NS",
                "time_first": "2013-07-19T16:22:00Z",
                "time_last": "2020-07-24T16:02:05Z"
            },
            {
                "bailiwick": "farsightsecurity.com",
                "count": 6350,
                "from_zone_file": false,
                "rdata": [
                    "66.160.140.81"
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "A",
                "time_first": "2013-09-25T15:37:03Z",
                "time_last": "2015-04-01T06:17:25Z"
            },
            {
                "bailiwick": "farsightsecurity.com",
                "count": 36770,
                "from_zone_file": false,
                "rdata": [
                    "104.244.13.104"
                ],
                "rrname": "farsightsecurity.com",
                "rrtype": "A",
                "time_first": "2015-04-01T14:17:52Z",
                "time_last": "2018-09-27T00:29:43Z"
            }
        ]
}

Workflows

  • Example Pre-Process Script:
inputs.owner_name = artifact.value
  • Example Post-Process Script:
dnsdb_records = results['dnsdb_records']

incident.addNote(helper.createRichText("{0} DNSDB RRSet Records Found".format(str(len(dnsdb_records)))))

Function – DNSDB Rate Limit

This function queries the “rate limit” endpoint to obtain information about the api key’s service limits and quotas.

  • See [Farsight DNSDB API Version 2 Documentation]({{ site.baseurl }}/dnsdb/dnsdb-apiv2/#service-limits-and-quotas) section on “service limits and quotas”.

Function – DNSDB Pivot

This function allows you to execute a series of queries against DNSDB. Each subsequent search “pivots” on the keys returned by the previous. You may pivot using the rrname, rdata, or raw_rdata field. This function takes a single argument, “pivot”, which is a json-encoded array of dictionaries. Each dictionary contains parameters for a DNSDB search, including a “function” field which may be any of “rdata”, “rrset”, or “flex”. Arguments for each search are the same as their corresponding Resilient function.

Required arguments:

  • rrset: owner_name
  • rdata: type, value
  • flex: key, method, value

Optional arguments:

  • limit, offset, time_first_before, time_first_after, time_last_before, time_last_after

The second through last dictionaries must each contain a “pivot” key which may be “rrname”, “rdata”, or “raw_rdata”, and indicates which field from the preceding query should be used as the key for the next.

  • Example:
inputs.pivot = """
[
\{\{"function": "rrset", "owner_name": "{0}", "rrtype": "{1}", "limit": {2},
    "time_first_before": "{3}", "time_last_after": "{4}"}},
\{\{"function": "rdata", "type": "ip", "pivot": "rdata"}}
]
""".format(artifact.value, "A", 10, time_first_before, time_last_after)

Additional Information

Rules

Rule NameObjectWorkflow Triggered
DNSDB RRSet Lookupartifactrrset_workflow
DNSDB RData Lookupartifactrdata_workflow
DNSDB Flex Lookupartifactflex_workflow
DNSDB Co-Located Host Lookupartifactdnsdb_colocated_host_workflow
DNSDB Historical Address Lookupartifactdnsdb_historical_address_workflow
DNSDB Historical Hosts Lookupartifactdnsdb_historical_host_workflow

Related Content

Farsight Security: A DomainTools Company logo in front of black-gray gradient
User Guides

The AXA User Guide

Read more
Farsight Security: A DomainTools Company logo in front of black-gray gradient
User Guides

The AXA Technical Overview

Read more
Farsight Security: A DomainTools Company logo in front of black-gray gradient
User Guides

The AXA API Reference

Read more
DomainTools Logo
Pricing Contact Login Support Request a Demo

© 2024 DomainTools

DomainTools® and DomainTools™ are owned by DomainTools, all rights reserved.

Privacy Policy    |    California Privacy Notice
Do Not Sell My Personal Information    |    Terms of Service    |    Sitemap