Cortex 101: Dissecting Observables a Hundred Ways

TheHive Project’s Code Chefs are glad to announce that, thanks to the precious contributions of the user community, Cortex has broken the one hundred analyzer mark.

Cortex-Analyzers version 1.14.0 is out and includes new analyzers, some improvements and some bug fixes.

New Analyzers

New and enhanced analyzers, described below, are:

  • Cisco Investigate by Cisco Umbrella Research @opendns
  • Proofpoint Forensics Lookup by Emmanuel Torquato
  • Proofpoint Threat Insight Forensics Analyzer by Emmanuel Torquato
  • RecordedFuture by jojoob
  • urlscan.io search by ninoseki
  • Google DNS over HTTP by 0xswitch
  • RTF files support in FileInfo by Nils Kuhnert
  • Datascan and Inetnum flavors in Onyphe analyzer by Pierre Baudry and Adrien Barchapt

Again, huge thanks for the awesome work that has been performed by all our contributors!

Investigate

Cisco Umbrella Investigate provides threat intelligence about domains and IP addresses accross the Internet. The analyzer can be used to query the Cisco Umbrella (formerly OpenDNS) API and get information about an IP or a domain name. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-investigate-short

sc-investigate-long
Cisco Investigate: short and long reports

Proofpoint Forensics Lookup

According to Proofpoint’s website, the Forensics API allows insight in detailed forensic evidences about individual threats or compaigns. The analyzer can be used to check observables against given indicators of compromise stored in the ProofPoint service.

Unfortunately, there are currently no sample report screenshots available, because TheHive’s Core Team does not have access to Proofpoint services. Also, due to the same reason, this analyzer could not be tested by us. If you have access to the service and can test the analyzer and/or provide report screenshots, please let us know.

RecordedFuture

This analyzer lets you get the latest risk data from RecordedFuture for a hash, domain or an IP address. It can be used to query the API and get information. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-recordedfuture-short

sc-recordedfuture-long
RecordedFuture: short and long reports

Urlscan.io search

Urlscan.io is a service that scans and analyzes websites. Submitted pages will be browsed like a regular user would do and every activity gets recorded. The analyzer submitted by ninoseki queries urlscan without initiating a scan which would be publicly visible on the website. Accepted datatypes for this analyzer are URL, domain, hash and IP.

The templates which display the results of the analyzer look like the following screenshots:

Screenshot from 2018-10-25 09-55-34

Screenshot from 2018-10-25 09-48-58
Urlscan.io: short and long reports

Google DNS over HTTP

This analyzer provides DNS information for an IP, a domain or a FQDN by making calls to Google DNS-over-HTTP (DoH). No API key is required.

Results are displayed in TheHive in the following manner:sc-googleDNS-short

sc-googleDNS-long
Google DNS: short and long reports

RTF files support in FileInfo

The FileInfo meta analyzer has been improved and now leverages the rtfobj tool provided in the Oletools suite by Decalage.

Results are displayed in TheHive in the following manner:

sc-fileinfo_RTF-short

sc-rtfobj-long
FileInfo with rtfobj: short and long reports

Datascan and Inetnum flavors in Onyphe analyzer

The Onyphe analyzer has been enhanced with two new flavors. Datascan provides information about known open ports on a specific IP, and Inetnum enumerates all known network information about the analyzed IP address.

An API key is required to use the analyzer and can be obtained by creating an account on the Onyphe website.

Results are displayed in TheHive in the following manner:

sc-onyphe-inetnum-short

sc-onyphe_inetnum-long
Onyphe  Inetnum: short and long reports

sc-onyphe_datascan-short

sc-onyphe_datascan-long
Onyphe Datascan: long and short reports

Bug fixes and enhancements

  • #248: Improve error msg when VT Get Report does not have an entry for
  • #323: Fix an issue with HybridAnalysis analyzer filenames handler
  • #329: Enhance PassiveTotal Passive DNS report
  • #348: Fix a typo in URLhaus’s long.html
  • #356 : Force the use of Python3 in MISP analyzer
  • #362: Fix file not found issue and empty result set in CERT.at passive DNS analyzer

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.14.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Hold your breath and git pull

Cortex-Analyzer 1.13.0 is out and includes new analyzers, some improvements and few bug fixes :

  • Hunter.io has been contributed by Rémi Allain (@remiallain)
  • DShield lookup contributed by Xavier Martens (@xme)
  • Pulsedive contributed by Nils Kuhnert (@3c7, TheHive Project)
  • FileInfo has been enhanced with Manalyze submodule for PE analysis ; by @lctrcl and Nils Kuhnert (@3c7)
  • Thanks to @silrumpa, the Fortiguard_URLCategory analyzer has been updated and let you configure categories and customize theirs maliciousness level
  • PhishingInitiative analyzer has been flavoured with a scan facility, thanks to Rémi Pointel

Huge thanks for the awesome work that has been performed by all our contributors!

Hunterio_DomainSearch

Hunter.io is a search engine that lets you find emails associated with a given domain name. The analyzer can be used to query the API and get a list of email addresses for a specific domain name. An API key is required to use this analyzer and can be obtained by registering on the website.

Results are displayed in TheHive in the following manner:

sc-hunterio-short

sc-hunterio-long
Hunterio_DomainSearch: short and long report samples

DShield_lookup

The analyzer lets you query the famous SANS Internet Storm Center (ISC) DShield API and look up IP address reputation. No API key is needed to run this analyzer.

Results are displayed in TheHive in the following manner:

sc-dshield-short
sc-dshield-long
DShield_lookup: short and long report templates

Pulsedive_GetIndicator

Pulsedive is a Threat Intelligence platform that allows you to enrich your observables. The analyzer can be used to query  the API and get information about a domain name,  hash, IP or URL. An API key is required to use this analyzer and can be acquired by creating an account on the webiste.

Results are displayed in TheHive in the following manner:

sc-pulsedive-short

sc-pulsedive-long
Pulsedive_GetIndicator: short and long report templates

Manalyze joins FileInfo

Manalyze is a tool developed by Ivan Kwiatkowski (@JusticeRage) that lets you analyze, operate PE (Portable Executable) and collect useful artifacts that help the analyst in determining its maliciousness.

@lctrcl wrote an analyzer that triggers Manalyze on a PE file and gives the analyst a useful report. Nils Kuhnert (@3c7) from TheHive Project then included this analyzer as a submodule into our FileInfo meta-analyzer for files ; it can run Manalyze from compiled binary, or, if your Cortex server is ready for that, through the right docker. If you decide to use the compiled binary, please follow instruction from Manalyze github page.

This submodule is disabled by default. To use it, you have to set some configuration in Cortex:

sc-fileinfo-config
FileInfo: configuration for Manalyze in Cortex

Results are displayed in TheHive in the following manner:

sc-fileinfo-short3

sc-fileinfo-long3
Manalyze submodule: short and long report templates

Fortiguard_URLCategory

This analyzer has been enhanced to let analysts choose categories considered as malicious or suspicious. It comes with a default configuration but you can setup your own by selecting the categories from the Fortiguard website.

sc-fortiguard-config
Fortiguard_URLCategory: default configuration for categories  in Cortex

Select which categories you want to be considered malicious or suspicious, and others will be considered by the analyzer as info. Analyzed observables that are not categorised by Fortigard service is considered as safe.

PhishingInitiative_Scan

PhishingInitiative has been enhanced with a new scan flavor. This let the analyst submit an URL to the webservice. An API key is needed to run this analyzer and can be obtained by registering on the website.

Results are displayed in TheHive in the following manner:

sc-phishinginitiative-short

sc-phishinginitiative-long
PhishingInitiative_Scan: short and long report samples

Bug fixes

  • #335 : fix a bug in MISP analyzer
  • #340 : fix short report template in C1fApp
  • #339 : fix short and mini reports for Domaintools Whois history flavor

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.13.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers//requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/
/requirements.txt; do sudo -H pip3 install -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Correction: September 25, 2018
The link to the report templates was incorrect.