Cortex-Analyzers 2.4.0: 138 Ways to Analyze, 10 Methods to Respond

Guess what? TheHive Project is still alive and well, as Saâd already mentioned in a previous blog post.

We’ve been certainly very busy lately, preparing the upcoming release of TheHive 4 and doing many other things beside working on our FOSS project. As a result, it took us a rather long time to merge several community contributions and reduce the sizeable pile of pull requests.

We would like to thank our contributors for their patience and we hope the cyberdefenders out there will enjoy the brand new Cortex-Analyzers 2.4.0 release, with many new analyzers, responders and some bug fixes & improvements, bringing the total to a whooping 138 analyzers (counting all flavors) and 10 responders!

Additionally, with this release, all analyzers are now using Python 3. No more Python 2 technodebt!

Photo by Saâd Kadhi

What’s New?

New Analyzers

8 new analyzers have been added to this release:

1 analyzer has new flavors:

New Responders

3 new responders have been added:

Overview of the New Analyzers

DomainToolsIris

This analyzer looks up domain names, IP addresses, e-mail addresses, and SSL hashes using the popular DomainTools Iris service API.

The analyzer comes in 2 flavors:

  • DomainToolsIris_Investigate: use DomainTools Iris API to investigate a domain.
  • DomainToolsIris_Pivot: use DomainTools Iris API to pivot on ssl_hash, ip, or email.

A valid DomainTools API integration subscription is needed to run this analyzer.

TheHive displays the analyzer results as follows:

DomainToolsIris short report
DomainToolsIris long report

EmailRep

The EmailRep analyzer checks the reputation of an email address against the emailrep.io database.

IPInfo

This analyzer accesses IP-centric features provided by ipinfo.io. While the EmailRep API can be used without a token for limited usage, the ipinfo.io analyzer requires the configuration of an API token before use.

Maltiverse

This analyzer lets you query the free Maltiverse Threat Intelligence platform for enrichment information about a particular hash, domain, ip or url.

TheHive displays the analyzer results as follows:

Maltiverse short report
Maltiverse long report

MalwareClustering

Andrea Garavaglia contributed this one a long time ago and we finally merged it into the Cortex-Analyzers repository. Andrea gave a talk about the background of this analyzer at the fourth MISP summit. You can watch it here.

In order to use the analyzer, you need to point it to a Neo4j server (you need to supply the host, port, login & password).

PaloAlto Autofocus

This analyzer lets you leverage PaloAlto Autofocus services. Provided you are an Autofocus customer and you have access to their API, you need to configure the analyzer with your username and a token key.

The analyzer comes with 3 flavors:

  • AUTOFOCUS_GetSampleAnalysis lets you request a full report for a given hash.
  • AUTOFOCUS_SearchIOC lets you research for samples linked to specific IoCs with datatypes like domain, fqdn, user-agent, imphash, ip, mutex, tag and url. Please note that mutex and tag are not default datatypes in TheHive. You need to create them in TheHive before you can leverage them.
  • AUTOFOCUS_SearchJSON lets you research for samples based on a complex JSON query.

Important: TheHive has no templates corresponding to this analyzer have been published yet. They will be provided in the near future.

SpamhausDBL

This analyzer performs reputation lookups of a domain or a fqdn against Spamhaus Domain Block List (DBL).

TheHive displays the analyzer results as follows:

SpamhausDBL short report
SpamhausDBL long report

TeamCymruMHR

This analyzer queries Team Cymru’s Malware Hash Registry for known malware hashes (MD5 or SHA-1). If it is malware and known by the service, it returns the last time it has been seen along with an approximate anti-virus detection percentage.

Overview of the New Responders

KnowBe4

This responder allows the integration between TheHive/Cortex and KnowBe4’s User Events API.
If a mail observable is tagged with a specified tag, corresponding to the responder’s configuration (e.g. phished), then the associated user will have a custom event added to their profile in KnowBe4.

A valid account on KnowBe4 and an API key are required to run this responder.

Minemeld

This responder sends observables you select to a Palo Alto Minemeld instance.

To run this responder, a MineMeld Threat Intelligence Sharing account is needed.

Wazuh

This responder performs actions on Wazuh, the open source security monitoring platform. It currently supports ad-hoc firewall blocking of ip observables.

Improvements

New PassiveTotal flavors

Thanks to Brandon Dixon, the PassiveTotal analyzer gains 3 new flavors, bringing the total to 11:

  • PassiveTotal_Trackers let you make tracker lookups on observables of type domain, fqdn and ip.
  • PassiveTotal_Host_Pairs let you make host pair lookups on observables of type domain, fqdn and ip.
  • PassiveTotal_Components lets you make components lookup on observables of type domain, fqdn and ip.

They come with their own report templates.

GreyNoise Analyzer

The analyzer has been updated to support GreyNoise API v2, thanks to the contribution of Whitney Champion (#562).

New Data Types Supported by Some Analyzers

  • VirusTotal_GetReporthas been updated to allow requests for observables of type fqdn.
  • Threatcrowd has been updated to allow requests for observables of type domain.
  • Shodan has been updated to allow requests for observables of type fqdn.

Fixes

  • [#602] The MISP analyzer was bumped to version 2.1 and is ready to use PyMISP 2.4.120.

Get It While Supply Lasts!

I’m Hype

If you are using the dockerized analyzers & responders, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button.

I’m Country

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

DigitalShadows Feeder 2.4.5

DigitalShadows2TH 2.4.5, a new version of the DigitalShadows feeder for TheHive, has been released with major improvements.

Source : https://www.bustle.com

Dockerise All The Things!

The most notable one is that you can now use it with docker. Run docker pull thehiveproject/ds2th and create a homedir for configuration and logs.

$ DS2TH_HOMEDIR = /opt/thehive_feeders/Digitalshadows2TH/
$ docker pull thehiveproject/ds2th:latest
$ mkdir -p $DS2TH_HOMEDIR/{config,log}
$ wget -O $DS2TH_HOMEDIR/config/config.py\
https://raw.githubusercontent.com/TheHive-Project/DigitalShadows2TH/master/config.py.template

Edit the config.py before runing the docker command below:

$ docker run --rm --net=host --mount\
type=bind,source="$DS2TH_HOMEDIR"/config,target=/app/config --mount type=bind,source="$DS2TH_HOMEDIR"/log,target=/app/log certbdf/ds2th <OPTIONS>
Quick Installation

If that sounds still complicated, worry not! We also wrote a little program that helps you prepare the environment and install everything. We ensured that it works well on Ubuntu 18.04. The program uses two environment variables to set up everything: FEEDERS_SYSACCOUNT and FEEDERS_HOMEDIR :

$ export FEEDERS_SYSACCOUNT=thehive
$ export FEEDERS_HOMEDIR=/opt/thehive_feeders

There are also sane, default settings in case you did not set any value. DigitalShadows2TH’s home directory will be set to /opt/thehive_feeders/DigitalShadows2TH. To use the script, run the following command line and follow the instructions:

 $ wget -qO- https://raw.githubusercontent.com/TheHive-Project/DigitalShadows2TH/mater/INSTALL/install_with_docker.sh | sudo -E bash -

Custom Case Templates

Previous versions of DigitalShadows2TH allowed only one case template to be associated with alerts created by the feeder in TheHive. Starting from DigitalShadows2TH 2.4.0, you can define a case template for each type of incidents raised by DigitalShadows in the configuration file.

The configuration pertaining to TheHive looks as follows:

TheHive = {
 'proxies':{
     'http': '',
     'https': ''
},
   'url':'THEHIVE_URL',
   'key':'THEHIVE_API_KEY',
   'templates': {
       'default':''
  }
}

Each incident type in DigitalShadows can be associated with a case template in TheHive, for example:

TheHive = {
  'proxies':{
      'http': '',
      'https': ''
  },
    'url':'THEHIVE_URL',
    'key':'THEHIVE_API_KEY',
    'templates': {
        'default':'MY_DEFAULT_CASE_TEMPLATE_FOR_DIGITALSHADOWS_INCIDENTS',
        'DATA_LEAKAGE': 'MY_DATA_LEAKAGE_CASE_TEMPLATE',
        'CYBER_THREAT': 'MY_CYBER_THREAT_CASE_TEMPLATE'
    }
}

A template can be defined for all the following DigitalShadows incident types:

  • DATA_LEAKAGE
  • CYBER_THREAT
  • PHYSICAL_SECURITY
  • SOCIAL_MEDIA_COMPLIANCE
  • BRAND_PROTECTION
  • INFRASTRUCTURE

A default template can be defined for DigitalShadows incidents. If no template is found for a specific incident type, the feeder looks for the default template. if no default template is found, an empty case will be created by when importing the alert.

Update or Install

If you are not using docker, just pull the repository and update your configuration file with the new templates part for TheHive.

Update your Repository
$ cd /opt/TheHive_feeders/DigitalShadows2TH/
$ git pull

The configuration file has changed, so you need to update yours before running the program. A new templates section has been added for TheHive and the path has changed. It is now in the config/ directory of the project.

Install and Use via the Code Repository
$ cd /opt/TheHive_feeders 
$ git clone https://github.com/TheHive-Project/DigitalShadows2TH.git

After that, follow the prerequisites and edit the configuration file. In /opt/TheHive_feeders/DigitalShadows2TH/config/ copy config.py.template to config.py and modify it.

Use cases and detailed configuration instructions can be found in the README file in the repository.

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

TheHive 3.2.0-RC1: The MISP Love Edition

Guess what? Our integration with MISP, the de facto standard for threat sharing, has just gotten better with our latest beta release: TheHive 3.2.0-RC1.

While you could synchronize TheHive with one or multiple MISP instances in earlier versions and select events using filters like their age, the number of attributes they contain or exclude those which are created by specific organisations or contain one or several black-listed tags, 3.2.0-RC1 adds the ability to whitelist tags, thus limiting the events that would show up in TheHive’s Alerts pane to only those which have been tagged with labels your SOC/CSIRT/CERT needs to act on. This can be very useful for example if your Cyber Threat Intelligence analysts pre-select or create events in MISP and tag for SOC consumption those that need to be acted on.

The Hive - Logo - Schéma - V1_Plan de travail 1.png
TheHive, Cortex, MISP: The Power DFIR & CTI Trio

To use this feature, use the whitelist.tags parameter in the MISP section of TheHive’s application.conf as described in the documentation.

This new version also adds the ability to create dashboards out of responder actions, log responder operations, and offers a confirmation dialog before running a responder to avoid noob over-clicks and errors made by seasoned incident handlers running low on caffeine.

TheHive 3.2.0-RC1 will also show you the description of an observable if any while hovering over one in the Observables tab. You can also see observable tags when previewing an alert in the Alerts pane.

Last but not least, some users reported severe problems when they enabled TLS/SSL directly on TheHive without resorting to a reverse proxy such as NGINX. Blame that on the crappy TLS support in Play framework ;-). So we highly recommend using a reverse proxy for that purpose, and delegate authentication to it if you are relying on X.509 authentication, as TheHive 3.2.0-RC1 allows you to. Please check the Single Sign-On on TheHive with X.509 Certificates guide for further information.

For additional details on this release, please check the full changelog.

Warning Capt’n Robinson!

The RC in 3.2.0-RC1 stands for Release Candidate. Please help us make a great stable release out of it by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release. You’ll find this release candidate in the pre-release, beta repositories.

Please check TheHive Installation guide for further details.

You got a problem?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cortex 101: Dissecting Observables a Hundred Ways

TheHive Project’s Code Chefs are glad to announce that, thanks to the precious contributions of the user community, Cortex has broken the one hundred analyzer mark.

Cortex-Analyzers version 1.14.0 is out and includes new analyzers, some improvements and some bug fixes.

New Analyzers

New and enhanced analyzers, described below, are:

  • Cisco Investigate by Cisco Umbrella Research @opendns
  • Proofpoint Forensics Lookup by Emmanuel Torquato
  • Proofpoint Threat Insight Forensics Analyzer by Emmanuel Torquato
  • RecordedFuture by jojoob
  • urlscan.io search by ninoseki
  • Google DNS over HTTP by 0xswitch
  • RTF files support in FileInfo by Nils Kuhnert
  • Datascan and Inetnum flavors in Onyphe analyzer by Pierre Baudry and Adrien Barchapt

Again, huge thanks for the awesome work that has been performed by all our contributors!

Investigate

Cisco Umbrella Investigate provides threat intelligence about domains and IP addresses accross the Internet. The analyzer can be used to query the Cisco Umbrella (formerly OpenDNS) API and get information about an IP or a domain name. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-investigate-short

sc-investigate-long
Cisco Investigate: short and long reports

Proofpoint Forensics Lookup

According to Proofpoint’s website, the Forensics API allows insight in detailed forensic evidences about individual threats or compaigns. The analyzer can be used to check observables against given indicators of compromise stored in the ProofPoint service.

Unfortunately, there are currently no sample report screenshots available, because TheHive’s Core Team does not have access to Proofpoint services. Also, due to the same reason, this analyzer could not be tested by us. If you have access to the service and can test the analyzer and/or provide report screenshots, please let us know.

RecordedFuture

This analyzer lets you get the latest risk data from RecordedFuture for a hash, domain or an IP address. It can be used to query the API and get information. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-recordedfuture-short

sc-recordedfuture-long
RecordedFuture: short and long reports

Urlscan.io search

Urlscan.io is a service that scans and analyzes websites. Submitted pages will be browsed like a regular user would do and every activity gets recorded. The analyzer submitted by ninoseki queries urlscan without initiating a scan which would be publicly visible on the website. Accepted datatypes for this analyzer are URL, domain, hash and IP.

The templates which display the results of the analyzer look like the following screenshots:

Screenshot from 2018-10-25 09-55-34

Screenshot from 2018-10-25 09-48-58
Urlscan.io: short and long reports

Google DNS over HTTP

This analyzer provides DNS information for an IP, a domain or a FQDN by making calls to Google DNS-over-HTTP (DoH). No API key is required.

Results are displayed in TheHive in the following manner:sc-googleDNS-short

sc-googleDNS-long
Google DNS: short and long reports

RTF files support in FileInfo

The FileInfo meta analyzer has been improved and now leverages the rtfobj tool provided in the Oletools suite by Decalage.

Results are displayed in TheHive in the following manner:

sc-fileinfo_RTF-short

sc-rtfobj-long
FileInfo with rtfobj: short and long reports

Datascan and Inetnum flavors in Onyphe analyzer

The Onyphe analyzer has been enhanced with two new flavors. Datascan provides information about known open ports on a specific IP, and Inetnum enumerates all known network information about the analyzed IP address.

An API key is required to use the analyzer and can be obtained by creating an account on the Onyphe website.

Results are displayed in TheHive in the following manner:

sc-onyphe-inetnum-short

sc-onyphe_inetnum-long
Onyphe  Inetnum: short and long reports

sc-onyphe_datascan-short

sc-onyphe_datascan-long
Onyphe Datascan: long and short reports

Bug fixes and enhancements

  • #248: Improve error msg when VT Get Report does not have an entry for
  • #323: Fix an issue with HybridAnalysis analyzer filenames handler
  • #329: Enhance PassiveTotal Passive DNS report
  • #348: Fix a typo in URLhaus’s long.html
  • #356 : Force the use of Python3 in MISP analyzer
  • #362: Fix file not found issue and empty result set in CERT.at passive DNS analyzer

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.14.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Cortex4py 2 is Out!

Cortex, a free, open source software allows security analysts and threat hunters to analyze and enrich observables (IP addresses, hashes, domains, …) collected in the course of an investigation or received from third parties, for example through MISP, the de facto standard for threat sharing.

On March 29, 2018, we released Cortex 2, a major improvement over the previous version which brought, among other cool features, authentication, caching, multi-tenancy (RBAC) and rate limiting. Instead of deploying several Cortex 1 instances behind reverse proxies which would implement authentification, administrators can deploy a single Cortex 2, create multiple organizations and serve the needs of various information security populations while enjoying extra features.

On May 31, 2018, we published a brand new API guide so that developers can take advantage of the powerful REST API of the product. Sadly, Cortex4py, the FOSS Python library we provide to interact with the API was not compatible with Cortex 2. Until today.

Thanks to the hard work of our dear Nabil Adouani, we are happy to announce the immediate availability of Cortex4py 2.0.0, a complete rewrite of the library in Python 3. Cortex4py 2.0.0 is fully compatible with Cortex 2. However, it doesn’t work with Cortex 1.

While TheHive, the highly popular free and open source Security Incident Response Platform (SIRP) we develop has native support for many Cortex 2 instances, Python developers can leverage Cortex4py to interact with Cortex 2, manage organizations, users, analyzer configurations and analyze observables at scale from alternative SIRPs, SIEMs or custom scripts thanks to the 83 analyzers Cortex 2 has as of June 18, 2018.

Screen Shot 2018-06-18 at 20.01.27.png
Cortex 2: there is more than one way to interact with it

Use It

To install Cortex4py, use PIP3:

$ sudo -H pip3 install cortex4py

If you are using Python on a Windows operating system, please forgo the sudo command.

Usage

Cortex4py 2 comes with a usage guide which includes many examples. For example, if you want to fetch the last 10 successful jobs that have been executed against domain names and display the result summaries of those 10 jobs you could write something like:

Screen Shot 2018-06-18 at 19.58.45.png
Sample Python3 code to retrieve Cortex analyzer results

Migrating from Cortex4py 1

If you have already written scripts using Cortex4py 1.x (for Cortex 1), we tried to keep the already available methods. However, we recommend you adapt your code to leverage the new Cortex4py 2 classes and methods as soon as feasible. Moreover, the existing scripts must be updated to support authentication if you intend to use them with Cortex 2. Please read the Cortex4py 2 usage guide for more information.

Support

Cortex 2.0.0 is brand new software. As such, it might contain bugs and limitations. If you find any or encounter problems, please ask on our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

There are More than 80 Ways to Analyze Them

TheHive Project Chefs are pleased to announce the immediate availability of Cortex-Analyzers 1.10.1. To install this new release and benefit from 11 new analyzers and some fixes:

$ cd /path/to/Cortex-Analyzers
$ sudo git pull
$ for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

After running these commands, read the Analyzer Requirements Guide,  log into the Cortex 2 Web UI as an orgAdmin, click on the Refresh Analyzers button in the Cortex Web UI, configure the new analyzers and enjoy!

click.png
Cortex: click on Refresh Analyzers after updating to the latest Cortex-analyzers version

If you are using TheHive, get the last version of  the report templates and import them into TheHive.

New Analyzers

We have added 11 analyzers to this release, bringing the total to 53 (83 if we count all the flavors):

  1. Crtsh: contributed by crackytsi
  2. Cybercrime-Tracker: contributed by ph34tur3
  3. FireEye iSIGHT: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  4. GreyNoise: contributed by Nclose
  5. IBM X-Force: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  6. Malwares: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  7. MnemonicPDNS: contributed by Michael Stensrud from the Nordic Financial CERT
  8. StaxxSearch: contributed by Robert Nixon
  9. StopForumSpam: contributed by Marc-André Doll from STARC (by EXAPROBE)
  10. ThreatCrowd: contributed by Rémi Allain from Cyberprotect
  11. Unshortenlink: contributed by Rémi Pointel from CERT-BDF

Crtsh

Get Crt.sh certificate transparency lists associated with a domain name. Crt.sh is an online service operated by the Comodo Certificate Authority.

The analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screen Shot 2018-06-05 at 16.46.22.png

Screen Shot 2018-06-05 at 16.46.31.png
TheHive: Crtsh — Short and Long Report Samples

Cybercrime-Tracker

Use the Cybercrime-tracker.net service to assess whether an IP address, URL, domain, or FQDN has a C2 (Command & Control) entry in its database.

This analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screenshot from 2018-06-05 14-43-54.png

Screenshot from 2018-06-05 14-43-37.png
TheHive: CyberCrime-Tracker — Short and Long Report Samples

FireEye iSIGHT

Leverage FireEye iSIGHT Threat Intelligence to qualify domains, IP addresses, hashes and URLs.

This analyzer comes in only one flavor. You need a valid FireEye iSIGHT Threat Intelligence subscription to use the analyzer. Retrieve the API key associated with your account and provide it as a value to the key parameter. Obtain the password associated with the API key and provide it as a value to the pwd parameter.

Fireeye.png
TheHive: FireEye iSIGHT — Long Report Sample (courtesy of Andrea Garavaglia)

GreyNoise

Determine whether an IP has known scanning activity using GreyNoise.

This analyzer comes in only one flavor. It can be used out of the box without configuration. However, if you make many requests, you need to obtain an API key. Please contact GreyNoise to ask for one. Once you get the API key, provide it as the value of the key parameter.

Screenshot from 2018-06-05 14-48-02.png

Screenshot from 2018-06-05 14-47-42.png
TheHive: GreyNoise — Short and Long Report Samples

IBM X-Force

Query domains, IPs, hashes and URLs against IBM X-Force Threat Intelligence sharing platform.

This analyzer comes in only one flavor. Access to IBM X-Force Threat Exchange requires an IBM ID. Once you have access to the service, supply the URL of the service as value for the url parameter, the API key associated with your account as value for the key parameter and the associated password as the value of the pwd parameter.

IBM-XForce.PNG
TheHive: IBM X-Force — Long Report Sample (courtesy of Andrea Garavaglia)

Malwares

Query Malwares.com and get reports on files, hashes, domain names and IP addresses.

The analyzer comes in two flavors:
– Malwares_pDNS_GetReport: get the latest Malwares report for a file,
hash, domain or an IP address.
– Malwares_pDNS_Scan: scan a file or URL.

You need to sign up for a Malwares.com account. An API key to use the service’s API should be associated with your account. Supply it as the value of the key parameter.

Screen Shot 2018-06-05 at 16.56.38.png

Screen Shot 2018-06-05 at 16.56.57.png
TheHive: Malwares_GetReport — Short and Long Report Samples

MnemonicPDNS

Query IP addresses and domain names against Mnemonic Passive DNS service.

This analyzer comes in two flavors:

  • Mnemonic_pDNS_Public: query Mnemonic’s public service.
  • Mnemonic_pDNS_Closed: query Mnemonic’s closed service.

When using the public service, the analyzer can be used out of the box with no further configuration. When using the closed service, you need to contact Mnemonic to get an API key which you’ll need to supply as the value of the key parameter.

Screenshot from 2018-06-05 14-46-40.png

Screenshot from 2018-06-05 14-46-19.png
TheHive: Mnemonic PDNS — Short and Long Report Samples

StaxxSearch

Fetch observable details from an Anomali STAXX instance.

This analyzer comes in only one flavor. You need to install an Anomali STAXX instance or to have access to one to use the analyzer. Supply the following parameters to the analyzer in order to use it:

  • auth_url: URL of the authentication endpoint.
  • query_url: URL of the intelligence endpoint.
  • username: the STAXX user name.
  • password: the STAXX password.
  • cert_check: boolean indicating whether the certificate of the endpoint must be checked or not.
  • cert_path: path to the CA on the system to validate the endpoint’s certificate if cert_check is true.

image0010.png

image002.png
TheHive: StaxxSearch — Short and Long Report Samples (courtesy of Robert Nixon)

StopForumSpam

Query StopForumSpam to check if an IP or email address is a known spammer.

You need to define the thresholds above which the analyzed observable should be marked as suspicious or malicious.

Screen Shot 2018-06-05 at 18.32.08.png

Screen Shot 2018-06-05 at 18.32.21.png
TheHive: StopForumSpam — Short and Long Report Samples

ThreatCrowd

Look up domains, mail and IP addresses on [ThreatCrowd(https://www.threatcrowd.org/), a service powered by AlienVault.

This analyzer comes in only one flavor. No configuration is needed. It can be used out of the box.

Screenshot from 2018-06-05 14-45-29.png

Screenshot from 2018-06-05 14-45-11.png
TheHive: Threatcrowd — Short and Long Report Samples

Unshortenlink

Follow redirects of shortened URLs to reveal the real ones.

This analyzer comes in only one flavor.

No configuration is required. It can be used out of the box.

Warning: using this analyzer without extra caution might lead to unexpected consequences. For example, if the URL you are seeking to unshorten is an attacker-controlled one, you may end up leaving undesired traces in the threat actor’s infrastructure logs. The TLP values Cortex allows you to configure to prevent the use of an analyzer if the TLP associated with an observable is above the authorized level won’t be of much help since Unshortenlink have to access the shortened URL. Please do not activate this analyzer unless you (and your fellow analysts) know what they are doing.

Pasted image at 2018_06_05 09_07 PM.png

Screen Shot 2018-06-05 at 21.05.45.png
TheHive: Unshortenlink — Short and Long Report Samples

Additional Enhancements

  • YARA analyzer had a bug which was fixed in version 1.9.7 of the Cortex-analyzers repository. If you install 1.10, you’ll obviously benefit from the fix 😉
  • A permission problem that prevented using the Cuckoo Sandbox analyzer was corrected (thanks Felix Bauer!)

Support

Something does not work as expected? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

CorrectionJune 6, 2018
An earlier version of this post mentioned that GreyNoise needs an API key. That’s only necessary if a certain level of requests are made. Also, to configure analyzers, you need to be orgAdmin.

Unveiling Cortex 2

TheHive Project’s Master Chefs are extremely happy to share, for free their latest recipe with the Cyber Threat Intelligence, Digital Forensics and Incident Response communities: Cortex 2.

As its predecessor, Cortex 2 is published under an AGPL v3 license and it introduces many important features that we brushed upon in a Dec 2017 post.

Screen Shot 2017-12-15 at 17.16.06
Cortex 2 — Architecture

Update: Cortex 2.0.1 was released since this post went live. It corrects a few bugs we uncovered in 2.0.0 as described in the changelog. Please install Cortex 2.0.1 instead of 2.0.0.

Authentication

Cortex 2 supports all the authentication methods that TheHive supports: LDAP, Active Directory, local accounts, API Keys, and X.509 SSO.

To connect your favorite Security Incident Response Platform with Cortex 2, you will need to update TheHive to Cerana 0.7 (TheHive 3.0.7) which was released today as well. This version fixes a regression pertaining to case templates introduced by Cerana 0.6 and is the first version to fully support Cortex 2’s API changes and authentication.

To make TheHive 3.0.7 analyze observables at scale through Cortex 2, you have to create an account on Cortex 2 with the read and analyze roles (see the next section) and generate the associated API Key. Next, feed the key in TheHive’s /etc/thehive/application.conf as described in the documentation et voilà !

TheHive 3.0.7 remains compatible with Cortex 1 and you can connect it to a mixed set of Cortex 1 and/or Cortex 2 instances with no issues.

Organizations, Analyzers and Rate Limiting

Cortex 2 introduces multi-tenancy through organizations and each organization can have its own set of users, with different roles, its own set of analyzers and, if necessary, rate limits that will prevent analysts from burning quotas.

Multi-tenancy has several interesting use cases. For instance, if you are the CSIRT or CERT of a large multinational organization with several regional teams, you can create an organization for each region within your constituency and enable the analyzers that they may need to use. Let’s assume that you bought a VirusTotal subscription that limits you to 5000 requests per month. You can configure the corresponding analyzers to give each region a fair share of that quota and keeping some requests for your own use.

In case you are a commercial CSIRT or an MSSP, you could do the same for your customers by installing only one Cortex 2 instance and creating an organization for each customer.

Screen Shot 2018-03-29 at 16.27.05.png
Configure an analyzer graphically and impose rate limits if necessary

User Roles

By default, Cortex 2 is shipped with the default cortex organization which sole purpose is to create other ones and manage the users within each organization and their associated powers. The cortex organization hosts all users with the superAdmin role and it cannot be used to configure or run analyzers.

As described in the new Quick Start Guide, after installing Cortex 2, updating its database and creating the first user who will have super admin powers, you’ll have to create your first organization and at least one user within that organization with orgAdmin rights.

Screen Shot 2018-03-29 at 16.33.02
Create an organization

You can then log out and log in using the orgAdmin account to create further users within that organization, enable and configure analyzers etc. Please note that no analyzer is enabled by default and you need at least v 1.9.0 of the cortex-analyzers repository. To update your set of analyzers to 1.9.0, please run git pull.

Screen Shot 2018-03-29 at 16.28.47
Manage users within an organization

Besides the superAdmin and orgAdmin roles, Cortex 2 introduces the read role which allows users to access analyzer reports and read them but not execute analyzers. For that, users need the analyze role (which implies the read role). orgAdmin users can also run analyzers. superAdmin users are limited to the default cortex organization. While they can create organizations and manage users within them, they cannot access analyzer configurations such as confidential API keys or job reports.

Screen Shot 2018-03-29 at 16.31.28
Job reports

Report Persistence and Caching

Cortex 2 relies on Elasticsearch 5.x to store many configuration items but also all the analyzer reports that have been generated. Unlike its predecessor, you won’t lose your existing reports should you need to restart the service or the host it is running on.

Cortex 2 also introduces report caching. By default the cache.job parameter is set to 10 minutes in /etc/cortex/application.conf. That means that if an analysis on a given observable with a defined TLP is requested and that a report has been previously generated in the last 10 minutes, Cortex 2 will serve that report instead of running a new analysis. This feature can help prevent soliciting analyzers, particularly those which require a subscription or have quotas, when there is no need to do so. Please note that this parameter is global to all the analyzers and all the organizations that are configured in the Cortex 2 instance. We do have plans to make it more granular in future versions.

Migrating from Cortex 1

If you are migrating from Cortex 1.x, we recommend that you:

  1. Save the configuration of your analyzers (which ones are enabled and what their configuration items are, such as users/passwords or API keys).
  2. Install Cortex 2.
  3. Edit /etc/cortex/application.conf to add the secret key as shown in Step 1 of the Quick Start Guide and point Cortex to the location of the analyzers.
  4. Follow the remaining steps of the Quick Start Guide to enable the analyzers you need and reinject their configuration.

What’s Next?

In the upcoming weeks, we will release a new version of Cortex4py in order to make it compatible with Cortex 2, continue the work we started with our MISP Project friends to support MISP attribute enrichment through Cortex 2 (MISP currently only supports enrichment using Cortex 1), and perform a long-overdue overhaul of our documentation.

Feeling Generous? Donate!

As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.

All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.

So if you are feeling generous, please contact us at support@thehive-project.org.

Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Correction: March 30, 2018
Instructions on how to update the cortex-analyzers have been added. Also, Cortex 2.0.1 was released to correct a few bugs in the previous version since this post went live.