New Year, New Analyzers

Dear fellow incident handlers and cybercrime fighters around the world, the galaxy, the known and the unknown universe, first and foremost, all TheHive Project’s team would like to wish a wonderful new year 2019 to you and to your cherished relatives. We truly hope that eagles, pandas, kittens, babars, bears and all sorts of animals will stay out of the way. And remember that you don’t need to go bankrupt by purchasing so-called Next Gen™ magical solutions that work only when there’s a full moon and the page number of the book you are currently reading is 42 to investigate threats 😉

We would like to begin the year by introducing version 1.15.0 of Cortex analyzers, bringing the total number of analyzers to a whopping 113! And thanks to Kyle Parrish, this release improves the Mailer responder to allow you to specify a custom port number for your SMTP server and adds a new one to blacklist observables on Cisco Umbrella utilizing the Enforcement API. The Cisco Umbrella Blacklister responder will then add the tag Umbrella:blockedto the observable.

Cortex-Analyzers 1.15.0 also include fixes and enhancements for Eml_Parser, IBM X-Force, Fortiguard, and Shodan. Most of these modifications were contributed by our continuously growing user community. Thanks to all of those who help us in our mission to provide free and open source security incident response tools to the masses!

Please read the relevant sections in the Cortex installation guide to install or update your analyzers and responders in order to benefit from all this sweet & tasty honey.

New Analyzers

The following analyzers have been added:

Cyberprotect

This analyzer lets you query the Cyberprotect ThreatScore service for domains and IP addresses. No configuration is needed and it can be used out of the box.

TheHive displays the analyzer results as follows:

Have I Been Pwned

The HIBP_Query analyzer lets you check email addresses on Have I Been Pwned. You can use an optional parameter to include unverified breaches in the search results. Otherwise, it can be used without any additional configuration.

When called from TheHive, results would display as such:

PatrOwl

As it name states, The Patrowl_GetReport analyzer will let you get the current PatrOwl report for a FQDN, a domain name or an IP address. You need a running PatrOwl instance or to have access to one to use the analyzer.

If you fire it from TheHive, it would display results as follows:

SecurityTrails

This analyzer comes in two flavors in order to get Whois data and Passive DNS details using SecurityTrails. To use both flavors, you will need an account for the service to retrieve the associated API key, which you need to configure the analyzers.

SecurityTrails_Passive_DNS displays results in TheHive as follows:

The Whois variant produces reports such as:

Cisco Umbrella

In addition to Cisco Umbrella Investigate, you can now query the Umbrella Reporting API for recent DNS queries and their status for a domain name using the new Umbrella_Report analyzer.

New Shodan Flavors

In addition to Shodan_Host and Shodan_Search, which allow you to obtain Shodan information on a host and the search results for a domain name, now you can get domain resolutions (Shodan_DNSResolve), obtain scan history results for an IP address (Shodan_Host_History), get information on a domain (Shodan_InfoDomain) and the reverse DNS resolutions for an IP address (Shodan_ReverseDNS).

DomainTools

The following DomainTools flavors were added to this release:

  • DomainTools_HostingHistory: get a list of historical registrant, name servers and IP addresses for a domain.
  • DomainTools_ReverseIPWhois: get a list of IP addresses which share the same registrant information. It applies to a mail, IP, or domain.

Moreover, please note that DomainTools_WhoisLookup now handles IP addresses in addition to domains and provides parsed results. DomainTools_WhoisLookup_IP is thus not needed anymore. Instead, DomainTools_WhoisLookupUnparsed has been added to do the same as DomainTools_WhoisLookup, except that the output results are unparsed.

Cortex 2.1.3: Security and Beyond

TheHive Project’s code Chefs are happy to announce the immediate availability of Cortex 2.1.3, a hotfix for your favorite observable and response engine, fresh out of the oven!

We highly recommend that you upgrade your existing installation to this new version as soon as feasible as it plugs a significant security vulnerability, kindly reported by Po-Hsing Wu. The vulnerability is a privilege escalation one which allows an orgadmin to create a superadmin user. The culprit has been punished by having to chant Perl mantras while doing a handstand on burning coals.

security_holes
Source : XKCD

Additionally, Cortex 2.1.3 fixes the following bugs:

  • #157: list and disable invalid responders
  • #152: enforce PAP when launching an analyzer from the Cortex Web UI
  • #147: add dig to the Cortex docker image as the SinkDB analyzer needs it
  • #146: the Cortex job list must display the PAP value
  • #145: fix the broken Web UI’s search function for job history

Pardon my French but do you speak English?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cortex 101: Dissecting Observables a Hundred Ways

TheHive Project’s Code Chefs are glad to announce that, thanks to the precious contributions of the user community, Cortex has broken the one hundred analyzer mark.

Cortex-Analyzers version 1.14.0 is out and includes new analyzers, some improvements and some bug fixes.

New Analyzers

New and enhanced analyzers, described below, are:

  • Cisco Investigate by Cisco Umbrella Research @opendns
  • Proofpoint Forensics Lookup by Emmanuel Torquato
  • Proofpoint Threat Insight Forensics Analyzer by Emmanuel Torquato
  • RecordedFuture by jojoob
  • urlscan.io search by ninoseki
  • Google DNS over HTTP by 0xswitch
  • RTF files support in FileInfo by Nils Kuhnert
  • Datascan and Inetnum flavors in Onyphe analyzer by Pierre Baudry and Adrien Barchapt

Again, huge thanks for the awesome work that has been performed by all our contributors!

Investigate

Cisco Umbrella Investigate provides threat intelligence about domains and IP addresses accross the Internet. The analyzer can be used to query the Cisco Umbrella (formerly OpenDNS) API and get information about an IP or a domain name. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-investigate-short

sc-investigate-long
Cisco Investigate: short and long reports

Proofpoint Forensics Lookup

According to Proofpoint’s website, the Forensics API allows insight in detailed forensic evidences about individual threats or compaigns. The analyzer can be used to check observables against given indicators of compromise stored in the ProofPoint service.

Unfortunately, there are currently no sample report screenshots available, because TheHive’s Core Team does not have access to Proofpoint services. Also, due to the same reason, this analyzer could not be tested by us. If you have access to the service and can test the analyzer and/or provide report screenshots, please let us know.

RecordedFuture

This analyzer lets you get the latest risk data from RecordedFuture for a hash, domain or an IP address. It can be used to query the API and get information. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-recordedfuture-short

sc-recordedfuture-long
RecordedFuture: short and long reports

Urlscan.io search

Urlscan.io is a service that scans and analyzes websites. Submitted pages will be browsed like a regular user would do and every activity gets recorded. The analyzer submitted by ninoseki queries urlscan without initiating a scan which would be publicly visible on the website. Accepted datatypes for this analyzer are URL, domain, hash and IP.

The templates which display the results of the analyzer look like the following screenshots:

Screenshot from 2018-10-25 09-55-34

Screenshot from 2018-10-25 09-48-58
Urlscan.io: short and long reports

Google DNS over HTTP

This analyzer provides DNS information for an IP, a domain or a FQDN by making calls to Google DNS-over-HTTP (DoH). No API key is required.

Results are displayed in TheHive in the following manner:sc-googleDNS-short

sc-googleDNS-long
Google DNS: short and long reports

RTF files support in FileInfo

The FileInfo meta analyzer has been improved and now leverages the rtfobj tool provided in the Oletools suite by Decalage.

Results are displayed in TheHive in the following manner:

sc-fileinfo_RTF-short

sc-rtfobj-long
FileInfo with rtfobj: short and long reports

Datascan and Inetnum flavors in Onyphe analyzer

The Onyphe analyzer has been enhanced with two new flavors. Datascan provides information about known open ports on a specific IP, and Inetnum enumerates all known network information about the analyzed IP address.

An API key is required to use the analyzer and can be obtained by creating an account on the Onyphe website.

Results are displayed in TheHive in the following manner:

sc-onyphe-inetnum-short

sc-onyphe_inetnum-long
Onyphe  Inetnum: short and long reports

sc-onyphe_datascan-short

sc-onyphe_datascan-long
Onyphe Datascan: long and short reports

Bug fixes and enhancements

  • #248: Improve error msg when VT Get Report does not have an entry for
  • #323: Fix an issue with HybridAnalysis analyzer filenames handler
  • #329: Enhance PassiveTotal Passive DNS report
  • #348: Fix a typo in URLhaus’s long.html
  • #356 : Force the use of Python3 in MISP analyzer
  • #362: Fix file not found issue and empty result set in CERT.at passive DNS analyzer

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.14.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

TheHive 3.1.2 & Cortex 2.1.2 Released

We could not leave for the week-end without issuing a minor release or two so here we go.

TheHive 3.1.2

Starting from TheHive 3.0.1, an administrator has the ability to configure Cortex job polling by defining the time between two polls thanks to the cortex.refreshDelay parameter as well as the number of consecutive failures before giving up (via cortex.MaxRetryOnError). However, these settings prevent the service from starting correctly. TheHive 3.1.2 corrects this issue.

Cortex 2.1.2

When running a job in Cortex with the exact same details, the function findSimilarJob is called. It should return results from any previous jobs, but in the latest versions (2.1.0, 2.1.1) it does not because of a change that went past our QA.

In a similar fashion, the GUI search function was broken. Cortex 2.1.2 fixes both issues.

Excuse my French but I Need Help

Keep calm. We speak French. So if you encounter any difficulty to update TheHive or Cortex, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are always ready to help as does our user community.

TheHive4py 1.5.1 Released

When you need to interact with TheHive’s REST API and you ain’t shy of working with Python, TheHive4py is the way to go. It’s a free, open source library we provide to allow you to easily create alert feeders, automate certain tasks like creating cases, assign them to analysts and much more. For example, Synapse, DigitalShadows2TH and Zerofox2TH leverage the library to send alerts to your favourite SIRP/SOAR.

Sometime ago, we decided that it was time to overhaul the whole library and we began working on version 2.0.0 which will be easier to use. It should also support the full set of TheHive’s REST API calls. In the meantime we decided to release version 1.5.0, shortly followed by version 1.5.1 to support some new functionality contributed by our user community and correct a few issues.

code_quality
Source : XKCD

New Features Introduced in 1.5.0

Bugfixes Introduced in 1.5.0

  • #80: Prevent max recursion depth exceeded error, contributed by Psynbiotik

New Features Introduced in 1.5.1

Important note: TheHive4py 1.5.1 does not work with TheHive 3.0.10 or earlier versions. Please stick with 1.5.0 if you are using those versions.

Updating/Installing

To update your existing package:

$ sudo pip install thehive4py --upgrade

If you are just getting started with TheHive4py, you can forgo the --upgrade at the end of the command above.

But I just Wanna Play!

If you’d like to play around with TheHive4py 1.5.1, TheHive 3.1.1., Cortex4py 2.0.1 and Cortex 2.1.1, please download the training VM.

Paris? Are you There?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. As usual, we’ll be more than happy to help!

Correction: October 12, 2018
As reported by Robin Hahling, TheHive 1.5.1 does not work with TheHive 3.0.10 or earlier versions.

Cortex-Analyzers 1.10.2

On June 6, 2018, we released Cortex-Analyzers 1.10, which contained 11 new analyzers, bringing the total to 83 programs. You read that correctly: 83 ways to assess and gain insight on observables collected during the course of an investigation or while performing threat intelligence thanks to Cortex, our free & open source analysis engine. One day after, we published version 1.10.1 which fixed the name of the Anomali STAXX reports for TheHive. Since then we uncovered a few additional issues which version 1.10.2 corrects:

  • #273 & #283: two encoding issues in the ThreatCrowd JSON definition file
  • #275: the GreyNoise mini-report for TheHive did not produce any information when no record was found
  • #278: the IBM X-Force analyzer forced the TLP in its configuration
  • #279: the VMRay should authorize self-signed certificates
  • #280: the name of the IBM X-Force Exchange report folder was incorrect preventing TheHive from displaying mini-reports or usefully laid out long ones

To install this hotfix release on your Cortex instance:

$ cd /path/to/Cortex-Analyzers
$ sudo git pull

Then log in to the Cortex Web UI as an orgAdmin and click on Refresh Analyzers:

click

If you are using TheHive, make sure to download the latest version of the report templates and import them into your instance.

Support

Something does not work as expected? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

There are More than 80 Ways to Analyze Them

TheHive Project Chefs are pleased to announce the immediate availability of Cortex-Analyzers 1.10.1. To install this new release and benefit from 11 new analyzers and some fixes:

$ cd /path/to/Cortex-Analyzers
$ sudo git pull
$ for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

After running these commands, read the Analyzer Requirements Guide,  log into the Cortex 2 Web UI as an orgAdmin, click on the Refresh Analyzers button in the Cortex Web UI, configure the new analyzers and enjoy!

click.png
Cortex: click on Refresh Analyzers after updating to the latest Cortex-analyzers version

If you are using TheHive, get the last version of  the report templates and import them into TheHive.

New Analyzers

We have added 11 analyzers to this release, bringing the total to 53 (83 if we count all the flavors):

  1. Crtsh: contributed by crackytsi
  2. Cybercrime-Tracker: contributed by ph34tur3
  3. FireEye iSIGHT: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  4. GreyNoise: contributed by Nclose
  5. IBM X-Force: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  6. Malwares: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  7. MnemonicPDNS: contributed by Michael Stensrud from the Nordic Financial CERT
  8. StaxxSearch: contributed by Robert Nixon
  9. StopForumSpam: contributed by Marc-André Doll from STARC (by EXAPROBE)
  10. ThreatCrowd: contributed by Rémi Allain from Cyberprotect
  11. Unshortenlink: contributed by Rémi Pointel from CERT-BDF

Crtsh

Get Crt.sh certificate transparency lists associated with a domain name. Crt.sh is an online service operated by the Comodo Certificate Authority.

The analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screen Shot 2018-06-05 at 16.46.22.png

Screen Shot 2018-06-05 at 16.46.31.png
TheHive: Crtsh — Short and Long Report Samples

Cybercrime-Tracker

Use the Cybercrime-tracker.net service to assess whether an IP address, URL, domain, or FQDN has a C2 (Command & Control) entry in its database.

This analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screenshot from 2018-06-05 14-43-54.png

Screenshot from 2018-06-05 14-43-37.png
TheHive: CyberCrime-Tracker — Short and Long Report Samples

FireEye iSIGHT

Leverage FireEye iSIGHT Threat Intelligence to qualify domains, IP addresses, hashes and URLs.

This analyzer comes in only one flavor. You need a valid FireEye iSIGHT Threat Intelligence subscription to use the analyzer. Retrieve the API key associated with your account and provide it as a value to the key parameter. Obtain the password associated with the API key and provide it as a value to the pwd parameter.

Fireeye.png
TheHive: FireEye iSIGHT — Long Report Sample (courtesy of Andrea Garavaglia)

GreyNoise

Determine whether an IP has known scanning activity using GreyNoise.

This analyzer comes in only one flavor. It can be used out of the box without configuration. However, if you make many requests, you need to obtain an API key. Please contact GreyNoise to ask for one. Once you get the API key, provide it as the value of the key parameter.

Screenshot from 2018-06-05 14-48-02.png

Screenshot from 2018-06-05 14-47-42.png
TheHive: GreyNoise — Short and Long Report Samples

IBM X-Force

Query domains, IPs, hashes and URLs against IBM X-Force Threat Intelligence sharing platform.

This analyzer comes in only one flavor. Access to IBM X-Force Threat Exchange requires an IBM ID. Once you have access to the service, supply the URL of the service as value for the url parameter, the API key associated with your account as value for the key parameter and the associated password as the value of the pwd parameter.

IBM-XForce.PNG
TheHive: IBM X-Force — Long Report Sample (courtesy of Andrea Garavaglia)

Malwares

Query Malwares.com and get reports on files, hashes, domain names and IP addresses.

The analyzer comes in two flavors:
– Malwares_pDNS_GetReport: get the latest Malwares report for a file,
hash, domain or an IP address.
– Malwares_pDNS_Scan: scan a file or URL.

You need to sign up for a Malwares.com account. An API key to use the service’s API should be associated with your account. Supply it as the value of the key parameter.

Screen Shot 2018-06-05 at 16.56.38.png

Screen Shot 2018-06-05 at 16.56.57.png
TheHive: Malwares_GetReport — Short and Long Report Samples

MnemonicPDNS

Query IP addresses and domain names against Mnemonic Passive DNS service.

This analyzer comes in two flavors:

  • Mnemonic_pDNS_Public: query Mnemonic’s public service.
  • Mnemonic_pDNS_Closed: query Mnemonic’s closed service.

When using the public service, the analyzer can be used out of the box with no further configuration. When using the closed service, you need to contact Mnemonic to get an API key which you’ll need to supply as the value of the key parameter.

Screenshot from 2018-06-05 14-46-40.png

Screenshot from 2018-06-05 14-46-19.png
TheHive: Mnemonic PDNS — Short and Long Report Samples

StaxxSearch

Fetch observable details from an Anomali STAXX instance.

This analyzer comes in only one flavor. You need to install an Anomali STAXX instance or to have access to one to use the analyzer. Supply the following parameters to the analyzer in order to use it:

  • auth_url: URL of the authentication endpoint.
  • query_url: URL of the intelligence endpoint.
  • username: the STAXX user name.
  • password: the STAXX password.
  • cert_check: boolean indicating whether the certificate of the endpoint must be checked or not.
  • cert_path: path to the CA on the system to validate the endpoint’s certificate if cert_check is true.

image0010.png

image002.png
TheHive: StaxxSearch — Short and Long Report Samples (courtesy of Robert Nixon)

StopForumSpam

Query StopForumSpam to check if an IP or email address is a known spammer.

You need to define the thresholds above which the analyzed observable should be marked as suspicious or malicious.

Screen Shot 2018-06-05 at 18.32.08.png

Screen Shot 2018-06-05 at 18.32.21.png
TheHive: StopForumSpam — Short and Long Report Samples

ThreatCrowd

Look up domains, mail and IP addresses on [ThreatCrowd(https://www.threatcrowd.org/), a service powered by AlienVault.

This analyzer comes in only one flavor. No configuration is needed. It can be used out of the box.

Screenshot from 2018-06-05 14-45-29.png

Screenshot from 2018-06-05 14-45-11.png
TheHive: Threatcrowd — Short and Long Report Samples

Unshortenlink

Follow redirects of shortened URLs to reveal the real ones.

This analyzer comes in only one flavor.

No configuration is required. It can be used out of the box.

Warning: using this analyzer without extra caution might lead to unexpected consequences. For example, if the URL you are seeking to unshorten is an attacker-controlled one, you may end up leaving undesired traces in the threat actor’s infrastructure logs. The TLP values Cortex allows you to configure to prevent the use of an analyzer if the TLP associated with an observable is above the authorized level won’t be of much help since Unshortenlink have to access the shortened URL. Please do not activate this analyzer unless you (and your fellow analysts) know what they are doing.

Pasted image at 2018_06_05 09_07 PM.png

Screen Shot 2018-06-05 at 21.05.45.png
TheHive: Unshortenlink — Short and Long Report Samples

Additional Enhancements

  • YARA analyzer had a bug which was fixed in version 1.9.7 of the Cortex-analyzers repository. If you install 1.10, you’ll obviously benefit from the fix 😉
  • A permission problem that prevented using the Cuckoo Sandbox analyzer was corrected (thanks Felix Bauer!)

Support

Something does not work as expected? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

CorrectionJune 6, 2018
An earlier version of this post mentioned that GreyNoise needs an API key. That’s only necessary if a certain level of requests are made. Also, to configure analyzers, you need to be orgAdmin.