TheHive 3.4.0 & Cortex 3.0.0 Released

For many months, we have been concentrating our efforts on TheHive 4, the next major version of your favourite Security Incident Response Platform, which we’ll finally provide RBAC (or multi-tenancy if you prefer), a feature that Cortex had for quite some time now.

Source : dilbert.com © Scott Adams

As you well know, both TheHive and Cortex rely on Elasticsearch (ES) for storage. The choice of ES made sense in the beginning of the project but as we added additional features and had new ideas to give you the best experience possible, we faced several ES quirks and shortcomings that proved challenging if not outright blocking for making our roadmap a reality, including RBAC implementation in TheHive, a far more complex endeavour than RBAC in Cortex. Transitioning from ES to graph databases was necessary and since we want our existing users to have a smooth migration path, TheHive 4 (the first release candidate should come out of the oven by the end of the year) will support both ES and graph databases.

But while we were focusing on that, we completely lost sight of the end of life of ES 5.6 so we wrote an apology to you, our dear users, back in May.

Shortly after, we released TheHive 3.4.0-RC1, to add support for ES 6 (with all the breaking changes it has introduced). We also did the same for Cortex with the release of Cortex 3.0.0-RC3. We also took that opportunity to clear out some AngularJS technodebt we had.

We then asked you to take them for a spin and report back any bugs you find given that both versions had to support ES 5.6 and ES 6 to allow for proper migration.

After a few rounds of release candidates, we are pleased to announce the immediate availability of TheHive 3.4.0 and Cortex 3.0.0 as stable releases.

Before upgrading your existing software to these new versions, please make sure to read the blog post we wrote back in June. We invite you to pay great attention to the regressions that we were forced to introduce because of ES 6.

You should also note that, in addition to ES 6 support, Cortex 3.0.0 supports fully dockerised analyzers and responders. We’ll elaborate on this in a future blog post soon.

Changelogs

If you are interested in some nitty-gritty details, we invite you to read the relevant changelogs since our last post on the subject:

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help as usual!

More than 120 ways of Analyzing your Observables

There’s a new version of Cortex-Analyzers in cybertown and its has an awesome, mind-blowing name and that’s… wait for it… wait for it: 2.1.0.

In this new release, we added two analyzers which bring the total number to more than 120:

We could not duly test DNSSinkhole since we do not have access to the associated service. So we would really appreciate it if you could test it and let us know whether it works or not.

Others analyzers have been fixed or improved:

DNSSinkhole

This analyzer lets you check if an IP address has been registered in your DNS sinkhole. TheHive displays the analyzer results as follows:

DNSSinkhole analyzer: long report
DNSSinkhole analyzer: short report

TalosReputation

This analyzer lets you determine whether an IP address has been reported as a threat on Cisco Talos Intelligence service. No special access to the service is required to run the analyzer.

TheHive displays the analyzer results as follows:

Talos Reputation: long report
Talos Reputation analyzer : short report

Crt.sh

This analyzer has been enriched to display SHA-1 fingerprints. The long report format has been updated to reflect this new information.

Crt.sh analyzer: long report

FileInfo

FileInfo has been updated and is now able to parse PDF files and extract IOCs such as URLs, hosts, domains, IPs, hashes and many more.The analyzer does also support the last version of the extract-msg library.

FileInfo analyzer: IOC Parser long report
FileInfo analyzer: IOC Parser short report

VirusTotal and Python3

The VirusTotal analyzer, including all its flavours, now uses Python3 and an updated virustotal-api library.

Yeti API key

An optional API key can now be configured and used by the Yeti analyzer.

Malwares_GetReport

A hash computation has been fixed in this analyzer.

EMLParser

A first fix has been introduced to avoid this analyzer to crash when there is no content-description in content_header, and a second has been added to correct a header display issue.

IBM XForce Lookup

The analyzer has been improved to allow users to add a trailing / at the end of the API URL without breaking everything.

Updating your Analyzers in Cortex 2.x

Each analyzer and responder comes with its own, pip compatible requirements.txt file. Run the following commands to update your Cortex analyzers to the latest version:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Dockerised Analyzers

Cortex 3.x gives you the opportunity to run dockerised analyzers and responders. This means that you no longer have to download all the git repository of Cortex-Analyzers and run lengthy commands to update your analyzers and responders.

If you want to use dockerised analyzers and responders, ensure that the URL of the catalog.json file corresponding to the Cortex-Analyzers repository is registered in application.conf. Please note that this won’t work if you are tracking the stable catalog.

After doing so, do not forget to login to Cortex as an orgadmin, click on the Refresh Analyzers button, then Disable and Enable again each analyzer and responder. Analyzer (and responder) updates should occur automatically as long as docker.autoUpdate is set to true in application.conf (this is the default setting).

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Searching for an Elastic? Here, Take 6!

As we announced on May 14, 2019, we have been working very hard to add Elasticsearch 6 support to TheHive and Cortex as Elasticsearch 5.x went the way of the dodo when Elastic plugged life support off this venerable version. We also took this occasion to upgrade AngularJS and its sub projects to 1.7.8, the latest 1.x version as of this writing. Additionally, Grunt build dependencies have also been updated to their latest compatible versions.

It took us more time than initially foreseen but hey, we all love deadlines. We all love the whooshing noise they make as they go by.

TheHive 3.4.0-RC1 and Cortex 3.0.0-RC3 are now available on every Internet pipe near you and before you take them for a spin to help us identify any issues to make the stable releases rock-solid, let us walk you through some important information. Relax and grab a drink (and send good wine our way, we can always use some!).

Source: https://dilbert.com/strip/1995-11-10

TheHive 3.4.0-RC1

In addition to ES5 and 6 support and the update of AngularJS, this version corrects a few bugs that were identified in the latest stable version (3.3.1) and adds a few features. The most important one in our opinion is the ability to import a file from a Cortex report. This requires Cortex 3.0.0-RC3. The full list of changes is available at the following location.

Prior to migrating to 3.4.0-RC1, please read the migration guide.

Cortex 3.0.0-RC3

ES5 and ES6 support, AngularJS et cetera et cetera. Well you know the song right? Not quite as Cortex 3.0.0 significantly facilitates analyzer and responder installation and updates, thanks to Docker as we touched upon in a blog post earlier this year.

As detailed in the Cortex migration guide, which we recommend you read thoroughly, you can migrate from Cortex 2 and keep using analyzers and responders the same way (using processes), use the new Docker-based analyzers and responders or mix and match between running processes and docker containers (but then, you gotta pay extra attention to configure properly which analyzer/responder runs in which fashion).

Moreover, if you use the new dockerised analyzers and responders, you will be able to choose if you want to have them autoupdated (that’s the default behaviour) and if so, pick the bleeding edge, potentially buggy versions, the minor releases or, if you are risk-averse, stick with stable ones.

Cortex 3.0.0-RC3 also adds the ability to retrieve files resulting from analyzer jobs and last but not least, corrects an information disclosure bug that allowed non-admin users to retrieve the details of other users through the API. The vulnerability was reported by Adam Maris so kudos to him!

Warning: Regressions Ahead!

As outlined in our previous post about these new versions:

  • TheHive 3.4.0-RC1 and Cortex 3.0.0-RC3 use HTTP transport (9200/tcp by default) to connect to Elasticsearch instead of its native binary protocol (9300/tcp by default).
  • SSL/TLS, including when using a client certificate, can be configured to connect securely to ES. However this has not been tested yet.
  • Support of X-Pack and Search Guard is discontinued for anything but basic and SSL client authentication, which would still work.

Caution: Performance May Take a Hit!

The parent-child relationships we use behind the scene in Elasticsearch could make queries significantly slower with ES 6 and in our limited testing, we had the impression that performance took a hit. So please be cautious there and we’d be grateful if you could report any sluggishness you notice during your tests of the new versions with ES6.

A Short Story of Getting Work Done: TheHive 3.3.0

Some of our die-hard fans noticed that we silently released TheHive 3.3.0 a few days ago, after six release candidates. Well. Silently won’t be the right word to use in this case as we are drowning under work and feature requests and we sometimes postpone communication in favour of getting true real work done.

So, without any further ado, we are happy to announce the official availability of our latest (and of course greatest) release of the most-advanced, next-gen, HI (Human Intelligence), gluten-free, (add here any keyword that you fancy to help us get the Gartner attention and land in the Magic Quadrant™), free and open source Security Incident Response Platform Security Orchestration Automation & Response Platform.

As stated earlier, TheHive 3.3.0 went through the largest number of release candidates to date in order to ensure it contains more features than bugs (or unexpected functionality as our dear Nabil call them sometimes).

Since RC5 which we have blogged about on Feb 26, 2019, below is an outline of the changes we made. Check TheHive Installation Guide for installation instructions.

The new Related Alerts tab in Case View introduced in 3.3.0-RC5

Fixed Issues

  • #899: fix a crashing issue encountered with Firefox
  • #907: dynamic (auto-refresh) of cases was broken in 3.3.0-RC5
  • #930: merging cases by CaseID was broken

Implemented Enhancements

  • #666: add support for filtering tags by prefix and wildcard search
  • #901: remove the possibility of creating cases from scratch (i.e. empty cases) when explicitly disabled by an admin
  • #908: add support for text-based widgets to the dashboards
  • #912: responders can now add tags to alerts when triggered thanks to the new AddTagToAlert operation

Looking for Help?

Something does not work as expected? Then please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org.

Since You are Here

TheHive and Cortex are a huge success. According to our estimates, there are about a hundred organisations of different sizes and locations using or testing them. And as the number of users grows, so does the number of features, professional service and support requests. 

We tried finding a solution to keep TheHive Project as healthy as possible. So we created Creative Source, a nonprofit organisation (NPO), in the hope that we could leverage it to hire more developers thanks to the generous donations of our large user community. Sadly, not everyone in this world is generous and altruistic. At the end, all but one company (yes, exactly one) trusted us enough to make a donation and get tailored services for its needs in return. Most of the others either did not reply to our proposals or explained that their procurement process does not accommodate working with NPOs.

As we informed you a few weeks ago, some members of our core team are finalising an alternative option to ensure not only the viability of TheHive and Cortex as FOSS products on the long run but the ability to provide professional training, support, and services without making highly bureaucratic, think-in-the-box-but-never-outside procurement departments freak out.

Expect to hear from us soon…

On Spring, Bees and Cortex-Analyzers 1.16.0

Spring is here and your favorite bees are busy buzzing flowers to prepare you the most palatable honey ever. In the meantime, we are pleased to announce the immediate availability of Cortex-Analyzers 1.16.0, which adds a new responder and three new analyzers to complete an already hefty collection, bringing the total to 117 analyzers and 3 responders!

Release Overview

One responder has been added:

  • FalconCustomIOC, contributed by Michael (#421). We don’t know the last name of Michael. That could be Jordan. Who knows?

Three analyzers have been added:

We could not duly test some of these additions due to lack of access to the associated services or to our legendary laziness. So we would really appreciate it if you could test them and let us know whether they work or not.

FalconCustomIOC

The FalconCustomIOC responder let you submit observables from alerts or cases to Crowdstrike Falcon Custom IOC service.

Crowdstrike Falcon is a paid service. An account and an API key are required to configure and run this responder.

AbuseIPDB

AbuseIPDB analyzer let you determine wether an IP has been reported as malicious or not to the AbuseIPDB web service.

An account and an API key is needed to configure and use this analyzer.

TheHive displays the analyzer results as follows:

AbuseIPDB: short report
AbuseIPDB: long report

BackscatterIO

The BackscatterIO analyzer lets you query the Backscatter.io service for IPs, networks or autonomous systems (AS). It comes in two flavors:

  • BackscatterIO_GetObservations: determine whether an observables has a known scanning activity
  • BackscatterIO_Enrichment: enrich your observables with additional information

TheHive displays this analyzer results as follow:

Backscatter.io:short report
Backscatter.io GetObservations: long report
Bascatter.io GetObservations: long report

SoltraEdge

SoltraEdge analyzer lets you query any observable against theSoltra Edge platform.

To configure and use this analyzer, an account, a token key and the base URL of a SoltraEdge server are needed.

TheHive displays this analyzers result as follow:

SoltraEdge: short report
SoltraEdge: long report

Get It While Supply Lasts!

Each analyzer and responder comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.16.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Here comes the Sun (and TheHive 3.3-RC5)

On February 10, 2019, we released TheHive 3.3-RC2. It contained new features such as bulk alert merging, alert sorting, observable tag autocompletion, exporting case tags to MISP & more. Since then your favourite French code Chefs have been beesy refining TheHive 3.3 through new release candidates while getting Cortex 3 ready for prime time.

Over the weekend, Nabil decided he was not working enough already during the week. So he drained his batteries to the very last drop to release TheHive 3.3-RC5 before he crashed headfirst into his bed for a long, reparative sleep. Cumulatively since RC2, we added several features and squashed 10 bugs as described below.

Note that release candidates are beta software. You can get TheHive 3.3-RC5 from the pre-release, beta repositories. As usual, we encourage you to test it and report any bugs or issues you spot so we can address them before the final release. 

Check TheHive Installation guide for further details.

New Features

  • #485: disable clickable widgets when editing a dashboard.
  • #820: provide direct access to MISP events for those alerts that have been generated from such events.
Direct access to the MISP event from which this alert has been generated from
  • #829: improve password-protected ZIP imports.
  • #831: add auto-completion to case and observable tags in their respective Details pages.
  • #838: when you add an observable that already exists in another case, indicate whether the existing observable is an IOC or not.
  • #870: provide a link to copy the alert ID to the clipboard which allows you to easily manipulate an alert through TheHive4py.
Now the Alert ID can be directly copied to the clipboard without resorting to your browser dev mode
  • #884: add a Related Alerts tab to the Case view.
The new Related Alerts tab in Case View
  • #888: add a new UI configuration admin section. One of the first use cases of this section consist in disabling creating empty cases (i.e. cases not associated with a template). It will be gradually improved with new use cases so speak your mind!
  • #893: disable the case template selection when trying to merge multiple alerts for which no case template exists.
  • #897: mouseover text for alert preview.

Fixed Bugs

  • #856: various fixes related to alert updates and, when applicable, associated cases.
  • #869: use the observable TLP instead of the case TLP to decide whether a Cortex responder could be executed or not.
  • #871: fix log messages for MISP synchronisation.
  • #874: fix typo in alert labels (credits: Zachary Priddy).
  • #877: fix Akka version mismatch (credits: Zachary Priddy).
  • #890: the Hide Empty Case button in the new UI admin section was broken.
  • #894: when clicking on a donut or a widget in the dashboards, this should take you to the search page with predefined filters, without generating errors.
  • #895: exclude responder jobs from search results.
  • #896: dashboard clicks are not correctly translated into tag filters.

Looking for Help?

Something does not work as expected? Then please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org.

New Year, New Analyzers

Dear fellow incident handlers and cybercrime fighters around the world, the galaxy, the known and the unknown universe, first and foremost, all TheHive Project’s team would like to wish a wonderful new year 2019 to you and to your cherished relatives. We truly hope that eagles, pandas, kittens, babars, bears and all sorts of animals will stay out of the way. And remember that you don’t need to go bankrupt by purchasing so-called Next Gen™ magical solutions that work only when there’s a full moon and the page number of the book you are currently reading is 42 to investigate threats 😉

We would like to begin the year by introducing version 1.15.0 of Cortex analyzers, bringing the total number of analyzers to a whopping 113! And thanks to Kyle Parrish, this release improves the Mailer responder to allow you to specify a custom port number for your SMTP server and adds a new one to blacklist observables on Cisco Umbrella utilizing the Enforcement API. The Cisco Umbrella Blacklister responder will then add the tag Umbrella:blockedto the observable.

Cortex-Analyzers 1.15.0 also include fixes and enhancements for Eml_Parser, IBM X-Force, Fortiguard, and Shodan. Most of these modifications were contributed by our continuously growing user community. Thanks to all of those who help us in our mission to provide free and open source security incident response tools to the masses!

Please read the relevant sections in the Cortex installation guide to install or update your analyzers and responders in order to benefit from all this sweet & tasty honey.

New Analyzers

The following analyzers have been added:

Cyberprotect

This analyzer lets you query the Cyberprotect ThreatScore service for domains and IP addresses. No configuration is needed and it can be used out of the box.

TheHive displays the analyzer results as follows:

Have I Been Pwned

The HIBP_Query analyzer lets you check email addresses on Have I Been Pwned. You can use an optional parameter to include unverified breaches in the search results. Otherwise, it can be used without any additional configuration.

When called from TheHive, results would display as such:

PatrOwl

As it name states, The Patrowl_GetReport analyzer will let you get the current PatrOwl report for a FQDN, a domain name or an IP address. You need a running PatrOwl instance or to have access to one to use the analyzer.

If you fire it from TheHive, it would display results as follows:

SecurityTrails

This analyzer comes in two flavors in order to get Whois data and Passive DNS details using SecurityTrails. To use both flavors, you will need an account for the service to retrieve the associated API key, which you need to configure the analyzers.

SecurityTrails_Passive_DNS displays results in TheHive as follows:

The Whois variant produces reports such as:

Cisco Umbrella

In addition to Cisco Umbrella Investigate, you can now query the Umbrella Reporting API for recent DNS queries and their status for a domain name using the new Umbrella_Report analyzer.

New Shodan Flavors

In addition to Shodan_Host and Shodan_Search, which allow you to obtain Shodan information on a host and the search results for a domain name, now you can get domain resolutions (Shodan_DNSResolve), obtain scan history results for an IP address (Shodan_Host_History), get information on a domain (Shodan_InfoDomain) and the reverse DNS resolutions for an IP address (Shodan_ReverseDNS).

DomainTools

The following DomainTools flavors were added to this release:

  • DomainTools_HostingHistory: get a list of historical registrant, name servers and IP addresses for a domain.
  • DomainTools_ReverseIPWhois: get a list of IP addresses which share the same registrant information. It applies to a mail, IP, or domain.

Moreover, please note that DomainTools_WhoisLookup now handles IP addresses in addition to domains and provides parsed results. DomainTools_WhoisLookup_IP is thus not needed anymore. Instead, DomainTools_WhoisLookupUnparsed has been added to do the same as DomainTools_WhoisLookup, except that the output results are unparsed.