Introduction

Several months,no, years ! after the first line of code – the first line was committed in 2016–, we are very excited and proud to announce the release of TheHive 4.0.

This means more than a major version for us. This was – and still is — like a completely new project, a new generation, a lot more challenging than before. We had to make the application climb a major step to introduce new key features, some we added in this version, others we have in mind for the future.

Objectives

The development of the second generation of TheHive, aka. version 4, was driven by three main objectives:

• Add support to multi-tenancy: allow 1 instance of TheHive to serve many teams and organisations
• Add support to Role Based Access Control to define fine grained user profiles
• Rethink the data model and structure to support the goals listed above (Moving from Elasticsearch as main persistence layer, to a data model designed as a graph).

Challenges

TheHive Project is thoroughly adopted by SOC, CERTs and CSIRT teams, who decided to go with TheHive Project since the first releases. It is worth noting that until today, TheHive has had a total of 52 releases since 2016.

Those teams helped the project by contributing to our QA, questions, feature requests etc… and our way of thinking drove us to not let them down, and we decided to produce a backward compatible software.

The way we have been working until now aims to make our community move smoothly from TheHIve 3 to TheHive 4.

Backward compatibility

This is the most difficult challenge we have had, but we have hard heads and soft hearts.

TheHive 4 is expected to be backward compatible, thanks to APIs v0. Yes, we provide versioned APIs having the same endpoints as TheHive 3, and producing the same results. Search APIs also support the same query language, except some corner cases like searching using the “_string” operator (which is tightly coupled to Elasticsearch query language, but we have working alternatives).

Performance concerns

Supporting backward compatibility might force you to accept complex designs. And TheHive 4 RC3 was a clear example of that limitation.

Many kind users who tested TheHive 4 RC3, raised performance issues, slow UI problems etc… And it was completely expected. We thank them for making such a pressure on us, we used it to boost the refactoring of the UI, which was using backward compatible APIs (unoptimized for the new data model and representation), specially to read data (listing cases or observables for example).

We can discuss the technical details of this hard point later, but it mainly relates to navigating through graph-based data using a document based query system, which is not optimised.

For example, if you want to search for list of observable of a given case, the ideal way of doing that on a graph-base model is to:

• Get the case by its ID, which is indexed (very fast operation)
• Navigate through case relation, to find its links of type observable

But the backward compatible query language works differently: It scans all the graphs to search for observables that have a case parent with a given ID, which has a slower performance in a graph-based database.

Multi-tenancy and RBAC

TheHive 4 comes with a special multi-tenancy support. It allows the following strategies:

• Use a siloed multi-tenancy: you can define many organisations, without allowing them to share data
• Use a collaborative multi-tenancy: you can define a set of organizations and allow them to collaborate on specific cases/tasks/observables, using custom defined user profiles (RBAC)

This feature is very powerful but has a cost: an expected performance overhead. For example, when scanning the graph of data to search for a list of cases, TheHive must return the cases of your organisation and the case you can have access to because of the sharing rule.

New foundations

TheHive 3 was based on a framework called Elastic4play, written by Thomas to abstract all the routines required by a web application written with play 2 and using Elasticsearch.

TheHive 4 has its own core framework: Scalligraph, built to handle the following features.

Scalligraph will be the foundation of the next major version of Cortex.

What’s new in 4.0 

TheHive 4.0 release has a significant amount of changes. We will quickly explain the most important, and you can refer to the change logs if you need to have more details.

UI Performance

This was the most important task of this release. As we mentioned above, we were using backward compatible APIs in RC3 release, and migrated 80% of the UI to use the APIs v1 which are optimised for the new graph-based and multi-tenant data model.

OAuth2 Support

This topic gave birth to many github issues, some of them related to TheHive’s UI not correctly redirecting authenticated users. OAuth2 support has been tested with many providers like: Okta, Keycloak, FusionAuth, Microsoft Azure, Office 365 and Google Gsuite.

Starting from this version, there is an API endpoint that handle all the authentication and redirections: /api/ssoLogin

Here is a configuration sample for MS Office 365

{
  name: oauth2
  clientId: "CLIENT_ID"
  clientSecret: "CLIENT_SECRET"
  redirectUri: "http://THEHIVE_URL/api/ssoLogin"
  responseType: code
  grantType: "authorization_code"
  authorizationUrl: "https://login.microsoftonline.com/TENANT/oauth2/v2.0/authorize"
  authorizationHeader: "Bearer "
  tokenUrl: "https://login.microsoftonline.com/TENANT/oauth2/v2.0/token"
  userUrl: "https://graph.microsoft.com/v1.0/me"
  scope: ["User.Read"]
  userIdField: "mail" 
}

You can find more details about the OAuth2 support in the authentication config documentation

Improved Analyzer and Responder selection

Analyzer selection when calling bulk observable analysis has been improved to show the possible analyzers per observable type.

For responders, the user experience has been improved as well, especially for instances with a big number of responders. The simple dropdown menu available to select responders has been replaced by a dialog allowing list filtering and scrolling:

Add bulk operations to case listing

Before this release, simple case updates required visiting the cases one by one and editing them. We added in this release a bulk edit feature, depending on user’s permissions on the selected cases

The same bulk editing component has been used to improve the same operations on observable list page.

Other noteworthy changes

We need to mention that the following changes have been included in TheHive 4.0 release:

• Add pagination and filtering to users administration
• Add back the UI configuration by organisation. The only available option is related to enabling/disabling the use of Empty Case.
• Show sharing summary in task and observable lists
• Improve alert preview dialog
• Add alert externalLink feature allowing the display of external links for any alert, not only MISP alerts.

Known limitations

Even after 49 closed Github Issues, there are still major topics to be addressed by the upcoming releases:

• Add back support to case merge which is not satisfying today. The challenge is to find the best to merge cases and make sure that it works in a profile-based multi-tenant design.
• Add full text search support. In older versions, TheHive benefited from the full text search capabilities of Elasticsearch. With the new database and persistence system, full text support requires adding a dedicated indexing layer.

Installing and testing TheHive 4.0

After months of testing versions, this official release means that we consider it ready for production purposes. If you’re new with TheHive, we recommend going with TheHive 4.0.

Several installation guides have already been published, suitable with the chosen operating system and installation type, and new are coming.

For testing and training purposes, a virtual machine with a simple configuration of TheHive 4.0 and Cortex 3.0.1, is also published and available starting from now. Please refer to the documentation for download and usage instructions.

Want to upgrade from TheHive 3.x ?

All changes brought to TheHive make the upgrade more challenging than installing the new package and watch the progress bar. To support you with the upgrade, a migration tool comes along with the application to shift your current version of TheHive to TheHive 4.0.

A dedicated guide has been published to help users with this significant task. We recommend using a new server aside from your production server to ensure everything works fine with the migration.

Future of TheHive 3.x

This major outcome doesn’t mean TheHive 3 end of life is reached. As previously announced, we plan to support this version for some time, our next milestone being to support Elasticsearch 7.x with a first Release Candidate.

How to report issues

Please open an issue on GitHub using the template made for TheHive4 if you’d like to report a bug on this version. We will monitor those closely and respond accordingly.