TheHive 3.4.0 & Cortex 3.0.0 Released

For many months, we have been concentrating our efforts on TheHive 4, the next major version of your favourite Security Incident Response Platform, which we’ll finally provide RBAC (or multi-tenancy if you prefer), a feature that Cortex had for quite some time now.

Source : dilbert.com © Scott Adams

As you well know, both TheHive and Cortex rely on Elasticsearch (ES) for storage. The choice of ES made sense in the beginning of the project but as we added additional features and had new ideas to give you the best experience possible, we faced several ES quirks and shortcomings that proved challenging if not outright blocking for making our roadmap a reality, including RBAC implementation in TheHive, a far more complex endeavour than RBAC in Cortex. Transitioning from ES to graph databases was necessary and since we want our existing users to have a smooth migration path, TheHive 4 (the first release candidate should come out of the oven by the end of the year) will support both ES and graph databases.

But while we were focusing on that, we completely lost sight of the end of life of ES 5.6 so we wrote an apology to you, our dear users, back in May.

Shortly after, we released TheHive 3.4.0-RC1, to add support for ES 6 (with all the breaking changes it has introduced). We also did the same for Cortex with the release of Cortex 3.0.0-RC3. We also took that opportunity to clear out some AngularJS technodebt we had.

We then asked you to take them for a spin and report back any bugs you find given that both versions had to support ES 5.6 and ES 6 to allow for proper migration.

After a few rounds of release candidates, we are pleased to announce the immediate availability of TheHive 3.4.0 and Cortex 3.0.0 as stable releases.

Before upgrading your existing software to these new versions, please make sure to read the blog post we wrote back in June. We invite you to pay great attention to the regressions that we were forced to introduce because of ES 6.

You should also note that, in addition to ES 6 support, Cortex 3.0.0 supports fully dockerised analyzers and responders. We’ll elaborate on this in a future blog post soon.

Changelogs

If you are interested in some nitty-gritty details, we invite you to read the relevant changelogs since our last post on the subject:

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help as usual!

An Apology

Dear Users,

We owe you an apology. We thought we would never need to support Elasticsearch 7 or even 6. We thought we could stick with the latest version of Elasticsearch 5 as the underlying storage and indexing engine for TheHive and Cortex until we would be able to complete the transition to a graph database. Moving to such a database is a necessity for your favourite open source, free Security Incident Response Platform and its analysis and orchestration companion, a necessity that has grown out of our frustration with Elasticsearch and its limitations, with the breaking changes that ES 6 introduced which forbid a smooth transition and puts a significant toll on an open source initiative such as ours.

We initially thought we could complete the transition by October of last year and finally offer you long-desired features such as RBAC and multi-tenancy as well as establish a solid ground to implement some exciting ideas that would help you lower the barrier to entry for junior analysts, save more time and concentrate on your work instead of having to master copy/paste between various interfaces or moving from one tool to the other.

Sadly, things did not play out the way we wanted. As TheHive and Cortex were adopted by more and more organisations, feature requests kept piling up and being generous bees, we have always strived to keep our users happy within the confines of our limited resources. Certainly, our user community helped us significantly by contributing a huge number of analyzers to Cortex in no time, making the total amount fly past the 100 landmark. However, we had to rely mostly on ourselves for heavy-duty backend work while steadily releasing new versions to satisfy the appetite for capabilities that sounded reasonable and feasible within a realistic, acceptable timeframe. Multi-tenancy and RBAC also proved more complex than initially foreseen and since we hate a half-baked recipe (blame it on our French culture and our love for delicious food), we did not want to rush things out and add flimsy ‘patch’ code.

Source : https://kininaru-korean.net/archives/10305

So we focused on supporting graph databases and working on multi-tenancy and RBAC. You certainly noticed our silence these past weeks. And we completely lost sight of the end of life of ES 5.6 until we realised recently that it was no longer supported by Elastic, not even in critical bug fix mode. When ES 7 was released on April 10, the death sentence of ES 5.6 was pronounced and its coffin permanently nailed.

We know this is a lot to stomach. Welcome to the Upside Down! But remember: keep calm. Help is already on the way and hopefully this time around the cops will arrive before the movie is over. We are shifting our priorities to release new major versions of TheHive and Cortex in order to use a supported version of ES. This work should take a few weeks at least. In the meantime, if you are using TheHive and Cortex with their own, standalone ES instance and you have implemented sane network security measures to shield ES against unwanted remote access, you should be fine.

We also took the opportunity to look at what other external code we rely on and that would need to be updated as well, to avoid falling in the EOL trap again. Glad we looked! The current versions of TheHive and Cortex both use AngularJS 1.5 (here, take a stone and throw it the Hulk’s way on Nabil’s forehead). We are going to update our frontends to use AngularJS 1.7.

We will come up imminently with a concrete action plan to address our embarrassing miscalculation. Meanwhile, please accept our sincere apologies and rest assured that we won’t let you down.

ごめんなさい 🙏🏼

The Mind-Boggling Implications of Multi-Tenancy

TheHive offers a powerful yet generic query API for all the data stored by the platform in the underlying Elasticsearch database.

Thanks to its DSL (Domain Specific Language), TheHive can handle complex search queries such as the following:

Among all the unassigned tasks, show me all those associated with cases which severity is high but also contain the highest number of observables which datatype is  ‘mail’

When faced with such complex queries, TheHive translates them using its DSL and sends them over to Elasticsearch to obtain the results. TheHive’s dashboards draw their power from such querties.

And while such capability is highly desirable in our opinion, a capability that we will further leverage to add a completely revamped search module in the upcoming Cerana 1 (TheHive 3.1) release, it greatly complicates RBAC (or multi-tenancy) in TheHive.

Screen Shot 2018-06-27 at 11.50.39.png
A Sneak Peek at the New Search Module of the Upcoming Cerana 1 (TheHive 3.1) Release

Indeed, in the RBAC world, the conversion of any search queries submitted to TheHive into an Elasticsearch one is fully dependent on the user context. The user view must be kept within the boundaries of the group or groups to which they belong. Each search filter,  each search parameter, must return only the results that the user can view.

The data scope needs to be clearly identified at the case level. To perform a search against task logs for example, TheHive will need to identify the parent task log, then identify the parent case and only then verify the scope. This is no small undertaking.

Similarities across cases or alerts, such as the Related Cases feature or the relationships between a given alert and existing cases, would need additional work that has not been clearly identified at this stage. But the difficulties do not stop there. Any element that has no clear relationship with case entities will have to be singled out and specific code would need to be added to limit access according to the RBAC rules. This will be clearly the case for the audit trail. Also, what should TheHive display when an analyst group is working on a case that shares observables with another one belonging to a different group? Shall it allow a limited view without any details so that groups may request from a super administrator to authorize both groups to collaborate on the investigation, something that distributed CERTs or SOCs in a large corporation may desire? Or shall it keep the data completely isolated as MSSPs which serve multiple customers with a single instance will require? We know the answer: make it configurable. But take a step back and think of the implications at the code (and security) level.

Contrary to the feature we added to Cortex 2, which allow multiple organizations to use a single Cortex instance, multi-tenancy in TheHive is a much more complex feature to implement and which is expected to have a significant impact on the platform’s performance. It will also need extreme caution to avoid blind spots that attackers (and not so innocent tenants) may exploit to circumvent scope limitations and extend their view to data they are not supposed to access. That’s why we had to delay it to Cerana 2 (TheHive 3.2), currently planned for the end of October 2018.

If you are well versed in Elasticsearch and Scala and willing to help implement this feature, please contact us at support@thehive-project.org.