TheHive 4 is Here, Finally!


We have been speaking about it for almost two years.

We have been making it for more than twelve months.

And the day finally came for TheHive 4, our latest and greatest version, to be unleashed! The Chefs behind TheHive Project’s Code Kitchen are very happy to announce the immediate availability of TheHive 4.0, Release Candidate 1 (or 4.0-RC1 or the cool geeks call it).

Source: Berserk, écranlarge

Please note that a release candidate is not considered stable and must not be used in production. And since we almost rewrote TheHive from the ground up to accommodate all the nifty features outlined in a previous blog post, not to mention a few others we will cover in the upcoming weeks, we strongly recommend to take it for a spin on a test environment and help us uncover and fix bugs so we can release a stable version by April or May 2020.

Hmmm… I’d Rather Wait for a Stable Version

That’s your right but please don’t complain that, once released, the stable version is so buggy that it crashed your entire SOC operation and drove down the valuation of cryptocurrencies.

OK, OK… You Convinced Me. Where Should I Start?

Good! Well first things first. At this time, we produced documentation in kind of a rush while minding bazillion other things at the same time. We still need to proof-read it and enhance it.

If you are a seasoned TheHive user/contributor and you know what you are doing, please start with the installation guides for Debian or RedHat like operating systems. Then read the Quick start guide.

Noob warning: if you are completely new to TheHive, please use the latest stable version (3.4). TheHive 4.0-RC1 adds non-negligible complexity to accommodate advanced features such as RBAC and multi-tenancy and we will be very busy taking feedback from the intermediate/advanced users of our platform to make sure the stable version is rock-solid before we can recommend it to beginners.

You can find all the documentation we manage to write (more is coming) in the dedicated TheHive4 area of TheHiveDocs repository:

I’ve Just Tried it and Webhooks are Missing!

Nice catch Eagle Eye! Indeed webhooks have not been integrated in RC1. They will make a reappearance in a future RC, before the stable release. We have integrated them into a new notification system that is almost finished but still needs some elbow grease.

But Are you Going to Maintain TheHive 3.4.x when 4.0 will be Released?

You should know that bees will never let you down unless you gas them with pesticides (i.e. non-constructive feedback) and exigences (don’t forget that this is FOSS and we try to do the best we can, right?). So TheHive 3.4.x is scheduled to be maintained around two years after the release of 4.0 as a stable version, unless Elasticsearch 6.x is EOL’ed before that. In which case, we will have no choice but phase out 3.4.x (moving to ES 7+ will require a lot of work that we can put elsewhere).

Help!!! TheHive 4.0-RC1 Does not Work!

Please open an issue on GitHub using the template made for TheHive4 if you’d like to report a bug on this version. We will monitor those closely and respond accordingly.

Correction: March 3, 2020 
A new section regarding webhooks was added. In addition, a few typos were corrected.

Cortex-Analyzers 2.5.0: 142 Analyzers, 16 Responders

Shortly after the release of Cortex-Analyzers 2.4.0, TheHive Project’s code Chefs are happy to announce Cortex-Analyzers 2.5.0, a new Cortex analyzer & responder release which brings the total to 142 analyzers and 16 responders, up from 138 and 10 respectively!

We’d like to thank all the contributors for their precious work which will certainly provide more options to fellow cyber defenders and cyber threat intelligence analysts for improving their efficiency and focus on what really matters.

Source: https://dilbert.com/strip/2007-08-03

What’s New?

New Analyzers

New Responders

Analyzers

Clamav

Clamav is a powerful and open source antivirus engine that allows writing custom signatures using Yara and sigtool. @Hestat contributed with this analyzer that permits to TheHive to communicate with a local clamav-daemon.

A detailed configuration guide is available on Hetstat’s website.

Clamav short report for safe and malicious samples

IPVoid

Contributed by @jdsnape, this analyzer leverages the IP reputation check of apivoid.com, the API of www.ipvoid.com. As you can probably guess by its name, this analyzer can be used to enrich ip observables.

In order to use this analyzer, an account and a valid subscription to apivoid.com are required. An API key needs then to be provided.

TheHive displays the analyzer results as follows:

IPVoid analyzer short report
IPVoid analyzer long report

ThreatResponse

This analyzer lets you leverage the Cisco Threat Response service. Query Threat Response for verdicts and sightings for observables of type domain, filename, fqdn, hash (MD5, SHA1, SHA256), ip and url.

The analyser report lets you pivot into a Threat Response investigation of an observable.

Combining it with AMP for Endpoints Responder

It will extract the connector GUIDs as new observables to enable seamless use of the AMP for Endpoints Responder if a target is returned from the AMP for Endpoints module. It requires the AMP for Endpoints module to be configured in Threat Response.

A valid Cisco ThreatResponse subscription is required, and you have to provide your client ID and password information to use this analyzer.

TheHive displays the analyzer results as follows:

ThreatResponse analyzer short report
ThreatResponser analyzer long report

ThreatGrid

This analyzer queries Cisco Threat Grid for file, url, or hash and deliver analysis report. It also lets you pivot into the Threat Grid report to access more information related to Behavioral indicators or TCP/IP stream.

A valid Cisco Threat Grid subscription is required, and you have to provide hostname and api key to use this analyzer.

TheHive displays the analyzer results as follows:

ThreatGrid analyzer short report
ThreatGrid analyzer long report

Responders

AMPForEndpoints

This responders performs several actions on Cisco AMP for Endpoints. It comes in 5 flavors:

  • AMPforEndpoints_IsolationStart: Start Host Isolation.
  • AMPforEndpoints_IsolationStop: Stop Host Isolation.
  • AMPforEndpoints_MoveGUID: Move Connector GUID to a new group.
  • AMPforEndpoints_SCDAdd: Add SHA256 to a Simple Custom Detection List. TheHive’s case ID and description are appended to the description
  • AMPforEndpoints_SCDRemove: Remove SHA256 from a Simple Custom Detetion List.

A valid Cisco AMP for Endpoints subscription is required, and you have to provide the client id, api key and several context information to use this responder.

Redmine

Redmine is a free and open source, web-based project management and issue tracking tool. It allows users to manage multiple projects and associated subprojects. 

This responder, contribuited by srilumpa, can be used to create an issue in the Redmine ticketing system from a case. It will use the case title as the issue subject and the case description as the issue body.

To set it up in Cortex, you will need:

  • To define a user to allow Cortex to connect to Redmine and with access to the various projects in which issues should be created
  • Define three custom fields in TheHive that will be used to select the project, the tracker and, optionally, the assignee of the issue. These fields can be free form or can be custom fields with preset values.
Custom fields in TheHive for Redmine integration

At the moment the responder has few capabilities. If you need any other integration feel free to discuss on the pull issue.

Cortex responder output and corresponding issue in Redmine

Fixes

  • Umbrella Investigate [#698]

Get It While Supply Lasts!

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates` menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Mum, Docker Docks Don’t Dock Well!

Soon after we released Cortex-Analyzers 2.4.0, Jérôme noticed that something was definitely wrong. And that something was plural.

As he set to retest a few things here and there, he realised that many docker images, for the latest and greatest analyzers and responders, were not automatically built. The code factory wasn’t working 😰

Source: https://www.flickr.com/photos/8021817@N07/6262431675

So he started digging. And the more he dug, the more bugs he discovered. Our user community also reported a few issues. He thought it was about time he opens that bottle of Aloxe-Corton, put a Makaya McCraven album to play on his turntable, and rolls his sleeves to address all these problems headfirst 🍷

After a few hours of intense work, he managed to fix the docker build process and release Cortex-Analyzers 2.4.1, a hotfix that corrects the following issues:

  • [#545] Message extraction using FileInfo doesn’t always work
  • [#610] The VirusTotal analyzer contains a typo which prevents it from running
  • [#614] Many analyzers fail to run due to incorrect permissions
  • [#619] Abuse Finder not working with docker after force usage of python3
  • [#620] Missing library prevented the build of the docker image corresponding to the new MalwareClustering analyzer

Finally, he took the opportunity to rename Palo Alto AUTOFOCUS analyzers to Autofocus, for consistency purposes.

Please refer to our previous blog post, pertaining to Cortex-Analyzers 2.4.0, for update instructions.

Thank you in advance for your understanding and happy cyberfighting! 💪🏼

Cortex-Analyzers 2.4.0: 138 Ways to Analyze, 10 Methods to Respond

Guess what? TheHive Project is still alive and well, as Saâd already mentioned in a previous blog post.

We’ve been certainly very busy lately, preparing the upcoming release of TheHive 4 and doing many other things beside working on our FOSS project. As a result, it took us a rather long time to merge several community contributions and reduce the sizeable pile of pull requests.

We would like to thank our contributors for their patience and we hope the cyberdefenders out there will enjoy the brand new Cortex-Analyzers 2.4.0 release, with many new analyzers, responders and some bug fixes & improvements, bringing the total to a whooping 138 analyzers (counting all flavors) and 10 responders!

Additionally, with this release, all analyzers are now using Python 3. No more Python 2 technodebt!

Photo by Saâd Kadhi

What’s New?

New Analyzers

8 new analyzers have been added to this release:

1 analyzer has new flavors:

New Responders

3 new responders have been added:

Overview of the New Analyzers

DomainToolsIris

This analyzer looks up domain names, IP addresses, e-mail addresses, and SSL hashes using the popular DomainTools Iris service API.

The analyzer comes in 2 flavors:

  • DomainToolsIris_Investigate: use DomainTools Iris API to investigate a domain.
  • DomainToolsIris_Pivot: use DomainTools Iris API to pivot on ssl_hash, ip, or email.

A valid DomainTools API integration subscription is needed to run this analyzer.

TheHive displays the analyzer results as follows:

DomainToolsIris short report
DomainToolsIris long report

EmailRep

The EmailRep analyzer checks the reputation of an email address against the emailrep.io database.

IPInfo

This analyzer accesses IP-centric features provided by ipinfo.io. While the EmailRep API can be used without a token for limited usage, the ipinfo.io analyzer requires the configuration of an API token before use.

Maltiverse

This analyzer lets you query the free Maltiverse Threat Intelligence platform for enrichment information about a particular hash, domain, ip or url.

TheHive displays the analyzer results as follows:

Maltiverse short report
Maltiverse long report

MalwareClustering

Andrea Garavaglia contributed this one a long time ago and we finally merged it into the Cortex-Analyzers repository. Andrea gave a talk about the background of this analyzer at the fourth MISP summit. You can watch it here.

In order to use the analyzer, you need to point it to a Neo4j server (you need to supply the host, port, login & password).

PaloAlto Autofocus

This analyzer lets you leverage PaloAlto Autofocus services. Provided you are an Autofocus customer and you have access to their API, you need to configure the analyzer with your username and a token key.

The analyzer comes with 3 flavors:

  • AUTOFOCUS_GetSampleAnalysis lets you request a full report for a given hash.
  • AUTOFOCUS_SearchIOC lets you research for samples linked to specific IoCs with datatypes like domain, fqdn, user-agent, imphash, ip, mutex, tag and url. Please note that mutex and tag are not default datatypes in TheHive. You need to create them in TheHive before you can leverage them.
  • AUTOFOCUS_SearchJSON lets you research for samples based on a complex JSON query.

Important: TheHive has no templates corresponding to this analyzer have been published yet. They will be provided in the near future.

SpamhausDBL

This analyzer performs reputation lookups of a domain or a fqdn against Spamhaus Domain Block List (DBL).

TheHive displays the analyzer results as follows:

SpamhausDBL short report
SpamhausDBL long report

TeamCymruMHR

This analyzer queries Team Cymru’s Malware Hash Registry for known malware hashes (MD5 or SHA-1). If it is malware and known by the service, it returns the last time it has been seen along with an approximate anti-virus detection percentage.

Overview of the New Responders

KnowBe4

This responder allows the integration between TheHive/Cortex and KnowBe4’s User Events API.
If a mail observable is tagged with a specified tag, corresponding to the responder’s configuration (e.g. phished), then the associated user will have a custom event added to their profile in KnowBe4.

A valid account on KnowBe4 and an API key are required to run this responder.

Minemeld

This responder sends observables you select to a Palo Alto Minemeld instance.

To run this responder, a MineMeld Threat Intelligence Sharing account is needed.

Wazuh

This responder performs actions on Wazuh, the open source security monitoring platform. It currently supports ad-hoc firewall blocking of ip observables.

Improvements

New PassiveTotal flavors

Thanks to Brandon Dixon, the PassiveTotal analyzer gains 3 new flavors, bringing the total to 11:

  • PassiveTotal_Trackers let you make tracker lookups on observables of type domain, fqdn and ip.
  • PassiveTotal_Host_Pairs let you make host pair lookups on observables of type domain, fqdn and ip.
  • PassiveTotal_Components lets you make components lookup on observables of type domain, fqdn and ip.

They come with their own report templates.

GreyNoise Analyzer

The analyzer has been updated to support GreyNoise API v2, thanks to the contribution of Whitney Champion (#562).

New Data Types Supported by Some Analyzers

  • VirusTotal_GetReporthas been updated to allow requests for observables of type fqdn.
  • Threatcrowd has been updated to allow requests for observables of type domain.
  • Shodan has been updated to allow requests for observables of type fqdn.

Fixes

  • [#602] The MISP analyzer was bumped to version 2.1 and is ready to use PyMISP 2.4.120.

Get It While Supply Lasts!

I’m Hype

If you are using the dockerized analyzers & responders, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button.

I’m Country

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Under the Mighty Hood of TheHive 4

We have been speaking about it for almost two years. We have been making it for more than twelve months. And the day (or rather the month in this case) has almost come for TheHive 4, our latest and greatest version, to be unleashed.

While the first release candidate should be published by the end of this month, we would like to cover some of the most important changes we introduced in a platform which we rewrote almost from the ground up (40,000 lines of Scala code and counting), while keeping the familiar look&feel our longtime users came to expect.In a previous blog post, we covered TheHiveFS, a nifty feature of TheHive4 that allows you to quickly access all files stored in TheHive directly from your investigation machine. It’s time now to get a look under the hood of THeHive 4.

My Time is Precious. TL;DR Please

A picture is worth a thousand words, right? Here you go then!

The Hive 4’s Brand New Architecture

I am Puzzled, can you Elaborate a Bit?

So, you are not in a hurry anymore? Fine. Here, grab a seat, a glass of Gevrey-Chambertin and tasty Burgundy snails. All set? Let’s start then!

TheHive 4 will be the first version to use a graph database instead of Elasticsearch. Yes, you read that correctly. TheHive 4 won’t support Elasticsearch anymore but fear not fearless cyberdefender. Your friendly bees will not leave you hanging. If you are already using TheHive 3.4.x, we will provide a migration tool that will move your existing data to the new storage system (with no losses or bit flips hopefully).

We haven’t decided to ditch Elasticsearch on a whim or because Thomas (Franco, not Chopitea nor the General) dropped his leftist hipster attitude for a tight, tailor-made dictator uniform straight out of Spain. For all its greatness, ES has some annoying limitations which prevented us from adding, in an elegant, haiku-like way important features such as multi-tenancy, RBAC and large file management, while laying the ground for the future (stop being curious, the future has not been invented yet and when we do invent it, we’ll let you know).

Using JanusGraph, TheHive 4 structures information in graphs and stores them in an Apache Cassandra database. All the files that you attach to task logs or add as observables are stored in a Hadoop Distributed File System (HDFS).

Thanks to this brand new architecture, TheHive 4 is horizontally scalable. You can add as many TheHive, Cassandra and HDFS nodes to your Security Incident Response Platform cluster and sustain whatever load you might be facing without a sweat. Who said FOSS can’t be ‘enterprise grade’ (whatever that means in marketing lingo)?

Tour d’Horizon of the Main Features

TheHive 4, boosted by all the passion and skills of Zen Master Franco and MC Adouani, will support, in addition to TheHiveFS:

  • Multi-tenancy
  • RBAC
  • 2FA
  • Web configuration
  • API versioning

We will cover some of these features in greater detail in future instalments. In the meantime, let’s take a ride in a helicopter and view the wonderful landscape laying before us from above. After you Messieurs-Dames, we are French gentlemen and gallantry is of the essence (except when we use the public transportation in Paris, then savages we become).

Multi-Tenancy

As in Cortex, you will be able to create multiple organisations within a single instance of TheHive 4. In addition, an organisation can decide to share a case or parts of it (say a task, some observables, etc.) with other organisations. That way, a peer organisation or a constituent can contribute to the investigation at hand, provide essential information, etc.

RBAC

TheHive 4 supports a large set of user permissions. Some pertain to administrators, others to users and there are also permissions that apply to connectors. For example, users can manage tasks but not observables. They can have the power to share a case or part of it with sister organisations and execute Cortex analyzers but not responders.

You will be able to create roles for users, and, at the organisational level, what we call shares. RBAC deserves its own blog post and we’ll get to it pretty soon.

2FA

Do you really want us to describe this one? Before you answer yes, we’d like to remind you that you are in a helicopter. Just sayin’.

‘They asked me to explain 2FA. So I helped them out of the helicopter. It was flying way above ground.’
Source: Berserk, FNAC.com

Web Configuration

Tired of using vi, Emacs or your favourite CLI editor for making configuration changes to TheHive’s application.conf? Tired of restarting the service to take into account those modifications? Then you will certainly go dance kizomba with Nabil all night long when we tell you that you don’t need to use vi & service (or whatever the kids are using these days) anymore!

Thanks to the new architecture, all the configuration will be stored in the underlying database and you will be able to edit it using the WebUI. TheHive will automatically take the changes into account and you won’t need to restart it.

We can feel your love here. Merci !

API Versioning

TheHive 4 adds API versioning and it will maintain backward compatibility with TheHive 3.4.x without preventing us from adding new features. TheHive4py will not be updated right away for TheHive 4 but thanks to the backward API compatibility, all existing feeders and programs that use the current version of TheHive4py will still work out of the box.

That’s all folks! Stay tuned for further news and, in the meantime, don’t be blue cuz’ the bees gonna take care of you.

TheHiveFS

TheHive Project’s Code Chefs, sweating under their toques, are working hard to deliver TheHive 4 as soon as feasible. The current target release date for the 1st release candidate (4.0-RC1) is Friday Feb 28, 2020.

While TheHive 4 will be the first release to support graph databases, multi-tenancy and Role-Based Access Control (RBAC), it will also have a nifty feature that can simplify the incident response and digital forensics workflows of our fellow cyberdefenders: TheHiveFS.

What is TheHiveFS?

Starting from TheHive 4, TheHive can be ‘mounted’ as a remote, WebDAV filesystem. The filesystem can be securely mounted if SSL/TLS is enabled.

Thanks to TheHiveFS, you can quickly access all files stored in TheHive directly from your investigation machine. This can speed up the time needed to triage and analyse evidence. 

What Types of Files Can I Access through TheHiveFS?

You can access, in read-only mode, all files attached to task logs and all observables which datatype is file, as long as you are allowed to do so. Indeed, TheHive 4 comes with RBAC so if, for example, you are not allowed to view a case or some file observables in a case, you won’t be able to access them using TheHiveFS, the same way as if you are using the WebUI.

Screenshot showing an analyst accessing file observables and files associated to tasks of case #40 using TheHiveFS

How Can I Mount TheHiveFS?

Assuming you have a WebDAV client, such as davfs2, use the following command line:

$ sudo mount -t davfs -o noexec https://myhiveinstance:9001/fs /mnt/dav/

You can also point your graphical file manager to:

dav(s)://myhiveinstance:9001/fs

You will need to authenticate using your username and password as if you were connecting to TheHive’s WebUI.

Mom, I’ve Just Stepped on a Landmine

Beware folks. When you download a file observable using TheHive’s WebUI, it will conveniently create a password-protected ZIP archive before handing you the file. This way, we avoid accidental double clicks that may lead to the infection and compromise of your workstation, which might reflect bad on you or force you to offer breakfast the next morning to all your fellow teammates.

There is no such protection if you use TheHiveFS. Let us repeat this so it sinks: there is no such protection if you use TheHiveFS.

If you mount TheHive’s filesystem and open by accident or by a great deal of will, as a true, hardcore fan of Russian roulette, a file observable that is in fact malware courtesy of your favourite bear, kitten, panda or eagle, you can’t blame your friendly bees. But we will empathise (and our empathy level is directly correlated to the amount of pains au chocolat you send our way).

You’ve been warned.

That Sounds Awesome! When Can I Try It?

As written above, you will be able to try TheHiveFS as soon as TheHive 4.0-RC1 is released and that’s currently planned for the end of February 2020.

You can cry, beg, try to bribe us with VC money, make the line at 3:00 AM in front of TheHive Store (there ain’t no such store, we are not Apple), this will not make us work any faster. But you can always cheer us up, hug us or just thank us. This means a lot to us and to the free, open source software flame we carry deep within our souls.

One More Thing…

While we aren’t Apple, we can mimic Steve to share one more information that will make TheHiveFS even more interesting by Q3-Q4 2020. We plan to add support for large file management in TheHive 4.1, the next major version after 4.0 as would Captain Obvious say. Thanks to this feature, you will be able to upload memory and disk images to TheHive and if your Internet line breaks, the upload will resume automatically. 

That’s all folks!

Cortex 3.0.1: The ‘Better Logging’ Edition

Lo and behold, we aren’t dead & TheHive Project ain’t toast! So, foremost, Happy New Year folks (we are still in January, right?)! We have some nice gifts coming up for you, gifts that have required very heavy-duty work. Of course, you might complain that we haven’t been responsive as of late but hey, there’s only so much we can do, right?

Happy New Year Folks! (Photo by Saâd Kadhi)

We’ll talk about those gifts in the upcoming weeks. In the meantime, there’s a new Cortex version in town and we urge you to upgrade to it, particularly if you consider deploying several Cortex nodes as a cluster. Indeed, Cortex 3.0.1 fixes a missing dependency that is required to set up such an architecture. Additionally (and this is the part where you should be paying attention), this version fixes the display of error messages pertaining to analyzer and responder operations, and also ensure that old responders and analyzers no longer show up once you clicked on that Refresh button.

Fixes and Enhancements

  • #244 Prevent the Play secret key from being displayed in the logs at startup. Nonetheless, you can still display it (for troubleshooting purposes or to make things easier for attackers that might have access to the logs and be interested in such a world-changing secret) by using the --show-secret option when starting Cortex
  • #243 fixes the display of error messages when analyzers & responders fail
  • #242 Remove references to Google Fonts
  • #238 The Docker image had 4 critical CVEs and 69 high CVEs (*cough*). Contributed by Micheal Hart
  • #239 Missing dependency for cluster bug
  • #234 fixes a bug where old, non-existent analysers were still showing in Cortex after an upgrade. Contributed by daskydasky
  • #241 Analyzer reports no output when it fails
  • #240 An encoding issue causes an invalid format in the catalog file
  • #230 Elastic4play has dropped the ES cluster configuration option. Contributed by Adeel Ahmad
  • #164 Handle second/minute-rates limits on Flavors and Analyzers

Running Into Trouble?

Shall you encounter any difficulty during the upgrade process, please ask on our user forum, get in touch with the community on Gitter, or send us an email at support@thehive-project.org.

TheHive 3.4.0 & Cortex 3.0.0 Released

For many months, we have been concentrating our efforts on TheHive 4, the next major version of your favourite Security Incident Response Platform, which we’ll finally provide RBAC (or multi-tenancy if you prefer), a feature that Cortex had for quite some time now.

Source : dilbert.com © Scott Adams

As you well know, both TheHive and Cortex rely on Elasticsearch (ES) for storage. The choice of ES made sense in the beginning of the project but as we added additional features and had new ideas to give you the best experience possible, we faced several ES quirks and shortcomings that proved challenging if not outright blocking for making our roadmap a reality, including RBAC implementation in TheHive, a far more complex endeavour than RBAC in Cortex. Transitioning from ES to graph databases was necessary and since we want our existing users to have a smooth migration path, TheHive 4 (the first release candidate should come out of the oven by the end of the year) will support both ES and graph databases.

But while we were focusing on that, we completely lost sight of the end of life of ES 5.6 so we wrote an apology to you, our dear users, back in May.

Shortly after, we released TheHive 3.4.0-RC1, to add support for ES 6 (with all the breaking changes it has introduced). We also did the same for Cortex with the release of Cortex 3.0.0-RC3. We also took that opportunity to clear out some AngularJS technodebt we had.

We then asked you to take them for a spin and report back any bugs you find given that both versions had to support ES 5.6 and ES 6 to allow for proper migration.

After a few rounds of release candidates, we are pleased to announce the immediate availability of TheHive 3.4.0 and Cortex 3.0.0 as stable releases.

Before upgrading your existing software to these new versions, please make sure to read the blog post we wrote back in June. We invite you to pay great attention to the regressions that we were forced to introduce because of ES 6.

You should also note that, in addition to ES 6 support, Cortex 3.0.0 supports fully dockerised analyzers and responders. We’ll elaborate on this in a future blog post soon.

Changelogs

If you are interested in some nitty-gritty details, we invite you to read the relevant changelogs since our last post on the subject:

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help as usual!

More than 120 ways of Analyzing your Observables

There’s a new version of Cortex-Analyzers in cybertown and its has an awesome, mind-blowing name and that’s… wait for it… wait for it: 2.1.0.

In this new release, we added two analyzers which bring the total number to more than 120:

We could not duly test DNSSinkhole since we do not have access to the associated service. So we would really appreciate it if you could test it and let us know whether it works or not.

Others analyzers have been fixed or improved:

DNSSinkhole

This analyzer lets you check if an IP address has been registered in your DNS sinkhole. TheHive displays the analyzer results as follows:

DNSSinkhole analyzer: long report
DNSSinkhole analyzer: short report

TalosReputation

This analyzer lets you determine whether an IP address has been reported as a threat on Cisco Talos Intelligence service. No special access to the service is required to run the analyzer.

TheHive displays the analyzer results as follows:

Talos Reputation: long report
Talos Reputation analyzer : short report

Crt.sh

This analyzer has been enriched to display SHA-1 fingerprints. The long report format has been updated to reflect this new information.

Crt.sh analyzer: long report

FileInfo

FileInfo has been updated and is now able to parse PDF files and extract IOCs such as URLs, hosts, domains, IPs, hashes and many more.The analyzer does also support the last version of the extract-msg library.

FileInfo analyzer: IOC Parser long report
FileInfo analyzer: IOC Parser short report

VirusTotal and Python3

The VirusTotal analyzer, including all its flavours, now uses Python3 and an updated virustotal-api library.

Yeti API key

An optional API key can now be configured and used by the Yeti analyzer.

Malwares_GetReport

A hash computation has been fixed in this analyzer.

EMLParser

A first fix has been introduced to avoid this analyzer to crash when there is no content-description in content_header, and a second has been added to correct a header display issue.

IBM XForce Lookup

The analyzer has been improved to allow users to add a trailing / at the end of the API URL without breaking everything.

Updating your Analyzers in Cortex 2.x

Each analyzer and responder comes with its own, pip compatible requirements.txt file. Run the following commands to update your Cortex analyzers to the latest version:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Dockerised Analyzers

Cortex 3.x gives you the opportunity to run dockerised analyzers and responders. This means that you no longer have to download all the git repository of Cortex-Analyzers and run lengthy commands to update your analyzers and responders.

If you want to use dockerised analyzers and responders, ensure that the URL of the catalog.json file corresponding to the Cortex-Analyzers repository is registered in application.conf. Please note that this won’t work if you are tracking the stable catalog.

After doing so, do not forget to login to Cortex as an orgadmin, click on the Refresh Analyzers button, then Disable and Enable again each analyzer and responder. Analyzer (and responder) updates should occur automatically as long as docker.autoUpdate is set to true in application.conf (this is the default setting).

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Searching for an Elastic? Here, Take 6!

As we announced on May 14, 2019, we have been working very hard to add Elasticsearch 6 support to TheHive and Cortex as Elasticsearch 5.x went the way of the dodo when Elastic plugged life support off this venerable version. We also took this occasion to upgrade AngularJS and its sub projects to 1.7.8, the latest 1.x version as of this writing. Additionally, Grunt build dependencies have also been updated to their latest compatible versions.

It took us more time than initially foreseen but hey, we all love deadlines. We all love the whooshing noise they make as they go by.

TheHive 3.4.0-RC1 and Cortex 3.0.0-RC3 are now available on every Internet pipe near you and before you take them for a spin to help us identify any issues to make the stable releases rock-solid, let us walk you through some important information. Relax and grab a drink (and send good wine our way, we can always use some!).

Source: https://dilbert.com/strip/1995-11-10

TheHive 3.4.0-RC1

In addition to ES5 and 6 support and the update of AngularJS, this version corrects a few bugs that were identified in the latest stable version (3.3.1) and adds a few features. The most important one in our opinion is the ability to import a file from a Cortex report. This requires Cortex 3.0.0-RC3. The full list of changes is available at the following location.

Prior to migrating to 3.4.0-RC1, please read the migration guide.

Cortex 3.0.0-RC3

ES5 and ES6 support, AngularJS et cetera et cetera. Well you know the song right? Not quite as Cortex 3.0.0 significantly facilitates analyzer and responder installation and updates, thanks to Docker as we touched upon in a blog post earlier this year.

As detailed in the Cortex migration guide, which we recommend you read thoroughly, you can migrate from Cortex 2 and keep using analyzers and responders the same way (using processes), use the new Docker-based analyzers and responders or mix and match between running processes and docker containers (but then, you gotta pay extra attention to configure properly which analyzer/responder runs in which fashion).

Moreover, if you use the new dockerised analyzers and responders, you will be able to choose if you want to have them autoupdated (that’s the default behaviour) and if so, pick the bleeding edge, potentially buggy versions, the minor releases or, if you are risk-averse, stick with stable ones.

Cortex 3.0.0-RC3 also adds the ability to retrieve files resulting from analyzer jobs and last but not least, corrects an information disclosure bug that allowed non-admin users to retrieve the details of other users through the API. The vulnerability was reported by Adam Maris so kudos to him!

Warning: Regressions Ahead!

As outlined in our previous post about these new versions:

  • TheHive 3.4.0-RC1 and Cortex 3.0.0-RC3 use HTTP transport (9200/tcp by default) to connect to Elasticsearch instead of its native binary protocol (9300/tcp by default).
  • SSL/TLS, including when using a client certificate, can be configured to connect securely to ES. However this has not been tested yet.
  • Support of X-Pack and Search Guard is discontinued for anything but basic and SSL client authentication, which would still work.

Caution: Performance May Take a Hit!

The parent-child relationships we use behind the scene in Elasticsearch could make queries significantly slower with ES 6 and in our limited testing, we had the impression that performance took a hit. So please be cautious there and we’d be grateful if you could report any sluggishness you notice during your tests of the new versions with ES6.