We released Cortex 2.1.0 as a release candidate back in July 31, 2018 along with TheHive 3.1.0-RC1. By then, the power duo which makes digital forensics, incident response and, to an extent, cyber threat intelligence, better, faster, happier, regular exercising gained the ability to perform active response.
We ate our own dog food for a couple of months. We found bugs. We added enhancements and we listened to the early adopters of these new major versions. And today we are thrilled to announce the availability of the stable release of Cortex 2.1.0 along with TheHive 3.1.0.
Cortex 2.1.0 restores the ability to query the analysis and response engine from MISP for enrichment purposes. A new version of the de facto standard for threat sharing should be released shortly as there are also some API-related issues on its side to make the integration fully working again.
Cortex 2.1.0 also gives you the ability to see the PAP (Permissible Actions Protocol) values for each analyzer as well as any custom cache values you might have configured.
Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!
TheHive Project’s Master Cooks are happy to announce the immediate availability of TheHive 3.1.0. This is the first release of your favourite SIRP (Security Incident Response Platform) or, if you fancy new buzzwords, SOAR (Security Orchestration, Automation & Response) that we put out as a release candidate to give sufficient time for our ever growing user community to test it and report any outstanding bug before publishing a stable version.
Indeed, TheHive 3.1.0 brings significant new functionalities that we detailed in previous blog posts. One of the most prominent features of this new major version is the support of responders through Cortex 2.1, also released today as a stable version.
Responders are similar to analyzers but instead of analyzing stuff, they allow you to respond to stuff. Put otherwise, they give you the ability to implement specific actions by a simple click from different elements in TheHive: alerts, cases, tasks, task logs and observables.
For instance, imagine a user in your constituency reporting a suspicious email. Using Synapse or an alternative alert feeder, the email reported by the user will automatically show up as an alert in your alert pane. Before starting working on it as a case, you preview it only to realise it is a scam and it does not warrant your time & effort. Still, you’d like to reply to the user.
In such a case, you could implement a responder that will not only send an email back to the user asking them to ignore such a scam but that can mark the alert as read. Using. A. Simple. Click. C’est beau n’est-ce pas ?
Going through all 71 (yes, 71) issues that have been closed with this release and the 3 RCs we published since July 31, 2018 will be terribly boring but you can read the full changelog while dipping your croissant in your espresso cup.
We’d rather encourage you to install this new version, which is as usual, AI-free, machine learning free, cyberbullshit-free, gluten-free, organic (well as much as free, open source software can be anyway), vegan (if you can eat it), and most importantly made with huge love and care for the SOC, CSIRT & CERT communities and other fellow cybercrime fighters. So go ahead and try it out. It won’t cost you a dime (or a franc if you are a French old timer).
Need Help?
Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!
After a nice summer break which allowed us to rest and enjoy Real Life™ (mostly) away from keyboards, screens and constant distractions and interruptions, we set to take into account the bug reports and feedback on TheHive 3.1.0-RC1 which we released a day or so before packing up for the mountains and elsewhere.
We are pleased to announce the immediate availability of Release Candidate 2 for TheHive 3.1.0. It contains numerous bug fixes and enhancements. You can read the full change log if you have nothing better to do with your life.
Among the changes we introduce in this new RC, we would like to highlight the following:
#652: the ability to set custom fields as mandatory.
#685: the quick case search box on the top navigation bar has been restored as it is highly useful for quick lookups without having to resort to the revamped search page.
#667: use alternative authentication methods when certificate authentification is enabled and the client does not present a certificate.
We encourage you to take TheHive 3.1.0-RC2 for a spin as quickly as you can and report any bug or issue so we can address them for the final release, scheduled in a couple of weeks. Cortex 2.1.0 is still at RC1 and we should be able to make a stable release at the same date as TheHive 3.1.0.
Important Note
We would like to remind you that starting from these versions, we have two release channels: a stable one that should be used for production systems and a pre-release channel that should be used to try the release candidates such as TheHive 3.1-RC2 and Cortex 2.1-RC1, and help us iron out bugs before adding them to the stable channel. Those who love living on the bleeding edge may be tempted by running the release candidates on their production environment given all the candy and icing we added. They are at liberty of doing so but we don’t want to hear anyone one whining about an RC that broke everything and beyond.
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Paris, France. The Sun is shining on the city of lights and temperatures are quite high, even for the summer season. Life is good. As a matter of fact, life is excellent.
TheHive Master Cooks are about to go on vacation for a few weeks. But before they pack up their Patagonia bags and leave the sandy beaches for those who enjoy them, preferring mountains, trails, walking and breathing fresh air with family and friends, they would like to make a significant contribution to help TheHive and Cortex users fight cyberattacks even better than they already do. And maybe convince those who don’t that free, open source software is not a joke or a geek fad.
We’d like to welcome to the stage our latest babies, which we are really proud of: TheHive 3.1 and Cortex 2.1, the new versions of the power duo which make digital forensics, incident response and, to an extent, cyber threat intelligence, better, faster, happier, regular exercising (well you know the Radiohead song so we’ll let you continue singing along) since early 2017.
While our project might seem very young, it is not. We’ve been working steadily on TheHive, using it (i.e. eating our own dog food) since early 2014 before releasing it at the end of 2016 once we were satisfied with it, as a token of gratitude to a community that helped us due our jobs in various ways. We then extracted what has become Cortex from its core to ship it as a separate product in February 2017. And we kept improving them at a steady piece for the collective benefit of incident responders, forensicators and threat analysts. And adoption has been rather spectacular. Thanks to all of our users for their love and support!
We believe we are at a moment where people could not brush us off anymore as amateurs. Try TheHive and Cortex, preferably with MISP and get a taste of what professional, free and open source software can be.
TheHive 3.1 and Cortex 2.1 are feature-packed and we won’t be able to cover them all in detail in a single blog post. Rather, we’d like to concentrate on a few important ones.
Stable, Pre-release Channels and New Repositories
We’d like to point out that, in order to improve our release process and given the number of features that we added, TheHive 3.1 and Cortex 2.1 are release candidates at this stage. So we encourage you to test them and report back any bugs or issues you encounter so we can address them and make the final releases as rock-solid as possible.
We have now two release channels: a stable one that should be used for production systems and a pre-release channel that should be used to try the release candidates such as TheHive 3.1-RC1 and Cortex 2.1-RC1, the subjects of this blog post, and help us iron out bugs before adding them to the stable channel. Those who love living on the bleeding edge may be tempted by running the release candidates on their production environment given all the candy and icing we added. They are at liberty of doing so but we don’t want to hear anyone one whining about an RC that broke everything and asking for their money back grin.
In previous releases of TheHive, whenever you configured a MISP instance, it was used to import events from and export cases to. Starting from TheHive 3.1.0, we added a purpose to the configuration file. By default, any added MISP instance will be used for import and export (ImportAndExport). However you can configure it to be used for importing events only (ImportOnly) or exporting cases only (ExportOnly).
Extended Events
When an analyst attempts to update a MISP event on which the account used by TheHive to connect to the MISP instance is not part of the original author’s organization, previous versions of TheHive will display a you do not have permission to do that error produced by MISP. Starting from TheHive 3.1, analysts have the ability to create a MISP extended event.
Task Grouping
Case tasks can now be associated with task groups. For example, you could create groups called Identification and Malware Analysis, Containment and Communication and add tasks to them. Of course, this new feature can be used when designing case templates as well.
Import Observables from Analyzer Output
If analyzers produce a set of artifacts in their output (which is the case of several existing ones), TheHive will give you the ability to select those artifacts very easily and add them to your case as observables.
ZIP File Upload
Austin Haigh contributed an important feature which will allow analysts to directly import password-protected ZIP files into a case. The code uses the supplied password when adding the archive to extract its contents and add them one by one to the existing set of observables. This is highly practical when you want to add suspicious files without risking an accidental click which would compromise your endpoint or having to unzip archives containing such files first then add them one by one to TheHive.
Revamped Search Page
The search page has been completely revamped as shown in the screenshot below:
The New Search Page
You can now select your search scope (cases, tasks, observables, alerts, analyser reports a.k.a. jobs or even the audit logs), apply filters and search TheHive without having to resort to complex, mind numbing Lucene syntax.
Responders and PAP
Last but not least, TheHive and Cortex offer you response capabilities (i.e. perform an action depending on the context) thanks to a new breed of programs called … wait for it … wait for it … responders. TADA!
Responders are very similar to analyzers. In fact we’ve taken the concept and extended it to apply to different elements in TheHive: alerts, cases, tasks, task logs, and observables of course.
Responders in Action
You can reuse almost the same principles that apply to analyzers to write your own responders and if you are feeling generous, contribute them to the community. To give you a head start, we published a sample Mailer responder which, when customized for your environment, should allow you to send emails to inform your fellow analysts that a case has been created and that their help is required. Another example could be the ability to respond to a suspicious email report from a user, which is displayed as an alert, that they can safely ignore the corresponding email.
Like an analyzer, a responder can have two or more service interaction files (or flavors) to allow it to perform different actions. For example, a Mailer responder can send messages using several body templates.
Thanks to our long-time friend Andras Iklody from MISP Project who brought that to our attention, responders (and analyzers starting from Cortex 2.1) support PAP, the Permissible Actions Protocol.
Running into Troubles?
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
TheHive offers a powerful yet generic query API for all the data stored by the platform in the underlying Elasticsearch database.
Thanks to its DSL (Domain Specific Language), TheHive can handle complex search queries such as the following:
Among all the unassigned tasks, show me all those associated with cases which severity is high but also contain the highest number of observables which datatype is ‘mail’
When faced with such complex queries, TheHive translates them using its DSL and sends them over to Elasticsearch to obtain the results. TheHive’s dashboards draw their power from such querties.
And while such capability is highly desirable in our opinion, a capability that we will further leverage to add a completely revamped search module in the upcoming Cerana 1 (TheHive 3.1) release, it greatly complicates RBAC (or multi-tenancy) in TheHive.
A Sneak Peek at the New Search Module of the Upcoming Cerana 1 (TheHive 3.1) Release
Indeed, in the RBAC world, the conversion of any search queries submitted to TheHive into an Elasticsearch one is fully dependent on the user context. The user view must be kept within the boundaries of the group or groups to which they belong. Each search filter, each search parameter, must return only the results that the user can view.
The data scope needs to be clearly identified at the case level. To perform a search against task logs for example, TheHive will need to identify the parent task log, then identify the parent case and only then verify the scope. This is no small undertaking.
Similarities across cases or alerts, such as the Related Cases feature or the relationships between a given alert and existing cases, would need additional work that has not been clearly identified at this stage. But the difficulties do not stop there. Any element that has no clear relationship with case entities will have to be singled out and specific code would need to be added to limit access according to the RBAC rules. This will be clearly the case for the audit trail. Also, what should TheHive display when an analyst group is working on a case that shares observables with another one belonging to a different group? Shall it allow a limited view without any details so that groups may request from a super administrator to authorize both groups to collaborate on the investigation, something that distributed CERTs or SOCs in a large corporation may desire? Or shall it keep the data completely isolated as MSSPs which serve multiple customers with a single instance will require? We know the answer: make it configurable. But take a step back and think of the implications at the code (and security) level.
Contrary to the feature we added to Cortex 2, which allow multiple organizations to use a single Cortex instance, multi-tenancy in TheHive is a much more complex feature to implement and which is expected to have a significant impact on the platform’s performance. It will also need extreme caution to avoid blind spots that attackers (and not so innocent tenants) may exploit to circumvent scope limitations and extend their view to data they are not supposed to access. That’s why we had to delay it to Cerana 2 (TheHive 3.2), currently planned for the end of October 2018.
If you are well versed in Elasticsearch and Scala and willing to help implement this feature, please contact us at support@thehive-project.org.
Cortex, a free, open source software allows security analysts and threat hunters to analyze and enrich observables (IP addresses, hashes, domains, …) collected in the course of an investigation or received from third parties, for example through MISP, the de facto standard for threat sharing.
On March 29, 2018, we released Cortex 2, a major improvement over the previous version which brought, among other cool features, authentication, caching, multi-tenancy (RBAC) and rate limiting. Instead of deploying several Cortex 1 instances behind reverse proxies which would implement authentification, administrators can deploy a single Cortex 2, create multiple organizations and serve the needs of various information security populations while enjoying extra features.
On May 31, 2018, we published a brand new API guide so that developers can take advantage of the powerful REST API of the product. Sadly, Cortex4py, the FOSS Python library we provide to interact with the API was not compatible with Cortex 2. Until today.
Thanks to the hard work of our dear Nabil Adouani, we are happy to announce the immediate availability of Cortex4py 2.0.0, a complete rewrite of the library in Python 3. Cortex4py 2.0.0 is fully compatible with Cortex 2. However, it doesn’t work with Cortex 1.
While TheHive, the highly popular free and open source Security Incident Response Platform (SIRP) we develop has native support for many Cortex 2 instances, Python developers can leverage Cortex4py to interact with Cortex 2, manage organizations, users, analyzer configurations and analyze observables at scale from alternative SIRPs, SIEMs or custom scripts thanks to the 83 analyzers Cortex 2 has as of June 18, 2018.
Cortex 2: there is more than one way to interact with it
Use It
To install Cortex4py, use PIP3:
$ sudo -H pip3 install cortex4py
If you are using Python on a Windows operating system, please forgo the sudo command.
Usage
Cortex4py 2 comes with a usage guide which includes many examples. For example, if you want to fetch the last 10 successful jobs that have been executed against domain names and display the result summaries of those 10 jobs you could write something like:
Sample Python3 code to retrieve Cortex analyzer results
Migrating from Cortex4py 1
If you have already written scripts using Cortex4py 1.x (for Cortex 1), we tried to keep the already available methods. However, we recommend you adapt your code to leverage the new Cortex4py 2 classes and methods as soon as feasible. Moreover, the existing scripts must be updated to support authentication if you intend to use them with Cortex 2. Please read the Cortex4py 2 usage guide for more information.
Support
Cortex 2.0.0 is brand new software. As such, it might contain bugs and limitations. If you find any or encounter problems, please ask on our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
On June 6, 2018, we released Cortex-Analyzers 1.10, which contained 11 new analyzers, bringing the total to 83 programs. You read that correctly: 83 ways to assess and gain insight on observables collected during the course of an investigation or while performing threat intelligence thanks to Cortex, our free & open source analysis engine. One day after, we published version 1.10.1 which fixed the name of the Anomali STAXX reports for TheHive. Since then we uncovered a few additional issues which version 1.10.2 corrects:
TheHive Project Chefs are pleased to announce the immediate availability of Cortex-Analyzers 1.10.1. To install this new release and benefit from 11 new analyzers and some fixes:
$ cd /path/to/Cortex-Analyzers
$ sudo git pull
$ for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done
After running these commands, read the Analyzer Requirements Guide, log into the Cortex 2 Web UI as an orgAdmin, click on the Refresh Analyzers button in the Cortex Web UI, configure the new analyzers and enjoy!
Cortex: click on Refresh Analyzers after updating to the latest Cortex-analyzers version
This analyzer comes in only one flavor. You need a valid FireEye iSIGHT Threat Intelligence subscription to use the analyzer. Retrieve the API key associated with your account and provide it as a value to the key parameter. Obtain the password associated with the API key and provide it as a value to the pwd parameter.
TheHive: FireEye iSIGHT — Long Report Sample (courtesy of Andrea Garavaglia)
GreyNoise
Determine whether an IP has known scanning activity using GreyNoise.
This analyzer comes in only one flavor. It can be used out of the box without configuration. However, if you make many requests, you need to obtain an API key. Please contact GreyNoise to ask for one. Once you get the API key, provide it as the value of the key parameter.
TheHive: GreyNoise — Short and Long Report Samples
IBM X-Force
Query domains, IPs, hashes and URLs against IBM X-Force Threat Intelligence sharing platform.
This analyzer comes in only one flavor. Access to IBM X-Force Threat Exchange requires an IBM ID. Once you have access to the service, supply the URL of the service as value for the url parameter, the API key associated with your account as value for the key parameter and the associated password as the value of the pwd parameter.
TheHive: IBM X-Force — Long Report Sample (courtesy of Andrea Garavaglia)
Malwares
Query Malwares.com and get reports on files, hashes, domain names and IP addresses.
The analyzer comes in two flavors:
– Malwares_pDNS_GetReport: get the latest Malwares report for a file,
hash, domain or an IP address.
– Malwares_pDNS_Scan: scan a file or URL.
You need to sign up for a Malwares.com account. An API key to use the service’s API should be associated with your account. Supply it as the value of the key parameter.
TheHive: Malwares_GetReport — Short and Long Report Samples
MnemonicPDNS
Query IP addresses and domain names against Mnemonic Passive DNS service.
This analyzer comes in two flavors:
Mnemonic_pDNS_Public: query Mnemonic’s public service.
When using the public service, the analyzer can be used out of the box with no further configuration. When using the closed service, you need to contact Mnemonic to get an API key which you’ll need to supply as the value of the key parameter.
TheHive: Mnemonic PDNS — Short and Long Report Samples
StaxxSearch
Fetch observable details from an Anomali STAXX instance.
This analyzer comes in only one flavor. You need to install an Anomali STAXX instance or to have access to one to use the analyzer. Supply the following parameters to the analyzer in order to use it:
auth_url: URL of the authentication endpoint.
query_url: URL of the intelligence endpoint.
username: the STAXX user name.
password: the STAXX password.
cert_check: boolean indicating whether the certificate of the endpoint must be checked or not.
cert_path: path to the CA on the system to validate the endpoint’s certificate if cert_check is true.
TheHive: StaxxSearch — Short and Long Report Samples (courtesy of Robert Nixon)
StopForumSpam
Query StopForumSpam to check if an IP or email address is a known spammer.
You need to define the thresholds above which the analyzed observable should be marked as suspicious or malicious.
TheHive: StopForumSpam — Short and Long Report Samples
ThreatCrowd
Look up domains, mail and IP addresses on [ThreatCrowd(https://www.threatcrowd.org/), a service powered by AlienVault.
This analyzer comes in only one flavor. No configuration is needed. It can be used out of the box.
TheHive: Threatcrowd — Short and Long Report Samples
Unshortenlink
Follow redirects of shortened URLs to reveal the real ones.
This analyzer comes in only one flavor.
No configuration is required. It can be used out of the box.
Warning: using this analyzer without extra caution might lead to unexpected consequences. For example, if the URL you are seeking to unshorten is an attacker-controlled one, you may end up leaving undesired traces in the threat actor’s infrastructure logs. The TLP values Cortex allows you to configure to prevent the use of an analyzer if the TLP associated with an observable is above the authorized level won’t be of much help since Unshortenlink have to access the shortened URL. Please do not activate this analyzer unless you (and your fellow analysts) know what they are doing.
TheHive: Unshortenlink — Short and Long Report Samples
Additional Enhancements
YARA analyzer had a bug which was fixed in version 1.9.7 of the Cortex-analyzers repository. If you install 1.10, you’ll obviously benefit from the fix 😉
A permission problem that prevented using the Cuckoo Sandbox analyzer was corrected (thanks Felix Bauer!)
Correction: June 6, 2018 An earlier version of this post mentioned that GreyNoise needs an API key. That’s only necessary if a certain level of requests are made. Also, to configure analyzers, you need to be orgAdmin.
Rejoice DFIR fighters! Your favorite Security Incident Response Platform has just gotten better. TheHive 3.0.10 (codename Cerana 0.10) is just out of the oven. Thomas and Nabil, as the true code Chefs they are, have fixed a number of issues and added a few enhancements in this release. Read on, update and defend your constituency!
Fixes
#446: keep observable reports when performing a case merge operation
#592: fix the error message when an observable is created with invalid data
#579: rotate application.log to keep it from becoming huge
#564: when submitting a job to Cortex, send the case ID as well
#563: poll the status of connectors (MISP, Cortex) every minute and update the UX display accordingly
#548: sort related cases by the amount of shared observables
#546: display start and end times for closed tasks
Support
Something does not work as expected? You have troubles installing or upgrading? You have spotted new bugs? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
We are proud to announce the immediate availability of Cerana 0.9 (TheHive 3.0.9) and Cortex 2.0.4. These hotfix releases address a number of issues and we encourage you to update your current installation at your earliest opportunity. For your comfort and sanity. Seriously.
We also took this opportunity to update Cortex analyzers to fix issues with CIRCL Passive SSL, Hybrid Analysis, and the Joe Sandbox URL Analysis template. Moreover, we have updated the cortexutils library to set the taxonomy level to info if it is invalid. To upgrade cortexutils to 1.2.4:
Note: the Bluecoat analyzer was removed since it does not comply with the updated Terms of Service of Symantec Web Pulse SiteReview. Symantec does no longer permit programmatic querying of the service.
Fixes in Cerana 0.9
#527: display long reports when the analyst clicks on the corresponding short reports. Meh!
#541: make the drop-down menu for case templates scroll when there is a truckload of them.
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: April 14, 2018 An earlier version of this post did not mention that the Bluecoat analyzer was removed in the latest Cortex Analyzers repository release.
