We are proud to announce the immediate availability of Cerana 0.9 (TheHive 3.0.9) and Cortex 2.0.4. These hotfix releases address a number of issues and we encourage you to update your current installation at your earliest opportunity. For your comfort and sanity. Seriously.
We also took this opportunity to update Cortex analyzers to fix issues with CIRCL Passive SSL, Hybrid Analysis, and the Joe Sandbox URL Analysis template. Moreover, we have updated the cortexutils library to set the taxonomy level to info if it is invalid. To upgrade cortexutils to 1.2.4:
Note: the Bluecoat analyzer was removed since it does not comply with the updated Terms of Service of Symantec Web Pulse SiteReview. Symantec does no longer permit programmatic querying of the service.
Fixes in Cerana 0.9
#527: display long reports when the analyst clicks on the corresponding short reports. Meh!
#541: make the drop-down menu for case templates scroll when there is a truckload of them.
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: April 14, 2018 An earlier version of this post did not mention that the Bluecoat analyzer was removed in the latest Cortex Analyzers repository release.
There’s a new version of your ultimate observable analysis engine in town : Cortex 2.0.3 is out!
Cortex 2.0.3 contains a few important enhancements over its predecessor and fixes a number bugs as described in the full changelog summarised below. So get it while it’s still hot out of the digital oven and let us know how tasty it is.
Source : Quickmeme.com
Implemented Enhancements
#81: reflect proxy changes in the global configuration at the analyzer level
#82: display invalid analyzers and let orgadmins delete them
#85: allow orgadmins to override the default global report cache.job period per analyzer through the Web UI
#75: a version upgrade of an analyzer makes all analyzers invisible in TheHive
#80: fix the analyzer configuration dialog to allow orgadmins to override the auto artifact extraction at the analyzer level
#83: hit Nabil on the head pretty hard until the analyzer refresh UI button works (well now it does so you can stop hitting poor Nabil’s head).
Analyzer Updates
We took the opportunity of a new release to make a few updates to the public analyzers. Cortex-Analyzers 1.9.3 contains the following changes:
Remove the Bluecoat analyzer to comply with the new ‘no scrapping’ ToS imposed by Symantec
Fix the default configuration of the Cymon Check IP analyzer
Fix the View all VT long template
Make the MISP Warning Lists Analyzer ignore case sensitivity when searching for hashes
Restrict the Abuse Finder and FileInfo analyzer dependencies to Python 2.7
You can read the full changelog if you like but if you want to enjoy the goods right away, git pull is your friend.
Support
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
After announcing Cortex 2.0.0 and TheHive 3.0.7, the first version of your favorite SIRP that is (supposedly) compatible with the brand-new version of Cortex, last week, we thought it was time to relax and enjoy the upcoming, long Easter weekend, the sunny sky of Paris (if you can pierce the veil of the Forever Grey Cloud™ that is hanging over the city of lights), and great jazz music. Heck, I even tweeted about it … only to be proven wrong by Life (and Murphy).
We literally field tested Cortex 2 for 3 weeks, we squashed bugs here and there, until almost the very last minute before the release. And yet, our QA needs to be improved by leaps and bounds as we had to release Cortex 2.0.1 one day after unveiling 2.0.0 to correct some additional bugs. And then some members of the core team and of our growing user community took it for a spin. And all hell broke lose. Well, almost 🙂
Session collisions (when TheHive and Cortex 2 are used on the same machine), analyzer malfunctions, connectivity problems … issues that were not identified during the testing phase, even in a production environment, where everything worked as expected. And we call this ‘Computer Science’. Right, right…
So we worked hard, took out our Code Hammer (it’s like Thor’s but cyber) and blasted away all the bugs that we found out or that were reported to us (arigato gozaimasu!) and we are happy to announce the immediate availability of Cortex 2.0.2, TheHive 3.0.8, Cortexutils 1.2.3 and Cortex-Analyzers 1.9.2.
TL;DR Install or upgrade Cortex 2.0.2, update Cortexutils, git pull the Cortex-analyzers repo to get the latest version of the repository, upgrade to TheHive 3.0.8, follow the Quick Start Guide and have a drink.
If you have time (which is admittedly quite scarce nowadays), please read on the changelogs:
As stated in the previous post, we will release a new version of Cortex4py in order to make it compatible with Cortex 2, continue the work we started with our MISP Project friends to support MISP attribute enrichment through Cortex 2 (MISP currently only supports enrichment using Cortex 1), and perform a long-overdue overhaul of our documentation. We will also release a brand new version of TheHive4py.
Last but not least, we’ll take a hard look at ourselves and our QA. You expect us from us high quality and we hold ourselves to high standards. And we will deliver.
Support
Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
TheHive Project’s Master Chefs are extremely happy to share, for free their latest recipe with the Cyber Threat Intelligence, Digital Forensics and Incident Response communities: Cortex 2.
As its predecessor, Cortex 2 is published under an AGPL v3 license and it introduces many important features that we brushed upon in a Dec 2017 post.
Cortex 2 — Architecture
Update: Cortex 2.0.1 was released since this post went live. It corrects a few bugs we uncovered in 2.0.0 as described in the changelog. Please install Cortex 2.0.1 instead of 2.0.0.
Authentication
Cortex 2 supports all the authentication methods that TheHive supports: LDAP, Active Directory, local accounts, API Keys, and X.509 SSO.
To connect your favorite Security Incident Response Platform with Cortex 2, you will need to update TheHive to Cerana 0.7 (TheHive 3.0.7) which was released today as well. This version fixes a regression pertaining to case templates introduced by Cerana 0.6 and is the first version to fully support Cortex 2’s API changes and authentication.
To make TheHive 3.0.7 analyze observables at scale through Cortex 2, you have to create an account on Cortex 2 with the read and analyze roles (see the next section) and generate the associated API Key. Next, feed the key in TheHive’s /etc/thehive/application.confas described in the documentationet voilà !
TheHive 3.0.7 remains compatible with Cortex 1 and you can connect it to a mixed set of Cortex 1 and/or Cortex 2 instances with no issues.
Organizations, Analyzers and Rate Limiting
Cortex 2 introduces multi-tenancy through organizations and each organization can have its own set of users, with different roles, its own set of analyzers and, if necessary, rate limits that will prevent analysts from burning quotas.
Multi-tenancy has several interesting use cases. For instance, if you are the CSIRT or CERT of a large multinational organization with several regional teams, you can create an organization for each region within your constituency and enable the analyzers that they may need to use. Let’s assume that you bought a VirusTotal subscription that limits you to 5000 requests per month. You can configure the corresponding analyzers to give each region a fair share of that quota and keeping some requests for your own use.
In case you are a commercial CSIRT or an MSSP, you could do the same for your customers by installing only one Cortex 2 instance and creating an organization for each customer.
Configure an analyzer graphically and impose rate limits if necessary
User Roles
By default, Cortex 2 is shipped with the default cortex organization which sole purpose is to create other ones and manage the users within each organization and their associated powers. The cortex organization hosts all users with the superAdmin role and it cannot be used to configure or run analyzers.
As described in the new Quick Start Guide, after installing Cortex 2, updating its database and creating the first user who will have super admin powers, you’ll have to create your first organization and at least one user within that organization with orgAdmin rights.
Create an organization
You can then log out and log in using the orgAdmin account to create further users within that organization, enable and configure analyzers etc. Please note that no analyzer is enabled by default and you need at least v 1.9.0 of the cortex-analyzers repository. To update your set of analyzers to 1.9.0, please run git pull.
Manage users within an organization
Besides the superAdmin and orgAdmin roles, Cortex 2 introduces the read role which allows users to access analyzer reports and read them but not execute analyzers. For that, users need the analyze role (which implies the read role). orgAdmin users can also run analyzers. superAdmin users are limited to the default cortex organization. While they can create organizations and manage users within them, they cannot access analyzer configurations such as confidential API keys or job reports.
Job reports
Report Persistence and Caching
Cortex 2 relies on Elasticsearch 5.x to store many configuration items but also all the analyzer reports that have been generated. Unlike its predecessor, you won’t lose your existing reports should you need to restart the service or the host it is running on.
Cortex 2 also introduces report caching. By default the cache.job parameter is set to 10 minutes in /etc/cortex/application.conf. That means that if an analysis on a given observable with a defined TLP is requested and that a report has been previously generated in the last 10 minutes, Cortex 2 will serve that report instead of running a new analysis. This feature can help prevent soliciting analyzers, particularly those which require a subscription or have quotas, when there is no need to do so. Please note that this parameter is global to all the analyzers and all the organizations that are configured in the Cortex 2 instance. We do have plans to make it more granular in future versions.
Migrating from Cortex 1
If you are migrating from Cortex 1.x, we recommend that you:
Save the configuration of your analyzers (which ones are enabled and what their configuration items are, such as users/passwords or API keys).
Install Cortex 2.
Edit /etc/cortex/application.conf to add the secret key as shown in Step 1 of the Quick Start Guide and point Cortex to the location of the analyzers.
Follow the remaining steps of the Quick Start Guide to enable the analyzers you need and reinject their configuration.
What’s Next?
In the upcoming weeks, we will release a new version of Cortex4py in order to make it compatible with Cortex 2, continue the work we started with our MISP Project friends to support MISP attribute enrichment through Cortex 2 (MISP currently only supports enrichment using Cortex 1), and perform a long-overdue overhaul of our documentation.
Feeling Generous? Donate!
As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.
All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.
Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: March 30, 2018 Instructions on how to update the cortex-analyzers have been added. Also, Cortex 2.0.1 was released to correct a few bugs in the previous version since this post went live.
Update: Cerana 0.5 (TheHive 3.0.5) was released on Feb 8, 2018 to fix a regression introduced by Cerana 0.4, pertaining to the observable mini-reports. Please install 3.0.5 instead.
Waiting for the menu, one of the numerous guests of TheHive Project’s world-renowned code restaurant grows impatient. He stands up and walks toward the kitchen, looking for a waiter to chastise. It’s certainly a free meal and, looking at the reviews on CodeAdvisor, a delicious one. But guests shouldn’t wait more than two minutes to get a copy of the menu, right?
As he approaches the door leading to the kitchen, it slams open and an all-smiling, all-French-looking man dressed in a bee costume comes out carrying hot plates. Peering at their content, the once angry guest cheers up as he stares down at Cerana 0.4 or TheHive 3.0.4 if you prefer. If the looks and smells of the dish match its taste, this will be a terrific meal.
As he sees how Cerana 0.4 prompted a change of heart of the guest, the waiter coming out of the kitchen where TheHive Chefs were busy creating this new recipe goes on describing its content.
‘Mon bon Monsieur‘, he starts in near-perfect Parisian-English accent, ‘this is the plat du jour, or the dish of the day if you prefer. Even though the mighty Chefs insist on calling it a hotfix, not only it fixes eleven bugs in previous versions but it adds four new features and two enhancements. Would you like me to describe them?’
The guest, feeling hungry, his mouth watery, but feeling bad about what he was about to say to the waiter just a few seconds ago nods. So the waiter goes on and describe how the latest release of TheHive will serve his hunger for efficient though free security incident response platforms even better.
Fix a nasty issue where the assignee list is not displayed when there are more than 20 users. This was impacting all drop down lists than contain dynamic data.
An alert is uniquely identified by a tuple: type; source; sourceRef. However, the type parameter was simply ignored. Nice fail (*cough*).
When a report template is not defined for a given analyzer, TheHive tries to load a default one that shows the raw job report. However, the path to that default template was broken.
Last but not least, filter MISP events using MISP tags & more before creating alerts. That way, you won’t have to deal with a mind-numbing number of alerts when you connect an event-loaded MISP instance to TheHive for the first time. Please check out the documentation to leverage this nifty feature.
At this point, the guest cannot control his lust so he snatches one of the plates out the waiter’s hands, runs back to his table and start installing Cerana 0.4 to unleash its power and use it to fight cybercrime. Bon appétit !
Feeling Generous? Donate!
As you know, we are a FOSS project and donations are always welcome to make our products even better for the community.
All donations go to Creative Source, the non-profit organization we have created, and we will use them to improve TheHive, Cortex & Hippocampe but also to develop (even better) integrations with other FOSS solutions such as MISP.
Creative Source can also provide so-called professional, entreprise-grade support, help integrating the products, train your analysts before they drain or assist you in specific areas such as developing in-house analyzers for Cortex.
Download & Get Down to Work
If you have an existing installation of TheHive, please follow the migration guide.
If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Correction: Feb 6, 2018 An earlier version of this post contained an incomplete sentence.
Version 1.4.0 of the Python API client for TheHive is now available. It is compatible with the freshly released Cerana (TheHive 3.0.0).
We’d like to thank Nick Pratley, a frequent contributor, Bill Murrin, Alexander Gödeke and “srilumpa” for their code additions and documentation.
To update your existing package:
$ sudo pip install thehive4py --upgrade
If you are just getting started with TheHive4py, you can forgo the --upgrade at the end of the command above.
New Features
#5: Add a method to update a case, contributed by Nick Pratley
#34: Add a get_task_logs method in order to obtain all the task logs associated with a given taskId. Contributed by Bill Murrin
#37: A new, very cool case helper class by Nick Pratley
#39: Add support for custom fields to the case model
#40: Ability to run a Cortex analyzer through the API by Alexander Gödeke
#45: Simplify case creation when using a template by providing just its name
#49: Add a query builder capability to support TheHive’s DSL query syntax
Paris? Are you There?
Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. As usual, we’ll be more than happy to help!
Update: 2 days after publishing this blog post, we’ve released Cerana 0.1 (TheHive 3.0.1) which fixes a number of issues. We encourage you to use 3.0.1 instead of 3.0.0.
The friendly honeybees at TheHive’s code kitchen were pretty busy lately even though winter came and temperatures have been close to zero Celsius in Paris, France. As we wrote a couple of weeks ago on this very blog, we are happy to announce Cerana to the world, available immediately.
Cerana or TheHive 3.0.0 is the latest (and obviously greatest) release of a now highly popular open source, free Security Incident Response Platform (or SIRP for short). Its flagship feature in comparison to previous releases is Dynamic Dashboards.
Dynamic Dashboards
Dynamic Dashboards replace the Statistics module in Cerana to allow you to explore the data available in Elasticsearch, which TheHive uses for storage, in many ways. For example, you can have a usage breakdown of Cortex analyzers, the number of open cases per assignee, the number of alerts per source (MISP, email notifications, DigitalShadows, Zerofox, Splunk, …), the number of observables that have been flagged as IOCs in a given time period, how many attributes were imported from MISP instances, top 10 tags of imported MISP attributes or incident categories.
Dynamic Dashboards
Dynamic Dashboards can be created by an analyst and kept private or shared with the other team members. Dashboards can also be exported and imported into another instance. This would facilitate community participation in the establishment of valuable data exploration graphs to drive DFIR activity and seek continuous improvement.
When you’ll migrate to Cerana, you won’t have to build dashboards from scratch. We recreated more or less those which were available under the Statistics view and included them in the Cerana build.
Cortex and MISP Health Status
Cerana will also allow you to monitor the health status of all the Cortex and MISP instances that it is connected to. In the bottom right corner of TheHive’s Web UI, the Cortex and MISP logos appear when you have configured the integration with those products as in previous releases. However, the logos will have a small outer circle which color will change depending on whether Cortex and/or MISP instances are reachable or not.
Cortex & MISP Health
If TheHive can’t reach N out of M Cortex/MISP instances, the outer circle will be orange. If it can’t reach all M instances, the circle will red. If everything is fine, the circle will be green. The exact status of each Cortex/MISP instance can be seen in the About page. And when you try to run analyzers on a Cortex which cannot be reached, TheHive will tell you so as well.
Cortex & MISP: Version & Status
Sighted IOCs
In previous releases of TheHive, observables can be flagged as IOCs. However, this doesn’t necessarily mean you’ve seen them in your network. Think for example of a suspicious attachment which you’ve submitted to Cuckoo or Joe Sandbox through Cortex. The analyzer returns some C2 addresses to which the sample tries to connect to. You’d be right to add those C2 addresses to your case and flag them as IOCs. Then you search for them in your proxy logs and you find connection attempts to one out of four. In previous versions, you’d add a seen label but this would be inconsistent among analysts. One may use found instead. Another will add a description and no labels.
To avoid such situations and give you a simple way to declare an IOC as seen, Cerana adds a sighted toggle which you can switch on/off. We will leverage this toggle in future versions to indicate sightings when sharing back cases to MISP.
Other Features and Improvements
Cerana contains numerous other features and improvements such as:
Case template import, export
The ability to assign default values to metrics and custom fields to case templates
The ability to assign by default tasks to their rightful owners in case templates
Show already known observables when previewing MISP events in the Alerts page
Add autonomous systems to the list of default datatypes
Single-sign on using X.509 certificates (in BETA currently)
We will update the documentation for Cerana in the upcoming weeks. So stay tuned.
Download & Get Down to Work
If you have an existing installation of TheHive, please follow the migration guide.
If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.
Support
Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Initially planned for Nov 17, 2017, Cerana, the next major release of TheHive, is delayed by a few days for three reasons: fixing a few minor but nonetheless irking bugs, quality assurance, and adding small but nice features that would have otherwise required a new database migration a short while after performing one during the upgrade to this new version.
The new release date for Cerana (TheHive 3.0.0) is Dec 5, 2017, the same day we’ll have our second joint workshop with the fine people of the MISP Project during the Botconf conference in Montpellier, France (food, wine, sightseeing… well you get the picture).
If we should mention a single major Cerana feature to convince you to install it or take it for a spin, that would be dynamic dashboards, with no hesitation.
While it was enough for a start, the Statistics module doesn’t take advantage of the underlying Elasticsearch storage and the many ways we can play with all the data that analysts keep feeding to TheHive. Not only that but what about custom fields, alerts, and so on? Enter Dynamic Dashboards.
Dynamic Dashboards – Alert types and sources
To put it simply, Cerana will allow you to analyze TheHive data (almost) any way you want and chart it using different options: how many alerts of a certain type have been received during a given period? Over all the cases that are recorded within TheHive, how many observables with a specific tag and flagged as IOCs are there? …
Dashboards can be private to an analyst, shared with fellow TheHive users, imported from another instance and exported. By adding the import/export feature, we hope to foster sharing within TheHive community where teams would impart useful dashboards to their peers. Graphs can also be saved as images to add to reports.
Dynamic Dashboards – Sources of observables
To alleviate upgrades, Cerana will come with a few dashboards out of the box to mimic the Statistics module hence you won’t lose existing functionality when you make the move. At this stage, we’d like to remind you that we only support the current release and the previous one. When Cerana will be published, we’ll obviously support it (genius, n’est-ce pas ?) as well as Mellifera 2.13.2. Nothing else.
Dynamic Dashboards – Case status, resolution and impact
Cerana will also give you the ability to import and export case templates, a feature that has been requested by our growing user base. This could be a first step towards a global repository where case templates can be shared, refined and created according to common standards, regulations or compliance requirements. Think LPM in France, NIS in Europe, GDPR, etc. Case templates will also be improved to contain default metrics values if needed and automatically assign tasks to given analysts.
Another addition worth mentioning is the sighted flag for IOCs. When an analyst flags an observable as IOC and as sighted, it means that observable is not simply something coming from a sandbox analysis (think C2) or from a 3rd party but was confirmed as being used by a threat actor in your network. In a later release, exporting cases to MISP instances will make use of this new flag to feed MISP attribute sightings. The sighted value will also be used in the future to improve alert previewing.
Last but not least, Cerana will supervise the ‘health’ of the Cortex and MISP instances it is integrated with. The Cortex and MISP logos at the bottom right corner of TheHive UI appear when integration with those products is enabled. They will also have a coloured circle to indicate health:
Green: TheHive can reach all of the configured Cortex/MISP instances.
Orange: TheHive cannot reach all of them.
Red: no instance can be reached.
There are other areas (the About page, the observable analysis buttons…) where the health of Cortex and/or MISP can be monitored.
Now, if you don’t mind, we have some coding to do. We’d better get back to it if we want to give you a luscious release. À bientôt !
Earlier today, the French (but nonetheless happy) Chefs of TheHive’s code kitchen released DigitalShadows2TH, an alert feeder for TheHive that can consume incidents and intel-incidents from Digital Shadows, a Threat Intelligence provider and feed them as alerts to your favorite Security Incident Response Platform.
We are glad to do the same for ZeroFOX, a social media monitoring platform, with Zerofox2TH. If you are a ZeroFOX customer with a valid API subscription and use TheHive for managing your security incidents and investigating them, you can now feed alerts generated by ZeroFOX to TheHive. Ain’t that joli?
Zerofox2TH is released under an AGPLv3 license (read: free and open source). To use it, you’ll need Python 3, the requests and pillow libraries as well as TheHive4py. You also need TheHive 2.13 or better, with an account on your SIRP that can create alerts.
Please read the README file to learn how to install, configure and run this alert feeder.
Need Help?
Something does not work as expected? No worries, we got you covered. Please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
Thanks to its REST API and alerting framework, TheHive can receive alerts from multiple sources: email notifications, SIEMs, IDS/IPS and, of course, one or several MISP instances.
While the integration with MISP is native and very easy to configure, teams need to develop their own code to feed alerts from other sources to TheHive, leveraging whenever possible TheHive4Py, a very handy Python library to interact with the API.
If you are a TheHive user and a Digital Shadows customer, you can now fetch any incident or intel-incident raised by their Searchlight service using DigitalShadows2TH, a free, open source alert feeder for TheHive freshly cooked by your friendly and so Frenchy Chefs behind TheHive Project.
To use DigitalShadows2TH, you’ll need Python 3, the requests library and TheHive4py. You also need a Digital Shadows subscription and TheHive 2.13 or better with an account on your SIRP that can create alerts.
Please read the README file to learn how to install, configure and run this alert feeder.
Need Help?
Something does not work as expected? No worries, we got you covered. Please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.
TheHive Project 