Cortex 2: a Sneak Peek

Unless you’ve been living in a cave with no Internet connection during the last year or so, you certainly know a thing or two about Cortex, TheHive’s perfect sidekick, which allows you to analyze observables, at scale, using its 30+ analyzers.

As of this writing, the latest version of Cortex is 1.1.4. Cortex can be queried using its Web UI for quick assessment of an observable. But the true power of Cortex is unleashed when the engine is queried through its REST API, either from TheHive (which can leverage multiple Cortex instances), from alternative SIRPs (Security Incident Response Platforms), Threat Intelligence Platforms and programs thanks to Cortex4py. Indeed, when Cortex is called through the API, it can analyze large sets of observables. Each analysis generates a job. Jobs are queued on first-created, first-executed basis.

However, Cortex 1 has three limitations:

  1. It does not support authentication. If you install it and don’t shield it from abuse (using a firewall for example), anyone can submit analysis jobs and consume your query quotas for subscription-based, commercial services, for example. Non-CSIRT/CERT/SOC personnel or threat actors can also view all the jobs you’ve executed (what observables you have analyzed, using which analyzers and what the associated results were).
  2. It does not support rate-limiting. All it takes to ruin your quotas is an unexperienced analyst who’d create a case in TheHive from a MISP event containing thousands of attributes, select them all from the newly created case, and run them through various Cortex analyzers.
  3. It has no persistence. If you restart the Cortex service or the host it runs on, all your analysis results will disappear. Please note that if you query Cortex from TheHive, the latter will keep a copy of all the reports generated by the analyzers.

Moreover, analyzer configuration is not as easy as we’d like it to be. Enters Cortex 2.

Authentication, Organizations, Configuration and Rate Limiting

Cortex 2, due for release in February 2018, almost a year after the release of the first version, will support all the authentication methods TheHive supports: LDAP, Active Directory, local accounts, API keys and/or SSO using X.509 certificates (an experimental feature as of this writing).

Once created, users will be associated to an organization. Each organization has its own configuration: which analyzers are enabled, associated API keys and/or authentication credentials for services (VirusTotal, PassiveTotal, MISP, …) and a query quota.

For example, if you have an overall quota on VT for 10,000 queries/month, you can limit the number of queries to 5000 for org A, 3000 for org B and leave 2000 for other uses. Rate limits can be configured per month or per day.

Screen Shot 2017-12-15 at 17.16.06
Cortex 2 — Architecture

More on Organizations

Organizations will be ideal for multi-tenant Cortex instances deployed, for a example, by the central CSIRT of a large company. They can then create orgs for their regional SOCs. Commercial teams such as MSSPs will also be able to use a single instance to serve all their customers.

Graphical Interface Enhancements

Administrators will not have to edit /etc/cortex/application.conf by hand to enable and configure analyzers per org. They will be able to do so from the Web UI. The Web UI will also allow them to manage users, orgs and authentication tokens when applicable.

Report Persistence and Freshness

Cortex 2 will use ES 5 for storage, like TheHive. That way, you will no longer lose your existing jobs when you reboot the Cortex host or restart the service. You will also be able to query historical results to monitor changes and so on. We will also add an optional parameter to make Cortex 2 to serve the latest report generated by an analyzer if it is called again, on the same observable in the last X seconds or minutes. That way, we’ll avoid running the same queries again and again for the same observable and thus consuming quotas and CPU and storage resources.

Pricing

Cortex 2 is a significant development over Cortex 1 … but it’ll still cost you nothing as it will remain free and open source. We could feel you itching when you started reading this paragraph. Chill out! But if you are willing to support the project, you can donate to Creative Source, the non-profit organization we have created to sustain TheHive, Cortex and Hippocampe in the long run. Interested? Contact us at support@thehive-project.org then.

Creative Source, a New Ally and a New Home

The Chefs who’ve been working hard to create delicious recipes in TheHive Project’s code kitchen are happy to announce the establishment of Creative Source, a non-profit organization, which aims to support TheHive, Cortex and Hippocampe.

Who’s behind this NPO?

Creative Source is co-managed by all the members of TheHive Project’s core team: Nabil Adouani, Thomas Franco, Danni Co, Saâd Kadhi and Jérôme Léonard. Work is in progress to provide Creative Source with a Web face.

What Will you Provide through It?

We have already started working with a couple of large organizations to provide trainings, limited support and assistance in Cortex analyzer development. All the money Creative Source is going to gain will serve to further support the project and keep refining our recipes to make them even more palatable.

If you are interested in funding the project, training your analysts or if you are looking for professional assistance with our products, please contact us at support@thehive-project.org.

Will TheHive, Cortex and Hippocampe Stay Free?

Don’t you dare ask that question! TheHive, Cortex and Hippocampe will stay free and open source in the foreseeable future as we are deeply committed in helping the global fight against cybercrime to the best of our abilities.

New Ally

We are also very happy to announce that Nils Kuhnert (a.k.a. @0x3c7 on Twitter), a longtime contributor, has now joined TheHive Project! We are no longer a pure French project, damn! 😉

Nils, who created many analyzers, will work mainly with Jérôme to deal with existing and new ones and absorb the numerous pull requests that have been piling up for many months. Welcome on board Nils!

New Home

IMG_4034
Author : Saâd Kadhi

To accommodate Nils and future members, our code and documentation will leave the lofty shelter of CERT-BDF‘s Github and move to https://github.com/orgs/TheHive-Project/  on Wed Dec 20, 2017. Save the date folks!

Featured

Introducing Cerana

Update: 2 days after publishing this blog post, we’ve released Cerana 0.1 (TheHive 3.0.1) which fixes a number of issues. We encourage you to use 3.0.1 instead of 3.0.0.

The friendly honeybees at TheHive’s code kitchen were pretty busy lately even though winter came and temperatures have been close to zero Celsius in Paris, France. As we wrote a couple of weeks ago on this very blog, we are happy to announce Cerana to the world, available immediately.

Cerana or TheHive 3.0.0 is the latest (and obviously greatest) release of a now highly popular open source, free Security Incident Response Platform (or SIRP for short). Its flagship feature in comparison to previous releases is Dynamic Dashboards.

Dynamic Dashboards

Dynamic Dashboards replace the Statistics module in Cerana to allow you to explore the data available in Elasticsearch, which TheHive uses for storage, in many ways. For example, you can have a usage breakdown of Cortex analyzers, the number of open cases per assignee, the number of alerts per source (MISP, email notifications, DigitalShadows, Zerofox, Splunk, …), the number of observables that have been flagged as IOCs in a given time period, how many attributes were imported from MISP instances, top 10 tags of imported MISP attributes or incident categories.

case3.png
Dynamic Dashboards

Dynamic Dashboards can be created by an analyst and kept private or shared with the other team members. Dashboards can also be exported and imported into another instance. This would facilitate community participation in the establishment of valuable data exploration graphs to drive DFIR activity and seek continuous improvement.

When you’ll migrate to Cerana, you won’t have to build dashboards from scratch. We recreated more or less those which were available under the Statistics view and included them in the Cerana build.

Cortex and MISP Health Status

Cerana will also allow you to monitor the health status of all the Cortex and MISP instances that it is connected to. In the bottom right corner of TheHive’s Web UI, the Cortex and MISP logos appear when you have configured the integration with those products as in previous releases. However, the logos will have a small outer circle which color will change depending on whether Cortex and/or MISP instances are reachable or not.

status
Cortex & MISP Health

If TheHive can’t reach N out of M Cortex/MISP instances, the outer circle will be orange. If it can’t reach all M instances, the circle will red. If everything is fine, the circle will be green. The exact status of each Cortex/MISP instance can be seen in the About page. And when you try to run analyzers on a Cortex which cannot be reached, TheHive will tell you so as well.

about
Cortex & MISP: Version & Status

Sighted IOCs

In previous releases of TheHive, observables can be flagged as IOCs. However, this doesn’t necessarily mean you’ve seen them in your network. Think for example of a suspicious attachment which you’ve submitted to Cuckoo or Joe Sandbox through Cortex. The analyzer returns some C2 addresses to which the sample tries to connect to. You’d be right to add those C2 addresses to your case and flag them as IOCs. Then you search for them in your proxy logs and you find connection attempts to one out of four. In previous versions, you’d add a seen label but this would be inconsistent among analysts. One may use found instead. Another will add a description and no labels.

To avoid such situations and give you a simple way to declare an IOC as seen, Cerana adds a sighted toggle which you can switch on/off. We will leverage this toggle in future versions to indicate sightings when sharing back cases to MISP.

Other Features and Improvements

Cerana contains numerous other features and improvements such as:

  • Case template import, export
  • The ability to assign default values to metrics and custom fields to case templates
  •  The ability to assign by default tasks to their rightful owners in case templates
  • Show already known observables when previewing MISP events in the Alerts page
  • Add autonomous systems to the list of default datatypes
  • Single-sign on using X.509 certificates (in BETA currently)

We will update the documentation for Cerana in the upcoming weeks. So stay tuned.

Download & Get Down to Work

If you have an existing installation of TheHive, please follow the migration guide.

If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cerana: a Sneak Peek

Initially planned for Nov 17, 2017, Cerana, the next major release of TheHive, is delayed by a few days for three reasons: fixing a few minor but nonetheless irking bugs, quality assurance, and adding small but nice features that would have otherwise required a new database migration a short while after performing one during the upgrade to this new version.

The new release date for Cerana (TheHive 3.0.0) is Dec 5, 2017, the same day we’ll have our second joint workshop with the fine people of the MISP Project during the Botconf conference in Montpellier, France (food, wine, sightseeing… well you get the picture).

If we should mention a single major Cerana feature to convince you to install it or take it for a spin, that would be dynamic dashboards, with no hesitation.

While it was enough for a start, the Statistics module doesn’t take advantage of the underlying Elasticsearch storage and the many ways we can play with all the data that analysts keep feeding to TheHive. Not only that but what about custom fields, alerts, and so on? Enter Dynamic Dashboards.

 

alerts.jpg
Dynamic Dashboards – Alert types and sources

To put it simply, Cerana will allow you to analyze TheHive data (almost) any way you want and chart it using different options: how many alerts of a certain type have been received during a given period? Over all the cases that are recorded within TheHive, how many observables with a specific tag and flagged as IOCs are there? …

 

Dashboards can be private to an analyst, shared with fellow TheHive users, imported from another instance and exported. By adding the import/export feature, we hope to foster sharing within TheHive community where teams would impart useful dashboards to their peers. Graphs can also be saved as images to add to reports.

observable_sources
Dynamic Dashboards – Sources of observables

To alleviate upgrades, Cerana will come with a few dashboards out of the box to mimic the Statistics module hence you won’t lose existing functionality when you make the move. At this stage, we’d like to remind you that we only support the current release and the previous one. When Cerana will be published, we’ll obviously support it (genius, n’est-ce pas ?) as well as Mellifera 2.13.2. Nothing else.

cases.png
Dynamic Dashboards – Case status, resolution and impact

Cerana will also give you the ability to import and export case templates, a feature that has been requested by our growing user base. This could be a first step towards a global repository where case templates can be shared, refined and created according to common standards, regulations or compliance requirements. Think LPM in France, NIS in Europe, GDPR, etc. Case templates will also be improved to contain default metrics values if needed and automatically assign tasks to given analysts.

Another addition worth mentioning is the sighted flag for IOCs. When an analyst flags an observable as IOC and as sighted, it means that observable is not simply something coming from a sandbox analysis (think C2) or from a 3rd party but was confirmed as being used by a threat actor in your network. In a later release, exporting cases to MISP instances will make use of this new flag to feed MISP attribute sightings. The sighted value will also be used in the future to improve alert previewing.

Last but not least, Cerana will supervise the ‘health’ of the Cortex and MISP instances it is integrated with. The Cortex and MISP logos at the bottom right corner of TheHive UI appear when integration with those products is enabled. They will also have a coloured circle to indicate health:

  • Green: TheHive can reach all of the configured Cortex/MISP instances.
  • Orange: TheHive cannot reach all of them.
  • Red: no instance can be reached.

There are other areas (the About page, the observable analysis buttons…) where the health of Cortex and/or MISP can be monitored.

Now, if you don’t mind, we have some coding to do. We’d better get back to it if we want to give you a luscious release. À bientôt !

Zerofox2TH: ZeroFOX Alert Feeder for TheHive

Earlier today, the French (but nonetheless happy) Chefs of TheHive’s code kitchen released DigitalShadows2TH, an alert feeder for TheHive that can consume incidents and intel-incidents from Digital Shadows, a Threat Intelligence provider and feed them as alerts to your favorite Security Incident Response Platform.

We are glad to do the same for ZeroFOX, a social media monitoring platform, with Zerofox2TH. If you are a ZeroFOX customer with a valid API subscription and use TheHive for managing your security incidents and investigating them, you can now feed alerts generated by ZeroFOX to TheHive. Ain’t that joli?

Zerofox2TH is released under an AGPLv3 license (read: free and open source). To use it, you’ll need Python 3, the requests and pillow libraries as well as TheHive4py. You also need TheHive 2.13 or better, with an account on your SIRP that can create alerts.

Please read the README file to learn how to install, configure and run this alert feeder.

Need Help?

Something does not work as expected? No worries, we got you covered. Please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Training VM Updated with Mellifera 13.2

Two days after the release of  Mellifera 13.2 (TheHive 2.13.2), we have updated the training VM with this version. You can download it from the following location:

https://drive.google.com/file/d/0B3G-Due88gfQMGZ2RjRlc1RfQ2M/view?usp=sharing

To ensure that your download went through nicely, check the file’s SHA256 hash which must be equal to the following value:

15dc0a1d1ef099abd852fefff3a12c1b752573c01b133fc6e643dd2fceb1d46f

The system’s login is thehive and the associated password is thehive1234.

Use It

You can start using TheHive & Cortex once the VM is started. To access TheHive, point your browser to the following URL:

http://IP_OF_VM:9000

For Cortex, the port is 9999:

http://IP_OF_VM:9999

Where to Go from Here?

Please read the associated documentation page to configure the services on your training virtual machine and plug it with MISP.

Need Help?

Something does not work as expected? No worries, we got you covered. Please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Mellifera 13.1 Released

Following the release of Mellifera 13 last week, some users reported problems getting the platform working correctly. They couldn’t browse a case’s tasks. TheHive Chefs reproduced the bug and corrected swiftly in Mellifera 13.1 (TheHive 2.13.1), which is now available. Please note that the identified bug happens only when you haven’t upgraded TheHive from an earlier version.

Is ES 2.x still supported?

Mellifera 13 introduced the support of Elasticsearch 5.x and has been thoroughly tested with version 5.5 (5.6 should be probably work just fine). Given the numerous changes between ES 2.x and ES 5.x, we do not support both versions. Hence, and starting from Mellifera 13, only ES 5.x is supported.

Download & Get Down to Work

If you have an existing installation of TheHive, please follow the migration guideThis is paramount to ensure a good transition from earlier versions. You have been warned.

If you are performing a fresh installation, read the installation guide corresponding to your needs and enjoy. Please note that you can install TheHive using an RPM or DEB package, use Docker, install it from a binary or build it from sources.

Support

Something does not work as expected? You have troubles installing or upgrading? No worries, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.