TheHive 3.2.0-RC1: The MISP Love Edition

Guess what? Our integration with MISP, the de facto standard for threat sharing, has just gotten better with our latest beta release: TheHive 3.2.0-RC1.

While you could synchronize TheHive with one or multiple MISP instances in earlier versions and select events using filters like their age, the number of attributes they contain or exclude those which are created by specific organisations or contain one or several black-listed tags, 3.2.0-RC1 adds the ability to whitelist tags, thus limiting the events that would show up in TheHive’s Alerts pane to only those which have been tagged with labels your SOC/CSIRT/CERT needs to act on. This can be very useful for example if your Cyber Threat Intelligence analysts pre-select or create events in MISP and tag for SOC consumption those that need to be acted on.

The Hive - Logo - Schéma - V1_Plan de travail 1.png
TheHive, Cortex, MISP: The Power DFIR & CTI Trio

To use this feature, use the whitelist.tags parameter in the MISP section of TheHive’s application.conf as described in the documentation.

This new version also adds the ability to create dashboards out of responder actions, log responder operations, and offers a confirmation dialog before running a responder to avoid noob over-clicks and errors made by seasoned incident handlers running low on caffeine.

TheHive 3.2.0-RC1 will also show you the description of an observable if any while hovering over one in the Observables tab. You can also see observable tags when previewing an alert in the Alerts pane.

Last but not least, some users reported severe problems when they enabled TLS/SSL directly on TheHive without resorting to a reverse proxy such as NGINX. Blame that on the crappy TLS support in Play framework ;-). So we highly recommend using a reverse proxy for that purpose, and delegate authentication to it if you are relying on X.509 authentication, as TheHive 3.2.0-RC1 allows you to. Please check the Single Sign-On on TheHive with X.509 Certificates guide for further information.

For additional details on this release, please check the full changelog.

Warning Capt’n Robinson!

The RC in 3.2.0-RC1 stands for Release Candidate. Please help us make a great stable release out of it by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release. You’ll find this release candidate in the pre-release, beta repositories.

Please check TheHive Installation guide for further details.

You got a problem?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

Cortex 101: Dissecting Observables a Hundred Ways

TheHive Project’s Code Chefs are glad to announce that, thanks to the precious contributions of the user community, Cortex has broken the one hundred analyzer mark.

Cortex-Analyzers version 1.14.0 is out and includes new analyzers, some improvements and some bug fixes.

New Analyzers

New and enhanced analyzers, described below, are:

  • Cisco Investigate by Cisco Umbrella Research @opendns
  • Proofpoint Forensics Lookup by Emmanuel Torquato
  • Proofpoint Threat Insight Forensics Analyzer by Emmanuel Torquato
  • RecordedFuture by jojoob
  • urlscan.io search by ninoseki
  • Google DNS over HTTP by 0xswitch
  • RTF files support in FileInfo by Nils Kuhnert
  • Datascan and Inetnum flavors in Onyphe analyzer by Pierre Baudry and Adrien Barchapt

Again, huge thanks for the awesome work that has been performed by all our contributors!

Investigate

Cisco Umbrella Investigate provides threat intelligence about domains and IP addresses accross the Internet. The analyzer can be used to query the Cisco Umbrella (formerly OpenDNS) API and get information about an IP or a domain name. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-investigate-short

sc-investigate-long
Cisco Investigate: short and long reports

Proofpoint Forensics Lookup

According to Proofpoint’s website, the Forensics API allows insight in detailed forensic evidences about individual threats or compaigns. The analyzer can be used to check observables against given indicators of compromise stored in the ProofPoint service.

Unfortunately, there are currently no sample report screenshots available, because TheHive’s Core Team does not have access to Proofpoint services. Also, due to the same reason, this analyzer could not be tested by us. If you have access to the service and can test the analyzer and/or provide report screenshots, please let us know.

RecordedFuture

This analyzer lets you get the latest risk data from RecordedFuture for a hash, domain or an IP address. It can be used to query the API and get information. An API key is required to use this analyzer.

Results are displayed in TheHive in the following manner:

sc-recordedfuture-short

sc-recordedfuture-long
RecordedFuture: short and long reports

Urlscan.io search

Urlscan.io is a service that scans and analyzes websites. Submitted pages will be browsed like a regular user would do and every activity gets recorded. The analyzer submitted by ninoseki queries urlscan without initiating a scan which would be publicly visible on the website. Accepted datatypes for this analyzer are URL, domain, hash and IP.

The templates which display the results of the analyzer look like the following screenshots:

Screenshot from 2018-10-25 09-55-34

Screenshot from 2018-10-25 09-48-58
Urlscan.io: short and long reports

Google DNS over HTTP

This analyzer provides DNS information for an IP, a domain or a FQDN by making calls to Google DNS-over-HTTP (DoH). No API key is required.

Results are displayed in TheHive in the following manner:sc-googleDNS-short

sc-googleDNS-long
Google DNS: short and long reports

RTF files support in FileInfo

The FileInfo meta analyzer has been improved and now leverages the rtfobj tool provided in the Oletools suite by Decalage.

Results are displayed in TheHive in the following manner:

sc-fileinfo_RTF-short

sc-rtfobj-long
FileInfo with rtfobj: short and long reports

Datascan and Inetnum flavors in Onyphe analyzer

The Onyphe analyzer has been enhanced with two new flavors. Datascan provides information about known open ports on a specific IP, and Inetnum enumerates all known network information about the analyzed IP address.

An API key is required to use the analyzer and can be obtained by creating an account on the Onyphe website.

Results are displayed in TheHive in the following manner:

sc-onyphe-inetnum-short

sc-onyphe_inetnum-long
Onyphe  Inetnum: short and long reports

sc-onyphe_datascan-short

sc-onyphe_datascan-long
Onyphe Datascan: long and short reports

Bug fixes and enhancements

  • #248: Improve error msg when VT Get Report does not have an entry for
  • #323: Fix an issue with HybridAnalysis analyzer filenames handler
  • #329: Enhance PassiveTotal Passive DNS report
  • #348: Fix a typo in URLhaus’s long.html
  • #356 : Force the use of Python3 in MISP analyzer
  • #362: Fix file not found issue and empty result set in CERT.at passive DNS analyzer

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.14.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -U -r $I || true; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

TheHive4py 1.5.1 Released

When you need to interact with TheHive’s REST API and you ain’t shy of working with Python, TheHive4py is the way to go. It’s a free, open source library we provide to allow you to easily create alert feeders, automate certain tasks like creating cases, assign them to analysts and much more. For example, Synapse, DigitalShadows2TH and Zerofox2TH leverage the library to send alerts to your favourite SIRP/SOAR.

Sometime ago, we decided that it was time to overhaul the whole library and we began working on version 2.0.0 which will be easier to use. It should also support the full set of TheHive’s REST API calls. In the meantime we decided to release version 1.5.0, shortly followed by version 1.5.1 to support some new functionality contributed by our user community and correct a few issues.

code_quality
Source : XKCD

New Features Introduced in 1.5.0

Bugfixes Introduced in 1.5.0

  • #80: Prevent max recursion depth exceeded error, contributed by Psynbiotik

New Features Introduced in 1.5.1

Important note: TheHive4py 1.5.1 does not work with TheHive 3.0.10 or earlier versions. Please stick with 1.5.0 if you are using those versions.

Updating/Installing

To update your existing package:

$ sudo pip install thehive4py --upgrade

If you are just getting started with TheHive4py, you can forgo the --upgrade at the end of the command above.

But I just Wanna Play!

If you’d like to play around with TheHive4py 1.5.1, TheHive 3.1.1., Cortex4py 2.0.1 and Cortex 2.1.1, please download the training VM.

Paris? Are you There?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. As usual, we’ll be more than happy to help!

Correction: October 12, 2018
As reported by Robin Hahling, TheHive 1.5.1 does not work with TheHive 3.0.10 or earlier versions.

Cortex 2.1.0: The Response Edition

We released Cortex 2.1.0 as a release candidate back in July 31, 2018 along with TheHive 3.1.0-RC1. By then, the power duo which makes digital forensics, incident response and, to an extent, cyber threat intelligence, better, faster, happier, regular exercising gained the ability to perform active response.

We ate our own dog food for a couple of months. We found bugs. We added enhancements and we listened to the early adopters of these new major versions. And today we are thrilled to announce the availability of the stable release of Cortex 2.1.0 along with TheHive 3.1.0.

Cortex 2.1.0 restores the ability to query the analysis and response engine from MISP for enrichment purposes. A new version of the de facto standard for threat sharing should be released shortly as there are also some API-related issues on its side to make the integration fully working again.

mighty-morphin-power-rangers.jpg
Source : Consequence of Sound

Cortex 2.1.0 also gives you the ability to see the PAP (Permissible Actions Protocol) values for each analyzer as well as any custom cache values you might have configured.

You can check out the full changelog and we highly encourage you to install this new version and let us know what you think of it.

Troubles?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

 

 

TheHive 3.1.0: Fresh out of the Oven

TheHive Project’s Master Cooks are happy to announce the immediate availability of TheHive 3.1.0. This is the first release of your favourite SIRP (Security Incident Response Platform) or, if you fancy new buzzwords, SOAR (Security Orchestration, Automation & Response) that we put out as a release candidate to give sufficient time for our ever growing user community to test it and report any outstanding bug before publishing a stable version.

Indeed, TheHive 3.1.0 brings significant new functionalities that we detailed in previous blog posts. One of the most prominent features of this new major version is the support of responders through Cortex 2.1, also released today as a stable version.

Responders are similar to analyzers but instead of analyzing stuff, they allow you to respond to stuff. Put otherwise, they give you the ability to implement specific actions by a simple click from different elements in TheHive: alerts, cases, tasks, task logs and observables.

For instance, imagine a user in your constituency reporting a suspicious email. Using Synapse or an alternative alert feeder, the email reported by the user will automatically show up as an alert in your alert pane. Before starting working on it as a case, you preview it only to realise it is a scam and it does not warrant your time & effort. Still, you’d like to reply to the user.

In such a case, you could implement a responder that will not only send an email back to the user asking them to ignore such a scam but that can mark the alert as read. Using. A. Simple. Click. C’est beau n’est-ce pas ?

Going through all 71 (yes, 71) issues that have been closed with this release and the 3 RCs we published since July 31, 2018 will be terribly boring but you can read the full changelog while dipping your croissant in your espresso cup.

We’d rather encourage you to install this new version, which is as usual, AI-free, machine learning free, cyberbullshit-free, gluten-free, organic (well as much as free, open source software can be anyway), vegan (if you can eat it), and most importantly made with huge love and care for the SOC, CSIRT & CERT communities and other fellow cybercrime fighters. So go ahead and try it out. It won’t cost you a dime (or a franc if you are a French old timer).

Caum6EmUMAA6vi8

Need Help?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

How to Cruise Ocean Threat Without Sinking Using TheHive 3.1 & Cortex 2.1

Paris, France. The Sun is shining on the city of lights and temperatures are quite high, even for the summer season. Life is good. As a matter of fact, life is excellent.

TheHive Master Cooks are about to go on vacation for a few weeks. But before they pack up their Patagonia bags and leave the sandy beaches for those who enjoy them, preferring mountains, trails, walking and breathing fresh air with family and friends, they would like to make a significant contribution to help TheHive and Cortex users fight cyberattacks even better than they already do. And maybe convince those who don’t that free, open source software is not a joke or a geek fad.

We’d like to welcome to the stage our latest babies, which we are really proud of: TheHive 3.1 and Cortex 2.1, the new versions of the power duo which make digital forensics, incident response and, to an extent, cyber threat intelligence, better, faster, happier, regular exercising (well you know the Radiohead song so we’ll let you continue singing along) since early 2017.

While our project might seem very young, it is not. We’ve been working steadily on TheHive, using it (i.e. eating our own dog food) since early 2014 before releasing it at the end of 2016 once we were satisfied with it, as a token of gratitude to a community that helped us due our jobs in various ways. We then extracted what has become Cortex from its core to ship it as a separate product in February 2017. And we kept improving them at a steady piece for the collective benefit of incident responders, forensicators and threat analysts. And adoption has been rather spectacular. Thanks to all of our users for their love and support!

We believe we are at a moment where people could not brush us off anymore as amateurs. Try TheHive and Cortex, preferably with MISP and get a taste of what professional, free and open source software can be.

TheHive 3.1 and Cortex 2.1 are feature-packed and we won’t be able to cover them all in detail in a single blog post. Rather, we’d like to concentrate on a few important ones.

Stable, Pre-release Channels and New Repositories

We’d like to point out that, in order to improve our release process and given the number of features that we added, TheHive 3.1 and Cortex 2.1 are release candidates at this stage. So we encourage you to test them and report back any bugs or issues you encounter so we can address them and make the final releases as rock-solid as possible.

We have now two release channels: a stable one that should be used for production systems and a pre-release channel that should be used to try the release candidates such as TheHive 3.1-RC1 and Cortex 2.1-RC1, the subjects of this blog post, and help us iron out bugs before adding them to the stable channel. Those who love living on the bleeding edge may be tempted by running the release candidates on their production environment given all the candy and icing we added. They are at liberty of doing so but we don’t want to hear anyone one whining about an RC that broke everything and asking for their money back grin.

We also moved our package and binary repositories to https://bintray.com/thehive-project and Docker images are now under thehiveproject.

Please check TheHive Installation and Cortex Installation guides for further details.

MISP with a Purpose

In previous releases of TheHive, whenever you configured a MISP instance, it was used to import events from and export cases to. Starting from TheHive 3.1.0, we added a purpose to the configuration file. By default, any added MISP instance will be used for import and export (ImportAndExport). However you can configure it to be used for importing events only (ImportOnly) or exporting cases only (ExportOnly).

Extended Events

When an analyst attempts to update a MISP event on which the account used by TheHive to connect to the MISP instance is not part of the original author’s organization, previous versions of TheHive will display a you do not have permission to do that error produced by MISP. Starting from TheHive 3.1, analysts have the ability to create a MISP extended event.

Task Grouping

Case tasks can now be associated with task groups. For example, you could create groups called Identification and Malware Analysis, Containment and Communication and add tasks to them. Of course, this new feature can be used when designing case templates as well.

Import Observables from Analyzer Output

If analyzers produce a set of artifacts in their output (which is the case of several existing ones), TheHive will give you the ability to select those artifacts very easily and add them to your case as observables.

ZIP File Upload

Austin Haigh contributed an important feature which will allow analysts to directly import password-protected ZIP files into a case. The code uses the supplied password when adding the archive to extract its contents and add them one by one to the existing set of observables. This is highly practical when you want to add suspicious files without risking an accidental click which would compromise your endpoint or having to unzip archives containing such files first then add them one by one to TheHive.

Revamped Search Page

The search page has been completely revamped as shown in the screenshot below:

41841154-d8d8fa5c-7867-11e8-8837-2a12a06a52a7.png
The New Search Page

You can now select your search scope (cases, tasks, observables, alerts, analyser reports a.k.a. jobs or even the audit logs), apply filters and search TheHive without having to resort to complex, mind numbing Lucene syntax.

Responders and PAP

Last but not least, TheHive and Cortex offer you response capabilities (i.e. perform an action depending on the context) thanks to a new breed of programs called … wait for it … wait for it … responders. TADA!

Responders are very similar to analyzers. In fact we’ve taken the concept and extended it to apply to different elements in TheHive: alerts, cases, tasks, task logs, and observables of course.

mailer_activeResponse.png
Responders in Action

You can reuse almost the same principles that apply to analyzers to write your own responders and if you are feeling generous, contribute them to the community. To give you a head start, we published a sample Mailer responder which, when customized for your environment, should allow you to send emails to inform your fellow analysts that a case has been created and that their help is required. Another example could be the ability to respond to a suspicious email report from a user, which is displayed as an alert, that they can safely ignore the corresponding email.

Like an analyzer, a responder can have two or more service interaction files (or flavors) to allow it to perform different actions. For example, a Mailer responder can send messages using several body templates.

Thanks to our long-time friend Andras Iklody from MISP Project who brought that to our attention, responders (and analyzers starting from Cortex 2.1) support PAP, the Permissible Actions Protocol.

Running into Troubles?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

 

 

 

Keep Them Coming: Hashdd, URLhaus & a Revamped FileInfo analyzer

We are happy to announce the immediate availability of a new major version of Cortex-Analyzers. Version 1.11.0 includes two brand new analyzers, several updates and a few bug fixes:

  • Hashdd, contributed by  iosonogio
  • URLhaus, contributed by  ninoseki
  • The Domaintools analyzer has been updated with two new flavors: Risk and Reputation
  • The VirusTotal analyzer can (finally!) get reports for URL observables
  • MsgParser and File_Info have been merged in a new, shiny, completely rewritten FileInfo analyzer

As we are approaching the 90 analyzers mark, we wholeheartedly thank our user community for continuously contributing new analyzers, testing them and helping us improve the existing ones.

Important Notice

We made significant changes in this release in the analyzers and short reports. Prior to Cortex-Analyzers 1.11.0, the summary() function in the analyzer code generates a result such as:

"taxonomies": [{
     "level": "suspicious",
     "namespace": "DT",
     "predicate": "Risk",
     "value": "\"10\""
 }]

Double quotes were included in the resulting value. We decided to update the summary() function and make it generate the same result without double quotes :

"taxonomies": [{
     "level": "suspicious",
     "namespace": "DT",
     "predicate": "Risk",
     "value": "10"
 }]

To display the double quotes in TheHive, we updated all short reports to include them:

<span class="label" ng-repeat="t in content.taxonomies" ng-class="{'info': 'label-info', 'safe': 'label-success', 'suspicious': 'label-warning', 'malicious':'label-danger'}[t.level]">
     {{t.namespace}}:{{t.predicate}}="{{t.value}}"
</span>

Hashdd

hashdd.com is a search engine for file hashes which automatically queries 3rd party services like VirusTotal and enriches the information provided based on the 3rd party data. The analyzer includes two flavors: Status and Detail. The first one is used to query hashdd without an API key for the threat level only. The latter produces additional meta information about the sample, but requires an API key.

Results are displayed in TheHive in the following manner:

Hashdd short report

Hashdd status long report

Hashdd detail long report
hashdd — short and long report samples

URLhaus

URLhaus, a service that shares the latest malware download URLs and reports those sites to their respective hosting companies, can now be queried for domains, URLs and hashes. If the observable is found, available information will be displayed as follows:

URLhaus short report

URLhaus long report
URLhaus – short and long report samples

Domaintools Risk and Reputation

New Risk and Reputation services from Domaintools have been added as new flavors to the existing Domaintools analyzer set.

Risk Evidence

The DomainTools Risk Score predicts the risk level and likely threats from a domain that has not been observed in malicious activities, by analyzing various properties of the domain as soon as it is registered.

sc-DTrisk-short

sc-DTrisk-long
Domaintools Risk Evidence – short and long report

Reputation

The Domaintools Reputation Score gives indications about how closely a domain is related to known bad domains, actors, and IPs.

sc-DTReputation-short

sc-DTReputation-long
Domaintools Reputation – short and long report samples

An All New FileInfo

FileInfo performs local static analysis of file observables. It has been completely rewritten from the ground up to be more flexible thus it can easily be enriched with new supported file types and analysis modules. We took this opportunity to merge MsgParser, in charge of extracting and displaying Outlook emails into FileInfo.

As of this release, FileInfo now supports PDF, PE, MS Office documents and Outlook .msg files. We also added support for DDE detection and link extraction in MS Office documents, thanks to Decalage who added this in Oletools since v0.52.

sc-FileInfo-short

sc-FileInfo-long2
FileInfo – short and long report samples

Bug fixes

  • #286 : we updated the way MISP analyzer validates its SSL configuration

  • #292 : we fixed the API URL of malwares.com in the Malwares analyzer

Get It While Supply Lasts!

Each analyzer comes with its own, pip compatible requirements.txt file. To update your Cortex analyzers to 1.11.0, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!