TheHive 3.2.0-RC1: The MISP Love Edition

Guess what? Our integration with MISP, the de facto standard for threat sharing, has just gotten better with our latest beta release: TheHive 3.2.0-RC1.

While you could synchronize TheHive with one or multiple MISP instances in earlier versions and select events using filters like their age, the number of attributes they contain or exclude those which are created by specific organisations or contain one or several black-listed tags, 3.2.0-RC1 adds the ability to whitelist tags, thus limiting the events that would show up in TheHive’s Alerts pane to only those which have been tagged with labels your SOC/CSIRT/CERT needs to act on. This can be very useful for example if your Cyber Threat Intelligence analysts pre-select or create events in MISP and tag for SOC consumption those that need to be acted on.

The Hive - Logo - Schéma - V1_Plan de travail 1.png
TheHive, Cortex, MISP: The Power DFIR & CTI Trio

To use this feature, use the whitelist.tags parameter in the MISP section of TheHive’s application.conf as described in the documentation.

This new version also adds the ability to create dashboards out of responder actions, log responder operations, and offers a confirmation dialog before running a responder to avoid noob over-clicks and errors made by seasoned incident handlers running low on caffeine.

TheHive 3.2.0-RC1 will also show you the description of an observable if any while hovering over one in the Observables tab. You can also see observable tags when previewing an alert in the Alerts pane.

Last but not least, some users reported severe problems when they enabled TLS/SSL directly on TheHive without resorting to a reverse proxy such as NGINX. Blame that on the crappy TLS support in Play framework ;-). So we highly recommend using a reverse proxy for that purpose, and delegate authentication to it if you are relying on X.509 authentication, as TheHive 3.2.0-RC1 allows you to. Please check the Single Sign-On on TheHive with X.509 Certificates guide for further information.

For additional details on this release, please check the full changelog.

Warning Capt’n Robinson!

The RC in 3.2.0-RC1 stands for Release Candidate. Please help us make a great stable release out of it by testing it as thoroughly as possible and reporting back any bugs or issues you encounter so we can address them before the final release. You’ll find this release candidate in the pre-release, beta repositories.

Please check TheHive Installation guide for further details.

You got a problem?

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

EmlParser: a New Cortex Analyzer for EML Files

The new EmlParser analyzer which we included in Cortex-Analyzers 1.12.0 leverages the eml_parser python library written by GOVCERT-LU. It parses EML email,  a MIME RFC 822 standard format, and extract all the information to help the analyst triage and investigate. EmlParser will prove very useful when analyzing observables imported from Synapse alerts.

You might notice that the analyzer’s requirements.txt installs the eml_parser library from one of our repositories. The original library dependencies contains file_magic library which brokes other analyzers that use python-magic. GOVCERT-LU is addressing this situation in their code but the installation process still considers file-magic as a mandatory library. We decided to consider it as an extra requirement.

Screen Shot 2018-07-26 at 08.19.11.png
Screen Shot 2018-07-26 at 08.19.31.png
EmlParser: short and long report samples

Get It While Supply Lasts!

To update your Cortex analyzers to 1.12.0, run the following commands:

cd path/to/Cortex-Analyzers

git pull

for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \

for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

Once done, do not forget to login to Cortex as an orgadmin and click on the Refresh Analyzers button.

Update TheHive Report Templates

If you are using TheHive, get the latest version of  the report templates and import them into TheHive.

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Unveiling Synapse

When we’re not busy cooking new features, we go back to the trenches and face incidents like many of our fellow analysts who read our publications and use our tools. To do so, we swap our chef toques for firefighter helmets, not only because such shiny headwear is cool, but mainly because incident response (IR) is, at its very heart, firefighting (minus all the dangerous stuff).

If you think about it, when handling incidents you can see everything from cats in trees (spam) to major fire (APT). Thankfully, there are more cats to bring down than fire to extinguish. That being said, a big herd of cats could be a serious threat to your organization, to your mental health or both.

We tend to forget that incident handlers are humans, not robots. Unlike our metal cyberfriends, we need diversity. We can’t risk insanity like Charlie Chaplin in Modern Times if we can avoid it. Unfortunately IR can be highly repetitive, especially if you only have cats to deal with.

Some could say ‘Nah, this is minor, nothing critical here’ but at some point, an analyst brainwashed by the same tasks again and again will be led to fault. In the worst case scenario, one could see an alert and immediately categorize it as false positive without any further consideration. Because ‘this alert is always a false positive’, until the day it is not…

Automation, a Solution?

Intuitively, we look in the direction of automation in order to minimize what we call ‘zombie’ tasks: highly repetitive and brainless tasks that need to be done. We believe that doing so will allow incident handlers to focus on the analysis and not on the tedious side of IR. Ultimately, we hope it will keep analysts stimulated and in a state of alert. Also, it should reduce time and effort spent on the low-hanging fruits.

One of the most dreary tasks in our opinion is to record the context around an incident.
What is the problem? When did it happen? What’s the origin? Who are the victims? How many are there? Answers to these questions let you have an overview of what is happening and are valuable to correlate incidents. So it is worth taking some minutes to add this information to your case. Sadly, most of the time it will look like a succession of ‘Ctrl+C; Alt+Tab; Ctrl+V’ from your incident source to TheHive. Exactly the kind of tasks we want to forego.

Specifications

Having identified the threat that apathetic analysts pose, the root cause (highly repetitive tasks) and a solution (automate the recording of incident context), the question of the implementation has been raised.

The first challenge to solve is the number of incident sources. Almost everything can trigger an incident: a firewall, an IDS, antivirus, SIEM, users, etc… So the application must be designed to accept several sources and must permit to easily integrate new ones. And instead of having to configure multiple alert feeders to supply alerts to TheHive, we would have only one. To some extent, it can be assimilated to a meta feeder.

And if the application works as intended, we still have a second challenge. Let’s say you, dear reader, and ourselves use the galaxy renowned Stargazer IDS. Maybe you’d like to include the full packet capture in the case but we wouldn’t. Using the same product doesn’t mean using it the same way. So we have a variety of sources and for each source, we have a variety of configurations and workflows. Hence any app we design needs to accept multiple configurations and workflows for any given source.

Finally: the third challenge. We want to make the most out of TheHive. Creating cases, creating alerts, assigning cases, adding logs, adding observables … all those actions are not an option.

Synapse

After several trials and failures, we came up with Synapse. Basically it is a Python 3 app which sits between TheHive and your incident sources:

Screen Shot 2018-07-18 at 09.54.28.png
Synapse Overview

To solve the first and third challenge, we rely on connectors. A connector is a Python object dedicated to interact with a security device. In the picture above, you can see the Exchange Connector and TheHive Connector. To extend the number of sources, you just have to develop the connector that corresponds to your device.

Regarding the second challenge, we rely on workflows. Workflows are python scripts who use connectors to automate repetitive tasks when tracking a case. Not happy with the current workflow? Develop your own using the connectors.

At this point, you probably wonder why there’s an API in the picture above. Well, the API is the link between the user and the workflows. By hitting a specific endpoint of the Synapse’s API, the corresponding workflow will be launched. That way the user can choose what to launch, especially if they are only interested in a particular workflow. Moreover, using an API allows us to listen to TheHive’s real-time stream and initiate some actions like closing a QRadar offense when the related case is solved.

At the moment, Synapse includes the Exchange connector and the associated Ews2Case workflow. The workflow features:

  • Case creation from emails
  • Case assignment
  • Adding email bodies to task logs
  • Adding email replies to the case
  • Adding email attachments as observables

And of course, everything is done to minimize the number of clicks! Check the workflow documentation to understand how it works under the hood.

We’re still working on the QRadar connector and the associated workflows but if you can’t wait, have a look at the work done by the community like pierrebarlet’s script.

Check it Out

As usual, Synapse is an open source and free software released under the AGPL (Affero General Public License).

Synapse has its own repository. Start with the user guide and read about the workflow you want to use as you’ll need to configure it.

Troubles?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Cortex4py 2 is Out!

Cortex, a free, open source software allows security analysts and threat hunters to analyze and enrich observables (IP addresses, hashes, domains, …) collected in the course of an investigation or received from third parties, for example through MISP, the de facto standard for threat sharing.

On March 29, 2018, we released Cortex 2, a major improvement over the previous version which brought, among other cool features, authentication, caching, multi-tenancy (RBAC) and rate limiting. Instead of deploying several Cortex 1 instances behind reverse proxies which would implement authentification, administrators can deploy a single Cortex 2, create multiple organizations and serve the needs of various information security populations while enjoying extra features.

On May 31, 2018, we published a brand new API guide so that developers can take advantage of the powerful REST API of the product. Sadly, Cortex4py, the FOSS Python library we provide to interact with the API was not compatible with Cortex 2. Until today.

Thanks to the hard work of our dear Nabil Adouani, we are happy to announce the immediate availability of Cortex4py 2.0.0, a complete rewrite of the library in Python 3. Cortex4py 2.0.0 is fully compatible with Cortex 2. However, it doesn’t work with Cortex 1.

While TheHive, the highly popular free and open source Security Incident Response Platform (SIRP) we develop has native support for many Cortex 2 instances, Python developers can leverage Cortex4py to interact with Cortex 2, manage organizations, users, analyzer configurations and analyze observables at scale from alternative SIRPs, SIEMs or custom scripts thanks to the 83 analyzers Cortex 2 has as of June 18, 2018.

Screen Shot 2018-06-18 at 20.01.27.png
Cortex 2: there is more than one way to interact with it

Use It

To install Cortex4py, use PIP3:

$ sudo -H pip3 install cortex4py

If you are using Python on a Windows operating system, please forgo the sudo command.

Usage

Cortex4py 2 comes with a usage guide which includes many examples. For example, if you want to fetch the last 10 successful jobs that have been executed against domain names and display the result summaries of those 10 jobs you could write something like:

Screen Shot 2018-06-18 at 19.58.45.png
Sample Python3 code to retrieve Cortex analyzer results

Migrating from Cortex4py 1

If you have already written scripts using Cortex4py 1.x (for Cortex 1), we tried to keep the already available methods. However, we recommend you adapt your code to leverage the new Cortex4py 2 classes and methods as soon as feasible. Moreover, the existing scripts must be updated to support authentication if you intend to use them with Cortex 2. Please read the Cortex4py 2 usage guide for more information.

Support

Cortex 2.0.0 is brand new software. As such, it might contain bugs and limitations. If you find any or encounter problems, please ask on our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

There are More than 80 Ways to Analyze Them

TheHive Project Chefs are pleased to announce the immediate availability of Cortex-Analyzers 1.10.1. To install this new release and benefit from 11 new analyzers and some fixes:

$ cd /path/to/Cortex-Analyzers
$ sudo git pull
$ for I in analyzers/*/requirements.txt; do sudo -H pip2 install -r $I; done && \
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -r $I || true; done

After running these commands, read the Analyzer Requirements Guide,  log into the Cortex 2 Web UI as an orgAdmin, click on the Refresh Analyzers button in the Cortex Web UI, configure the new analyzers and enjoy!

click.png
Cortex: click on Refresh Analyzers after updating to the latest Cortex-analyzers version

If you are using TheHive, get the last version of  the report templates and import them into TheHive.

New Analyzers

We have added 11 analyzers to this release, bringing the total to 53 (83 if we count all the flavors):

  1. Crtsh: contributed by crackytsi
  2. Cybercrime-Tracker: contributed by ph34tur3
  3. FireEye iSIGHT: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  4. GreyNoise: contributed by Nclose
  5. IBM X-Force: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  6. Malwares: contributed by Davide Arcuri and Andrea Garavaglia from LDO-CERT
  7. MnemonicPDNS: contributed by Michael Stensrud from the Nordic Financial CERT
  8. StaxxSearch: contributed by Robert Nixon
  9. StopForumSpam: contributed by Marc-André Doll from STARC (by EXAPROBE)
  10. ThreatCrowd: contributed by Rémi Allain from Cyberprotect
  11. Unshortenlink: contributed by Rémi Pointel from CERT-BDF

Crtsh

Get Crt.sh certificate transparency lists associated with a domain name. Crt.sh is an online service operated by the Comodo Certificate Authority.

The analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screen Shot 2018-06-05 at 16.46.22.png

Screen Shot 2018-06-05 at 16.46.31.png
TheHive: Crtsh — Short and Long Report Samples

Cybercrime-Tracker

Use the Cybercrime-tracker.net service to assess whether an IP address, URL, domain, or FQDN has a C2 (Command & Control) entry in its database.

This analyzer comes in only one flavor. No configuration is required. It can be used out of the box.

Screenshot from 2018-06-05 14-43-54.png

Screenshot from 2018-06-05 14-43-37.png
TheHive: CyberCrime-Tracker — Short and Long Report Samples

FireEye iSIGHT

Leverage FireEye iSIGHT Threat Intelligence to qualify domains, IP addresses, hashes and URLs.

This analyzer comes in only one flavor. You need a valid FireEye iSIGHT Threat Intelligence subscription to use the analyzer. Retrieve the API key associated with your account and provide it as a value to the key parameter. Obtain the password associated with the API key and provide it as a value to the pwd parameter.

Fireeye.png
TheHive: FireEye iSIGHT — Long Report Sample (courtesy of Andrea Garavaglia)

GreyNoise

Determine whether an IP has known scanning activity using GreyNoise.

This analyzer comes in only one flavor. It can be used out of the box without configuration. However, if you make many requests, you need to obtain an API key. Please contact GreyNoise to ask for one. Once you get the API key, provide it as the value of the key parameter.

Screenshot from 2018-06-05 14-48-02.png

Screenshot from 2018-06-05 14-47-42.png
TheHive: GreyNoise — Short and Long Report Samples

IBM X-Force

Query domains, IPs, hashes and URLs against IBM X-Force Threat Intelligence sharing platform.

This analyzer comes in only one flavor. Access to IBM X-Force Threat Exchange requires an IBM ID. Once you have access to the service, supply the URL of the service as value for the url parameter, the API key associated with your account as value for the key parameter and the associated password as the value of the pwd parameter.

IBM-XForce.PNG
TheHive: IBM X-Force — Long Report Sample (courtesy of Andrea Garavaglia)

Malwares

Query Malwares.com and get reports on files, hashes, domain names and IP addresses.

The analyzer comes in two flavors:
– Malwares_pDNS_GetReport: get the latest Malwares report for a file,
hash, domain or an IP address.
– Malwares_pDNS_Scan: scan a file or URL.

You need to sign up for a Malwares.com account. An API key to use the service’s API should be associated with your account. Supply it as the value of the key parameter.

Screen Shot 2018-06-05 at 16.56.38.png

Screen Shot 2018-06-05 at 16.56.57.png
TheHive: Malwares_GetReport — Short and Long Report Samples

MnemonicPDNS

Query IP addresses and domain names against Mnemonic Passive DNS service.

This analyzer comes in two flavors:

  • Mnemonic_pDNS_Public: query Mnemonic’s public service.
  • Mnemonic_pDNS_Closed: query Mnemonic’s closed service.

When using the public service, the analyzer can be used out of the box with no further configuration. When using the closed service, you need to contact Mnemonic to get an API key which you’ll need to supply as the value of the key parameter.

Screenshot from 2018-06-05 14-46-40.png

Screenshot from 2018-06-05 14-46-19.png
TheHive: Mnemonic PDNS — Short and Long Report Samples

StaxxSearch

Fetch observable details from an Anomali STAXX instance.

This analyzer comes in only one flavor. You need to install an Anomali STAXX instance or to have access to one to use the analyzer. Supply the following parameters to the analyzer in order to use it:

  • auth_url: URL of the authentication endpoint.
  • query_url: URL of the intelligence endpoint.
  • username: the STAXX user name.
  • password: the STAXX password.
  • cert_check: boolean indicating whether the certificate of the endpoint must be checked or not.
  • cert_path: path to the CA on the system to validate the endpoint’s certificate if cert_check is true.

image0010.png

image002.png
TheHive: StaxxSearch — Short and Long Report Samples (courtesy of Robert Nixon)

StopForumSpam

Query StopForumSpam to check if an IP or email address is a known spammer.

You need to define the thresholds above which the analyzed observable should be marked as suspicious or malicious.

Screen Shot 2018-06-05 at 18.32.08.png

Screen Shot 2018-06-05 at 18.32.21.png
TheHive: StopForumSpam — Short and Long Report Samples

ThreatCrowd

Look up domains, mail and IP addresses on [ThreatCrowd(https://www.threatcrowd.org/), a service powered by AlienVault.

This analyzer comes in only one flavor. No configuration is needed. It can be used out of the box.

Screenshot from 2018-06-05 14-45-29.png

Screenshot from 2018-06-05 14-45-11.png
TheHive: Threatcrowd — Short and Long Report Samples

Unshortenlink

Follow redirects of shortened URLs to reveal the real ones.

This analyzer comes in only one flavor.

No configuration is required. It can be used out of the box.

Warning: using this analyzer without extra caution might lead to unexpected consequences. For example, if the URL you are seeking to unshorten is an attacker-controlled one, you may end up leaving undesired traces in the threat actor’s infrastructure logs. The TLP values Cortex allows you to configure to prevent the use of an analyzer if the TLP associated with an observable is above the authorized level won’t be of much help since Unshortenlink have to access the shortened URL. Please do not activate this analyzer unless you (and your fellow analysts) know what they are doing.

Pasted image at 2018_06_05 09_07 PM.png

Screen Shot 2018-06-05 at 21.05.45.png
TheHive: Unshortenlink — Short and Long Report Samples

Additional Enhancements

  • YARA analyzer had a bug which was fixed in version 1.9.7 of the Cortex-analyzers repository. If you install 1.10, you’ll obviously benefit from the fix 😉
  • A permission problem that prevented using the Cuckoo Sandbox analyzer was corrected (thanks Felix Bauer!)

Support

Something does not work as expected? No worries, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

CorrectionJune 6, 2018
An earlier version of this post mentioned that GreyNoise needs an API key. That’s only necessary if a certain level of requests are made. Also, to configure analyzers, you need to be orgAdmin.

Cerana 0.9 and Cortex 2.0.4 are Out!

We are proud to announce the immediate availability of Cerana 0.9 (TheHive 3.0.9) and Cortex 2.0.4. These hotfix releases address a number of issues and we encourage you to update your current installation at your earliest opportunity. For your comfort and sanity. Seriously.

We also took this opportunity to update Cortex analyzers to fix issues with CIRCL Passive SSL, Hybrid Analysis, and the Joe Sandbox URL Analysis template. Moreover, we have updated the cortexutils library to set the taxonomy level to info if it is invalid. To upgrade cortexutils​ to 1.2.4:

sudo pip install -U cortexutils && sudo pip3 install -U cortexutils

To update your Cortex analyzers:

cd /path/to/Cortex-analyzers && git pull

Note: the Bluecoat analyzer was removed since it does not comply with the updated Terms of Service of Symantec Web Pulse SiteReview. Symantec does no longer permit programmatic querying of the service.

Fixes in Cerana 0.9

  • #527: display long reports when the analyst clicks on the corresponding short reports. Meh!
  • #541: make the drop-down menu for case templates scroll when there is a truckload of them.
  • #452: prevent WSAPI failure.
  • #531: fix naming inconsistencies in the Live Stream.
  • #530: correct an error when trying to analyze a filename using the Hybrid Analysis analyzer.
  • #543: generate an error if unable to contact Cortex.
  • #518: merge observable sightings when merging cases.
  • #535: fix the tag color of the PhishTank analyzer which was transparent under certain conditions.

Fixes in Cortex 2.0.4

  • #89: let a read,analyze user change or display their API key.
  • #91: sort analyzers by name.
  • #92: redirect users to the index page when they click on the Cortex logo.
  • #93: under the Organization > Configurations page, the UI displays wrong green checkmarks for empty configurations.
  • #94: orgadmin users are not able to update their organization’s users after the users are created. The UI doesn’t display any error message.
  • #95: avoid ‘lax programming’, Nabil style😜, and strictly filter the list of analyzers in the Run dialog.
  • #90: fix Python dependency errors in docker.

Support

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.

CorrectionApril 14, 2018
An earlier version of this post did not mention that the Bluecoat analyzer was removed in the latest Cortex Analyzers repository release.

Cortex 2.0.3 Released and Analyzer Updates

There’s a new version of your ultimate observable analysis engine in town : Cortex 2.0.3 is out!

Cortex 2.0.3 contains a few important enhancements over its predecessor and fixes a number bugs as described in the full changelog summarised below. So get it while it’s still hot out of the digital oven and let us know how tasty it is.

analyzeallthethings
Source : Quickmeme.com

Implemented Enhancements

  • #81: reflect proxy changes in the global configuration at the analyzer level
  • #82: display invalid analyzers and let orgadmins delete them
  • #85: allow orgadmins to override the default global report cache.job period per analyzer through the Web UI
  • #86: allow a job to run with arbitrary parameters

Fixed Bugs

  • #75: a version upgrade of an analyzer makes all analyzers invisible in TheHive
  • #80: fix the analyzer configuration dialog to allow orgadmins to override the auto artifact extraction at the analyzer level
  • #83: hit Nabil on the head pretty hard until the analyzer refresh UI button works (well now it does so you can stop hitting poor Nabil’s head).

Analyzer Updates

We took the opportunity of a new release to make a few updates to the public analyzers. Cortex-Analyzers 1.9.3 contains the following changes:

  • Remove the Bluecoat analyzer to comply with the new ‘no scrapping’ ToS imposed by Symantec
  • Fix the default configuration of the Cymon Check IP analyzer
  • Fix the View all VT long template
  • Make the MISP Warning Lists Analyzer ignore case sensitivity when searching for hashes
  • Restrict the Abuse Finder and FileInfo analyzer dependencies to Python 2.7

You can read the full changelog if you like but if you want to enjoy the goods right away, git pull is your friend.

Support

Something does not work as expected? You have troubles installing or upgrading? Spotted new bugs? No worries, please open issues on GitHub or comment on existing ones, join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We are here to help.