TheHive Reloaded: 4.1.0 is out

We are proud and excited to annonce the availability of TheHive v4.1.0. This release is the new beginning of TheHive’s future, because all the upcoming features and enhancements will be based on this version, without the brakes.

4.1.0 is out after a significant work done during the last 6 months, interrupted by multiple events like:

  • supporting ES 7 in TheHive 3 and Cortex
  • supporting ES 7.11 in TheHive 3 and Cortex
  • 5 patch releases of TheHive 4.0
  • recent OVH datacenter incident (but this one was quickly fixed)

Our community is waiting for it, as we announced it will fix the performance issues that many of you were facing. The reason is that TheHive 4.0.x doesn’t include a database indexation component resulting on poor data fetching performance.

We are glad about this release, it includes tons of new features and improvements, across all the application.

The first and maybe, the main one, is the database indexation (#1731) that will significantly improve performance of queries such as lists, filters and sorting. As a consequence, you should experience a smoother browsing and usage of the application.

Amongst all other news features, we introduced:

  • Support for MISP taxonomies, in addition to custom tags and tag colour support;
  • Support for Case TTPs specified by MITRE ATT&CK framework;
  • Better and improved Case merging and deleting capabilities;
  • A customisable date and time format
  • A refined Case Templates management UI
  • Enhanced Case list UI with bulk actions
  • Enhanced Task list UI with bulk actions
  • A platform status page to get an overview of the platform health
  • Support for migration from TheHive 3.5.1 to 4.1
  • Support for authentication on Webhook endpoints

For those of you who want to know the full details, here is the change log

A word about the documentation

As you might know, TheHive has a documentation repository that includes content for TheHive 3 and TheHive 4, but the structure needs to be redefined. For that reason, we introduced a new documentation website that aims to be the single starting point of all the docs we produce for TheHive Projects’ products.

In the upcoming days, we will mark the old docs repositories as ARCHIVED and we will maintain the new docs repository.

To visit the new documentation website, go to
https://docs.thehive-project.org

More about the new features

Database indexation

Until now, we knew TheHive suffered from a performance issue with the database.

Technically, TheHive uses basic index mechanism embedded in JanusGraph. This indexes are simple to use and manage but they contain limitations. They only support equality lookups and cannot be used for sort (for example, this is not possible to simply look for cases with severity > LOW) . However, within TheHive, almost all lists – lists of cases, lists of alerts, tasks …- are sorted. So, getting a list means a scan of all the elements of the list, which have a heavy performance impact, particularly if the list is long.

In order to solve this issue, TheHive 4.1.0 comes with a new index engine, and indexes have to be stored outside the database. As a consequence you need to define and setup dedicated storage for these indexes.

  • If TheHive is used in a cluster mode, all servers must connect to a common index engine, which can be in a cluster mode or not. In this case, a new component must be installed for index management. Elasticsearch should be used, only as an index engine. For clusters, Elasticsearch is not used like it was with TheHive 3.x when it stored data.
  • If TheHive is used on a standalone server, Elasticsearch can be used, but a file based index engine – Lucene – can be preferred. The latter solution has less infrastructure impact – no component to install – and only requires configuration that indicate where index data is stored in the filesystem.

For more details about database indexing configuration, please refer to the documentation

MISP taxonomies support

TheHive 4 handles tags in a completely different way than TheHive 3, even if the APIs are still considering tags as simple strings. For example when you create a Case from TheHiv4py, you specify tags as a simple array of strings.

Behind the scene, TheHive transforms the tags into objects, with a namespace, a predicate and an empty value. For example if your create a case with a free tag src=mailbox, TheHive 4 creates a new Tag object as following:

{
    "namespace": "_freetag",
    "predicate": "src=mailbox",
    "value": null,
    "colour": "DEFAULT_COLOUR_FROM_THE_CONFIG"
}

This is where we introduced the support of MISP Taxonomies, allowing admin users with `manageTaxonomy` permission, to import tags defined in machinetag.json format, from the “Administration > Taxonomies page”

Taxonomies import dialog

Note: importing the full MISP taxonomies library can take some time (1 minute)

Once imported, you need to enable the taxonomies you need for use cases. All the taxonomies are disables by default. Enabling a taxonomy, make its tags available to all the organisations on the platform, so analysts can use the to tag Cases, Alerts, Observables.

Tags selector

Taxonomy tags can also bee used from any tag filter in Case, Alert and Observable lists:

Tag filter

Custom tags support

Custom tags or free tags, is the way we call free text tags associated with TheHive objects. Internally, custom tags are included into organisation related spécial taxonomy called `freetags`.

Custom tags are not shared across organisations, so existing users defining sensitive data in tags won’t suffer any data leakage issue.

In general, custom tags in TheHive are not supposed to user data and information. It’s not a best practice to use email adresses as tags. Custom fields are the right place for this type of data.

This topic needs a dedicated blog post, to share the best practices of using tags.

So, how do an org admin manage the custom tags? Well, TheHive 4.1 comes with a new UI section, under the organisation management page, allowing:

  • Listing all the custom tags
  • Filtering and sorting
  • Updating custom tag values and colours
  • Deleting custom tags
  • Displaying custom tag usage (# of cases, # of alerts, # of observables and # of case templates)
Custom tags management

For users migrating from TheHive 3, all the existing tags are imported as custom tags

Tactics, Techniques & Procedures, with MITRE ATT&CK

Supporting MITRE ATT&CK framework is one of the feature that has been postponed many time from TheHive. It’s a no brainer feature that any blue team oriented product must include.

In TheHive 4.1, we allow:

  • Importing the official MITRE ATT&CK attack patterns collection as defined in https://github.com/mitre/cti. In this version support the entreprise catalog, but the future goal is to allow managing multiple catalogs in a misp-galaxy-like manner.
  • Defining TTPs associated with TheHive Cases

Attack Pattern management

From the administration page, any user with managePattern permission is able to have access to a page where patterns can be imported, filtered, viewed.

Import attack patterns
Attck Pattern management page

Case TTPs

In addition to Tasks and Observables, in TheHive 4.1, you can associated TTPs to your Cases. TTPs objects are defined by:

  • A tactic
  • A technique or sub-technique
  • An occur date
  • An optional procedure description
Add a TTP to a Case

The Case TTPs are displayed in a dedicated tab on the Case details page, the same way as Tasks and Observables, with filtering and sorting capabilities.

List of TTPs published in the report SANDWORM INTRUSION SET
CAMPAIGN TARGETING
CENTREON SYSTEMS
” by CERT-FR

This screenshot, showcases the tactic colours we use, thanks to Paul Tol’s blogpost

Case list TTP related improvements

One more feature: Case list now show the number of TTPs associated with each Case, with a link to the TTPs list page.

Case overview

Customise date and time

Since the very beginning, TheHive displays dates with the Month-Day-Year format. After 5 years, you can now customise the way you want date and time be displayed in all views of the UI.

This can be configured at the Organisation level, by defining the preferred format in the UI Configuration view. This required a user with org-admin profile or any profile with manageConfig permission.

UI Configuration view

Improved Case templates management

Case template management UI has been rewritten to allow more capabilities, required when you start having a growing number of templates:

  • Filtering
  • Sorting
  • Display dates
  • Display number of tasks and custom fields
Case template management UI
Case template editing UI

Case merging

This is a feature that has been removed from TheHive 4.0 as it required a design update to take into account the multi-tenancy support.

In TheHive 4.1, case merging has the same UI:

  • Go to a case details page
  • Click merge
  • Select the case to merge data into
  • Validate

The difference in TheHive 4.1 is that, merging two cases, removes the originating cases, and create a new one with all the merged data.

Platform status page

This feature aims to help understand the issues related to TheHive health status. It contains details about:

  • database schema version
  • status of database indexes
  • status of database integrity checks

It also allows:

  • exporting a JSON report of the health status including more details then what is displayed on the UI
  • reindexing the database
  • triggering the database integrity checks

This page is available to super admin users and contains cross organisations data

Platform status

Webhook authentication options

In TheHive 4.1, you can define authentication configuration for your Webhooks. For more details, please refer to the documentation website

KNOWN ISSUE: the auth property in Webhook definitions, in application.conf file is REQUIRED, so if your webhook doesn’t need authentication, then just add

auth: {type: "none"}

for each Webhook definition

Updating from TheHive 4.0

Due to the new database indexation feature, updating from 4.0.x requires some attention. Depending on your type of installation – standalone server or cluster – your setup will need to be upgraded accordingly.

  • On standalone servers, you will have to define a new local folder to store indexes;
  • For a cluster, you will need to locate an Elasticsearch instance to use it as index.

Finally, update the /etc/thehive/application.conf configuration file, install the new version of the application and restart the service.

more detailed information regarding this update can be found here:

https://docs.thehive-project.org/thehive/operations/update/

Indexes related to existing data will be created at the first start after the update to TheHive 4.1.0. Depending on the size of your database, this process can take a long time.

Upgrading from TheHive 3.x

If you are still using TheHive 3.x and want to migrate to TheHive 4.1, then you need to take a look to the following supported paths:

Target versionRequired source version
TheHive 4.1.xTheHive 3.5.1
TheHive 4.0.x3.4.x < TheHive < 3.5.1
Supported migration paths

For more details, please refer to the documentation website

How to install

If you starts using TheHive with this version, we recommend having a look at our documentation site, and particularly to the installation and configuration section which is up to date and contains all instructions to install & configure TheHive 4.1.0.

Docker

If you use TheHive as a docker container, you can refer to the Docker-Templates repository that has a TheHive 4 up-to-date docker-compose configurations.

How to report issues

Please open an issue on GitHub with the dedicated template for TheHive 4. We will monitor them closely and respond accordingly. 

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Discord, or send us an email at support@thehive-project.org. We will be more than happy to help!

Compatibility issues with Elasticsearch update

Dear TheHive and Cortex users,

If you are running TheHive v3.5.0 and / or Cortex v3.1.0, the underlying database is Elasticsearch v7.x.

Elastic recently released two new versions: v7.11.0 and v7.11.1. After some initial feedback and investigations, we found that the new releases introduce changes that break the compatibility with our products – TheHive 3.5.0 and Cortex 3.1.0.

Therefore, please DO NOT upgrade your current database to Elasticsearch v7.11.x as no rollback is possible. Elasticsearch v7.11.x breaks the installation process as well as the update process.

If you are in the process of installing or updating to Cortex v3.1.0 or TheHive v3.5.0, you need to specify the exact working version of Elasticsearch to use:

  • For Debian packages: “apt install elasticsearch=7.10.2”
  • FOR RPM packages: “yum install elasticsearch-7.10.2-1”

We are currently running deeper investigations and are planning to release updated versions as soon as possible for Cortex v3.1.0 and for TheHive 3.5.0.

We will keep you informed, stay tuned!

It’s not Patch Friday… TheHive 4.0.2 released

Last Friday, our team released a significant number of changes and fixes, improving TheHive 4. The community was very reactive and hurried to test it. Today we are announcing a patch release to fix annoying issues we decided to quickly patch: welcome to 4.0.2.

The major issue is related to Alert bulk merging which is part of the big refactoring of the Alert listing section.

Well, s**t happens, but as a gentlemen, we owe you a feature, or two 🙂

What’s new?

In addition to the Alert bulk merging issue, we fixed bugs related to:

  • Migration
  • Filtering observables by attachement attributes
  • Backward compatible APIs

Curious to read the detailed change log? Here your go.

TheHiveFS, do you remember?

It stands for TheHive File System, a feature we released with TheHive 4.0-RC1. In today’s milestone we are improving the security of this feature by adding a new user permission, required to have access to TheHiveFS features.

The newly created accessTheHiveFS permission is included by default in both org-admin and analyst user profiles.

Refined editable fields, for a better UX

In TheHive UI, editing objects relies on editable fields instead of dedicated form to edit objects. This means you can update a case title by clicking the Blue Pencil icon displayed when you mouse-over the case title for exemple.

In this release we refactored all the editable fields to provide a better user experience:

  • You no longer need to click just on the `Blue Pencil` icon to switch to edit mode, you just need to click on any value, for example the Case assignee field, or the Observable tags field:
Editable fields on mouse over events

Editable fields have now a `Clear` option, allowing users to unset the value of an attribute:

Editable custom fields can be set empty

This new improvement benefits to the custom fields sections in Case and Alert details sections, as showcased above.

Configurable layout of custom fields

In older TheHive versions, custom fields were displayed using a single column. Cases with big number of custom fields produced a long scrolling Case details pages, so we decided in TheHive 4 to use a 3-column layout to reduce the resulting scroll fatigue.

Now some users are complaining because of long custom fields values not being correctly displayed. So we decided to let users choose their preferred layout.

1-column layout to display custom fields in a Case details page
2-columns layout

How to install/update ?

Our installation guides are up-to-date.

Important note

If you are updating an existing instance of TheHive 4.0.0, ensure to read the installation guide. Since our recent release of TheHive 3.5.0, we updated our repositories. The codename of repositories containing all new released packages is now named release, instead of stable. So update your apt or yum repository file.

Have you got a minute to let us know how you use TheHive ?

We would love to hear from you (10 questions < 1minute) : https://t.co/l5osjvQ1k7

Many thanks for taking the time 🙏

How to report issues

Please open an issue on GitHub with the dedicated template for TheHive 4. We will monitor them closely and respond accordingly. 

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Discord, or send us an email at support@thehive-project.org. We will be more than happy to help!

Cortex-Analyzers 2.8.0: to infinity and beyond!

Thanks to the community and all the contributors, this release comes with 1 new Analyzer, 2 new Responders, lots of improvements and bug fixes.

But there is more news from the front.

Starting from this milestone, bugfixes and new Analyzers or Responders should be released in a smoother way as we are improving few processes. Some changes and recommandations should appear in the next days for submission, and our release process will be improved to fix bugs easier and release new code faster.

We also plan to offer a better documentation. We already started to publish information regarding each Analyzer and Responder. This is a work in progress, and it will be updated with the current requirements guide.

DomainToolsIris documentation page

For each Analyzer and Responder, a page details the purpose of each flavors, the configuration required and even some screenshots from report samples. It will also allow developers to share their own notes if wanted.

New Analyzers

New Responders

Improvements

  • Refactor Onyphe using new v2 api (#736)
  • Improvement in Shodan: add vulns in template and taxonomies (#772 & #776)
  • Improvement in Mailer responder: tasks support and auth (#764, #737, #379)
  • Improvement in SinkDb: support for new api with new dataTypes supported (#483, #498, #756)

Analyzers

LastInfoSec

LastInfoSec offers innovative and automated solutions to collect data, refine it and turn it into useful and actionable information, quickly available to improve the protection, detection and investigation capabilities of companies and government organizations.

TheHive displays the analyzer results as follows:

Short template for LastInfoSec Report
Long Template for LastInfoSec Report

Onyphe

An important work has been made on Onyphe Analyzer to support APIv2. All 7 flavors from older version have been removed and merged into only one flavor named “Onyphe_Summary”. An API key is still needed to query Onyphe API.

TheHive displays the analyzer results as follows:

Onyphe_Summary short report
Onyphe_Summary long report

Responders

Sendgrid

Sendgrid is a customer communication platform for transactional and marketing email used when you have to ensure that your notifications and transactional emails are delivered quickly and securely.

This analyzer works like the Mailer one, but relying on SendGrid external service to delivery emails.

In order to use the service please follow the instruction being careful to the verify your email address.

VirusTotalDownloader

This responders runs on Observables of type “hash” and allows analyst to download corresponding file from VirusTotal. Once downloaded, the file is added to the current case observables in TheHive.

In order to use this responder, a Premium API key from VirusTotal is needed. An API key from TheHive is also needed to upload the file in the observables list.

Use the responder on the hash to add the sample in your Observables

Fixes and Improvements

  • Fix: some analyzer uses invalid “email” dataType (#799)
  • Fix in MalwareBazaar: wrong dataTypes in config (#794)
  • Fix in PhishTank: the JSON object must be str, not ‘bytes’ (#786)
  • Fix in VMRay: fix error in parsing and workflow (#785 & #784)
  • Fix in Wazuh: ipaddress import missing (#778)
  • Fix in Minemeld Responder: requests missing in requirements (#774)
  • Fix in WOT: moving to new endpoint (#771)
  • DomainTools Iris: update api urls (#760)
  • Fix in ThreatResponse: module_type key removed from response (#759)
  • Fix in Abuse_Finder: pythonwhois dependency (#742)

Get It While Supply Lasts!

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates` menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

TheHive 4.0-RC3, a new kid on the block

Three weeks ago, on May 6th, we announced the availability of TheHive 4.0 RC-2 release and the active community made the usual hard work of testing the release to find bugs and ask for enhancements.

Special thanks to Christopher, alias crackytsi who has already created 122 Github issues, 11 of them are just for 4.0-RC-3 milestone.

We are thrilled to present our third release candidate before the final release of TheHive 4. As the RC-2, this milestone brings new features and fixes a nice set of issues.

This blog post will focus on the following new features:

  • Multi-orgs users
  • Switch organisation
  • OAuth2
  • Migration tool performance
  • Case sharing overview

If you are curious about all the issues that have been addressed, you can read the full changelog

Multi-orgs users

This feature doesn’t introduce UI changes, but it allows a superadmin or an org-admin to add an existing user to an organisation.

Users in TheHive 4 are identified by their email addresses, so when an administrator adds a new user, with an email address that already exists, TheHive 4 links that existing user to the organisation being updated.

This ends up with a single User record on the database, linked to multiple organisations. Thanks to the new graph data model. This means the given user has:

  • the same credentials
  • the same api key, if enabled
  • the same 2FA settings, if enables
  • the same status (locked or not)

With that being said, the user can have a different profile for on the organisation (s)he belongs to.

What happens when a user is logged in?

As we mentioned earlier, a user belonging to several organisations, has the same authentication settings, and after the login, his/her workspace is opened with the context of the first organisation (s)he has been created on.

For example, if John was firstly created on the *SocLevel2* organisation, and was later attached to *CTI* organisation, then after signing in, the user is redirected to the workspace of *SocLevel2* organisation.

Future improvements

We will consider allowing the user to define a default organisation to be displayed juste after the login. We are examining the possibility to allow the user to define a default organisation to be displayed just after the login process. Hopefully, we will be able to add this feature in TheHive 4.0.0 release.

Switch organisation

This feature empowers the multi-tenancy capabilities brought to you by TheHive 4. Following what has been showcased above, how can a user, who belongs to more than one organisation, switch between his/her tenants?

The UI introduces a simple feature, available to “multi-org” users only, as a button on the right hand side of the page’s header, aka. the navigation bar.

The switch organisation action button

This button is just hidden for users who belong to a single organisation.

Once clicked, that button show a dialog that displays the following details:

  • user’s organisations
  • user’s profile on each organisation
  • the current organisation

Clicking on an item of this list, refreshes the page by loading the context of the selected organisation, and the UI behaves like if the user was logged in a a member of that selected organisation.

Very useful.

Switch organisation dialog

OAuth2

We had a considerable amount of users asking for SSO and OAuth support in TheHive. We tried to make it more robust in TheHive 4, and let it rely on a redirectUri provided by the backend (/api/ssoLogin) instead of the old redirectUri that some OAuth providers don’t support (index.html/#!/ssoLogin).

In TheHive 4.0 RC-2, OAuth 2 partially worked, and failed to redirect the user to the home page after the authentication success. Yes, sorry for that.

We spent some time testing the new implementation. We will devote some blog posts to it, but firstly, here is a working example relying on Keycloak

auth {
  providers: [
    {name: session}               # required !
    {name: basic, realm: thehive}
    {name: local}
    {name: key}    
    {
      ##############
      # Keycloak
      ##############
      name: oauth2
      clientId: "CLIENT_ID"
      clientSecret: "CLIENT_SECRET" # or empty
      redirectUri: "http://THEHIVE/api/ssoLogin"
      responseType: "code"
      grantType: "authorization_code"
      authorizationUrl: "http://KEYCLOAK/auth/realms/TENANT/protocol/openid-connect/auth"
      authorizationHeader: "Bearer"
      tokenUrl: "http://KEYCLOAK/auth/realms/TENANT/protocol/openid-connect/token"
      userUrl: "http://KEYCLOAK/auth/realms/TENANT/protocol/openid-connect/userinfo"
      scope: ["openid", "email"]
      userIdField: "email"
    }
  ]
}

After a question asked on Twitter, we tried to test our OAuth implementation with the providers mentioned in the answers, and we have successfully tested:

Migration tool performance

The migration tool we implemented in TheHive 4.0 RC-2 suffered from important performance issues as a result of our desire for a clean design.

In fact, enabling database locks during a parallelised and asynchronous processing of the migration operations produce a migration tool with poor performance.

We changed the strategy, by disabling locks and programmatically handling duplicates if they happen. This ended by a significant improvement of performance

We hope you can test it and provide us with your feedback.

Case sharing overview

Case sharing is the most important feature that the multi-tenancy support adds to TheHive. Allowing users to quickly spot if a case is owned or is coming from a share (made by another organisation) improves the user’s experience.

The other handy information is: the number of organisations having access to a certain case

Case list with sharing indicators

This screenshot shows all the case sharing related UI element:

  • The blue line, indicates that the case is coming from another organisation
  • The green line, indicates that the case is owned by the current organisation
  • The red line, highlights the column that show the number of organisation having access to the corresponding case

How to report issues

Please open an issue on GitHub using the template made for TheHive4 if you’d like to report a bug on this version. We will monitor those closely and respond accordingly.

Cortex-Analyzers 2.7.0: 5 Analyzers, 1 Responder

Good morning (or evening if you are on that side of the planet) folks!

We had a very busy week, packed with announcements. First, we released TheHive 4.0-RC2 which you’ve certainly taken to test, then we announced two patch releases for TheHive 3.4. And guess what? Here are some additional Cortex analyzers, a responder and a number of fixes and improvements for existing ones, bringing the total to a whopping 146 analyzers and 18 responders!

New Analyzers

New Responders

Analyzers

ANY.RUN

ANY.RUN is a malware sandbox service in the cloud. By using this analyzer, an analyst can submit a suspicious file or URL to the service for analysis and get a report. The report can contain various information such as:

  • Interactive access
  • Research threats by filter in public submissions
  • File and URL dynamic analysis
  • Mitre ATT&CK mapping
  • Detailed malware reports
ANY.RUN short report
ANY.RUN long report

CyberChef

CyberChef is a simple, intuitive web app for carrying out all manner of “cyber” operations within a web browser. These operations include simple encoding like XOR or Base64, more complex encryption like AES, DES and Blowfish, creating binary and hexdumps, compression and decompression of data, calculating hashes and checksums, IPv6 and X.509 parsing, changing character encodings, and much more.

This analyzer connects to a CyberChef-server and comes in 3 flavors:

  • CyberChef_FromBase64, that takes Base64 strings as input for CyberChef-server
  • CyberChef_FromCharCode, that takes CharCode as input for CyberChef-server and run this recipe
  • CyberChef_FromHex, that takes Hex strings as input for CyberChef-server

TheHive displays the analyzer results as follows:

CyberChef short report
image
CyberChef long report

MalwareBazaar

MalwareBazaar is a project from abuse.ch with the goal of sharing malware samples with the infosec community, AV vendors and threat intelligence providers.

This analyzer allows analysts to query the API of this service on observables of types ip, domain, fqdn, url, and hash.

TheHive displays the analyzer results as follows:

MalwareBazaar short report
MalwareBazaar long report

OpenCTI

OpenCTI is an open source platform allowing organisations to manage their Cyber Threat Intelligence knowledge and observables. It has been created in order to structure, store, organise and visualise technical and non-technical information about cyber threats.

This analyzer allows an analyst to query the API and request for information about observables of types domain, ip, url, fqdn, uri_path, user-agent, hash, email, mail, mail_subject, registry, regexp, filename and other.

TheHive displays the analyzer results as follows:

OpenCTI short report
OpenCTI long report

MISPWarningLists reloaded (need for speed aka DB support)

The previous version of this analyzer basically used to clone the MISP Warning lists repository and do a lookup in downloaded files. This can be very long to complete.

This new version introduces the optional support of PostgreSQL:

  • To store warning lists, in a similar way to the NSRL (National Software Reference Library) Analyzer.
  • Make lookups through these lists faster.

If you want to benefit from the performance boost, using a PostgreSQL server to store the data, you can simply install the requirements.txt, feed the database and configure the connection in the configuration as well:

  • First, sync with the misp-warninglists GitHub repository
  • In the analyzer folder, use the program warninglists_create_db.py to import the warning lists in PostgreSQL. Before running, edit the program file and update the path of where your lists are stored (warninglists_path = "misp-warninglists/**/list.json")
  • You can schedule these jobs (ex. with cron): first, sync a folder with the repository, and then run the program to update the database.

Once done, configure the analyzer with the conn parameter to connect to the database, or, if you prefer to continue using the previous behaviour and do your lookups in files, just specify the path of the folder:

MISPWarningList Configuration example

Templates have also been updated, and TheHive displays the analyzer results as follows:

MISPWarningList short report
MISPWarningList long report

Responders

RT4-CreateTicket

RT4 (Request Tracker) is a ticketing system. With this responder, an analyst can create a ticket in RT. CaseID is submitted to RT as a reference.

Unfortunately, like for some other analyzers and responders, we have not been able to test this responder on our side. Please feel free to share your feedback with us and also with Michael Davis, who we would like to thank for the hard work and for having shared this responder with the community.

Fixes and Improvements

  • Fix Inconsistent Key References in Shodan Analyzer (#748)
  • Fix SSL & python3 for Yeti Analyzer (#468 , #708)
  • Fix bug in Emlparser Analyzer (#730)
  • Fix in Shodan Analyzer: Inconsistent Key References (#748)
  • Support python3 in DNSDB Analyzer (#613)
  • Support APIKey for EmailRep Analyzer (#750)
  • Improvement: EmlParser now extracts some useful IOCs (#710)

Get It While Supply Lasts!

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates` menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

New TheHive 3.4 Patch Releases

Last month (that should be… April… we are kinda losing track of time during the confinement), we made silently 2 patch releases for TheHive 3.4, our current stable version even if we have our hands full of soap and bleach as we are working on the eagerly awaited TheHive 4.0.0 final release: 3.4.1, shortly followed by 3.4.2. Your lovely bees are truly committed at keeping TheHive 3 branch buzzing well after 4.0.0 is out.

As usual, we’d like to start by thank the community for bringing the issues they discover to our attention. This is definitely one of the best contributions that we can get from you!

A simple way to help any open source project

3.4.1 Release

Released on April 25, 2020, 3.4.1 mainly fixed some docker-related issues as well as problems with OAuth2 and MISP integration, in addition to a few bugs, as described in the changelog.

Implemented Enhancements

  • Docker: TheHive fails to connect to Elasticsearch (NoNodeAvailableException) #854
  • Improved support for OpenID connect and OAuth2 #1110
  • TheHive’s Docker entrypoint logs the Play secret key at startup (… looking elsewhere hoping not to attract too much attention on this one) #1177
  • Configure TheHive’s first run using Docker Compose #1199
  • TheHive’s docker containers should be orchestration-ready #1204
  • MISP synchronisation: any attribute having the to_ids flag will be imported as ioc by TheHive. In the same way, when you export a case to MISP, observables which have the ioc flag on will become MISP attributes for which to_ids is true #1273

Closed Issues

  • Include Dockerfile in root of project #1222
  • Docker user daemon with id 1 causes permission issues with local #1227

Fixed Bugs

  • Fix MISP sync issues related to Docker #866
  • Owner is case-sensitive on API calls and should be lowercased #928
  • Bug: Observable without data breaks display of observables #1080
  • Docker-Compose Elasticsearch incompatibility #1140
  • Analyzers that take more than 10 minutes run into timeout #1156
  • TheHive 3.4.0 migration log errors ([error] m.Migration – Failed to create dashboard) #1202
  • Computed metrics are not compatible with the painless scripting language #1210
  • OAuth2 Bearer header should be of the format “Authorization Bearer” ? #1228
  • Health API endpoint returns warning when everything is OK #1233
  • Job submission sometimes fails when there are multiple Cortex servers #1272

3.4.2 Release

3.4.1 introduced a regression which was spotted few hours after it has been made public. 3.4.2 fixes t the problem.

It also adds a quick improvement allowing users to have access to error messages returned by Cortex Responder calls.

Display of a failed responder jobs, in case details page

Implemented Enhancements

  • Providing output details for Responders #962

Fixed Bugs

  • File observables in alert are not created in case #1292
  • Analyzer’s artifacts tags and message are not kept when importing observables #1285

Running Into Trouble?

Shall you encounter any difficulty, please join our  user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help as usual!

TheHive 4.0-RC2, Fresh out of the Oven

Shortly after the release of TheHive 4.0 RC-1 in February 2020, many members of our community tested it and provided great feedback, spotting issues here and there. We would like to wholeheartedly thank all of those who, like us, want to make TheHive 4.0 a great, rock-solid release!

We are now happy to unveil the 2nd release candidate. It fixes many bugs and introduce – or reintroduce – some new (and old) features :-). In this blog post, we will concentrate on the following features:

  • 2FA
  • Age of cases
  • Reintroduction of webhooks

Please read the changelog for a more comprehensive view, including bugfixes.

And since the COVID-19 crisis is here to stay for quite some time, we don’t want you to rediscover boredom, a dreadful feeling long forgotten thanks to the continued stream of notifications, solicitations and attention-grabbing, 280 chars ‘thoughts’. So instead of getting bored, we invite you to test TheHive 4.0-RC2 to the best extent possible and, should you encounter any issue, please let us know. We want to issue the final release during the summer so that everyone can have it just in time for their forthcoming vacations at home!

2FA

Two factor authentication was initially scheduled for the final release. We changed our minds and decided to offer you the possibility to test this feature right away to gather your feedback and improvement ideas before we finish up baking the final recipe.

Users can enable 2FA from their account. To enable it, first go to your account Settings and check Enable Multi-Factor Authentication.

Once done, you are invited to use your preferred TOTP application (Google Authenticator, Authy, Microsoft Authenticator etc.) to scan the QR code or the code underneath it. Your 2FA will generate A TOTP that you should supply in the MFA Code area. If it is valid, 2FA will be activated.

Important notes:

  1. If a user loses access to their TOTP application, only an administrator can restore access to their account.
  2. If an org administrator loses access to their TOTP application and they are the only administrator for that org, only a super admin can restore access to their account.
  3. If a super admin loses access to their TOTP application and they are the only super admin of the instance, they should pack up their things and look for another job. That or use a magic DB command to restore access to their account. We’ll update the documentation accordingly.
  4. The current implementation of 2FA does not support backup codes or alternate authentication methods should a user loses access to their TOTP application. However, we are considering adding backup codes to the final release.
  5. 2FA cannot be enforced by default for all users at this stage. It is thus of rather marginal value. However, an org admin can see from the UI who did not activate it and pester them until they do. In the same way, a super admin can do the same for org admins, other super admins and mere users. We are updating the documentation to add an API query that will allow you to list all users who did not activate 2FA.
  6. We will consider making 2FA mandatory in TheHive 4.1.
2FA configuration view

Next time you log in, you will need to supply the TOTP verification code in addition to your login and password.

TOTP verification code required at login

Age of Cases

A new information regarding case duration has been added in the list of cases and in case view, so you can easily keep an eye on how old your cases are and activate your escalation procedures etc. if necessary.

Age of Cases in list view
Age of a Case in Case view

Webhooks are back!

TheHive 4.0-RC1 was released without webhooks. They have been reintroduced in this version. You can now configure TheHive 4.0 to use them, but also filter data sent to the remote server by Organisation.

How to report issues

Please open an issue on GitHub using the template made for TheHive4 if you’d like to report a bug on this version. We will monitor those closely and respond accordingly.

Cortex-Analyzers 2.6.0: 146 Analyzers, 18 Responders

Amidst the ongoing COVID-19 crisis, we managed to release Cortex-Analyzers 2.6.0, which includes 4 new Analyzers, 2 new Responders, and a large number of bug fixes and improvements.

We’d like to thank all the contributors for their awesome work!

We truly appreciate the time they generously give away for helping our fellow cyberdefenders out there protect their environments against attackers who are also in lockdown mode. Attackers who, instead of playing board games or chess, are playing with our nerves and the hordes of teleworkers who are willing to click on anything that provide the ‘latest and greatest COVID-19 information’ or which can help them do their jobs (like this wonderful ‘Zoon’ video-conferencing application 😋).

Les Temps modernes - Film (1936) - SensCritique
Source: senscritique.com

What’s New?

New Analyzers

New Responders

Analyzers

DomainTools Iris

The Investigate flavour was missing from the DomainToolsIris analyzer that was included in Cortex-Analyzers 2.4.0. This is now fixed. This new flavour can be used to gather interesting information on a domain.

TheHive displays the analyzer results as follows:

DomainToolsIris_Investigate short reports
DomainToolsIris_Investigate long report

IntezerCommunity

Intezer Analyze™ is a cloud-based malware analysis service that provides an extensive understanding of any executable file by comparing code on a massive scale to a comprehensive database of malware and trusted software. 

This analyzer can be used to submit a file to the Intezer service for analysis.

TheHive displays the analyzer results as follows:

IntezerCommunity short report
IntezerCommunity long report

NSRL

The National Software Reference Library (NSRL) is designed to collect software from various sources and incorporate file profiles computed from this software into a Reference Data Set (RDS) of information. The RDS can be used by law enforcement, government, and industry organisations to review files on a computer by matching file profiles in the RDS. This will help alleviate much of the effort involved in determining which files are important as evidence on computers or file systems that have been seized as part of criminal investigations.

In order to use this analyzer, you must download and extract NSRLFile files from the NIST website. You can pick multiple files but you need to rename them in order to understand which file contains the required information.

All files are called NSRLFile.txt, renaming them permit to understand in which file the record has been found.

The analyzer can operate in 2 different ways with 2 completely different performance profiles (we’re speaking around 30 secs vs 0.05 sec):

  1. lookup in plain files
  2. lookup in a database

If you are planning to use this analyzer for many searches, then the second option is suggested and we provide a script to help you parse, validate and insert data in a PostgreSQL database. If you choose this option, consider that the DB size can be around 4 times bigger than plain files.

NSRL Lookup short template
NSRL Lookup long report

UrlScan.io

The URLScan.io analyzer has been updated with a new Scan flavour. Until now, this analyzer allowed to request report regarding a url, domain, fqdn observable. With this new flavour, anyone with a valid API key, which can be obtained for free, can request a scan on observables of the same type.

UrlScan.io short template
UrlScan.io long template

Responders

DomainToolsIris_CheckMaliciousTags

Depending on the reports generated by the DomainToolsIris analyzer, this responder adds a tag at the Case and Observable level if something malicious is found. This responder can be updated to add more custom actions depending on your needs and environment.

DomainToolsIris_AddRiskyDNSTag

Depending on on the reports generated by the DomainToolsIris analyzer, this responder adds a tag at the Case and Observable level if one of the domain observables is considered risky. This responder can be updated to add more custom actions depending on your needs and environment.

Fixes and Improvements

  • Improve TalosReputation analyzer (#521)
  • MISP WarningList analyzer fixed (#538)
  • Error fixed in ThreatCrowd (#518)
  • Encoding related bug fixed in Mailer 1_0 (#573)
  • API has changed: temporary fix for Crt_sh_Transparency_Logs_1_0 (#594)
  • Analyzers missing cortexutils in requirements (#695)
  • New mime types for Office documents in FileInfo (#705)
  • UmbrellaBlacklister analyzer now support fqdn and url observables (#547)
  • URLHaus analyzer support fqdn observables (#556)
  • Abuselpdb now support APIv2 (#618)

Get It While Supply Lasts!

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates` menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!

Cortex-Analyzers 2.5.0: 142 Analyzers, 16 Responders

Shortly after the release of Cortex-Analyzers 2.4.0, TheHive Project’s code Chefs are happy to announce Cortex-Analyzers 2.5.0, a new Cortex analyzer & responder release which brings the total to 142 analyzers and 16 responders, up from 138 and 10 respectively!

We’d like to thank all the contributors for their precious work which will certainly provide more options to fellow cyber defenders and cyber threat intelligence analysts for improving their efficiency and focus on what really matters.

Source: https://dilbert.com/strip/2007-08-03

What’s New?

New Analyzers

New Responders

Analyzers

Clamav

Clamav is a powerful and open source antivirus engine that allows writing custom signatures using Yara and sigtool. @Hestat contributed with this analyzer that permits to TheHive to communicate with a local clamav-daemon.

A detailed configuration guide is available on Hetstat’s website.

Clamav short report for safe and malicious samples

IPVoid

Contributed by @jdsnape, this analyzer leverages the IP reputation check of apivoid.com, the API of www.ipvoid.com. As you can probably guess by its name, this analyzer can be used to enrich ip observables.

In order to use this analyzer, an account and a valid subscription to apivoid.com are required. An API key needs then to be provided.

TheHive displays the analyzer results as follows:

IPVoid analyzer short report
IPVoid analyzer long report

ThreatResponse

This analyzer lets you leverage the Cisco Threat Response service. Query Threat Response for verdicts and sightings for observables of type domain, filename, fqdn, hash (MD5, SHA1, SHA256), ip and url.

The analyser report lets you pivot into a Threat Response investigation of an observable.

Combining it with AMP for Endpoints Responder

It will extract the connector GUIDs as new observables to enable seamless use of the AMP for Endpoints Responder if a target is returned from the AMP for Endpoints module. It requires the AMP for Endpoints module to be configured in Threat Response.

A valid Cisco ThreatResponse subscription is required, and you have to provide your client ID and password information to use this analyzer.

TheHive displays the analyzer results as follows:

ThreatResponse analyzer short report
ThreatResponser analyzer long report

ThreatGrid

This analyzer queries Cisco Threat Grid for file, url, or hash and deliver analysis report. It also lets you pivot into the Threat Grid report to access more information related to Behavioral indicators or TCP/IP stream.

A valid Cisco Threat Grid subscription is required, and you have to provide hostname and api key to use this analyzer.

TheHive displays the analyzer results as follows:

ThreatGrid analyzer short report
ThreatGrid analyzer long report

Responders

AMPForEndpoints

This responders performs several actions on Cisco AMP for Endpoints. It comes in 5 flavors:

  • AMPforEndpoints_IsolationStart: Start Host Isolation.
  • AMPforEndpoints_IsolationStop: Stop Host Isolation.
  • AMPforEndpoints_MoveGUID: Move Connector GUID to a new group.
  • AMPforEndpoints_SCDAdd: Add SHA256 to a Simple Custom Detection List. TheHive’s case ID and description are appended to the description
  • AMPforEndpoints_SCDRemove: Remove SHA256 from a Simple Custom Detetion List.

A valid Cisco AMP for Endpoints subscription is required, and you have to provide the client id, api key and several context information to use this responder.

Redmine

Redmine is a free and open source, web-based project management and issue tracking tool. It allows users to manage multiple projects and associated subprojects. 

This responder, contribuited by srilumpa, can be used to create an issue in the Redmine ticketing system from a case. It will use the case title as the issue subject and the case description as the issue body.

To set it up in Cortex, you will need:

  • To define a user to allow Cortex to connect to Redmine and with access to the various projects in which issues should be created
  • Define three custom fields in TheHive that will be used to select the project, the tracker and, optionally, the assignee of the issue. These fields can be free form or can be custom fields with preset values.
Custom fields in TheHive for Redmine integration

At the moment the responder has few capabilities. If you need any other integration feel free to discuss on the pull issue.

Cortex responder output and corresponding issue in Redmine

Fixes

  • Umbrella Investigate [#698]

Get It While Supply Lasts!

If you are still using the old-style way of installing analyzers and responders, run the following commands:

cd path/to/Cortex-Analyzers
git pull
for I in analyzers/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done
for I in responders/*/requirements.txt; do sudo -H pip3 install -U -r $I || true; done

Once done, ensure to refresh your analyzers and responders in the Cortex WebUI. Connect as an orgadmin and go to the Organization menu. Click on the Analyzers tab and click on the Refresh analyzers button. Do the same for the Responders tab: click on the Refresh responders button. Refer to the online Cortex documentation for further details.

Update TheHive Report Templates

If you are using TheHive, you must import the new report templates in your instance as follows:

  • download the updated package
  • log in TheHive using an administrator account
  • go to Admin > Report templates` menu
  • click on Import templates button and select the downloaded package

Running Into Trouble?

Shall you encounter any difficulty, please join our user forum, contact us on Gitter, or send us an email at support@thehive-project.org. We will be more than happy to help!