In Cyber Threat Intelligence, the collection and analysis of Indicators of Compromise (IOCs) is critical because they provide valuable information that can help organisations detect and respond to cyber threats more effectively.
IOCs are pieces of evidence or artefacts that indicate the presence of a threat, such as a virus, malware or malicious activity, within a system or network. Examples of IOCs include IP addresses, domain names, file hashes, URLs and email addresses. By collecting and analysing these IOCs, CTI analysts can identify patterns, trends and characteristics of cyber threats, which can help in the development of defensive measures.
Harvesting IOCs involves collecting data from a variety of sources, such as threat intelligence feeds, open source intelligence, and malware repositories. Once collected, these IOCs are analysed to determine their relevance, the severity of the threat, and the potential impact on the organisation’s systems and networks.
In the following list, I have compiled some useful tools for harvesting and analysing IOCs, ranging from open source tools to commercial solutions.
An open-source framework for receiving and redistributing abuse feeds and threat intel.
Share and collaborate in developing Threat Intelligence.
Tool to gather Threat Intelligence indicators from publicly available sources.
Pull intelligence per file hash. The output is in CSV format and sent to STDOUT so the data can be saved or piped into another program.
Pull intelligence per host. Like Fileintel, the output is in CSV format and sent to STDOUT so the data can be saved or piped into another program.
A tool for CERTs for processing incident data using a message queue.IntelMQ can be used for - automated incident handling - situational awareness - automated notifications - as data collector for other tools - etc.
A free editor from Fireeyefor XML IOC files, that allows:
Manipulation of the logical structures that define the IOC
Application of meta-information to IOCs, including detailed descriptions or arbitrary labels
Conversion of IOCs into XPath filters
Management of lists of “terms” used within IOCs
Advanced Indicator of Compromise (IOC) extractor, Python library and command-line tool: extracts URLs, IP addresses, MD5/SHA hashes, email addresses, and YARA rules from text corpora. It includes some encoded and “defanged” IOCs in the output, and optionally decodes/refangs them.
Python library for working with OpenIOC objects, from Mandiant.
Malware/IOC ingestion and processing engine, that enriches collected data.
At this time, the following feeds are supported:
Malware Information Sharing Platform curated by The MISP Project. The objective of MISP is to foster the sharing of structured information within the security community and abroad. MISP provides functionalities to support the exchange of information but also the consumption of said information by Network Intrusion Detection Systems (NIDS), LIDS but also log analysis tools, SIEMs.
Free, community-driven threat intelligence platform collecting IOCs from open-source feeds.
Formerly RiskIQ: research, connect, tag and share IPs and domains.
Aggregates security threats from a number of sources, and outputs to Syslog CEF, Snort Signatures, Iptables rules, hosts.deny, etc.
Build automated threat intel pipelines sourcing from Twitter, RSS, GitHub, and more.
A Python script to monitor and generate alerts based on IOCs indexed by a set of Google Custom Search Engines.
Data visualization and statistical analysis of Threat Intelligence feeds.