Ingesting InsecureWeb Threat Intelligence with Wazuh SIEM for Real-Time IOC Detection
Introduction
Threat Intelligence plays a critical role in modern Security Operations Centers (SOCs) by providing organizations with actionable information about malicious IP addresses, domains, hostnames, and malware hashes actively used by threat actors. Integrating external threat intelligence feeds into a Security Information and Event Management (SIEM) platform enables security teams to detect and respond to threats more effectively.
This project demonstrates the integration of InsecureWeb Threat Intelligence feeds with Wazuh SIEM to enhance threat detection capabilities. By importing threat intelligence indicators into Wazuh’s CDB (Constant Database) lists and creating custom correlation rules, the SIEM can automatically identify malicious activity observed within collected logs and generate high-priority alerts.
The implementation includes:
- Downloading InsecureWeb threat intelligence feeds
- Importing Indicators of Compromise (IOCs) into Wazuh CDB lists
- Automating feed updates using Python
- Creating custom Wazuh detection rules
- Generating alerts when malicious indicators appear in monitored logs
- Validating detections through simulated attack scenarios
Understanding InsecureWeb Threat Intelligence
Link: https://insecureweb.com/
InsecureWeb provides various cyber threat intelligence datasets that include:
- Blacklisted IP addresses
- Malicious domains
- Suspicious hostnames
- MD5 malware hashes
- SHA256 malware hashes
These indicators are continuously updated and can be used to identify known malicious infrastructure associated with cybercriminal operations, malware campaigns, phishing attacks, and botnet activities.
Within the InsecureWeb portal, the Threat Intelligence section provides access to downloadable IOC feeds and integration options.
The platform allows security teams to retrieve cumulative threat intelligence lists and integrate them into security monitoring solutions such as Wazuh.
Why Integrate Threat Intelligence with Wazuh?
Wazuh is an open-source SIEM and XDR platform capable of:
- Log analysis
- Threat detection
- File integrity monitoring
- Vulnerability assessment
- Security event correlation
- Incident response
By enriching Wazuh with external threat intelligence, analysts gain the ability to:
Detect Known Threat Actors
Logs containing IPs or domains that already exist in threat intelligence databases can be immediately flagged.
Improve Detection Accuracy
Threat intelligence provides context that reduces false positives and increases confidence in generated alerts.
Accelerate Incident Response
Security teams can quickly identify malicious communication attempts and take appropriate remediation actions.
Enhance SOC Visibility
Combining internal telemetry with external intelligence creates a more comprehensive security monitoring environment.
Threat Intelligence Feed Acquisition
The integration begins by accessing the InsecureWeb Threat Intelligence portal.
Within the Ingesting section, multiple downloadable IOC feeds are available.
Examples include:
Blacklisted IP Feeds
Used for identifying communication with known malicious IP addresses.
Malicious Domain Feeds
Used for detecting suspicious DNS requests and domain-based threats.
Hostname Feeds
Used to identify malicious hostnames observed in network activity.
MD5 Hash Feeds
Used for detecting known malware samples.
SHA256 Hash Feeds
Used for identifying advanced malware artifacts using stronger cryptographic hashes.
The platform provides downloadable feed URLs that can be consumed programmatically.
Creating the Wazuh Threat Intelligence Environment
A dedicated working directory was created for managing InsecureWeb integrations.
sudo mkdir -p /opt/wazuh-insecureweb
cd /opt/wazuh-insecureweb
touch update_insecureweb.py
chmod 750 update_insecureweb.py
chown wazuh:wazuh update_insecureweb.pyThis directory stores the automation script responsible for downloading and updating threat intelligence indicators.
Developing the Threat Intelligence Update Script
A custom Python script named:
update_insecureweb.pywas developed to automate the retrieval and processing of threat intelligence feeds.
The script imports required modules such as:
#!/usr/bin/env python3
import argparse
import gzip
import logging
import os
import re
import shutil
import subprocess
import tarfile
import tempfile
from pathlib import Path
import requests
FEEDS = {
"ip": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/ip",
"cdb_file": "insecureweb-ip",
},
"domain": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/domain",
"cdb_file": "insecureweb-domain",
},
"hostname": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/hostname",
"cdb_file": "insecureweb-hostname",
},
"md5": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/md5",
"cdb_file": "insecureweb-md5",
},
"sha256": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/sha256",
"cdb_file": "insecureweb-sha256",
},
}
WAZUH_LIST_DIR = "/var/ossec/etc/lists"
BACKUP_DIR = "/var/ossec/etc/lists/insecureweb-backups"
IP_RE = re.compile(r"^(?:\d{1,3}\.){3}\d{1,3}(?:/\d{1,2})?$")
MD5_RE = re.compile(r"^[a-fA-F0-9]{32}$")
SHA256_RE = re.compile(r"^[a-fA-F0-9]{64}$")
def download_feed(url, output_path):
logging.info("Downloading %s", url)
with requests.get(url, stream=True, timeout=120) as response:
response.raise_for_status()
with output_path.open("wb") as file:
for chunk in response.iter_content(chunk_size=1024 * 1024):
if chunk:
file.write(chunk)
def safe_extract_tar_gz(archive_path, extract_dir):
with tarfile.open(archive_path, "r:gz") as tar:
for member in tar.getmembers():
target = extract_dir / member.name
if not str(target.resolve()).startswith(str(extract_dir.resolve())):
raise RuntimeError(f"Unsafe tar path detected: {member.name}")
tar.extractall(extract_dir)
def read_file(path):
if path.name.endswith(".gz"):
with gzip.open(path, "rt", errors="ignore") as file:
yield from file
else:
with path.open("r", errors="ignore") as file:
yield from file
def extract_lines_from_archive(archive_path):
with tempfile.TemporaryDirectory(prefix="insecureweb_extract_") as tmp:
extract_dir = Path(tmp)
safe_extract_tar_gz(archive_path, extract_dir)
for file_path in extract_dir.rglob("*"):
if file_path.is_file():
yield from read_file(file_path)
def normalize_indicator(raw_line, feed_type):
line = raw_line.strip()
if not line or line.startswith("#"):
return None
line = line.split(":", 1)[0].strip()
line = line.strip("'\"[](),;")
line = re.split(r"[\s,]+", line)[0].strip().lower()
if not line:
return None
if feed_type == "ip":
return line if IP_RE.match(line) else None
if feed_type in {"domain", "hostname"}:
line = line.rstrip(".")
if "." not in line:
return None
if "/" in line or "://" in line:
return None
return line
if feed_type == "md5":
return line if MD5_RE.match(line) else None
if feed_type == "sha256":
return line if SHA256_RE.match(line) else None
return None
def collect_indicators(archive_path, feed_type):
indicators = set()
for raw_line in extract_lines_from_archive(archive_path):
indicator = normalize_indicator(raw_line, feed_type)
if indicator:
indicators.add(indicator)
return indicators
def set_wazuh_permissions(path):
try:
shutil.chown(path, user="wazuh", group="wazuh")
os.chmod(path, 0o660)
except Exception as error:
logging.warning(
"Failed to set ownership/permissions on %s: %s",
path,
error,
)
def prepare_backup_dir():
backup_path = Path(BACKUP_DIR)
backup_path.mkdir(parents=True, exist_ok=True)
try:
shutil.chown(backup_path, user="wazuh", group="wazuh")
os.chmod(backup_path, 0o770)
except Exception as error:
logging.warning(
"Failed to set backup directory ownership/permissions: %s",
error,
)
return backup_path
def write_cdb_list(indicators, destination):
destination.parent.mkdir(parents=True, exist_ok=True)
backup_path = prepare_backup_dir()
if destination.exists():
backup_file = backup_path / f"{destination.name}.bak"
shutil.copy2(destination, backup_file)
set_wazuh_permissions(backup_file)
tmp_file = destination.with_suffix(".tmp")
with tmp_file.open("w") as file:
for indicator in sorted(indicators):
file.write(f"{indicator}:\n")
os.chmod(tmp_file, 0o660)
os.replace(tmp_file, destination)
set_wazuh_permissions(destination)
def restart_wazuh():
logging.info("Restarting wazuh-manager")
subprocess.run(["systemctl", "restart", "wazuh-manager"], check=True)
def main():
parser = argparse.ArgumentParser(
description="Download InsecureWeb Threat Intel feeds and update Wazuh CDB lists."
)
parser.add_argument(
"--restart-wazuh",
action="store_true",
help="Restart wazuh-manager after updating CDB lists.",
)
args = parser.parse_args()
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
)
list_dir = Path(WAZUH_LIST_DIR)
list_dir.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="insecureweb_feeds_") as tmp:
tmp_dir = Path(tmp)
for feed_type, feed in FEEDS.items():
archive_path = tmp_dir / f"{feed_type}_level1.list.tar.gz"
destination = list_dir / feed["cdb_file"]
try:
download_feed(feed["url"], archive_path)
indicators = collect_indicators(archive_path, feed_type)
if not indicators:
logging.error(
"No indicators found for %s. Existing CDB list not changed.",
feed_type,
)
continue
write_cdb_list(indicators, destination)
logging.info(
"Updated %s with %d indicators",
destination,
len(indicators),
)
except Exception as error:
logging.exception("Failed processing %s: %s", feed_type, error)
if args.restart_wazuh:
restart_wazuh()
logging.info("Done")
if __name__ == "__main__":
main()The script defines multiple feed sources:
FEEDS = {
"ip": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/ip",
"cdb_file": "insecureweb-ip",
},
"domain": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/domain",
"cdb_file": "insecureweb-domain",
},
"hostname": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/hostname",
"cdb_file": "insecureweb-hostname",
},
"md5": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/md5",
"cdb_file": "insecureweb-md5",
},
"sha256": {
"url": "https://apis.threatwinds.com/api/feeds/v1/download/list/level1/accumulative/sha256",
"cdb_file": "insecureweb-sha256",
},
}Cron every 24 hours:
0 2 * * * /usr/bin/python3 /opt/wazuh-insecureweb/update_insecureweb_cdb.py --restart-wazuhThe script auto downloads, auto extracts .tar.gz, updates the Wazuh CDB list files, and because it uses a temporary directory, the downloaded .tar.gz files are auto deleted after the script finishes.
Each feed points to a corresponding InsecureWeb API endpoint.
The script performs the following operations:
- Download feed archives
- Extract feed contents
- Convert indicators into Wazuh-compatible format
- Automaticly Store indicators in CDB list files
- Build corresponding
.cdbdatabases - Auto add files “.cdb” permissions
- Replace previous threat intelligence lists
- Log update status
Automatic Feed Import into Wazuh
After executing the script:
python3 update_insecureweb.pyWazuh successfully downloaded and processed:
- IP indicators
- Domain indicators
- Hostname indicators
- MD5 indicators
- SHA256 indicators
The script reported successful imports, including thousands of indicators from each category.
Example output:
Updated insecureweb-ip with 1325029 indicators
Updated insecureweb-domain with 127339 indicators
Updated insecureweb-hostname with 370 indicators
Updated insecureweb-md5 with 60670 indicators
Updated insecureweb-sha256 with 146 indicatorsThis confirms successful ingestion of threat intelligence data into the SIEM environment.
Verifying Imported Threat Intelligence Lists
After execution, the generated files appeared under:
/var/ossec/etc/lists/Created files included:
insecureweb-ip
insecureweb-domain
insecureweb-hostname
insecureweb-md5
insecureweb-sha256Alongside compiled database files:
insecureweb-ip.cdb
insecureweb-domain.cdb
insecureweb-hostname.cdb
insecureweb-md5.cdb
insecureweb-sha256.cdbThe presence of both raw lists and compiled databases confirms successful Wazuh integration.
Registering Threat Intelligence Lists in Wazuh
The newly created IOC lists were registered inside the Wazuh Manager configuration.
File:
/var/ossec/etc/ossec.confAdded entries:
<list>etc/lists/insecureweb-ip</list>
<list>etc/lists/insecureweb-domain</list>
<list>etc/lists/insecureweb-hostname</list>
<list>etc/lists/insecureweb-md5</list>
<list>etc/lists/insecureweb-sha256</list>Save the configuration and restart wazuh-manager
This configuration enables Wazuh to load the threat intelligence feeds during startup.
Verifying CDB List Availability
Inside the Wazuh Dashboard, the imported lists became visible under:
Server Management → CDB Lists
The following lists were successfully registered:
- insecureweb-domain
- insecureweb-hostname
- insecureweb-ip
- insecureweb-md5
- insecureweb-sha256
This validated that Wazuh recognized the imported threat intelligence databases.
Creating Custom Detection Rules
Custom correlation rules were created within:
insecureweb_rules.xmlThe rules leverage Wazuh’s CDB lookup functionality.
Example:
<group name="insecureweb,threat_intel,">
<!-- Malicious source IP -->
<rule id="110001" level="12">
<if_sid>5716</if_sid>
<list field="srcip" lookup="address_match_key">etc/lists/insecureweb-ip</list>
<description>Malicious Source IP Found in InsecureWeb Threat Intel: $(srcip)</description>
<group>insecureweb_ip,malicious_ip,</group>
</rule>
<!-- Malicious destination IP -->
<rule id="110002" level="12">
<list field="dstip" lookup="address_match_key">etc/lists/insecureweb-ip</list>
<description>Malicious Destination IP Found in InsecureWeb Threat Intel: $(dstip)</description>
<group>insecureweb_ip,malicious_ip,</group>
</rule>
<!-- Malicious domain / host -->
<rule id="110003" level="12">
<list field="host" lookup="match_key">etc/lists/insecureweb-domain</list>
<description>Malicious Domain Found in InsecureWeb Threat Intel: $(host)</description>
<group>insecureweb_domain,malicious_domain,</group>
</rule>
<!-- Malicious DNS query -->
<rule id="110004" level="12">
<list field="dns.query" lookup="match_key">etc/lists/insecureweb-domain</list>
<description>Malicious DNS Query Found in InsecureWeb Threat Intel: $(dns.query)</description>
<group>insecureweb_domain,malicious_domain,dns,</group>
</rule>
<!-- Malicious hostname -->
<rule id="110005" level="12">
<list field="hostname" lookup="match_key">etc/lists/insecureweb-hostname</list>
<description>Malicious Hostname Found in InsecureWeb Threat Intel: $(hostname)</description>
<group>insecureweb_hostname,malicious_hostname,</group>
</rule>
<!-- Malicious MD5 -->
<rule id="110006" level="13">
<list field="md5" lookup="match_key">etc/lists/insecureweb-md5</list>
<description>Malicious MD5 Hash Found in InsecureWeb Threat Intel: $(md5)</description>
<group>insecureweb_hash,malicious_file,</group>
</rule>
<!-- Malicious SHA256 -->
<rule id="110007" level="13">
<list field="sha256" lookup="match_key">etc/lists/insecureweb-sha256</list>
<description>Malicious SHA256 Hash Found in InsecureWeb Threat Intel: $(sha256)</description>
<group>insecureweb_hash,malicious_file,</group>
</rule>
</group>After adding rules restart wazuh-manager
This rule performs the following:
- Waits for SSH authentication events.
- Extracts the source IP.
- Searches the IP within the InsecureWeb threat database.
- Generates a critical alert if a match exists.
Similar rules can be created for:
- Destination IPs
- Domains
- DNS Queries
- Hostnames
- MD5 Hashes
- SHA256 Hashes
You can also add <if_group> options in rules to correlate with specific groups.
Wazuh CDB List Reference: https://documentation.wazuh.com/current/user-manual/ruleset/cdb-list.html
Note: Also you can adjust these rules according to your requirements.
Detection Validation and Testing
To validate the integration, a known malicious IP address:
99.89.126.104was identified within the imported threat intelligence database.
Verification through the CDB list confirmed the IP existed inside:
insecureweb-ipGenerating Test Logs
A simulated SSH authentication failure log was created containing the malicious source IP.
Example log:
Jun 21 01:02:02 host sshd[1234]:
Failed none for Moiz from 99.89.126.104 port 22 ssh2The log was then written into:
/var/log/insecure_web.logand monitored by Wazuh.
Wazuh Alert Generation
Once Wazuh processed the log:
- The SSH decoder extracted the source IP.
- The custom rule checked the IP against the InsecureWeb database.
- A match was identified.
- Wazuh generated a level 12 security alert.
Generated alert:
Malicious Source IP Found in InsecureWeb Threat Intel:
99.89.126.104Alert details included:
- Rule ID: 110001
- Rule Level: 12
- Agent: Kali
- Source IP: 99.89.126.104
- Location: /var/log/insecure_web.log
Security Benefits of the Integration
This integration significantly improves the effectiveness of Wazuh by enabling:
IOC-Based Detection
Immediate identification of known malicious infrastructure.
Real-Time Alerting
Threats are detected as soon as matching indicators appear in logs.
Automated Intelligence Updates
Threat feeds remain current without manual intervention.
Enhanced SOC Operations
Security analysts gain contextual intelligence for investigations.
Reduced Mean Time to Detect (MTTD)
Known threats can be identified within seconds of observation.
Conclusion
Ingesting InsecureWeb Threat Intelligence with Wazuh SIEM provides a powerful threat detection capability by combining external intelligence with internal log monitoring. Through automated feed retrieval, CDB list generation, custom rule creation, and real-time IOC correlation, Wazuh becomes capable of identifying malicious IP addresses, domains, hostnames, and malware hashes as soon as they appear within monitored environments.
The successful detection of a malicious SSH source IP demonstrates how threat intelligence enrichment can transform raw log data into actionable security alerts. This implementation strengthens organizational security posture, enhances SOC visibility, and enables faster response to cyber threats through intelligence-driven detection mechanisms.
Join Wazuh Ambassador Program:
https://wazuh.com/ambassadors-program/
