The database underneath your SIEM determines how fast you can investigate an incident, how much log history you can afford to keep, and how much hardware you need to run it. Elasticsearch has been the default for log storage for a decade, but ClickHouse has emerged as a compelling alternative for security workloads. Here is a technical comparison of the two for SIEM use.
The core distinction comes down to how each system stores data. Elasticsearch is a document store built on Apache Lucene, designed for full-text search. It indexes every field to make arbitrary text queries fast. ClickHouse is a columnar database built for online analytical processing (OLAP), designed to scan and aggregate enormous volumes of structured data quickly.
Security logs are mostly structured, time-series data — timestamps, IP addresses, usernames, event codes, ports. You query them with time-bounded filters and aggregations ("show me all failed logins from this subnet in the last 6 hours, grouped by source IP"). This is an analytical workload, which is exactly what columnar databases like ClickHouse are built for.
For the analytical queries that dominate security investigation — filtering and aggregating over time ranges across billions of rows — ClickHouse is typically significantly faster, often by an order of magnitude. Its columnar storage means a query touching three fields only reads those three columns from disk, not the entire document. Vectorised query execution processes data in batches that fit CPU cache.
Elasticsearch excels at a different thing: ranked full-text relevance search ("find log lines containing this rare string, ranked by relevance"). For pure free-text needle-in-haystack searches, Elasticsearch's inverted index is excellent. But that is a minority of security analytics work.
The practical impact
During an active incident, an analyst runs dozens of iterative queries refining a hypothesis. The difference between each query taking 8 seconds versus 0.5 seconds is the difference between fluid investigation and frustrating context-switching. Query latency compounds across an investigation.
This is where ClickHouse delivers its most dramatic advantage for security teams. Columnar storage compresses far better than row or document storage, because adjacent values in a column are similar and compress efficiently. ClickHouse routinely achieves 10:1 compression ratios on log data — sometimes much higher.
Elasticsearch, because it indexes every field for search, typically has substantial storage overhead — the indexes themselves can be as large as the raw data. For a security team that wants to retain a year or more of logs for compliance and threat hunting, the storage cost difference between the two systems is enormous. On ClickHouse, long-term retention that would be financially prohibitive on Elasticsearch becomes affordable on standard hardware.
Running a production Elasticsearch cluster for SIEM is a specialist skill. You manage shard allocation, index lifecycle policies, heap sizing (the JVM is notoriously sensitive), hot-warm-cold tiering, and cluster rebalancing. Many organisations dedicate engineers specifically to keeping their ELK stack healthy.
ClickHouse is generally simpler to operate at the scale most security teams need. It has no JVM heap to tune, a smaller operational surface, and predictable resource usage. For a self-hosted SIEM that needs to run reliably without a dedicated platform team, this matters enormously.
nPro uses ClickHouse as its storage engine specifically because security log analytics is an OLAP workload. The combination of fast time-bounded aggregations, high compression for affordable long-term retention, and lower operational complexity makes it the right foundation for a self-hosted SIEM that has to run reliably on customer hardware without a dedicated database team.
This is one of the core architectural differences between nPro and Elasticsearch-based SIEMs — see our nPro vs Elastic comparison for the full picture, including how it affects deployment time and total cost of ownership.
To be fair: if your primary use case is genuinely full-text search across unstructured documents — application logs with rich free-text messages where you frequently search for arbitrary strings — Elasticsearch's inverted index is excellent and may be the better fit. Some large organisations also run both: ClickHouse for high-volume structured security telemetry, Elasticsearch for specific full-text search needs. The right choice depends on your actual query patterns.
Sub-second queries on billions of events. Self-hosted, 5-minute deploy.
Related: What is a SIEM? · nPro vs Elastic · Deploy on Ubuntu