Log retention is one of the most consequential and most frequently mishandled decisions in running a SIEM. Retain too little and you cannot investigate an incident or satisfy an auditor; retain too much on the wrong architecture and storage costs spiral. This guide covers how long to keep security logs and how to architect retention so it is both compliant and affordable.
The single most important statistic in incident response is attacker dwell time. Industry data consistently shows median dwell times measured in weeks to months. If your logs only go back 30 days but the attacker was present for 90, your investigation hits a wall exactly where it matters most.
The retention paradox
The logs you most need during a breach are often the oldest — from when the initial compromise happened, weeks or months before you noticed. Short retention saves money right up until the moment it costs you the entire investigation.
Compliance: PCI-DSS requires at least one year, with three months immediately available. ISO 27001 expects retention aligned to a risk assessment. Regional regimes — Malaysia PDPA, Singapore PDPA, Indonesia UU PDP — expect you to evidence controls and reconstruct breaches. Forensics: retention should cover realistic dwell times; mature teams retain 12 months or more. Cost: bounded by what you can afford to store and query.
Authentication and identity logs: 90 days hot, 12 to 24 months total. Firewall and network: 30 to 90 days hot, 12 months total. Endpoint: 90 days hot, 12 months total. Critical audit logs for compliance: up to 7 years where required. These are starting points driven by your specific obligations and risk assessment.
The key to affordable long retention is tiering: a hot tier (30 to 90 days) in fast storage for real-time detection; a warm tier still queryable on cheaper storage; and a cold tier of compressed long-term archive for compliance and deep forensics.
The economics come down to compression. A SIEM on a columnar store like ClickHouse — which nPro uses — typically achieves around 10:1 compression. A year of logs that would occupy 10TB raw fits in roughly 1TB. On platforms with high per-GB overhead, the same retention is often prohibitive, forcing teams to cut retention precisely when they should extend it. See our ClickHouse vs Elasticsearch comparison.
The cloud SIEM retention trap
Cloud SIEMs charging per GB make long retention expensive, so teams shorten it to control the bill — unknowingly destroying future forensic capability. A self-hosted SIEM with strong compression removes this pressure: you set retention based on risk, not invoice anxiety.
nPro 10:1 compression makes year-plus retention affordable. Self-hosted, no per-GB fees.