Game-Theoretic Analysis of Adaptive Intrusion Detection Systems under Detection Uncertainty

Abstract

As computer networks expand and cyberattacks grow more sophisticated, Building Intrusion Detection Systems (IDS) that can withstand intelligent, adaptive adversaries has become a core problem in network security. Much of the existing literature assumes attacker behavior that is static or fully specified in advance, ignoring the uncertainty inherent to any real detection process, including false alarms and missed detections. Such simplifications make it difficult to capture how a defensive system and a learning attacker actually behave against each other over time. We address this gap with a game-theoretic framework, built on repeated games, that models the ongoing interaction between an IDS and an intelligent attacker. Both players are represented as autonomous agents that follow memory-one strategies and adjust their behavior through gradual learning. To make the model more realistic, detection uncertainty enters the security payoffs directly through the True Positive Rate (TPR) and the False Positive Rate (FPR), and a separate control parameter captures how quickly the attacker adapts relative to the defender, so we can study fast-adapting attacker scenarios. Numerical simulations produce a phase diagram over the space of attacker learning speed and IDS detection capability, revealing three distinct regimes: attacker dominance, adaptive competition, and a secure state. Faster-learning attackers push the security boundary outward, so the IDS needs stronger detection capability to stay in the secure regime. These findings offer practical guidance for designing and tuning adaptive  IDS  in dynamic, hostile network environments.