research article on cyber security

• Search by keyword
• Search by citation

Page 1 of 5

Cloud EMRs auditing with decentralized ( t ,  n )-threshold ownership transfer

In certain cloud Electronic Medical Records (EMRs) applications, the data ownership may need to be transferred. In practice, not only the data but also the auditing ability should be transferred securely and e...

• View Full Text

SIFT: Sifting file types—application of explainable artificial intelligence in cyber forensics

Artificial Intelligence (AI) is being applied to improve the efficiency of software systems used in various domains, especially in the health and forensic sciences. Explainable AI (XAI) is one of the fields of...

Modelling user notification scenarios in privacy policies

The processing of personal data gives a rise to many privacy concerns, and one of them is to ensure the transparency of data processing to end users. Usually this information is communicated to them using priv...

FLSec-RPL: a fuzzy logic-based intrusion detection scheme for securing RPL-based IoT networks against DIO neighbor suppression attacks

The Internet of Things (IoT) has gained popularity and is widely used in modern society. The growth in the sizes of IoT networks with more internet-connected devices has led to concerns regarding privacy and s...

New partial key exposure attacks on RSA with additive exponent blinding

Partial key exposure attacks present a significant threat to RSA-type cryptosystems. These attacks factorize the RSA modulus by utilizing partial knowledge of the decryption exponent, which is typically reveal...

Dynamic group fuzzy extractor

The group fuzzy extractor allows group users to extract and reproduce group cryptographic keys from their individual non-uniform random sources. It can be easily used in group-oriented cryptographic applicatio...

EvilPromptFuzzer: generating inappropriate content based on text-to-image models

Text-to-image (TTI) models provide huge innovation ability for many industries, while the content security triggered by them has also attracted wide attention. Considerable research has focused on content secu...

ProcSAGE: an efficient host threat detection method based on graph representation learning

Advanced Persistent Threats (APTs) achieves internal networks penetration through multiple methods, making it difficult to detect attack clues solely through boundary defense measures. To address this challeng...

Lightweight ring-neighbor-based user authentication and group-key agreement for internet of drones

As mobile internet and Internet of Things technologies continue to advance, the application scenarios of peer-to-peer Internet of Drones (IoD) are becoming increasingly diverse. However, the development of IoD...

A multi-channel spatial information feature based human pose estimation algorithm

Human pose estimation is an important task in computer vision, which can provide key point detection of human body and obtain bone information. At present, human pose estimation is mainly utilized for detectio...

TVRAVNF: an efficient low-cost TEE-based virtual remote attestation scheme for virtual network functions

With the continuous advancement of virtualization technology and the widespread adoption of 5G networks, the application of the Network Function Virtualization (NFV) architecture has become increasingly popula...

Efficient post-quantum secure deterministic wallet scheme

Since the advent of Bitcoin, cryptocurrencies have gained substantial popularity, and crypto wallets have evolved into the predominant tool for safeguarding and managing cryptographic keys to access cryptocurrenc...

Classification of DDoS attack traffic on SDN network environment using deep learning

Distributed Denial of Service (DDoS) attack is a major threat to the Internet of Things (IoT), Software Defined Networks (SDN), and Cloud Computing Networks. Due to the tremendous applications of IoT networks,...

Revisiting frequency-smoothing encryption: new security definitions and efficient construction

Deterministic encryption (DET) allows for fast retrieval of encrypted information, but it would cause significant leakage of frequency information of the underlying data, which results in an array of inference...

GLDOC: detection of implicitly malicious MS-Office documents using graph convolutional networks

Nowadays, the malicious MS-Office document has already become one of the most effective attacking vectors in APT attacks. Though many protection mechanisms are provided, they have been proved easy to bypass, a...

Revealing the exploitability of heap overflow through PoC analysis

The exploitable heap layouts are used to determine the exploitability of heap vulnerabilities in general-purpose applications. Prior studies have focused on using fuzzing-based methods to generate more exploit...

Threshold ring signature: generic construction and logarithmic size instantiation

A ring signature is a variant of normal digital signature and protects the privacy of a specific signer in the sense that a ring signature can be verified, but the signer’s identity can only be traced to a lim...

FedSHE: privacy preserving and efficient federated learning with adaptive segmented CKKS homomorphic encryption

Unprotected gradient exchange in federated learning (FL) systems may lead to gradient leakage-related attacks. CKKS is a promising approximate homomorphic encryption scheme to protect gradients, owing to its u...

A privacy-preserving image retrieval scheme with access control based on searchable encryption in media cloud

With the popularity of the media cloud computing industry, individuals and organizations outsource image computation and storage to the media cloud server to reduce the storage burden. Media images usually con...

Improved homomorphic evaluation for hash function based on TFHE

Homomorphic evaluation of hash functions offers a solution to the challenge of data integrity authentication in the context of homomorphic encryption. The earliest attempt to achieve homomorphic evaluation of ...

An empirical study of reflection attacks using NetFlow data

Reflection attacks are one of the most intimidating threats organizations face. A reflection attack is a special type of distributed denial-of-service attack that amplifies the amount of malicious traffic by u...

Phishing behavior detection on different blockchains via adversarial domain adaptation

Despite the growing attention on blockchain, phishing activities have surged, particularly on newly established chains. Acknowledging the challenge of limited intelligence in the early stages of new chains, we...

Ensemble learning based anomaly detection for IoT cybersecurity via Bayesian hyperparameters sensitivity analysis

The Internet of Things (IoT) integrates more than billions of intelligent devices over the globe with the capability of communicating with other connected devices with little to no human intervention. IoT enab...

CommanderUAP: a practical and transferable universal adversarial attacks on speech recognition models

Most of the adversarial attacks against speech recognition systems focus on specific adversarial perturbations, which are generated by adversaries for each normal example to achieve the attack. Universal adver...

Enhancing fairness of trading environment: discovering overlapping spammer groups with dynamic co-review graph optimization

Within the thriving e-commerce landscape, some unscrupulous merchants hire spammer groups to post misleading reviews or ratings, aiming to manipulate public perception and disrupt fair market competition. This...

In-depth Correlation Power Analysis Attacks on a Hardware Implementation of CRYSTALS-Dilithium

During the standardisation process of post-quantum cryptography, NIST encourages research on side-channel analysis for candidate schemes. As the recommended lattice signature scheme, CRYSTALS-Dilithium, when i...

Atomic cross-chain swap based on private key exchange

Atomic Cross-Chain Swap (ACCS) is one important topic in cryptocurrency, where users can securely and trustlessly exchange assets between two different blockchains. However, most known ACCS schemes assume spec...

HSS: enhancing IoT malicious traffic classification leveraging hybrid sampling strategy

Using deep learning models to deal with the classification tasks in network traffic offers a new approach to address the imbalanced Internet of Things malicious traffic classification problems. However, the em...

Key derivable signature and its application in blockchain stealth address

Stealth address protocol (SAP) is widely used in blockchain to achieve anonymity. In this paper, we formalize a key derivable signature scheme (KDS) to capture the functionality and security requirements of SA...

Polar code-based secure transmission with higher message rate combining channel entropy and computational entropy

The existing physical layer security schemes, which are based on the key generation model and the wire-tap channel model, achieve security by utilizing channel reciprocity entropy and noise entropy, respective...

Dissecting zero trust: research landscape and its implementation in IoT

As a progressive security strategy, the zero trust model has attracted notable attention and importance within the realm of network security, especially in the context of the Internet of Things (IoT). This pap...

Study of smart grid cyber-security, examining architectures, communication networks, cyber-attacks, countermeasure techniques, and challenges

Smart Grid (SG) technology utilizes advanced network communication and monitoring technologies to manage and regulate electricity generation and transport. However, this increased reliance on technology and co...

A multi-agent adaptive deep learning framework for online intrusion detection

The network security analyzers use intrusion detection systems (IDSes) to distinguish malicious traffic from benign ones. The deep learning-based (DL-based) IDSes are proposed to auto-extract high-level featur...

Iterative and mixed-spaces image gradient inversion attack in federated learning

As a distributed learning paradigm, federated learning is supposed to protect data privacy without exchanging users’ local data. Even so, the gradient inversion attack , in which the adversary can reconstruct the ...

Winternitz stack protocols for embedded systems and IoT

This paper proposes and evaluates a new bipartite post-quantum digital signature protocol based on Winternitz chains and an  oracle. Mutually mistrustful Alice and Bob are able to agree and sign a series of do...

Joint contrastive learning and belief rule base for named entity recognition in cybersecurity

Named Entity Recognition (NER) in cybersecurity is crucial for mining information during cybersecurity incidents. Current methods rely on pre-trained models for rich semantic text embeddings, but the challenge...

DTA: distribution transform-based attack for query-limited scenario

In generating adversarial examples, the conventional black-box attack methods rely on sufficient feedback from the to-be-attacked models by repeatedly querying until the attack is successful, which usually res...

A survey on lattice-based digital signature

Lattice-based digital signature has become one of the widely recognized post-quantum algorithms because of its simple algebraic operation, rich mathematical foundation and worst-case security, and also an impo...

Shorter ZK-SNARKs from square span programs over ideal lattices

Zero-knowledge succinct non-interactive arguments of knowledge (zk-SNARKs) are cryptographic protocols that offer efficient and privacy-preserving means of verifying NP language relations and have drawn consid...

Revocable and verifiable weighted attribute-based encryption with collaborative access for electronic health record in cloud

The encryption of user data is crucial when employing electronic health record services to guarantee the security of the data stored on cloud servers. Attribute-based encryption (ABE) scheme is considered a po...

Maxwell’s Demon in MLP-Mixer: towards transferable adversarial attacks

Models based on MLP-Mixer architecture are becoming popular, but they still suffer from adversarial examples. Although it has been shown that MLP-Mixer is more robust to adversarial attacks compared to convolu...

Practical solutions in fully homomorphic encryption: a survey analyzing existing acceleration methods

Fully homomorphic encryption (FHE) has experienced significant development and continuous breakthroughs in theory, enabling its widespread application in various fields, like outsourcing computation and secure...

A circuit area optimization of MK-3 S-box

In MILCOM 2015, Kelly et al. proposed the authentication encryption algorithm MK-3, which applied the 16-bit S-box. This paper aims to implement the 16-bit S-box with less circuit area. First, we classified th...

Intrusion detection system for controller area network

The rapid expansion of intra-vehicle networks has increased the number of threats to such networks. Most modern vehicles implement various physical and data-link layer technologies. Vehicles are becoming incre...

CT-GCN+: a high-performance cryptocurrency transaction graph convolutional model for phishing node classification

Due to the anonymous and contract transfer nature of blockchain cryptocurrencies, they are susceptible to fraudulent incidents such as phishing. This poses a threat to the property security of users and hinder...

Enhanced detection of obfuscated malware in memory dumps: a machine learning approach for advanced cybersecurity

In the realm of cybersecurity, the detection and analysis of obfuscated malware remain a critical challenge, especially in the context of memory dumps. This research paper presents a novel machine learning-bas...

BRITD: behavior rhythm insider threat detection with time awareness and user adaptation

Researchers usually detect insider threats by analyzing user behavior. The time information of user behavior is an important concern in internal threat detection.

F3l: an automated and secure function-level low-overhead labeled encrypted traffic dataset construction method for IM in Android

Fine-grained function-level encrypted traffic classification is an essential approach to maintaining network security. Machine learning and deep learning have become mainstream methods to analyze traffic, and ...

WAS: improved white-box cryptographic algorithm over AS iteration

The attacker in white-box model has full access to software implementation of a cryptographic algorithm and full control over its execution environment. In order to solve the issues of high storage cost and in...

Full-round impossible differential attack on shadow block cipher

Lightweight block ciphers are the essential encryption algorithm for devices with limited resources. Its goal is to ensure the security of data transmission through resource-constrained devices. Impossible dif...

• Editorial Board
• Sign up for article alerts and news from this journal

Affiliated with

The Institute of Information Engineering (IIE) is a national research institute in Beijing that specializes in comprehensive research on theories and applications related to information technology.

IIE strives to be a leading global academic institution by creating first-class research platforms and attracting top researchers. It also seeks to become an important national strategic power in the field of information technology.

IIE’s mission is to promote China’s innovation and industrial competitiveness by advancing information science, standards, and technology in ways that enhance economic security and public safety as well as improve our quality of life.

Read more..

The journal is indexed by

• EI Compendex
• Emerging Sources Citation Index
• EBSCO Discovery Service
• Institute of Scientific and Technical Information of China
• Google Scholar
• Norwegian Register for Scientific Journals and Series
• OCLC WorldCat Discovery Service
• ProQuest-ExLibris Primo
• ProQuest-ExLibris Summon
• TD Net Discovery Service
• UGC-CARE List (India)

Annual Journal Metrics

Citation Impact 2023 Journal Impact Factor: 3.9 5-year Journal Impact Factor: 4.9 Source Normalized Impact per Paper (SNIP): 1.587 SCImago Journal Rank (SJR): 1.136

Speed 2023 Submission to first editorial decision (median days): 8 Submission to acceptance (median days): 95

Usage 2023 Downloads: 408,523 Altmetric mentions: 15

• ISSN: 2523-3246 (electronic)

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Journal Proposal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Journal Description

Journal of cybersecurity and privacy.

• Open Access — free for readers, with article processing charges (APC) paid by authors or their institutions.
• High Visibility:  indexed within  Scopus ,  EBSCO , and  other databases .
• Rapid Publication: manuscripts are peer-reviewed and a first decision is provided to authors approximately 32.4 days after submission; acceptance to publication is undertaken in 4.6 days (median values for papers published in this journal in the first half of 2024).
• Journal Rank:  CiteScore - Q1 ( Computer Science (miscellaneous) )
• Recognition of Reviewers: APC discount vouchers, optional signed peer review, and reviewer names published annually in the journal.
• Companion journal: Sensors .

Latest Articles

Journal Menu

• Aims & Scope
• Editorial Board
• Reviewer Board
• Topical Advisory Panel
• Instructions for Authors

Special Issues

• Sections & Collections
• Article Processing Charge
• Indexing & Archiving
• Most Cited & Viewed
• Journal Statistics
• Journal History
• Editorial Office

Journal Browser

• arrow_forward_ios Forthcoming issue arrow_forward_ios Current issue
• Vol. 4 (2024)
• Vol. 3 (2023)
• Vol. 2 (2022)
• Vol. 1 (2021)

Highly Accessed Articles

Latest books, e-mail alert.

Conferences

Topical collections, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Programs submenu

Regions submenu, topics submenu, press briefing: previewing the quad leaders summit and the high-level week of unga 79, what are the impacts of gps jamming and spoofing on civilians, ending forced labor: partnerships for a path forward.

• Abshire-Inamori Leadership Academy
• Aerospace Security Project
• Africa Program
• Americas Program
• Arleigh A. Burke Chair in Strategy
• Asia Maritime Transparency Initiative
• Asia Program
• Australia Chair
• Brzezinski Chair in Global Security and Geostrategy
• Brzezinski Institute on Geostrategy
• Chair in U.S.-India Policy Studies
• China Power Project
• Chinese Business and Economics
• Defending Democratic Institutions
• Defense-Industrial Initiatives Group
• Defense 360
• Defense Budget Analysis
• Diversity and Leadership in International Affairs Project
• Economics Program
• Emeritus Chair in Strategy
• Energy Security and Climate Change Program
• Europe, Russia, and Eurasia Program
• Freeman Chair in China Studies
• Futures Lab
• Geoeconomic Council of Advisers
• Global Food and Water Security Program
• Global Health Policy Center
• Hess Center for New Frontiers
• Human Rights Initiative
• Humanitarian Agenda
• Intelligence, National Security, and Technology Program

International Security Program

• Japan Chair
• Kissinger Chair
• Korea Chair
• Langone Chair in American Leadership
• Middle East Program
• Missile Defense Project
• Project on Critical Minerals Security
• Project on Fragility and Mobility
• Project on Nuclear Issues
• Project on Prosperity and Development
• Project on Trade and Technology
• Renewing American Innovation
• Scholl Chair in International Business
• Smart Women, Smart Power
• Southeast Asia Program
• Stephenson Ocean Security Project

Strategic Technologies Program

• Sustainable Development and Resilience Initiative
• Wadhwani Center for AI and Advanced Technologies
• Warfare, Irregular Threats, and Terrorism Program
• All Regions
• Australia, New Zealand & Pacific
• Middle East
• Russia and Eurasia
• American Innovation
• Civic Education
• Climate Change

Cybersecurity

• Defense Budget and Acquisition
• Defense and Security
• Energy and Sustainability
• Food Security
• Gender and International Security
• Geopolitics
• Global Health
• Human Rights
• Humanitarian Assistance
• Intelligence
• International Development
• Maritime Issues and Oceans
• Missile Defense
• Nuclear Issues
• Transnational Threats
• Water Security

Led by the Strategic Technologies Program and the International Security Program , CSIS’s cybersecurity portfolio covers cyber warfare, encryption, military cyber capacity, hacking, financial terrorism, and more.

Photo: Adrian Grosu/Adobe Stock

AI and Advanced Technologies in the Fight: Combatant Command and Service Collaboration

The CSIS Wadhwani Center for AI and Advanced Technologies hosted a discussion on the warfighter’s adoption of emerging technologies, featuring the Chief Technology Officers of CENTCOM, the Department of the Navy, and the Chief of Staff of the Army.

Transcript — September 13, 2024

Blackout Scorecard

Commentary by James Andrew Lewis — July 25, 2024

A Russian Bot Farm Used AI to Lie to Americans. What Now?

Commentary by Emily Harding — July 16, 2024

Why the United Nations Is Chasing Its Tail on Cybersecurity

Commentary by James Andrew Lewis — July 16, 2024

Latest Podcasts

Nvidia's Earnings Report, Gallium and Germanium Export Controls, and OpenAI's National Security Demo

Podcast Episode by Gregory C. Allen and H. Andrew Schwartz — September 5, 2024

How to Make Cyber Policy a Headline in Brussels

Podcast Episode by James Andrew Lewis and Christopher Painter — August 27, 2024

Our Biggest Fight: Reclaiming Liberty, Humanity, and Dignity in the Digital Age

Podcast Episode by H. Andrew Schwartz and Frank H. McCourt, Jr. — August 5, 2024

Inside the State Department’s Cyber Strategy: A Conversation with Adam Segal

Podcast Episode by James Andrew Lewis and Christopher Painter — July 26, 2024

Past Events

The Cyber Safety Review Board: Reflecting on the Past & Charting the Future

Cyber Leaders Series: The future of cyber on the African continent; a conversation with Kenya's PS Tanui

The New Era of U.S.-Japan Strategic Cooperation: A Dialogue with Japanese Lawmakers

Cyber Incident Reporting in the Communications Sector

Counterspace Trends: An Evolving Global Landscape

Shaping the Future of Federal Cybersecurity: Insights from FCEBs

Related programs.

James Andrew Lewis

Suzanne Spaulding

Emily Harding

Clayton Swope

All cybersecurity content, type open filter submenu.

• Article (159)
• Event (149)
• Expert/Staff (33)
• Podcast Episode (96)
• Report (67)

Article Type open filter submenu

Report type open filter submenu, region open filter submenu.

• Afghanistan (5)
• Americas (43)
• Australia, New Zealand, and Pacific (6)
• Caribbean Security (1)
• Central Asia (1)
• Eastern Europe (12)
• Europe (40)
• European Union (18)
• Middle East (13)
• North Africa (1)
• North America (62)
• Russia (36)
• Russia and Eurasia (29)
• South America (7)
• Southeast Asia (13)
• Sub-Saharan Africa (4)
• The South Caucasus (1)

A Discussion on the UN Cybercrime Convention

Join CSIS for a discussion on the impact of the United Nations Convention on Cybercrime and some of the myths surrounding it.

Event — October 4, 2024

Symposium: AI in the Department of Justice

 This Artificial Intelligence and Justice Symposium will look at how the U.S. Department of Justice is responsibly approaching AI and its use in law enforcement activities.  

Event — October 2, 2024

Cyber Leaders Series: Winning the Cyber War; a Conversation with Israel Soong

Please join CSIS virtually on Friday, September 20th, from 9:00 a.m. to 10:00 a.m. EDT for a conversation with Israel Soong, National Security Council Director of East Asia & Pacific Cyber Policy. 

Event — September 20, 2024

CSIS is pleased to host an event examining how combatant commands and services collaborate to drive the adoption of AI for the warfighter on September 13.

Event — September 13, 2024

Looking Beyond TikTok – The Risks of Temu

CSIS affiliate, Diane Rinaldo, examines the privacy concerns and national security risks of the popular e-commerce app Temu.  

Blog Post by Diane Rinaldo — September 11, 2024

Please join the CSIS Defending Democratic Institutions Project on Monday, September 9 at 9:15 AM for a conversation on The Cyber Safety Review Board: Reflecting on the Past & Charting the Future with Robert Silvers, the Under Secretary for Policy at the Department of Homeland Security. 

Event — September 9, 2024

In this episode, we discuss Nvidia's earnings report and its implications for the AI industry, the impact of China's Gallium and Germanium export controls on the global semiconductor competition, and why OpenAI is demonstrating its capabilities for the national security community.

Jim Lewis and Chris Painter talk to Estonia's former ambassador at large for cyber diplomacy Heli Tirmaa-Klaar.

Revisiting Australia's Encryption Landscape

This piece provides a timeline and overview of Australia's encryption legislation amid new debates of its effectiveness in intelligence and law enforcement. 

Blog Post by Taylar Rajic — August 20, 2024

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 May 2023

A holistic and proactive approach to forecasting cyber threats

• Zaid Almahmoud 1 ,
• Paul D. Yoo 1 ,
• Omar Alhussein 2 ,
• Ilyas Farhat 3 &
• Ernesto Damiani 4 , 5  

Scientific Reports volume  13 , Article number:  8049 ( 2023 ) Cite this article

6682 Accesses

10 Citations

2 Altmetric

Metrics details

• Computer science
• Information technology

Traditionally, cyber-attack detection relies on reactive, assistive techniques, where pattern-matching algorithms help human experts to scan system logs and network traffic for known virus or malware signatures. Recent research has introduced effective Machine Learning (ML) models for cyber-attack detection, promising to automate the task of detecting, tracking and blocking malware and intruders. Much less effort has been devoted to cyber-attack prediction, especially beyond the short-term time scale of hours and days. Approaches that can forecast attacks likely to happen in the longer term are desirable, as this gives defenders more time to develop and share defensive actions and tools. Today, long-term predictions of attack waves are mostly based on the subjective perceptiveness of experienced human experts, which can be impaired by the scarcity of cyber-security expertise. This paper introduces a novel ML-based approach that leverages unstructured big data and logs to forecast the trend of cyber-attacks at a large scale, years in advance. To this end, we put forward a framework that utilises a monthly dataset of major cyber incidents in 36 countries over the past 11 years, with new features extracted from three major categories of big data sources, namely the scientific research literature, news, blogs, and tweets. Our framework not only identifies future attack trends in an automated fashion, but also generates a threat cycle that drills down into five key phases that constitute the life cycle of all 42 known cyber threats.

Similar content being viewed by others

Knowledge mining of unstructured information: application to cyber domain

Machine learning partners in criminal networks

A novel hybrid feature selection and ensemble-based machine learning approach for botnet detection

Introduction.

Running a global technology infrastructure in an increasingly de-globalised world raises unprecedented security issues. In the past decade, we have witnessed waves of cyber-attacks that caused major damage to governments, organisations and enterprises, affecting their bottom lines 1 . Nevertheless, cyber-defences remained reactive in nature, involving significant overhead in terms of execution time. This latency is due to the complex pattern-matching operations required to identify the signatures of polymorphic malware 2 , which shows different behaviour each time it is run. More recently, ML-based models were introduced relying on anomaly detection algorithms. Although these models have shown a good capability to detect unknown attacks, they may classify benign behaviour as abnormal 3 , giving rise to a false alarm.

We argue that data availability can enable a proactive defense, acting before a potential threat escalates into an actual incident. Concerning non-cyber threats, including terrorism and military attacks, proactive approaches alleviate, delay, and even prevent incidents from arising in the first place. Massive software programs are available to assess the intention, potential damages, attack methods, and alternative options for a terrorist attack 4 . We claim that cyber-attacks should be no exception, and that nowadays we have the capabilities to carry out proactive, low latency cyber-defenses based on ML 5 .

Indeed, ML models can provide accurate and reliable forecasts. For example, ML models such as AlphaFold2 6 and RoseTTAFold 7 can predict a protein’s three-dimensional structure from its linear sequence. Cyber-security data, however, poses its unique challenges. Cyber-incidents are highly sensitive events and are usually kept confidential since they affect the involved organisations’ reputation. It is often difficult to keep track of these incidents, because they can go unnoticed even by the victim. It is also worth mentioning that pre-processing cyber-security data is challenging, due to characteristics such as lack of structure, diversity in format, and high rates of missing values which distort the findings.

When devising a ML-based method, one can rely on manual feature identification and engineering, or try and learn the features from raw data. In the context of cyber-incidents, there are many factors ( i.e. , potential features) that could lead to the occurrence of an attack. Wars and political conflicts between countries often lead to cyber-warfare 8 , 9 . The number of mentions of a certain attack appearing in scientific articles may correlate well with the actual incident rate. Also, cyber-attacks often take place on holidays, anniversaries and other politically significant dates 5 . Finding the right features out of unstructured big data is one of the key strands of our proposed framework.

The remainder of the paper is structured as follows. The “ Literature review ” section presents an overview of the related work and highlights the research gaps and our contributions. The “ Methods ” section describes the framework design, including the construction of the dataset and the building of the model. The “ Results ” section presents the validation results of our model, the trend analysis and forecast, and a detailed description of the developed threat cycle. Lastly, the “ Discussion ” section offers a critical evaluation of our work, highlighting its strengths and limitations, and provides recommendations for future research.

Literature review

In recent years, the literature has extensively covered different cyber threats across various application domains, and researchers have proposed several solutions to mitigate these threats. In the Social Internet of Vehicles (SIoV), one of the primary concerns is the interception and tampering of sensitive information by attackers 10 . To address this, a secure authentication protocol has been proposed that utilises confidential computing environments to ensure the privacy of vehicle-generated data. Another application domain that has been studied is the privacy of image data, specifically lane images in rural areas 11 . The proposed methodology uses Error Level Analysis (ELA) and artificial neural network (ANN) algorithms to classify lane images as genuine or fake, with the U-Net model for lane detection in bona fide images. The final images are secured using the proxy re-encryption technique with RSA and ECC algorithms, and maintained using fog computing to protect against forgery.

Another application domain that has been studied is the security of Wireless Mesh Networks (WMNs) in the context of the Internet of Things (IoT) 12 . WMNs rely on cooperative forwarding, making them vulnerable to various attacks, including packet drop/modification, badmouthing, on-off, and collusion attacks. To address this, a novel trust mechanism framework has been proposed that differentiates between legitimate and malicious nodes using direct and indirect trust computation. The framework utilises a two-hop mechanism to observe the packet forwarding behaviour of neighbours, and a weighted D-S theory to aggregate recommendations from different nodes. While these solutions have shown promising results in addressing cyber threats, it is important to anticipate the type of threat that may arise to ensure that the solutions can be effectively deployed. By proactively identifying and anticipating cyber threats, organisations can better prepare themselves to protect their systems and data from potential attacks.

While we are relatively successful in detecting and classifying cyber-attacks when they occur 13 , 14 , 15 , there has been a much more limited success in predicting them. Some studies exist on short-term predictive capability 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , such as predicting the number or source of attacks to be expected in the next hours or days. The majority of this work performs the prediction in restricted settings ( e.g. , against a specific entity or organisation) where historical data are available 18 , 19 , 25 . Forecasting attack occurrences has been attempted by using statistical methods, especially when parametric data distributions could be assumed 16 , 17 , as well as by using ML models 20 . Other methods adopt a Bayesian setting and build event graphs suitable for estimating the conditional probability of an attack following a given chain of events 21 . Such techniques rely on libraries of predefined attack graphs: they can identify the known attack most likely to happen, but are helpless against never-experienced-before, zero-day attacks.

Other approaches try to identify potential attackers by using network entity reputation and scoring 26 . A small but growing body of research explores the fusion of heterogeneous features (warning signals) to forecast cyber-threats using ML. Warning signs may include the number of mentions of a victim organisation on Twitter 18 , mentions in news articles about the victim entity 19 , and digital traces from dark web hacker forums 20 . Our literature review is summarised in Table 1 .

Forecasting the cyber-threats that will most likely turn into attacks in the medium and long term is of significant importance. It not only gives to cyber-security agencies the time to evaluate the existing defence measures, but also assists them in identifying areas where to develop preventive solutions. Long-term prediction of cyber-threats, however, still relies on the subjective perceptions of human security experts 27 , 28 . Unlike a fully automated procedure based on quantitative metrics, the human-based approach is prone to bias based on scientific or technical interests 29 . Also, quantitative predictions are crucial to scientific objectivity 30 . In summary, we highlight the following research gaps:

Current research primarily focuses on detecting ( i.e. , reactive) rather than predicting cyber-attacks ( i.e. , proactive).

Available predictive methods for cyber-attacks are mostly limited to short-term predictions.

Current predictive methods for cyber-attacks are limited to restricted settings ( e.g. , a particular network or system).

Long-term prediction of cyber-attacks is currently performed by human experts, whose judgement is subjective and prone to bias and disagreement.

Research contributions

Our objective is to fill these research gaps by a proactive, long-term, and holistic approach to attack prediction. The proposed framework gives cyber-security agencies sufficient time to evaluate existing defence measures while also providing objective and accurate representation of the forecast. Our study is aimed at predicting the trend of cyber-attacks up to three years in advance, utilising big data sources and ML techniques. Our ML models are learned from heterogeneous features extracted from massive, unstructured data sources, namely, Hackmageddon 9 , Elsevier 31 , Twitter 32 , and Python APIs 33 . Hackmageddon provides more than 15, 000 records of global cyber-incidents since the year 2011, while Elsevier API offers access to the Scopus database, the largest abstract and citation database of peer-reviewed literature with over 27,000,000 documents 34 . The number of relevant tweets we collected is around 9 million. Our study covers 36 countries and 42 major attack types. The proposed framework not only provides the forecast and categorisation of the threats, but also generates a threat life-cycle model, whose the five key phases underlie the life cycle of all 42 known cyber-threats. The key contribution of this study consists of the following:

A novel dataset is constructed using big unstructured data ( i.e. , Hackmageddon) including news and government advisories, in addition to Elsevier, Twitter, and Python API. The dataset comprises monthly counts of cyber-attacks and other unique features, covering 42 attack types across 36 countries.

Our proactive approach offers long-term forecasting by predicting threats up to 3 years in advance.

Our approach is holistic in nature, as it does not limit itself to specific entities or regions. Instead, it provides projections of attacks across 36 countries situated in diverse parts of the world.

Our approach is completely automated and quantitative, effectively addressing the issue of bias in human predictions and providing a precise forecast.

By analysing past and predicted future data, we have classified threats into four main groups and provided a forecast of 42 attacks until 2025.

The first threat cycle is proposed, which delineates the distinct phases in the life cycle of 42 cyber-attack types.

The framework of forecasting cyber threats

The architecture of our framework for forecasting cyber threats is illustrated in Fig. 1 . As seen in the Data Sources component (l.h.s), to harness all the relevant data and extract meaningful insights, our framework utilises various sources of unstructured data. One of our main sources is Hackmageddon, which includes massive textual data on major cyber-attacks (approx. 15,334 incidents) dating back to July 2011. We refer to the monthly number of attacks in the list as the Number of Incidents (NoI). Also, Elsevier’s Application Programming Interface (API) gives access to a very large corpus of scientific articles and data sets from thousands of sources. Utilising this API, we obtained the Number of Mentions (NoM) ( e.g. , monthly) of each attack that appeared in the scientific publications. This NoM data is of particular importance as it can be used as the ground truth for attack types that do not appear in Hackmageddon. During the preliminary research phase, we examined all the potentially relevant features and noticed that wars/political conflicts are highly correlated to the number of cyber-events. These data were then extracted via Twitter API as Armed Conflict Areas/Wars (ACA). Lastly, as attacks often take place around holidays, Python’s holidays package was used to obtain the number of public holidays per month for each country, which is referred to as Public Holidays (PH).

To ensure the accuracy and quality of Hackmageddon data, we validated it using the statistics from official sources across government, academia, research institutes and technology organisations. For a ransomware example, the Cybersecurity & Infrastructure Security Agency stated in their 2021 trend report that cybersecurity authorities in the United States, Australia, and the United Kingdom observed an increase in sophisticated, high-impact ransomware incidents against critical infrastructure organisations globally 35 . The WannaCry attack in the dataset was also validated with Ghafur et al ’s 1 statement in their article: “WannaCry ransomware attack was a global epidemic that took place in May 2017”.

An example of an entry in the Hackmageddon dataset is shown in Table 2 . Each entry includes the incident date, the description of the attack, the attack type, and the target country. Data pre-processing (Fig. 1 ) focused on noise reduction through imputing missing values ( e.g. , countries), which were often observed in the earlier years. We were able to impute these values from the description column or occasionally, by looking up the entity location using Google.

The textual data were quantified via our Word Frequency Counter (WFC), which counted the number of each attack type per month as in Table 3 . Cumulative Aggregation (CA) obtained the number of attacks for all countries combined and an example of a data entry after transformation includes the month, and the number of attacks against each country (and all countries combined) for each attack type. By adding features such as NoM, ACA, and PH, we ended up having additional features that we appended to the dataset as shown in Table 4 . Our final dataset covers 42 common types of attacks in 36 countries. The full list of attacks is provided in Table 5 . The list of the countries is given in Supplementary Table S1 .

To analyse and investigate the main characteristics of our data, an exploratory analysis was conducted focusing on the visualisation and identification of key patterns such as trend and seasonality, correlated features, missing data and outliers. For seasonal data, we smoothed out the seasonality so that we could identify the trend while removing the noise in the time series 36 . The smoothing type and constants were optimised along with the ML model (see Optimisation for details). We applied Stochastic selection of Features (SoF) to find the subset of features that minimises the prediction error, and compared the univariate against the multivariate approach.

For the modelling, we built a Bayesian encoder-decoder Long Short-Term Memory (B-LSTM) network. B-LSTM models have been proposed to predict “perfect wave” events like the onset of stock market “bear” periods on the basis of multiple warning signs, each having different time dynamics 37 . Encoder-decoder architectures can manage inputs and outputs that both consist of variable-length sequences. The encoder stage encodes a sequence into a fixed-length vector representation (known as the latent representation). The decoder prompts the latent representation to predict a sequence. By applying an efficient latent representation, we train the model to consider all the useful warning information from the input sequence - regardless of its position - and disregard the noise.

Our Bayesian variation of the encoder-decoder LSTM network considers the weights of the model as random variables. This way, we extract epistemic uncertainty via (approximate) Bayesian inference, which quantifies the prediction error due to insufficient information 38 . This is an important parameter, as epistemic uncertainty can be reduced by better intelligence, i.e. , by acquiring more samples and new informative features. Details are provided in “ Bayesian long short-term memory ” section.

Our overall analytical platform learns an operational model for each attack type. Here, we evaluated the model’s performance in predicting the threat trend 36 months in advance. A newly modified symmetric Mean Absolute Percentage Error (M-SMAPE) was devised as the evaluation metric, where we added a penalty term that accounts for the trend direction. More details are provided in the “ Evaluation metrics ” section.

Feature extraction

Below, we provide the details of the process that transforms raw data into numerical features, obtaining the ground truth NoI and the additional features NoM, ACA and PH.

NoI: The number of daily incidents in Hackmageddon was transformed from the purely unstructured daily description of attacks along with the attack and country columns, to the monthly count of incidents for each attack in each country. Within the description, multiple related attacks may appear, which are not necessarily in the attack column. Let \(E_{x_i}\) denote the set of entries during the month \(x_i\) in Hackmageddon dataset. Let \(a_j\) and \(c_k\) denote the j th attack and k th country. Then NoI can be expressed as follows:

where \(Z(a_j,c_k,e)\) is a function that evaluates to 1 if \(a_j\) appears either in the description or in the attack columns of entry e and \(c_k\) appears in the country column of e . Otherwise, the function evaluates to 0. Next, we performed CA to obtain the monthly count of attacks in all countries combined for each attack type as follows:

NoM: We wrote a Python script to query Elsevier API for the number of mentions of each attack during each month 31 . The search covers the title, abstract and keywords of published research papers that are stored in Scopus database 39 . Let \(P_{x_i}\) denote the set of research papers in Scopus published during the month \(x_i\) . Also, let \(W_{p}\) denote the set of words in the title, abstract and keywords of research paper p . Then NoM can be expressed as follows:

where \(U(w,a_j)\) evaluates to 1 if \(w=a_j\) , and to 0 otherwise.

ACA: Using Twitter API in Python 32 , we wrote a query to obtain the number of tweets with keywords related to political conflicts or military attacks associated with each country during each month. The keywords used for each country are summarised in Supplementary Table S2 , representing our query. Formally, let \(T_{x_i}\) denote the set of all tweets during the month \(x_i\) . Then ACA can be expressed as follows:

where \(Q(t,c_k)\) evaluates to 1 if the query in Supplementary Table S2 evaluates to 1 given t and \(c_k\) . Otherwise, it evaluates to 0.

PH: We used the Python holidays library 33 to count the number of days that are considered public holidays in each country during each month. More formally, this can be expressed as follows:

where \(H(d,c_k)\) evaluates to 1 if the day d in the country \(c_k\) is a public holiday, and to 0 otherwise. In ( 4 ) and ( 5 ), CA was used to obtain the count for all countries combined as in ( 2 ).

Data integration

Based on Eqs. ( 1 )–( 5 ), we obtain the following columns for each month:

NoI_C: The number of incidents for each attack type in each country ( \(42 \times 36\) columns) [Hackmageddon].

NoI: The total number of incidents for each attack type (42 columns) [Hackmageddon].

NoM: The number of mentions of each attack type in research articles (42 columns) [Elsevier].

ACA_C: The number of tweets about wars and conflicts related to each country (36 columns) [Twitter].

ACA: The total number of tweets about wars and conflicts (1 column) [Twitter].

PH_C: The number of public holidays in each country (36 columns) [Python].

PH: The total number of public holidays (1 column) [Python].

In the aforementioned list of columns, the name enclosed within square brackets denotes the source of data. By matching and combining these columns, we derive our monthly dataset, wherein each row represents a distinct month. A concrete example can be found in Tables 3 and 4 , which, taken together, constitute a single observation in our dataset. The dataset can be expanded through the inclusion of other monthly features as supplementary columns. Additionally, the dataset may be augmented with further samples as additional monthly records become available. Some suggestions for extending the dataset are provided in the “ Discussion ” section.

Data smoothing

We tested multiple smoothing methods and selected the one that resulted in the model with the lowest M-SMAPE during the hyper-parameter optimisation process. The methods we tested include exponential smoothing (ES), double exponential smoothing (DES) and no smoothing (NS). Let \(\alpha \) be the smoothing constant. Then the ES formula is:

where \(D(x_{i})\) denotes the original data at month \(x_{i}\) . For the DES formula, let \(\alpha \) and \(\beta \) be the smoothing constants. We first define the level \(l(x_{i})\) and the trend \(\tau (x_{i})\) as follows:

then, DES is expressed as follows:

The smoothing constants ( \(\alpha \) and \(\beta \) ) in the aforementioned methods are chosen as the predictive results of the ML model that gives the lowest M-SMAPE during the hyper-parameter optimisation process. Supplementary Fig. S5 depicts an example for the DES result.

Bayesian long short-term memory

LSTM is a type of recurrent neural network (RNN) that uses lagged observations to forecast the future time steps 30 . It was introduced as a solution to the so-called vanishing/exploding gradient problem of traditional RNNs 40 , where the partial derivative of the loss function may suddenly approach zero at some point of the training. In LSTM, the input is passed to the network cell, which combines it with the hidden state and cell state values from previous time steps to produce the next states. The hidden state can be thought of as a short-term memory since it stores information from recent periods in a weighted manner. On the other hand, the cell state is meant to remember all the past information from previous intervals and store them in the LSTM cell. The cell state thus represents the long-term memory.

LSTM networks are well-suited for time-series forecasting, due to their proficiency in retaining both long-term and short-term temporal dependencies 41 , 42 . By leveraging their ability to capture these dependencies within cyber-attack data, LSTM networks can effectively recognise recurring patterns in the attack time-series. Moreover, the LSTM model is capable of learning intricate temporal patterns in the data and can uncover inter-correlations between various variables, making it a compelling option for multivariate time-series analysis 43 .

Given a sequence of LSTM cells, each processing a single time-step from the past, the final hidden state is encoded into a fixed-length vector. Then, a decoder uses this vector to forecast future values. Using such architecture, we can map a sequence of time steps to another sequence of time steps, where the number of steps in each sequence can be set as needed. This technique is referred to as encoder-decoder architecture.

Because we have relatively short sequences within our refined data ( e.g. , 129 monthly data points over the period from July 2011 to March 2022), it is crucial to extract the source of uncertainty, known as epistemic uncertainty 44 , which is caused by lack of knowledge. In principle, epistemic uncertainty can be reduced with more knowledge either in the form of new features or more samples. Deterministic (non-stochastic) neural network models are not adequate to this task as they provide point estimates of model parameters. Rather, we utilise a Bayesian framework to capture epistemic uncertainty. Namely, we adopt the Monte Carlo dropout method proposed by Gal et al. 45 , who showed that the use of non-random dropout neurons during ML training (and inference) provides a Bayesian approximation of the deep Gaussian processes. Specifically, during the training of our LSTM encoder-decoder network, we applied the same dropout mask at every time-step (rather than applying a dropout mask randomly from time-step to time-step). This technique, known as recurrent dropout is readily available in Keras 46 . During the inference phase, we run trained model multiple times with recurrent dropout to produce a distribution of predictive results. Such prediction is shown in Fig. 4 .

Figure 2 shows our encoder-decoder B-LSTM architecture. The hidden state and cell state are denoted respectively by \(h_{i}\) and \(C_{i}\) , while the input is denoted by \(X_{i}\) . Here, the length of the input sequence (lag) is a hyper-parameter tuned to produce the optimal model, where the output is a single time-step. The number of cells ( i.e. , the depth of each layer) is tuned as a hyper-parameter in the range between 25 and 200 cells. Moreover, we used one or two layers, tuning the number of layers to each attack type. For the univariate model we used a standard Rectified Linear Unit (ReLU) activation function, while for the multivariate model we used a Leaky ReLU. Standard ReLU computes the function \(f(x)=max(0,x)\) , thresholding the activation at zero. In the multivariate case, zero-thresholding may generate the same ReLU output for many input vectors, making the model convergence slower 47 . With Leaky ReLU, instead of defining ReLU as zero when \(x < 0\) , we introduce a negative slope \(\alpha =0.2\) . Additionally, we used recurrent dropout ( i.e. , arrows in red as shown in Fig. 2 ), where the probability of dropping out is another hyper-parameter that we tune as described above, following Gal’s method 48 . The tuned dropout value is maintained during the testing and prediction as previously mentioned. Once the final hidden vector \(h_{0}\) is produced by the encoder, the Repeat Vector layer is used as an adapter to reshape it from the bi-dimensional output of the encoder ( e.g. , \(h_{0}\) ) to the three-dimensional input expected by the decoder. The decoder processes the input and produces the hidden state, which is then passed to a dense layer to produce the final output.

Each time-step corresponds to a month in our model. Since the model is learnt to predict a single time-step (single month), we use a sliding window during the prediction phase to forecast 36 (monthly) data points. In other words, we predict a single month at each step, and the predicted value is fed back for the prediction of the following month. This concept is illustrated in the table shown in Fig. 2 . Utilising a single time-step in the model’s output minimises the size of the sliding window, which in turn allows for training with as many observations as possible with such limited data.

The difference between the univariate and multivariate B-LSTMs is that the latter carries additional features in each time-step. Thus, instead of passing a scalar input value to the network, we pass a vector of features including the ground truth at each time-step. The model predicts a vector of features as an output, from which we retrieve the ground truth, and use it along with the other predicted features as an input to predict the next time-step.

Evaluation metrics

The evaluation metric SMAPE is a percentage (or relative) error based accuracy measure that judges the prediction performance purely on how far the predicted value is from the actual value 49 . It is expressed by the following formula:

where \(F_{t}\) and \(A_{t}\) denote the predicted and actual values at time t . This metric returns a value between 0% and 100%. Given that our data has zero values in some months ( e.g. , emerging threats), the issue of division by zero may arise, a problem that often emerges when using standard MAPE (Mean Absolute Percentage Error). We find SMAPE to be resilient to this problem, since it has both the actual and predicted values in the denominator.

Recall that our model aims to predict a curve (corresponding to multiple time steps). Using plain SMAPE as the evaluation metric, the “best” model may turn out to be simply a straight line passing through the same points of the fluctuating actual curve. However, this is undesired in our case since our priority is to predict the trend direction (or slope) over its intensity or value at a certain point. We hence add a penalty term to SMAPE that we apply when the height of the predicted curve is relatively smaller than that of the actual curve. This yields the modified SMAPE (M-SMAPE). More formally, let I ( V ) be the height of the curve V , calculated as follows:

where n is the curve width or the number of data points. Let A and F denote the actual and predicted curves. We define M-SMAPE as follows:

where \(\gamma \) is a penalty constant between 0 and 1, and d is another constant \(\ge \) 1. In our experiment, we set \(\gamma \) to 0.3, and d to 3, as we found these to be reasonable values by trial and error. We note that the range of possible values of M-SMAPE is between 0% and (100 + 100 \(\gamma \) )% after this modification. By running multiple experiments we found out that the modified evaluation metric is more suitable for our scenario, and therefore was adopted for the model’s evaluation.

Optimisation

On average, our model was trained on around 67% of the refined data, which is equivalent to approximately 7.2 years. We kept the rest, approximately 33% (3 years + lag period), for validation. These percentages may slightly differ for different attack types depending on the optimal lag period selected.

For hyper-parameter optimisation, we performed a random search with 60 iterations, to obtain the set of features, smoothing methods and constants, and model’s hyper-parameters that results in the model with the lowest M-SMAPE. Random search is a simple and efficient technique for hyper-parameter optimisation, with advantages including efficiency, flexibility, robustness, and scalability. The technique has been studied extensively in the literature and was found to be superior to grid search in many cases 50 . For each set of hyper-parameters, the model was trained using the mean squared error (MSE) as the loss function, and while using ADAM as the optimisation algorithm 51 . Then, the model was validated by forecasting 3 years while using M-SMAPE as the evaluation metric, and the average performance was recorded over 3 different seeds. Once the set of hyper-parameters with the minimum M-SMAPE was obtained, we used it to train the model on the full data, after which we predicted the trend for the next 3 years (until March, 2025).

The first group of hyper-parameters is the subset of features in the case of the multivariate model. Here, we experimented with each of the 3 features separately (NoM, ACA or PH) along with the ground truth (NoI), in addition to the combination of all features. The second group is the smoothing methods and constants. The set of methods includes ES, DES and NS, as previously discussed. The set of values for the smoothing constant \(\alpha \) ranges from 0.05 to 0.7 while the set of values for the smoothing constant \(\beta \) (for DES) ranges from 0.3 to 0.7. Next is the optimisation of the lag period with values that range from 1 to 12 months. This is followed by the model’s hyper-parameters which include the learning rate with values that range from \(6\times 10^{-4}\) to \(1\times 10^{-2}\) , the number of epochs with values between 30 and 200, the number of layers in the range 1 to 2, the number of units in the range 25 to 200, and the recurrent dropout value between 0.2 and 0.5. The range of these values was obtained from the literature and the online code repositories 52 .

Validation and comparative analysis

The results of our model’s validation are provided in Fig. 3 and Table 5 . As shown in Fig. 3 , the predicted data points are well aligned with the ground truth. Our models successfully predicted the next 36 months of all the attacks’ trends with an average M-SMAPE of 0.25. Table 5 summarises the validation results of univariate and multivariate approaches using B-LSTM. The results show that with approximately 69% of all the attack types, the multivariate approach outperformed the univariate approach. As seen in Fig. 3 , the threats that have a consistent increasing or emerging trend seemed to be more suitable for the univariate approach, while threats that have a fluctuating or decreasing trend showed less validation error when using the multivariate approach. The feature of ACA resulted in the best model for 33% of all the attack types, which makes it among the three most informative features that can boost the prediction performance. The PH accounts for 17% of all the attacks followed by NoM that accounts for 12%.

We additionally compared the performance of the proposed model B-LSTM with other models namely LSTM and ARIMA. The comparison covers the univariate and multivariate approaches of LSTM and B-LSTM, with two features in the case of multivariate approach namely NoI and NoM. The comparison is in terms of the Mean Absolute Percentage Error (MAPE) when predicting four common attack types, namely DDoS, Password Attack, Malware, and Ransomware. A comparison table is provided in Supplementary Table S3 . The results illustrate the superiority of the B-LSTM model for most of the attack types.

Trends analysis

The forecast of each attack trend until the end of the first quarter of 2025 is given in Supplementary Figs. S1 – S4 . By visualising the historical data of each attack as well as the prediction for the next three years, we were able to analyse the overall trend of each attack. The attacks generally follow 4 types of trends: (1) rapidly increasing, (2) overall increasing, (3) emerging and (4) decreasing. The names of attacks for each category are provided in Fig. 4 .

The first trend category is the rapidly increasing trend (Fig. 4 a—approximately 40% of the attacks belong to this trend. We can see that the attacks belonging to this category have increased dramatically over the past 11 years. Based on the model’s prediction, some of these attacks will exhibit a steep growth until 2025. Examples include session hijacking, supply chain, account hijacking, zero-day and botnet. Some of the attacks under this category have reached their peak, have recently started stabilising, and will probably remain steady over the next 3 years. Examples include malware, targeted attack, dropper and brute force attack. Some attacks in this category, after a recent increase, are likely to level off in the next coming years. These are password attack, DNS spoofing and vulnerability-related attacks.

The second trend category is the overall increasing trend as seen in Fig. 4 b. Approximately 31% of the attacks seem to follow this trend. The attacks under this category have a slower rate of increase over the years compared to the attacks in the first category, with occasional fluctuations as can be observed in the figure. Although some of the attacks show a slight recent decline ( e.g. , malvertising, keylogger and URL manipulation), malvertising and keylogger are likely to recover and return to a steady state while URL manipulation is projected to continue a smooth decline. Other attacks typical of “cold” cyber-warfare like Advanced Persistent Threats (APT) and rootkits are already recovering from a small drop and will likely to rise to a steady state by 2025. Spyware and data breach have already reached their peak and are predicted to decline in the near future.

Next is the emerging trend as shown in Fig. 4 c. These are the attacks that started to grow significantly after the year 2016, although many of them existed much earlier. In our study, around 17% of the attacks follow this trend. Some attacks have been growing steeply and are predicted to continue this trend until 2025. These are Internet of Things (IoT) device attack and deepfake. Other attacks have also been increasing rapidly since 2016, however, are likely to slow down after 2022. These include ransomware and adversarial attacks. Interestingly, some attacks that emerged after 2016 have already reached the peak and recently started a slight decline ( e.g. , cryptojacking and WannaCry ransomware attack). It is likely that WannaCry will become relatively steady in the coming years, however, cryptojacking will probably continue to decline until 2025 thanks to the rise of proof-of-stake consensus mechanisms 53 .

The fourth and last trend category is the decreasing trend (Fig. 4 d—only 12% of the attacks follow this trend. Some attacks in this category peaked around 2012, and have been slowly decreasing since then ( e.g. , SQL Injection and defacement). The drive-by attack also peaked in 2012, however, had other local peaks in 2016 and 2018, after which it declined noticeably. Cross-site scripting (XSS) and pharming had their peak more recently compared to the other attacks, however, have been smoothly declining since then. All the attacks under this category are predicted to become relatively stable from 2023 onward, however, they are unlikely to disappear in the next 3 years.

The threat cycle

This large-scale analysis involving the historical data and the predictions for the next three years enables us to come up with a generalisable model that traces the evolution and adoption of the threats as they pass through successive stages. These stages are named by the launch, growth, maturity, trough and stability/decline. We refer to this model as The Threat Cycle (or TTC), which is depicted in Fig. 5 . In the launch phase, few incidents start appearing for a short period. This is followed by a sharp increase in terms of the number of incidents, growth and visibility as more and more cyber actors learn and adopt this new attack. Usually, the attacks in the launch phase are likely to have many variants as observed in the case of the WannaCry attack in 2017. At some point, the number of incidents reaches a peak where the attack enters the maturity phase, and the curve becomes steady for a while. Via the trough (when the attack experiences a slight decline as new security measures seem to be very effective), some attacks recover and adapt to the security defences, entering the slope of plateau, while others continue to smoothly decline although they do not completely disappear ( i.e. , slope of decline). It is worth noting that the speed of transition between the different phases may vary significantly between the attacks.

As seen in Fig. 5 , the attacks are placed on the cycle based on the slope of their current trend, while considering their historical trend and prediction. In the trough phase, we can see that the attacks will either follow the slope of plateau or the slope of decline. Based on the predicted trend in the blue zone in Fig. 4 , we were able to indicate the future direction for some of the attacks close to the split point of the trough using different colours (blue or red). Brute force, malvertising, the Distributed Denial-of-Service attack (DDoS), insider threat, WannaCry and phishing are denoted in blue meaning that these are likely on their way to the slope of plateau. In the first three phases, it is usually unclear and difficult to predict whether a particular attack will reach the plateau or decline, thus, denoted in grey.

There are some similarities and differences between TTC and the well-known Gartner hype cycle (GHC) 54 . A standard GHC is shown in a vanishing green colour in Fig. 5 . As TTC is specific to cyber threats, it has a much wider peak compared to GHC. Although both GHC and TTC have a trough phase, the threats decline slightly (while significant drop in GHC) as they exit their maturity phase, after which they recover and move to stability (slope of plateau) or decline.

Many of the attacks in the emerging category are observed in the growth phase. These include IoT device attack, deepfake and data poisoning. While ransomwares (except WannaCry) are in the growth phase, WannaCry already reached the trough, and is predicted to follow the slope of plateau. Adversarial attack has just entered the maturity stage, and cryptojacking is about to enter the trough. Although adversarial attack is generally regarded as a growing threat, interestingly, this machine-based prediction and introspection shows that it is maturing. The majority of the rapidly increasing threats are either in the growth or in the maturity phase. The attacks in the growth phase include session hijacking, supply chain, account hijacking, zero-day and botnet. The attacks in the maturity phase include malware, targeted attack, vulnerability-related attacks and Man-In-The-Middle attack (MITM). Some rapidly increasing attacks such as phishing, brute force, and DDoS are in the trough and are predicted to enter the stability. We also observe that most of the attacks in the category of overall increasing threats have passed the growth phase and are mostly branching to the slope of plateau or the slope of decline, while few are still in the maturity phase ( e.g. , spyware). All of the decreasing threats are on the slope of decline. These include XSS, pharming, drive-by, defacement and SQL injection.

Highlights and limitations

This study presents the development of a ML-based proactive approach for long-term prediction of cyber-attacks offering the ability to communicate effectively with the potential attacks and the relevant security measures in an early stage to plan for the future. This approach can contribute to the prevention of an incident by allowing more time to develop optimal defensive actions/tools in a contested cyberspace. Proactive approaches can also effectively reduce uncertainty when prioritising existing security measures or initiating new security solutions. We argue that cyber-security agencies should prioritise their resources to provide the best possible support in preventing fastest-growing attacks that appear in the launch phase of TTC or the attacks in the categories of the rapidly increasing or emerging trend as in Fig. 4 a and c based on the predictions in the coming years.

In addition, our fully automated approach is promising to overcome the well-known issues of human-based analysis, above all expertise scarcity. Given the absence of the possibility of analysing with human’s subjective bias while following a purely quantitative procedure and data, the resulting predictions are expected to have lower degree of subjectivity, leading to consistencies within the subject. By fully automating this analytic process, the results are reproducible and can potentially be explainable with help of the recent advancements in Explainable Artificial Intelligence.

Thanks to the massive data volume and wide geographic coverage of the data sources we utilised, this study covers every facet of today’s cyber-attack scenario. Our holistic approach performs the long-term prediction on the scale of 36 countries, and is not confined to a specific region. Indeed, cyberspace is limitless, and a cyber-attack on critical infrastructure in one country can affect the continent as a whole or even globally. We argue that our Threat Cycle (TTC) provides a sound basis to awareness of and investment in new security measures that could prevent attacks from taking place. We believe that our tool can enable a collective defence effort by sharing the long-term predictions and trend analysis generated via quantitative processes and data and furthering the analysis of its regional and global impacts.

Zero-day attacks exploit a previously unknown vulnerability before the developer has had a chance to release a patch or fix for the problem 55 . Zero-day attacks are particularly dangerous because they can be used to target even the most secure systems and go undetected for extended periods of time. As a result, these attacks can cause significant damage to an organisation’s reputation, financial well-being, and customer trust. Our approach takes the existing research on using ML in the field of zero-day attacks to another level, offering a more proactive solution. By leveraging the power of deep neural networks to analyse complex, high-dimensional data, our approach can help agencies to prepare ahead of time, in-order to prevent the zero-day attack from happening at the first place, a problem that there is no existing fix for it despite our ability to detect it. Our results in Fig. 4 a suggest that zero-day attack is likely to continue a steep growth until 2025. If we know this information, we can proactively invest on solutions to prevent it or slow down its rise in the future, since after all, the ML detection approaches may not be alone sufficient to reduce its effect.

A limitation of our approach is its reliance on a restricted dataset that encompasses data since 2011 only. This is due to the challenges we encountered in accessing confidential and sensitive information. Extending the prediction phase requires the model to make predictions further into the future, where there may be more variability and uncertainty. This could lead to a decrease in prediction accuracy, especially if the underlying data patterns change over time or if there are unforeseen external factors that affect the data. While not always the case, this uncertainty is highlighted by the results of the Bayesian model itself as it expresses this uncertainty through the increase of the confidence interval over time (Fig. 3 a and b). Despite incorporating the Bayesian model to tackle the epistemic uncertainty, our model could benefit substantially from additional data to acquire a comprehensive understanding of past patterns, ultimately improving its capacity to forecast long-term trends. Moreover, an augmented dataset would allow ample opportunity for testing, providing greater confidence in the model’s resilience and capability to generalise.

Further enhancements can be made to the dataset by including pivotal dates (such as anniversaries of political events and war declarations) as a feature, specifically those that experience a high frequency of cyber-attacks. Additionally, augmenting the dataset with digital traces that reflect the attackers’ intentions and motivations obtained from the dark web would be valuable. Other informative features could facilitate short-term prediction, specifically to forecast the on-set of each attack.

Future work

Moving forward, future research can focus on augmenting the dataset with additional samples and informative features to enhance the model’s performance and its ability to forecast the trend in the longer-term. Also, the work opens a new area of research that focuses on prognosticating the disparity between the trend of cyber-attacks and the associated technological solutions and other variables, with the aim of guiding research investment decisions. Subsequently, TTC could be improved by adopting another curve model that can visualise the current development of relevant security measures. The threat trend categories (Fig. 4 ) and TTC (Fig. 5 ) show how attacks will be visible in the next three years and more, however, we do not know where the relevant security measures will be. For example, data poisoning is an AI-targeted adversarial attack that attempts to manipulate the training dataset to control the prediction behaviour of a machine-learned model. From the scientific literature data ( e.g. , Scopus), we could analyse the published articles studying the data poisoning problem and identify the relevant keywords of these articles ( e.g. , Reject on Negative Impact (RONI) and Probability of Sufficiency (PS)). RONI and PS are typical methods used for detecting poisonous data by evaluating the effect of individual data points on the performance of the trained model. Likewise, the features that are informative, discriminating or uncertainty-reducing for knowing how the relevant security measures evolve exist within such online sources in the form of author’s keywords, number of citations, research funding, number of publications, etc .

The workflow and architecture of forecasting cyber threats. The ground truth of Number of Incidents (NoI) was extracted from Hackmageddon which has over 15,000 daily records of cyber incidents worldwide over the past 11 years. Additional features were obtained including the Number of Mentions (NoM) of each attack in the scientific literature using Elsevier API which gives access to over 27 million documents. The number of tweets about Armed Conflict Areas/Wars (ACA) was also obtained using Twitter API for each country, with a total of approximately 9 million tweets. Finally, the number of Public Holidays (PH) in each country was obtained using the holidays library in Python. The data preparation phase includes data re-formatting, imputation and quantification using Word Frequency Counter (WFC) to obtain the monthly occurrence of attacks per country and Cumulative Aggregation (CA) to obtain the sum for all countries. The monthly NoM, ACA and PHs were quantified and aggregated using CA. The numerical features were then combined and stored in the refined database. The percentages in the refined database are based on the contribution of each data source. In the exploratory analysis phase, the analytic platform analyses the trend and performs data smoothing using Exponential Smoothing (ES), Double Exponential Smoothing (DES) and No Smoothing (NS). The smoothing methods and Smoothing Constants (SCs) were chosen for each attack followed by the Stochastic Selection of Features (SoF). In the model development phase, the meta data was partitioned into approximately 67% for training and 33% for testing. The models were learned using the encoder-decoder architecture of the Bayesian Long Short-Term Memory (B-LSTM). The optimisation component finds the set of hyper-parameters that minimises the error (i.e., M-SMAPE), which is then used for learning the operational models. In the forecasting phase, we used the operational models to predict the next three years’ NoIs. Analysing the predicted data, trend types were identified and attacks were categorised into four different trends. The slope of each attack was then measured and the Magnitude of Slope (MoS) was analysed. The final output is The Threat Cycle (TTC) illustrating the attacks trend, status, and direction in the next 3 years.

The encoder-decoder architecture of Bayesian Long Short-Term Memory (B-LSTM). \(X_{i}\) stands for the input at time-step i . \(h_{i}\) stands for the hidden state, which stores information from the recent time steps (short-term). \(C_{i}\) stands for the cell state, which stores all processed information from the past (long-term). The number of input time steps in the encoder is a variable tuned as a hyper-parameter, while the output in the decoder is a single time-step. The depth and number of layers are another set of hyper-parameters tuned during the model optimisation. The red arrows indicate a recurrent dropout maintained during the testing and prediction. The figure shows an example for an input with time lag=6 and a single layer. The final hidden state \(h_{0}\) produced by the encoder is passed to the Repeat Vector layer to convert it from 2 dimensional output to 3 dimensional input as expected by the decoder. The decoder processes the input and produces the final hidden state \(h_{1}\) . This hidden state is finally passed to a dense layer to produce the output. The table illustrates the concept of sliding window method used to forecast multiple time steps during the testing and prediction (i.e., using the output at a time-step as an input to forecast the next time-step). Using this concept, we can predict as many time steps as needed. In the table, an output vector of 6 time steps was predicted.

The B-LSTM validation results of predicting the number of attacks from April, 2019 to March, 2022. (U) indicates an univariate model while (M) indicates a multivariate model. ( a ) Botnet attack with M-SMAPE=0.03. ( b ) Brute force attack with M-SMAPE=0.13. ( c ) SQL injection attack with M-SMAPE=0.04 using the feature of NoM. ( d ) Targeted attack with M-SMAPE=0.06 using the feature of NoM. Y axis is normalised in the case of multivariate models to account for the different ranges of feature values.

A bird’s eye view of threat trend categories. The period of the trend plots is between July, 2011 and March, 2025, with the period between April, 2022 and March, 2025 forecasted using B-LSTM. ( a ) Among rapidly increasing threats, as observed in the forecast period, some threats are predicted to continue a sharp increase until 2025 while others will probably level off. ( b ) Threats under this category have overall been increasing while fluctuating over the past 11 years. Recently, some of the overall increasing threats slightly declined however many of those are likely to recover and level off by 2025. ( c ) Emerging threats that began to appear and grow sharply after the year 2016, and are expected to continue growing at this increasing rate, while others are likely to slow down or stabilise by 2025. ( d ) Decreasing threats that peaked in the earlier years and have slowly been declining since then. This decreasing group are likely to level off however probably will not disappear in the coming 3 years. The Y axis is normalised to account for the different ranges of values across different attacks. The 95% confidence interval is shown for each threat prediction.

The threat cycle (TTC). The attacks go through 5 stages, namely, launch, growth, maturity trough, and stability/decline. A standard Gartner hype cycle (GHC) is shown with a vanishing green colour for a comparison to TTC. Both GHC and TTC have a peak, however, TTC’s peak is much wider with a slightly less steep curve during the growth stage. Some attacks in TTC do not recover after the trough and slide into the slope of decline. TTC captures the state of each attack in 2022, where the colour of each attack indicates which slope it would follow (e.g., plateau or decreasing) based on the predictive results until 2025. Within the trough stage, the attacks (in blue dot) are likely to arrive at the slope of plateau by 2025. The attacks (in red dot) will probably be on the slope of decline by 2025. The attacks with unknown final destination are coloured in grey.

Data availability

As requested by the journal, the data used in this paper is available to editors and reviewers upon request. The data will be made publicly available and can be accessed at the following link after the paper is published. https://github.com/zaidalmahmoud/Cyber-threat-forecast .

Ghafur, S. et al. A retrospective impact analysis of the wannacry cyberattack on the NHS. NPJ Digit. Med. 2 , 1–7 (2019).

Article   Google Scholar  

Alrzini, J. R. S. & Pennington, D. A review of polymorphic malware detection techniques. Int. J. Adv. Res. Eng. Technol. 11 , 1238–1247 (2020).

Google Scholar  

Lazarevic, A., Ertoz, L., Kumar, V., Ozgur, A. & Srivastava, J. A comparative study of anomaly detection schemes in network intrusion detection. In: Proceedings of the 2003 SIAM International Conference on Data Mining , 25–36 (SIAM, 2003).

Kebir, O., Nouaouri, I., Rejeb, L. & Said, L. B. Atipreta: An analytical model for time-dependent prediction of terrorist attacks. Int. J. Appl. Math. Comput. Sci. 32 , 495–510 (2022).

MATH   Google Scholar  

Anticipating cyber attacks: There’s no abbottabad in cyber space. Infosecurity Magazine https://www.infosecurity-magazine.com/white-papers/anticipating-cyber-attacks (2015).

Jumper, J. et al. Highly accurate protein structure prediction with alphafold. Nature 596 , 583–589 (2021).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Baek, M. et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science 373 , 871–876 (2021).

Gibney, E. et al. Where is russia’s cyberwar? researchers decipher its strategy. Nature 603 , 775–776 (2022).

Article   ADS   CAS   PubMed   Google Scholar  

Passeri, P. Hackmageddon data set. Hackmageddon https://www.hackmageddon.com (2022).

Chen, C.-M. et al. A provably secure key transfer protocol for the fog-enabled social internet of vehicles based on a confidential computing environment. Veh. Commun. 39 , 100567 (2023).

Nagasree, Y. et al. Preserving privacy of classified authentic satellite lane imagery using proxy re-encryption and UAV technologies. Drones 7 , 53 (2023).

Kavitha, A. et al. Security in IoT mesh networks based on trust similarity. IEEE Access 10 , 121712–121724 (2022).

Salih, A., Zeebaree, S. T., Ameen, S., Alkhyyat, A. & Shukur, H. M A survey on the role of artificial intelligence, machine learning and deep learning for cybersecurity attack detection. In: 2021 7th International Engineering Conference “Research and Innovation amid Global Pandemic” (IEC) , 61–66 (IEEE, 2021).

Ren, K., Zeng, Y., Cao, Z. & Zhang, Y. Id-rdrl: A deep reinforcement learning-based feature selection intrusion detection model. Sci. Rep. 12 , 1–18 (2022).

Liu, X. & Liu, J. Malicious traffic detection combined deep neural network with hierarchical attention mechanism. Sci. Rep. 11 , 1–15 (2021).

Werner, G., Yang, S. & McConky, K. Time series forecasting of cyber attack intensity. In Proceedings of the 12th Annual Conference on Cyber and Information Security Research , 1–3 (2017).

Werner, G., Yang, S. & McConky, K. Leveraging intra-day temporal variations to predict daily cyberattack activity. In 2018 IEEE International Conference on Intelligence and Security Informatics (ISI) , 58–63 (IEEE, 2018).

Okutan, A., Yang, S. J., McConky, K. & Werner, G. Capture: cyberattack forecasting using non-stationary features with time lags. In 2019 IEEE Conference on Communications and Network Security (CNS) , 205–213 (IEEE, 2019).

Munkhdorj, B. & Yuji, S. Cyber attack prediction using social data analysis. J. High Speed Netw. 23 , 109–135 (2017).

Goyal, P. et al. Discovering signals from web sources to predict cyber attacks. arXiv preprint arXiv:1806.03342 (2018).

Qin, X. & Lee, W. Attack plan recognition and prediction using causal networks. In 20th Annual Computer Security Applications Conference , 370–379 (IEEE, 2004).

Husák, M. & Kašpar, J. Aida framework: real-time correlation and prediction of intrusion detection alerts. In: Proceedings of the 14th international conference on availability, reliability and security , 1–8 (2019).

Liu, Y. et al. Cloudy with a chance of breach: Forecasting cyber security incidents. In: 24th USENIX Security Symposium (USENIX Security 15) , 1009–1024 (2015).

Malik, J. et al. Hybrid deep learning: An efficient reconnaissance and surveillance detection mechanism in sdn. IEEE Access 8 , 134695–134706 (2020).

Bilge, L., Han, Y. & Dell’Amico, M. Riskteller: Predicting the risk of cyber incidents. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security , 1299–1311 (2017).

Husák, M., Bartoš, V., Sokol, P. & Gajdoš, A. Predictive methods in cyber defense: Current experience and research challenges. Futur. Gener. Comput. Syst. 115 , 517–530 (2021).

Stephens, G. Cybercrime in the year 2025. Futurist 42 , 32 (2008).

Adamov, A. & Carlsson, A. The state of ransomware. Trends and mitigation techniques. In EWDTS , 1–8 (2017).

Shoufan, A. & Damiani, E. On inter-rater reliability of information security experts. J. Inf. Secur. Appl. 37 , 101–111 (2017).

Cha, Y.-O. & Hao, Y. The dawn of metamaterial engineering predicted via hyperdimensional keyword pool and memory learning. Adv. Opt. Mater. 10 , 2102444 (2022).

Article   CAS   Google Scholar  

Elsevier research products apis. Elsevier Developer Portal https://dev.elsevier.com (2022).

Twitter api v2. Developer Platform https://developer.twitter.com/en/docs/twitter-api (2022).

holidays 0.15. PyPI. The Python Package Index https://pypi.org/project/holidays/ (2022).

Visser, M., van Eck, N. J. & Waltman, L. Large-scale comparison of bibliographic data sources: Scopus, web of science, dimensions, crossref, and microsoft academic. Quant. Sci. Stud. 2 , 20–41 (2021).

2021 trends show increased globalized threat of ransomware. Cybersecurity and Infrastructure Security Agency https://www.cisa.gov/uscert/ncas/alerts/aa22-040a (2022).

Lai, K. K., Yu, L., Wang, S. & Huang, W. Hybridizing exponential smoothing and neural network for financial time series predication. In International Conference on Computational Science , 493–500 (Springer, 2006).

Huang, B., Ding, Q., Sun, G. & Li, H. Stock prediction based on Bayesian-lstm. In Proceedings of the 2018 10th International Conference on Machine Learning and Computing , 128–133 (2018).

Mae, Y., Kumagai, W. & Kanamori, T. Uncertainty propagation for dropout-based Bayesian neural networks. Neural Netw. 144 , 394–406 (2021).

Article   PubMed   Google Scholar  

Scopus preview. Scopus https://www.scopus.com/home.uri (2022).

Jia, P., Chen, H., Zhang, L. & Han, D. Attention-lstm based prediction model for aircraft 4-d trajectory. Sci. Rep. 12 (2022).

Chandra, R., Goyal, S. & Gupta, R. Evaluation of deep learning models for multi-step ahead time series prediction. IEEE Access 9 , 83105–83123 (2021).

Gers, F. A., Schmidhuber, J. & Cummins, F. Learning to forget: Continual prediction with lstm. Neural Comput. 12 , 2451–2471 (2000).

Article   CAS   PubMed   Google Scholar  

Sagheer, A. & Kotb, M. Unsupervised pre-training of a deep lstm-based stacked autoencoder for multivariate time series forecasting problems. Sci. Rep. 9 , 1–16 (2019).

Article   ADS   Google Scholar  

Swiler, L. P., Paez, T. L. & Mayes, R. L. Epistemic uncertainty quantification tutorial. In Proceedings of the 27th International Modal Analysis Conference (2009).

Gal, Y. & Ghahramani, Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. arXiv preprint arXiv:1506.02142v6 (2016).

Chollet, F. Deep Learning with Python , 2 edn. (Manning Publications, 2017).

Xu, J., Li, Z., Du, B., Zhang, M. & Liu, J. Reluplex made more practical: Leaky relu. In 2020 IEEE Symposium on Computers and Communications (ISCC) , 1–7 (IEEE, 2020).

Gal, Y., Hron, J. & Kendall, A. Concrete dropout. Adv. Neural Inf. Process. Syst. 30 (2017).

Shcherbakov, M. V. et al. A survey of forecast error measures. World Appl. Sci. J. 24 , 171–176 (2013).

Bergstra, J. & Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13 (2012).

Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Commun. ACM 60 , 84–90 (2017).

Shifferaw, Y. & Lemma, S. Limitations of proof of stake algorithm in blockchain: A review. Zede J. 39 , 81–95 (2021).

Dedehayir, O. & Steinert, M. The hype cycle model: A review and future directions. Technol. Forecast. Soc. Chang. 108 , 28–41 (2016).

Abri, F., Siami-Namini, S., Khanghah, M. A., Soltani, F. M. & Namin, A. S. Can machine/deep learning classifiers detect zero-day malware with high accuracy?. In 2019 IEEE International Conference on Big Data (Big Data) , 3252–3259 (IEEE, 2019).

Download references

Acknowledgements

The authors are grateful to the DASA’s machine learning team for their invaluable discussions and feedback, and special thanks to the EBTIC, British Telecom’s (BT) cyber security team for their constructive criticism on this work.

Author information

Authors and affiliations.

Department of Computer Science and Information Systems, University of London, Birkbeck College, London, United Kingdom

Zaid Almahmoud & Paul D. Yoo

Huawei Technologies Canada, Ottawa, Canada

Omar Alhussein

Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Canada

Ilyas Farhat

Department of Computer Science, Università degli Studi di Milano, Milan, Italy

Ernesto Damiani

Center for Cyber-Physical Systems (C2PS), Khalifa University, Abu Dhabi, United Arab Emirates

You can also search for this author in PubMed   Google Scholar

Contributions

Z.A., P.D.Y, I.F., and E.D. were in charge of the framework design and theoretical analysis of the trend analysis and TTC. Z.A., O.A., and P.D.Y. contributed to the B-LSTM design and experiments. O.A. proposed the concepts of B-LSTM. All of the authors contributed to the discussion of the framework design and experiments, and the writing of this paper. P.D.Y. proposed the big data approach and supervised the whole project.

Corresponding author

Correspondence to Paul D. Yoo .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Almahmoud, Z., Yoo, P.D., Alhussein, O. et al. A holistic and proactive approach to forecasting cyber threats. Sci Rep 13 , 8049 (2023). https://doi.org/10.1038/s41598-023-35198-1

Download citation

Received : 21 December 2022

Accepted : 14 May 2023

Published : 17 May 2023

DOI : https://doi.org/10.1038/s41598-023-35198-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Integrating ai-driven threat intelligence and forecasting in the cyber security exercise content generation lifecycle.

• Alexandros Zacharis
• Vasilios Katos
• Constantinos Patsakis

International Journal of Information Security (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Computer Science and Engineering
• Computer Security and Reliability
• Cyber Security

Cyber Security Threats and Vulnerabilities: A Systematic Mapping Study

• January 2020
• Arabian Journal for Science and Engineering 45(2)

• Universiy of Roehampton

• King Fahd University of Petroleum and Minerals

• Taylor's University

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Dhuha Sabri Ghazi
• Hamood Shehab Hamid
• Mhammed Joudah Zaiter
• Ahmed Sabri Ghazi Behadili

• Graduate Researcher
• Zaman Farhana
• Shahab Uddin

• Hesham A. Hefny
• Nagy R. Darwish

• Thiha Naing
• Noor Ul Amin
• Hira Arshad

• Abdullah Akbar
• Shahzad Latif
• Juvi Bharti
• Sarpreet Singh
• Ihab Abdelwahab
• Vanshita Jain

• David Budgen
• O. Pearl Brereton

• COMPUT NETW

• Daniel T. Kuehl

• Ahmed H. Anjariny
• Shakeel A. Habib

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Cyber risk and cybersecurity: a systematic review of data availability

Frank cremer.

1 University of Limerick, Limerick, Ireland

Barry Sheehan

Michael fortmann.

2 TH Köln University of Applied Sciences, Cologne, Germany

Arash N. Kia

Martin mullins, finbarr murphy, stefan materne, associated data.

Cybercrime is estimated to have cost the global economy just under USD 1 trillion in 2020, indicating an increase of more than 50% since 2018. With the average cyber insurance claim rising from USD 145,000 in 2019 to USD 359,000 in 2020, there is a growing necessity for better cyber information sources, standardised databases, mandatory reporting and public awareness. This research analyses the extant academic and industry literature on cybersecurity and cyber risk management with a particular focus on data availability. From a preliminary search resulting in 5219 cyber peer-reviewed studies, the application of the systematic methodology resulted in 79 unique datasets. We posit that the lack of available data on cyber risk poses a serious problem for stakeholders seeking to tackle this issue. In particular, we identify a lacuna in open databases that undermine collective endeavours to better manage this set of risks. The resulting data evaluation and categorisation will support cybersecurity researchers and the insurance industry in their efforts to comprehend, metricise and manage cyber risks.

Supplementary Information

The online version contains supplementary material available at 10.1057/s41288-022-00266-6.

Introduction

Globalisation, digitalisation and smart technologies have escalated the propensity and severity of cybercrime. Whilst it is an emerging field of research and industry, the importance of robust cybersecurity defence systems has been highlighted at the corporate, national and supranational levels. The impacts of inadequate cybersecurity are estimated to have cost the global economy USD 945 billion in 2020 (Maleks Smith et al. 2020 ). Cyber vulnerabilities pose significant corporate risks, including business interruption, breach of privacy and financial losses (Sheehan et al. 2019 ). Despite the increasing relevance for the international economy, the availability of data on cyber risks remains limited. The reasons for this are many. Firstly, it is an emerging and evolving risk; therefore, historical data sources are limited (Biener et al. 2015 ). It could also be due to the fact that, in general, institutions that have been hacked do not publish the incidents (Eling and Schnell 2016 ). The lack of data poses challenges for many areas, such as research, risk management and cybersecurity (Falco et al. 2019 ). The importance of this topic is demonstrated by the announcement of the European Council in April 2021 that a centre of excellence for cybersecurity will be established to pool investments in research, technology and industrial development. The goal of this centre is to increase the security of the internet and other critical network and information systems (European Council 2021 ).

This research takes a risk management perspective, focusing on cyber risk and considering the role of cybersecurity and cyber insurance in risk mitigation and risk transfer. The study reviews the existing literature and open data sources related to cybersecurity and cyber risk. This is the first systematic review of data availability in the general context of cyber risk and cybersecurity. By identifying and critically analysing the available datasets, this paper supports the research community by aggregating, summarising and categorising all available open datasets. In addition, further information on datasets is attached to provide deeper insights and support stakeholders engaged in cyber risk control and cybersecurity. Finally, this research paper highlights the need for open access to cyber-specific data, without price or permission barriers.

The identified open data can support cyber insurers in their efforts on sustainable product development. To date, traditional risk assessment methods have been untenable for insurance companies due to the absence of historical claims data (Sheehan et al. 2021 ). These high levels of uncertainty mean that cyber insurers are more inclined to overprice cyber risk cover (Kshetri 2018 ). Combining external data with insurance portfolio data therefore seems to be essential to improve the evaluation of the risk and thus lead to risk-adjusted pricing (Bessy-Roland et al. 2021 ). This argument is also supported by the fact that some re/insurers reported that they are working to improve their cyber pricing models (e.g. by creating or purchasing databases from external providers) (EIOPA 2018 ). Figure  1 provides an overview of pricing tools and factors considered in the estimation of cyber insurance based on the findings of EIOPA ( 2018 ) and the research of Romanosky et al. ( 2019 ). The term cyber risk refers to all cyber risks and their potential impact.

An overview of the current cyber insurance informational and methodological landscape, adapted from EIOPA ( 2018 ) and Romanosky et al. ( 2019 )

Besides the advantage of risk-adjusted pricing, the availability of open datasets helps companies benchmark their internal cyber posture and cybersecurity measures. The research can also help to improve risk awareness and corporate behaviour. Many companies still underestimate their cyber risk (Leong and Chen 2020 ). For policymakers, this research offers starting points for a comprehensive recording of cyber risks. Although in many countries, companies are obliged to report data breaches to the respective supervisory authority, this information is usually not accessible to the research community. Furthermore, the economic impact of these breaches is usually unclear.

As well as the cyber risk management community, this research also supports cybersecurity stakeholders. Researchers are provided with an up-to-date, peer-reviewed literature of available datasets showing where these datasets have been used. For example, this includes datasets that have been used to evaluate the effectiveness of countermeasures in simulated cyberattacks or to test intrusion detection systems. This reduces a time-consuming search for suitable datasets and ensures a comprehensive review of those available. Through the dataset descriptions, researchers and industry stakeholders can compare and select the most suitable datasets for their purposes. In addition, it is possible to combine the datasets from one source in the context of cybersecurity or cyber risk. This supports efficient and timely progress in cyber risk research and is beneficial given the dynamic nature of cyber risks.

Cyber risks are defined as “operational risks to information and technology assets that have consequences affecting the confidentiality, availability, and/or integrity of information or information systems” (Cebula et al. 2014 ). Prominent cyber risk events include data breaches and cyberattacks (Agrafiotis et al. 2018 ). The increasing exposure and potential impact of cyber risk have been highlighted in recent industry reports (e.g. Allianz 2021 ; World Economic Forum 2020 ). Cyberattacks on critical infrastructures are ranked 5th in the World Economic Forum's Global Risk Report. Ransomware, malware and distributed denial-of-service (DDoS) are examples of the evolving modes of a cyberattack. One example is the ransomware attack on the Colonial Pipeline, which shut down the 5500 mile pipeline system that delivers 2.5 million barrels of fuel per day and critical liquid fuel infrastructure from oil refineries to states along the U.S. East Coast (Brower and McCormick 2021 ). These and other cyber incidents have led the U.S. to strengthen its cybersecurity and introduce, among other things, a public body to analyse major cyber incidents and make recommendations to prevent a recurrence (Murphey 2021a ). Another example of the scope of cyberattacks is the ransomware NotPetya in 2017. The damage amounted to USD 10 billion, as the ransomware exploited a vulnerability in the windows system, allowing it to spread independently worldwide in the network (GAO 2021 ). In the same year, the ransomware WannaCry was launched by cybercriminals. The cyberattack on Windows software took user data hostage in exchange for Bitcoin cryptocurrency (Smart 2018 ). The victims included the National Health Service in Great Britain. As a result, ambulances were redirected to other hospitals because of information technology (IT) systems failing, leaving people in need of urgent assistance waiting. It has been estimated that 19,000 cancelled treatment appointments resulted from losses of GBP 92 million (Field 2018 ). Throughout the COVID-19 pandemic, ransomware attacks increased significantly, as working from home arrangements increased vulnerability (Murphey 2021b ).

Besides cyberattacks, data breaches can also cause high costs. Under the General Data Protection Regulation (GDPR), companies are obliged to protect personal data and safeguard the data protection rights of all individuals in the EU area. The GDPR allows data protection authorities in each country to impose sanctions and fines on organisations they find in breach. “For data breaches, the maximum fine can be €20 million or 4% of global turnover, whichever is higher” (GDPR.EU 2021 ). Data breaches often involve a large amount of sensitive data that has been accessed, unauthorised, by external parties, and are therefore considered important for information security due to their far-reaching impact (Goode et al. 2017 ). A data breach is defined as a “security incident in which sensitive, protected, or confidential data are copied, transmitted, viewed, stolen, or used by an unauthorized individual” (Freeha et al. 2021 ). Depending on the amount of data, the extent of the damage caused by a data breach can be significant, with the average cost being USD 392 million 1 (IBM Security 2020 ).

This research paper reviews the existing literature and open data sources related to cybersecurity and cyber risk, focusing on the datasets used to improve academic understanding and advance the current state-of-the-art in cybersecurity. Furthermore, important information about the available datasets is presented (e.g. use cases), and a plea is made for open data and the standardisation of cyber risk data for academic comparability and replication. The remainder of the paper is structured as follows. The next section describes the related work regarding cybersecurity and cyber risks. The third section outlines the review method used in this work and the process. The fourth section details the results of the identified literature. Further discussion is presented in the penultimate section and the final section concludes.

Related work

Due to the significance of cyber risks, several literature reviews have been conducted in this field. Eling ( 2020 ) reviewed the existing academic literature on the topic of cyber risk and cyber insurance from an economic perspective. A total of 217 papers with the term ‘cyber risk’ were identified and classified in different categories. As a result, open research questions are identified, showing that research on cyber risks is still in its infancy because of their dynamic and emerging nature. Furthermore, the author highlights that particular focus should be placed on the exchange of information between public and private actors. An improved information flow could help to measure the risk more accurately and thus make cyber risks more insurable and help risk managers to determine the right level of cyber risk for their company. In the context of cyber insurance data, Romanosky et al. ( 2019 ) analysed the underwriting process for cyber insurance and revealed how cyber insurers understand and assess cyber risks. For this research, they examined 235 American cyber insurance policies that were publicly available and looked at three components (coverage, application questionnaires and pricing). The authors state in their findings that many of the insurers used very simple, flat-rate pricing (based on a single calculation of expected loss), while others used more parameters such as the asset value of the company (or company revenue) or standard insurance metrics (e.g. deductible, limits), and the industry in the calculation. This is in keeping with Eling ( 2020 ), who states that an increased amount of data could help to make cyber risk more accurately measured and thus more insurable. Similar research on cyber insurance and data was conducted by Nurse et al. ( 2020 ). The authors examined cyber insurance practitioners' perceptions and the challenges they face in collecting and using data. In addition, gaps were identified during the research where further data is needed. The authors concluded that cyber insurance is still in its infancy, and there are still several unanswered questions (for example, cyber valuation, risk calculation and recovery). They also pointed out that a better understanding of data collection and use in cyber insurance would be invaluable for future research and practice. Bessy-Roland et al. ( 2021 ) come to a similar conclusion. They proposed a multivariate Hawkes framework to model and predict the frequency of cyberattacks. They used a public dataset with characteristics of data breaches affecting the U.S. industry. In the conclusion, the authors make the argument that an insurer has a better knowledge of cyber losses, but that it is based on a small dataset and therefore combination with external data sources seems essential to improve the assessment of cyber risks.

Several systematic reviews have been published in the area of cybersecurity (Kruse et al. 2017 ; Lee et al. 2020 ; Loukas et al. 2013 ; Ulven and Wangen 2021 ). In these papers, the authors concentrated on a specific area or sector in the context of cybersecurity. This paper adds to this extant literature by focusing on data availability and its importance to risk management and insurance stakeholders. With a priority on healthcare and cybersecurity, Kruse et al. ( 2017 ) conducted a systematic literature review. The authors identified 472 articles with the keywords ‘cybersecurity and healthcare’ or ‘ransomware’ in the databases Cumulative Index of Nursing and Allied Health Literature, PubMed and Proquest. Articles were eligible for this review if they satisfied three criteria: (1) they were published between 2006 and 2016, (2) the full-text version of the article was available, and (3) the publication is a peer-reviewed or scholarly journal. The authors found that technological development and federal policies (in the U.S.) are the main factors exposing the health sector to cyber risks. Loukas et al. ( 2013 ) conducted a review with a focus on cyber risks and cybersecurity in emergency management. The authors provided an overview of cyber risks in communication, sensor, information management and vehicle technologies used in emergency management and showed areas for which there is still no solution in the literature. Similarly, Ulven and Wangen ( 2021 ) reviewed the literature on cybersecurity risks in higher education institutions. For the literature review, the authors used the keywords ‘cyber’, ‘information threats’ or ‘vulnerability’ in connection with the terms ‘higher education, ‘university’ or ‘academia’. A similar literature review with a focus on Internet of Things (IoT) cybersecurity was conducted by Lee et al. ( 2020 ). The review revealed that qualitative approaches focus on high-level frameworks, and quantitative approaches to cybersecurity risk management focus on risk assessment and quantification of cyberattacks and impacts. In addition, the findings presented a four-step IoT cyber risk management framework that identifies, quantifies and prioritises cyber risks.

Datasets are an essential part of cybersecurity research, underlined by the following works. Ilhan Firat et al. ( 2021 ) examined various cybersecurity datasets in detail. The study was motivated by the fact that with the proliferation of the internet and smart technologies, the mode of cyberattacks is also evolving. However, in order to prevent such attacks, they must first be detected; the dissemination and further development of cybersecurity datasets is therefore critical. In their work, the authors observed studies of datasets used in intrusion detection systems. Khraisat et al. ( 2019 ) also identified a need for new datasets in the context of cybersecurity. The researchers presented a taxonomy of current intrusion detection systems, a comprehensive review of notable recent work, and an overview of the datasets commonly used for assessment purposes. In their conclusion, the authors noted that new datasets are needed because most machine-learning techniques are trained and evaluated on the knowledge of old datasets. These datasets do not contain new and comprehensive information and are partly derived from datasets from 1999. The authors noted that the core of this issue is the availability of new public datasets as well as their quality. The availability of data, how it is used, created and shared was also investigated by Zheng et al. ( 2018 ). The researchers analysed 965 cybersecurity research papers published between 2012 and 2016. They created a taxonomy of the types of data that are created and shared and then analysed the data collected via datasets. The researchers concluded that while datasets are recognised as valuable for cybersecurity research, the proportion of publicly available datasets is limited.

The main contributions of this review and what differentiates it from previous studies can be summarised as follows. First, as far as we can tell, it is the first work to summarise all available datasets on cyber risk and cybersecurity in the context of a systematic review and present them to the scientific community and cyber insurance and cybersecurity stakeholders. Second, we investigated, analysed, and made available the datasets to support efficient and timely progress in cyber risk research. And third, we enable comparability of datasets so that the appropriate dataset can be selected depending on the research area.

Methodology

Process and eligibility criteria.

The structure of this systematic review is inspired by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework (Page et al. 2021 ), and the search was conducted from 3 to 10 May 2021. Due to the continuous development of cyber risks and their countermeasures, only articles published in the last 10 years were considered. In addition, only articles published in peer-reviewed journals written in English were included. As a final criterion, only articles that make use of one or more cybersecurity or cyber risk datasets met the inclusion criteria. Specifically, these studies presented new or existing datasets, used them for methods, or used them to verify new results, as well as analysed them in an economic context and pointed out their effects. The criterion was fulfilled if it was clearly stated in the abstract that one or more datasets were used. A detailed explanation of this selection criterion can be found in the ‘Study selection’ section.

Information sources

In order to cover a complete spectrum of literature, various databases were queried to collect relevant literature on the topic of cybersecurity and cyber risks. Due to the spread of related articles across multiple databases, the literature search was limited to the following four databases for simplicity: IEEE Xplore, Scopus, SpringerLink and Web of Science. This is similar to other literature reviews addressing cyber risks or cybersecurity, including Sardi et al. ( 2021 ), Franke and Brynielsson ( 2014 ), Lagerström (2019), Eling and Schnell ( 2016 ) and Eling ( 2020 ). In this paper, all databases used in the aforementioned works were considered. However, only two studies also used all the databases listed. The IEEE Xplore database contains electrical engineering, computer science, and electronics work from over 200 journals and three million conference papers (IEEE 2021 ). Scopus includes 23,400 peer-reviewed journals from more than 5000 international publishers in the areas of science, engineering, medicine, social sciences and humanities (Scopus 2021 ). SpringerLink contains 3742 journals and indexes over 10 million scientific documents (SpringerLink 2021 ). Finally, Web of Science indexes over 9200 journals in different scientific disciplines (Science 2021 ).

A search string was created and applied to all databases. To make the search efficient and reproducible, the following search string with Boolean operator was used in all databases: cybersecurity OR cyber risk AND dataset OR database. To ensure uniformity of the search across all databases, some adjustments had to be made for the respective search engines. In Scopus, for example, the Advanced Search was used, and the field code ‘Title-ABS-KEY’ was integrated into the search string. For IEEE Xplore, the search was carried out with the Search String in the Command Search and ‘All Metadata’. In the Web of Science database, the Advanced Search was used. The special feature of this search was that it had to be carried out in individual steps. The first search was carried out with the terms cybersecurity OR cyber risk with the field tag Topic (T.S. =) and the second search with dataset OR database. Subsequently, these searches were combined, which then delivered the searched articles for review. For SpringerLink, the search string was used in the Advanced Search under the category ‘Find the resources with all of the words’. After conducting this search string, 5219 studies could be found. According to the eligibility criteria (period, language and only scientific journals), 1581 studies were identified in the databases:

• Scopus: 135
• Springer Link: 548
• Web of Science: 534

An overview of the process is given in Fig.  2 . Combined with the results from the four databases, 854 articles without duplicates were identified.

Literature search process and categorisation of the studies

Study selection

In the final step of the selection process, the articles were screened for relevance. Due to a large number of results, the abstracts were analysed in the first step of the process. The aim was to determine whether the article was relevant for the systematic review. An article fulfilled the criterion if it was recognisable in the abstract that it had made a contribution to datasets or databases with regard to cyber risks or cybersecurity. Specifically, the criterion was considered to be met if the abstract used datasets that address the causes or impacts of cyber risks, and measures in the area of cybersecurity. In this process, the number of articles was reduced to 288. The articles were then read in their entirety, and an expert panel of six people decided whether they should be used. This led to a final number of 255 articles. The years in which the articles were published and the exact number can be seen in Fig.  3 .

Distribution of studies

Data collection process and synthesis of the results

For the data collection process, various data were extracted from the studies, including the names of the respective creators, the name of the dataset or database and the corresponding reference. It was also determined where the data came from. In the context of accessibility, it was determined whether access is free, controlled, available for purchase or not available. It was also determined when the datasets were created and the time period referenced. The application type and domain characteristics of the datasets were identified.

This section analyses the results of the systematic literature review. The previously identified studies are divided into three categories: datasets on the causes of cyber risks, datasets on the effects of cyber risks and datasets on cybersecurity. The classification is based on the intended use of the studies. This system of classification makes it easier for stakeholders to find the appropriate datasets. The categories are evaluated individually. Although complete information is available for a large proportion of datasets, this is not true for all of them. Accordingly, the abbreviation N/A has been inserted in the respective characters to indicate that this information could not be determined by the time of submission. The term ‘use cases in the literature’ in the following and supplementary tables refers to the application areas in which the corresponding datasets were used in the literature. The areas listed there refer to the topic area on which the researchers conducted their research. Since some datasets were used interdisciplinarily, the listed use cases in the literature are correspondingly longer. Before discussing each category in the next sections, Fig.  4 provides an overview of the number of datasets found and their year of creation. Figure  5 then shows the relationship between studies and datasets in the period under consideration. Figure  6 shows the distribution of studies, their use of datasets and their creation date. The number of datasets used is higher than the number of studies because the studies often used several datasets (Table ​ (Table1). 1 ).

Distribution of dataset results

Correlation between the studies and the datasets

Distribution of studies and their use of datasets

Percentage contribution of datasets for each place of origin

Most of the datasets are generated in the U.S. (up to 58.2%). Canada and Australia rank next, with 11.3% and 5% of all the reviewed datasets, respectively.

Additionally, to create value for the datasets for the cyber insurance industry, an assessment of the applicability of each dataset has been provided for cyber insurers. This ‘Use Case Assessment’ includes the use of the data in the context of different analyses, calculation of cyber insurance premiums, and use of the information for the design of cyber insurance contracts or for additional customer services. To reasonably account for the transition of direct hyperlinks in the future, references were directed to the main websites for longevity (nearest resource point). In addition, the links to the main pages contain further information on the datasets and different versions related to the operating systems. The references were chosen in such a way that practitioners get the best overview of the respective datasets.

Case datasets

This section presents selected articles that use the datasets to analyse the causes of cyber risks. The datasets help identify emerging trends and allow pattern discovery in cyber risks. This information gives cybersecurity experts and cyber insurers the data to make better predictions and take appropriate action. For example, if certain vulnerabilities are not adequately protected, cyber insurers will demand a risk surcharge leading to an improvement in the risk-adjusted premium. Due to the capricious nature of cyber risks, existing data must be supplemented with new data sources (for example, new events, new methods or security vulnerabilities) to determine prevailing cyber exposure. The datasets of cyber risk causes could be combined with existing portfolio data from cyber insurers and integrated into existing pricing tools and factors to improve the valuation of cyber risks.

A portion of these datasets consists of several taxonomies and classifications of cyber risks. Aassal et al. ( 2020 ) propose a new taxonomy of phishing characteristics based on the interpretation and purpose of each characteristic. In comparison, Hindy et al. ( 2020 ) presented a taxonomy of network threats and the impact of current datasets on intrusion detection systems. A similar taxonomy was suggested by Kiwia et al. ( 2018 ). The authors presented a cyber kill chain-based taxonomy of banking Trojans features. The taxonomy built on a real-world dataset of 127 banking Trojans collected from December 2014 to January 2016 by a major U.K.-based financial organisation.

In the context of classification, Aamir et al. ( 2021 ) showed the benefits of machine learning for classifying port scans and DDoS attacks in a mixture of normal and attack traffic. Guo et al. ( 2020 ) presented a new method to improve malware classification based on entropy sequence features. The evaluation of this new method was conducted on different malware datasets.

To reconstruct attack scenarios and draw conclusions based on the evidence in the alert stream, Barzegar and Shajari ( 2018 ) use the DARPA2000 and MACCDC 2012 dataset for their research. Giudici and Raffinetti ( 2020 ) proposed a rank-based statistical model aimed at predicting the severity levels of cyber risk. The model used cyber risk data from the University of Milan. In contrast to the previous datasets, Skrjanc et al. ( 2018 ) used the older dataset KDD99 to monitor large-scale cyberattacks using a cauchy clustering method.

Amin et al. ( 2021 ) used a cyberattack dataset from the Canadian Institute for Cybersecurity to identify spatial clusters of countries with high rates of cyberattacks. In the context of cybercrime, Junger et al. ( 2020 ) examined crime scripts, key characteristics of the target company and the relationship between criminal effort and financial benefit. For their study, the authors analysed 300 cases of fraudulent activities against Dutch companies. With a similar focus on cybercrime, Mireles et al. ( 2019 ) proposed a metric framework to measure the effectiveness of the dynamic evolution of cyberattacks and defensive measures. To validate its usefulness, they used the DEFCON dataset.

Due to the rapidly changing nature of cyber risks, it is often impossible to obtain all information on them. Kim and Kim ( 2019 ) proposed an automated dataset generation system called CTIMiner that collects threat data from publicly available security reports and malware repositories. They released a dataset to the public containing about 640,000 records from 612 security reports published between January 2008 and 2019. A similar approach is proposed by Kim et al. ( 2020 ), using a named entity recognition system to extract core information from cyber threat reports automatically. They created a 498,000-tag dataset during their research (Ulven and Wangen 2021 ).

Within the framework of vulnerabilities and cybersecurity issues, Ulven and Wangen ( 2021 ) proposed an overview of mission-critical assets and everyday threat events, suggested a generic threat model, and summarised common cybersecurity vulnerabilities. With a focus on hospitality, Chen and Fiscus ( 2018 ) proposed several issues related to cybersecurity in this sector. They analysed 76 security incidents from the Privacy Rights Clearinghouse database. Supplementary Table 1 lists all findings that belong to the cyber causes dataset.

Impact datasets

This section outlines selected findings of the cyber impact dataset. For cyber insurers, these datasets can form an important basis for information, as they can be used to calculate cyber insurance premiums, evaluate specific cyber risks, formulate inclusions and exclusions in cyber wordings, and re-evaluate as well as supplement the data collected so far on cyber risks. For example, information on financial losses can help to better assess the loss potential of cyber risks. Furthermore, the datasets can provide insight into the frequency of occurrence of these cyber risks. The new datasets can be used to close any data gaps that were previously based on very approximate estimates or to find new results.

Eight studies addressed the costs of data breaches. For instance, Eling and Jung ( 2018 ) reviewed 3327 data breach events from 2005 to 2016 and identified an asymmetric dependence of monthly losses by breach type and industry. The authors used datasets from the Privacy Rights Clearinghouse for analysis. The Privacy Rights Clearinghouse datasets and the Breach level index database were also used by De Giovanni et al. ( 2020 ) to describe relationships between data breaches and bitcoin-related variables using the cointegration methodology. The data were obtained from the Department of Health and Human Services of healthcare facilities reporting data breaches and a national database of technical and organisational infrastructure information. Also in the context of data breaches, Algarni et al. ( 2021 ) developed a comprehensive, formal model that estimates the two components of security risks: breach cost and the likelihood of a data breach within 12 months. For their survey, the authors used two industrial reports from the Ponemon institute and VERIZON. To illustrate the scope of data breaches, Neto et al. ( 2021 ) identified 430 major data breach incidents among more than 10,000 incidents. The database created is available and covers the period 2018 to 2019.

With a direct focus on insurance, Biener et al. ( 2015 ) analysed 994 cyber loss cases from an operational risk database and investigated the insurability of cyber risks based on predefined criteria. For their study, they used data from the company SAS OpRisk Global Data. Similarly, Eling and Wirfs ( 2019 ) looked at a wide range of cyber risk events and actual cost data using the same database. They identified cyber losses and analysed them using methods from statistics and actuarial science. Using a similar reference, Farkas et al. ( 2021 ) proposed a method for analysing cyber claims based on regression trees to identify criteria for classifying and evaluating claims. Similar to Chen and Fiscus ( 2018 ), the dataset used was the Privacy Rights Clearinghouse database. Within the framework of reinsurance, Moro ( 2020 ) analysed cyber index-based information technology activity to see if index-parametric reinsurance coverage could suggest its cedant using data from a Symantec dataset.

Paté-Cornell et al. ( 2018 ) presented a general probabilistic risk analysis framework for cybersecurity in an organisation to be specified. The results are distributions of losses to cyberattacks, with and without considered countermeasures in support of risk management decisions based both on past data and anticipated incidents. The data used were from The Common Vulnerability and Exposures database and via confidential access to a database of cyberattacks on a large, U.S.-based organisation. A different conceptual framework for cyber risk classification and assessment was proposed by Sheehan et al. ( 2021 ). This framework showed the importance of proactive and reactive barriers in reducing companies’ exposure to cyber risk and quantifying the risk. Another approach to cyber risk assessment and mitigation was proposed by Mukhopadhyay et al. ( 2019 ). They estimated the probability of an attack using generalised linear models, predicted the security technology required to reduce the probability of cyberattacks, and used gamma and exponential distributions to best approximate the average loss data for each malicious attack. They also calculated the expected loss due to cyberattacks, calculated the net premium that would need to be charged by a cyber insurer, and suggested cyber insurance as a strategy to minimise losses. They used the CSI-FBI survey (1997–2010) to conduct their research.

In order to highlight the lack of data on cyber risks, Eling ( 2020 ) conducted a literature review in the areas of cyber risk and cyber insurance. Available information on the frequency, severity, and dependency structure of cyber risks was filtered out. In addition, open questions for future cyber risk research were set up. Another example of data collection on the impact of cyberattacks is provided by Sornette et al. ( 2013 ), who use a database of newspaper articles, press reports and other media to provide a predictive method to identify triggering events and potential accident scenarios and estimate their severity and frequency. A similar approach to data collection was used by Arcuri et al. ( 2020 ) to gather an original sample of global cyberattacks from newspaper reports sourced from the LexisNexis database. This collection is also used and applied to the fields of dynamic communication and cyber risk perception by Fang et al. ( 2021 ). To create a dataset of cyber incidents and disputes, Valeriano and Maness ( 2014 ) collected information on cyber interactions between rival states.

To assess trends and the scale of economic cybercrime, Levi ( 2017 ) examined datasets from different countries and their impact on crime policy. Pooser et al. ( 2018 ) investigated the trend in cyber risk identification from 2006 to 2015 and company characteristics related to cyber risk perception. The authors used a dataset of various reports from cyber insurers for their study. Walker-Roberts et al. ( 2020 ) investigated the spectrum of risk of a cybersecurity incident taking place in the cyber-physical-enabled world using the VERIS Community Database. The datasets of impacts identified are presented below. Due to overlap, some may also appear in the causes dataset (Supplementary Table 2).

Cybersecurity datasets

General intrusion detection.

General intrusion detection systems account for the largest share of countermeasure datasets. For companies or researchers focused on cybersecurity, the datasets can be used to test their own countermeasures or obtain information about potential vulnerabilities. For example, Al-Omari et al. ( 2021 ) proposed an intelligent intrusion detection model for predicting and detecting attacks in cyberspace, which was applied to dataset UNSW-NB 15. A similar approach was taken by Choras and Kozik ( 2015 ), who used machine learning to detect cyberattacks on web applications. To evaluate their method, they used the HTTP dataset CSIC 2010. For the identification of unknown attacks on web servers, Kamarudin et al. ( 2017 ) proposed an anomaly-based intrusion detection system using an ensemble classification approach. Ganeshan and Rodrigues ( 2020 ) showed an intrusion detection system approach, which clusters the database into several groups and detects the presence of intrusion in the clusters. In comparison, AlKadi et al. ( 2019 ) used a localisation-based model to discover abnormal patterns in network traffic. Hybrid models have been recommended by Bhattacharya et al. ( 2020 ) and Agrawal et al. ( 2019 ); the former is a machine-learning model based on principal component analysis for the classification of intrusion detection system datasets, while the latter is a hybrid ensemble intrusion detection system for anomaly detection using different datasets to detect patterns in network traffic that deviate from normal behaviour.

Agarwal et al. ( 2021 ) used three different machine learning algorithms in their research to find the most suitable for efficiently identifying patterns of suspicious network activity. The UNSW-NB15 dataset was used for this purpose. Kasongo and Sun ( 2020 ), Feed-Forward Deep Neural Network (FFDNN), Keshk et al. ( 2021 ), the privacy-preserving anomaly detection framework, and others also use the UNSW-NB 15 dataset as part of intrusion detection systems. The same dataset and others were used by Binbusayyis and Vaiyapuri ( 2019 ) to identify and compare key features for cyber intrusion detection. Atefinia and Ahmadi ( 2021 ) proposed a deep neural network model to reduce the false positive rate of an anomaly-based intrusion detection system. Fossaceca et al. ( 2015 ) focused in their research on the development of a framework that combined the outputs of multiple learners in order to improve the efficacy of network intrusion, and Gauthama Raman et al. ( 2020 ) presented a search algorithm based on Support Vector machine to improve the performance of the detection and false alarm rate to improve intrusion detection techniques. Ahmad and Alsemmeari ( 2020 ) targeted extreme learning machine techniques due to their good capabilities in classification problems and handling huge data. They used the NSL-KDD dataset as a benchmark.

With reference to prediction, Bakdash et al. ( 2018 ) used datasets from the U.S. Department of Defence to predict cyberattacks by malware. This dataset consists of weekly counts of cyber events over approximately seven years. Another prediction method was presented by Fan et al. ( 2018 ), which showed an improved integrated cybersecurity prediction method based on spatial-time analysis. Also, with reference to prediction, Ashtiani and Azgomi ( 2014 ) proposed a framework for the distributed simulation of cyberattacks based on high-level architecture. Kirubavathi and Anitha ( 2016 ) recommended an approach to detect botnets, irrespective of their structures, based on network traffic flow behaviour analysis and machine-learning techniques. Dwivedi et al. ( 2021 ) introduced a multi-parallel adaptive technique to utilise an adaption mechanism in the group of swarms for network intrusion detection. AlEroud and Karabatis ( 2018 ) presented an approach that used contextual information to automatically identify and query possible semantic links between different types of suspicious activities extracted from network flows.

Intrusion detection systems with a focus on IoT

In addition to general intrusion detection systems, a proportion of studies focused on IoT. Habib et al. ( 2020 ) presented an approach for converting traditional intrusion detection systems into smart intrusion detection systems for IoT networks. To enhance the process of diagnostic detection of possible vulnerabilities with an IoT system, Georgescu et al. ( 2019 ) introduced a method that uses a named entity recognition-based solution. With regard to IoT in the smart home sector, Heartfield et al. ( 2021 ) presented a detection system that is able to autonomously adjust the decision function of its underlying anomaly classification models to a smart home’s changing condition. Another intrusion detection system was suggested by Keserwani et al. ( 2021 ), which combined Grey Wolf Optimization and Particle Swam Optimization to identify various attacks for IoT networks. They used the KDD Cup 99, NSL-KDD and CICIDS-2017 to evaluate their model. Abu Al-Haija and Zein-Sabatto ( 2020 ) provide a comprehensive development of a new intelligent and autonomous deep-learning-based detection and classification system for cyberattacks in IoT communication networks that leverage the power of convolutional neural networks, abbreviated as IoT-IDCS-CNN (IoT-based Intrusion Detection and Classification System using Convolutional Neural Network). To evaluate the development, the authors used the NSL-KDD dataset. Biswas and Roy ( 2021 ) recommended a model that identifies malicious botnet traffic using novel deep-learning approaches like artificial neural networks gutted recurrent units and long- or short-term memory models. They tested their model with the Bot-IoT dataset.

With a more forensic background, Koroniotis et al. ( 2020 ) submitted a network forensic framework, which described the digital investigation phases for identifying and tracing attack behaviours in IoT networks. The suggested work was evaluated with the Bot-IoT and UINSW-NB15 datasets. With a focus on big data and IoT, Chhabra et al. ( 2020 ) presented a cyber forensic framework for big data analytics in an IoT environment using machine learning. Furthermore, the authors mentioned different publicly available datasets for machine-learning models.

A stronger focus on a mobile phones was exhibited by Alazab et al. ( 2020 ), which presented a classification model that combined permission requests and application programme interface calls. The model was tested with a malware dataset containing 27,891 Android apps. A similar approach was taken by Li et al. ( 2019a , b ), who proposed a reliable classifier for Android malware detection based on factorisation machine architecture and extraction of Android app features from manifest files and source code.

Literature reviews

In addition to the different methods and models for intrusion detection systems, various literature reviews on the methods and datasets were also found. Liu and Lang ( 2019 ) proposed a taxonomy of intrusion detection systems that uses data objects as the main dimension to classify and summarise machine learning and deep learning-based intrusion detection literature. They also presented four different benchmark datasets for machine-learning detection systems. Ahmed et al. ( 2016 ) presented an in-depth analysis of four major categories of anomaly detection techniques, which include classification, statistical, information theory and clustering. Hajj et al. ( 2021 ) gave a comprehensive overview of anomaly-based intrusion detection systems. Their article gives an overview of the requirements, methods, measurements and datasets that are used in an intrusion detection system.

Within the framework of machine learning, Chattopadhyay et al. ( 2018 ) conducted a comprehensive review and meta-analysis on the application of machine-learning techniques in intrusion detection systems. They also compared different machine learning techniques in different datasets and summarised the performance. Vidros et al. ( 2017 ) presented an overview of characteristics and methods in automatic detection of online recruitment fraud. They also published an available dataset of 17,880 annotated job ads, retrieved from the use of a real-life system. An empirical study of different unsupervised learning algorithms used in the detection of unknown attacks was presented by Meira et al. ( 2020 ).

New datasets

Kilincer et al. ( 2021 ) reviewed different intrusion detection system datasets in detail. They had a closer look at the UNS-NB15, ISCX-2012, NSL-KDD and CIDDS-001 datasets. Stojanovic et al. ( 2020 ) also provided a review on datasets and their creation for use in advanced persistent threat detection in the literature. Another review of datasets was provided by Sarker et al. ( 2020 ), who focused on cybersecurity data science as part of their research and provided an overview from a machine-learning perspective. Avila et al. ( 2021 ) conducted a systematic literature review on the use of security logs for data leak detection. They recommended a new classification of information leak, which uses the GDPR principles, identified the most widely publicly available dataset for threat detection, described the attack types in the datasets and the algorithms used for data leak detection. Tuncer et al. ( 2020 ) presented a bytecode-based detection method consisting of feature extraction using local neighbourhood binary patterns. They chose a byte-based malware dataset to investigate the performance of the proposed local neighbourhood binary pattern-based detection method. With a different focus, Mauro et al. ( 2020 ) gave an experimental overview of neural-based techniques relevant to intrusion detection. They assessed the value of neural networks using the Bot-IoT and UNSW-DB15 datasets.

Another category of results in the context of countermeasure datasets is those that were presented as new. Moreno et al. ( 2018 ) developed a database of 300 security-related accidents from European and American sources. The database contained cybersecurity-related events in the chemical and process industry. Damasevicius et al. ( 2020 ) proposed a new dataset (LITNET-2020) for network intrusion detection. The dataset is a new annotated network benchmark dataset obtained from the real-world academic network. It presents real-world examples of normal and under-attack network traffic. With a focus on IoT intrusion detection systems, Alsaedi et al. ( 2020 ) proposed a new benchmark IoT/IIot datasets for assessing intrusion detection system-enabled IoT systems. Also in the context of IoT, Vaccari et al. ( 2020 ) proposed a dataset focusing on message queue telemetry transport protocols, which can be used to train machine-learning models. To evaluate the performance of machine-learning classifiers, Mahfouz et al. ( 2020 ) created a dataset called Game Theory and Cybersecurity (GTCS). A dataset containing 22,000 malware and benign samples was constructed by Martin et al. ( 2019 ). The dataset can be used as a benchmark to test the algorithm for Android malware classification and clustering techniques. In addition, Laso et al. ( 2017 ) presented a dataset created to investigate how data and information quality estimates enable the detection of anomalies and malicious acts in cyber-physical systems. The dataset contained various cyberattacks and is publicly available.

In addition to the results described above, several other studies were found that fit into the category of countermeasures. Johnson et al. ( 2016 ) examined the time between vulnerability disclosures. Using another vulnerabilities database, Common Vulnerabilities and Exposures (CVE), Subroto and Apriyana ( 2019 ) presented an algorithm model that uses big data analysis of social media and statistical machine learning to predict cyber risks. A similar databank but with a different focus, Common Vulnerability Scoring System, was used by Chatterjee and Thekdi ( 2020 ) to present an iterative data-driven learning approach to vulnerability assessment and management for complex systems. Using the CICIDS2017 dataset to evaluate the performance, Malik et al. ( 2020 ) proposed a control plane-based orchestration for varied, sophisticated threats and attacks. The same dataset was used in another study by Lee et al. ( 2019 ), who developed an artificial security information event management system based on a combination of event profiling for data processing and different artificial network methods. To exploit the interdependence between multiple series, Fang et al. ( 2021 ) proposed a statistical framework. In order to validate the framework, the authors applied it to a dataset of enterprise-level security breaches from the Privacy Rights Clearinghouse and Identity Theft Center database. Another framework with a defensive aspect was recommended by Li et al. ( 2021 ) to increase the robustness of deep neural networks against adversarial malware evasion attacks. Sarabi et al. ( 2016 ) investigated whether and to what extent business details can help assess an organisation's risk of data breaches and the distribution of risk across different types of incidents to create policies for protection, detection and recovery from different forms of security incidents. They used data from the VERIS Community Database.

Datasets that have been classified into the cybersecurity category are detailed in Supplementary Table 3. Due to overlap, records from the previous tables may also be included.

This paper presented a systematic literature review of studies on cyber risk and cybersecurity that used datasets. Within this framework, 255 studies were fully reviewed and then classified into three different categories. Then, 79 datasets were consolidated from these studies. These datasets were subsequently analysed, and important information was selected through a process of filtering out. This information was recorded in a table and enhanced with further information as part of the literature analysis. This made it possible to create a comprehensive overview of the datasets. For example, each dataset contains a description of where the data came from and how the data has been used to date. This allows different datasets to be compared and the appropriate dataset for the use case to be selected. This research certainly has limitations, so our selection of datasets cannot necessarily be taken as a representation of all available datasets related to cyber risks and cybersecurity. For example, literature searches were conducted in four academic databases and only found datasets that were used in the literature. Many research projects also used old datasets that may no longer consider current developments. In addition, the data are often focused on only one observation and are limited in scope. For example, the datasets can only be applied to specific contexts and are also subject to further limitations (e.g. region, industry, operating system). In the context of the applicability of the datasets, it is unfortunately not possible to make a clear statement on the extent to which they can be integrated into academic or practical areas of application or how great this effort is. Finally, it remains to be pointed out that this is an overview of currently available datasets, which are subject to constant change.

Due to the lack of datasets on cyber risks in the academic literature, additional datasets on cyber risks were integrated as part of a further search. The search was conducted on the Google Dataset search portal. The search term used was ‘cyber risk datasets’. Over 100 results were found. However, due to the low significance and verifiability, only 20 selected datasets were included. These can be found in Table 2  in the “ Appendix ”.

Summary of Google datasets

The results of the literature review and datasets also showed that there continues to be a lack of available, open cyber datasets. This lack of data is reflected in cyber insurance, for example, as it is difficult to find a risk-based premium without a sufficient database (Nurse et al. 2020 ). The global cyber insurance market was estimated at USD 5.5 billion in 2020 (Dyson 2020 ). When compared to the USD 1 trillion global losses from cybercrime (Maleks Smith et al. 2020 ), it is clear that there exists a significant cyber risk awareness challenge for both the insurance industry and international commerce. Without comprehensive and qualitative data on cyber losses, it can be difficult to estimate potential losses from cyberattacks and price cyber insurance accordingly (GAO 2021 ). For instance, the average cyber insurance loss increased from USD 145,000 in 2019 to USD 359,000 in 2020 (FitchRatings 2021 ). Cyber insurance is an important risk management tool to mitigate the financial impact of cybercrime. This is particularly evident in the impact of different industries. In the Energy & Commodities financial markets, a ransomware attack on the Colonial Pipeline led to a substantial impact on the U.S. economy. As a result of the attack, about 45% of the U.S. East Coast was temporarily unable to obtain supplies of diesel, petrol and jet fuel. This caused the average price in the U.S. to rise 7 cents to USD 3.04 per gallon, the highest in seven years (Garber 2021 ). In addition, Colonial Pipeline confirmed that it paid a USD 4.4 million ransom to a hacker gang after the attack. Another ransomware attack occurred in the healthcare and government sector. The victim of this attack was the Irish Health Service Executive (HSE). A ransom payment of USD 20 million was demanded from the Irish government to restore services after the hack (Tidy 2021 ). In the car manufacturing sector, Miller and Valasek ( 2015 ) initiated a cyberattack that resulted in the recall of 1.4 million vehicles and cost manufacturers EUR 761 million. The risk that arises in the context of these events is the potential for the accumulation of cyber losses, which is why cyber insurers are not expanding their capacity. An example of this accumulation of cyber risks is the NotPetya malware attack, which originated in Russia, struck in Ukraine, and rapidly spread around the world, causing at least USD 10 billion in damage (GAO 2021 ). These events highlight the importance of proper cyber risk management.

This research provides cyber insurance stakeholders with an overview of cyber datasets. Cyber insurers can use the open datasets to improve their understanding and assessment of cyber risks. For example, the impact datasets can be used to better measure financial impacts and their frequencies. These data could be combined with existing portfolio data from cyber insurers and integrated with existing pricing tools and factors to better assess cyber risk valuation. Although most cyber insurers have sparse historical cyber policy and claims data, they remain too small at present for accurate prediction (Bessy-Roland et al. 2021 ). A combination of portfolio data and external datasets would support risk-adjusted pricing for cyber insurance, which would also benefit policyholders. In addition, cyber insurance stakeholders can use the datasets to identify patterns and make better predictions, which would benefit sustainable cyber insurance coverage. In terms of cyber risk cause datasets, cyber insurers can use the data to review their insurance products. For example, the data could provide information on which cyber risks have not been sufficiently considered in product design or where improvements are needed. A combination of cyber cause and cybersecurity datasets can help establish uniform definitions to provide greater transparency and clarity. Consistent terminology could lead to a more sustainable cyber market, where cyber insurers make informed decisions about the level of coverage and policyholders understand their coverage (The Geneva Association 2020).

In addition to the cyber insurance community, this research also supports cybersecurity stakeholders. The reviewed literature can be used to provide a contemporary, contextual and categorised summary of available datasets. This supports efficient and timely progress in cyber risk research and is beneficial given the dynamic nature of cyber risks. With the help of the described cybersecurity datasets and the identified information, a comparison of different datasets is possible. The datasets can be used to evaluate the effectiveness of countermeasures in simulated cyberattacks or to test intrusion detection systems.

In this paper, we conducted a systematic review of studies on cyber risk and cybersecurity databases. We found that most of the datasets are in the field of intrusion detection and machine learning and are used for technical cybersecurity aspects. The available datasets on cyber risks were relatively less represented. Due to the dynamic nature and lack of historical data, assessing and understanding cyber risk is a major challenge for cyber insurance stakeholders. To address this challenge, a greater density of cyber data is needed to support cyber insurers in risk management and researchers with cyber risk-related topics. With reference to ‘Open Science’ FAIR data (Jacobsen et al. 2020 ), mandatory reporting of cyber incidents could help improve cyber understanding, awareness and loss prevention among companies and insurers. Through greater availability of data, cyber risks can be better understood, enabling researchers to conduct more in-depth research into these risks. Companies could incorporate this new knowledge into their corporate culture to reduce cyber risks. For insurance companies, this would have the advantage that all insurers would have the same understanding of cyber risks, which would support sustainable risk-based pricing. In addition, common definitions of cyber risks could be derived from new data.

The cybersecurity databases summarised and categorised in this research could provide a different perspective on cyber risks that would enable the formulation of common definitions in cyber policies. The datasets can help companies addressing cybersecurity and cyber risk as part of risk management assess their internal cyber posture and cybersecurity measures. The paper can also help improve risk awareness and corporate behaviour, and provides the research community with a comprehensive overview of peer-reviewed datasets and other available datasets in the area of cyber risk and cybersecurity. This approach is intended to support the free availability of data for research. The complete tabulated review of the literature is included in the Supplementary Material.

This work provides directions for several paths of future work. First, there are currently few publicly available datasets for cyber risk and cybersecurity. The older datasets that are still widely used no longer reflect today's technical environment. Moreover, they can often only be used in one context, and the scope of the samples is very limited. It would be of great value if more datasets were publicly available that reflect current environmental conditions. This could help intrusion detection systems to consider current events and thus lead to a higher success rate. It could also compensate for the disadvantages of older datasets by collecting larger quantities of samples and making this contextualisation more widespread. Another area of research may be the integratability and adaptability of cybersecurity and cyber risk datasets. For example, it is often unclear to what extent datasets can be integrated or adapted to existing data. For cyber risks and cybersecurity, it would be helpful to know what requirements need to be met or what is needed to use the datasets appropriately. In addition, it would certainly be helpful to know whether datasets can be modified to be used for cyber risks or cybersecurity. Finally, the ability for stakeholders to identify machine-readable cybersecurity datasets would be useful because it would allow for even clearer delineations or comparisons between datasets. Due to the lack of publicly available datasets, concrete benchmarks often cannot be applied.

Below is the link to the electronic supplementary material.

Biographies

is a PhD student at the Kemmy Business School, University of Limerick, as part of the Emerging Risk Group (ERG). He is researching in joint cooperation with the Institute for Insurance Studies (ivwKöln), TH Köln, where he is working as a Research Assistant at the Cologne Research Centre for Reinsurance. His current research interests include cyber risks, cyber insurance and cybersecurity. Frank is a Fellow of the Chartered Insurance Institute (FCII) and a member of the German Association for Insurance Studies (DVfVW).

is a Lecturer in Risk and Finance at the Kemmy Business School at the University of Limerick. In his research, Dr Sheehan investigates novel risk metrication and machine learning methodologies in the context of insurance and finance, attentive to a changing private and public emerging risk environment. He is a researcher with significant insurance industry and academic experience. With a professional background in actuarial science, his research uses machine-learning techniques to estimate the changing risk profile produced by emerging technologies. He is a senior member of the Emerging Risk Group (ERG) at the University of Limerick, which has long-established expertise in insurance and risk management and has continued success within large research consortia including a number of SFI, FP7 and EU H2020 research projects. In particular, he contributed to the successful completion of three Horizon 2020 EU-funded projects, including PROTECT, Vision Inspired Driver Assistance Systems (VI-DAS) and Cloud Large Scale Video Analysis (Cloud-LSVA).

is a Professor at the Institute of Insurance at the Technical University of Cologne. His activities include teaching and research in insurance law and liability insurance. His research focuses include D&O, corporate liability, fidelity and cyber insurance. In addition, he heads the Master’s degree programme in insurance law and is the Academic Director of the Automotive Insurance Manager and Cyber Insurance Manager certificate programmes. He is also chairman of the examination board at the Institute of Insurance Studies.

Arash Negahdari Kia

is a postdoctoral Marie Cuire scholar and Research Fellow at the Kemmy Business School (KBS), University of Limerick (UL), a member of the Lero Software Research Center and Emerging Risk Group (ERG). He researches the cybersecurity risks of autonomous vehicles using machine-learning algorithms in a team supervised by Dr Finbarr Murphy at KBS, UL. For his PhD, he developed two graph-based, semi-supervised algorithms for multivariate time series for global stock market indices prediction. For his Master’s, he developed neural network models for Forex market prediction. Arash’s other research interests include text mining, graph mining and bioinformatics.

is a Professor in Risk and Insurance at the Kemmy Business School, University of Limerick. He worked on a number of insurance-related research projects, including four EU Commission-funded projects around emerging technologies and risk transfer. Prof. Mullins maintains strong links with the international insurance industry and works closely with Lloyd’s of London and XL Catlin on emerging risk. His work also encompasses the area of applied ethics as it pertains to new technologies. In the field of applied ethics, Dr Mullins works closely with the insurance industry and lectures on cultural and technological breakthroughs of high societal relevance. In that respect, Dr Martin Mullins has been appointed to a European expert group to advise EIOPA on the development of digital responsibility principles in insurance.

is Executive Dean Kemmy Business School. A computer engineering graduate, Finbarr worked for over 10 years in investment banking before returning to academia and completing his PhD in 2010. Finbarr has authored or co-authored over 70 refereed journal papers, edited books and book chapters. His research has been published in leading research journals in his discipline, such as Nature Nanotechnology, Small, Transportation Research A-F and the Review of Derivatives Research. A former Fulbright Scholar and Erasmus Mundus Exchange Scholar, Finbarr has delivered numerous guest lectures in America, mainland Europe, Israel, Russia, China and Vietnam. His research interests include quantitative finance and, more recently, emerging technological risk. Finbarr is currently engaged in several EU H2020 projects and with the Irish Science Foundation Ireland.

(FCII) has held the Chair of Reinsurance at the Institute of Insurance of TH Köln since 1998, focusing on the efficiency of reinsurance, industrial insurance and alternative risk transfer (ART). He studied mathematics and computer science with a focus on artificial intelligence and researched from 1988 to 1991 at the Fraunhofer Institute for Autonomous Intelligent Systems (AiS) in Schloß Birlinghoven. From 1991 to 2004, Prof. Materne worked for Gen Re (formerly Cologne Re) in various management positions in Germany and abroad, and from 2001 to 2003, he served as General Manager of Cologne Re of Dublin in Ireland. In 2008, Prof. Materne founded the Cologne Reinsurance Research Centre, of which he is the Director. Current issues in reinsurance and related fields are analysed and discussed with practitioners, with valuable contacts through the ‘Förderkreis Rückversicherung’ and the organisation of the annual Cologne Reinsurance Symposium. Prof. Materne holds various international supervisory boards, board of directors and advisory board mandates at insurance and reinsurance companies, captives, InsurTechs, EIOPA, as well as at insurance-scientific institutions. He also acts as an arbitrator and party representative in arbitration proceedings.

Open Access funding provided by the IReL Consortium.

Declarations

On behalf of all authors, the corresponding author states that there is no conflict of interest.

1 Average cost of a breach of more than 50 million records.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Aamir M, Rizvi SSH, Hashmani MA, Zubair M, Ahmad J. Machine learning classification of port scanning and DDoS attacks: A comparative analysis. Mehran University Research Journal of Engineering and Technology. 2021; 40 (1):215–229. doi: 10.22581/muet1982.2101.19. [ CrossRef ] [ Google Scholar ]
• Aamir M, Zaidi SMA. DDoS attack detection with feature engineering and machine learning: The framework and performance evaluation. International Journal of Information Security. 2019; 18 (6):761–785. doi: 10.1007/s10207-019-00434-1. [ CrossRef ] [ Google Scholar ]
• Aassal A, El S, Baki A. Das, Verma RM. An in-depth benchmarking and evaluation of phishing detection research for security needs. IEEE Access. 2020; 8 :22170–22192. doi: 10.1109/ACCESS.2020.2969780. [ CrossRef ] [ Google Scholar ]
• Abu Al-Haija Q, Zein-Sabatto S. An efficient deep-learning-based detection and classification system for cyber-attacks in IoT communication networks. Electronics. 2020; 9 (12):26. doi: 10.3390/electronics9122152. [ CrossRef ] [ Google Scholar ]
• Adhikari U, Morris TH, Pan SY. Applying Hoeffding adaptive trees for real-time cyber-power event and intrusion classification. IEEE Transactions on Smart Grid. 2018; 9 (5):4049–4060. doi: 10.1109/tsg.2017.2647778. [ CrossRef ] [ Google Scholar ]
• Agarwal A, Sharma P, Alshehri M, Mohamed AA, Alfarraj O. Classification model for accuracy and intrusion detection using machine learning approach. PeerJ Computer Science. 2021 doi: 10.7717/peerj-cs.437. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Agrafiotis Ioannis, Nurse Jason R.C., Goldsmith M, Creese S, Upton D. A taxonomy of cyber-harms: Defining the impacts of cyber-attacks and understanding how they propagate. Journal of Cybersecurity. 2018; 4 :tyy006. doi: 10.1093/cybsec/tyy006. [ CrossRef ] [ Google Scholar ]
• Agrawal A, Mohammed S, Fiaidhi J. Ensemble technique for intruder detection in network traffic. International Journal of Security and Its Applications. 2019; 13 (3):1–8. doi: 10.33832/ijsia.2019.13.3.01. [ CrossRef ] [ Google Scholar ]
• Ahmad, I., and R.A. Alsemmeari. 2020. Towards improving the intrusion detection through ELM (extreme learning machine). CMC Computers Materials & Continua 65 (2): 1097–1111. 10.32604/cmc.2020.011732.
• Ahmed M, Mahmood AN, Hu JK. A survey of network anomaly detection techniques. Journal of Network and Computer Applications. 2016; 60 :19–31. doi: 10.1016/j.jnca.2015.11.016. [ CrossRef ] [ Google Scholar ]
• Al-Jarrah OY, Alhussein O, Yoo PD, Muhaidat S, Taha K, Kim K. Data randomization and cluster-based partitioning for Botnet intrusion detection. IEEE Transactions on Cybernetics. 2016; 46 (8):1796–1806. doi: 10.1109/TCYB.2015.2490802. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Al-Mhiqani MN, Ahmad R, Abidin ZZ, Yassin W, Hassan A, Abdulkareem KH, Ali NS, Yunos Z. A review of insider threat detection: Classification, machine learning techniques, datasets, open challenges, and recommendations. Applied Sciences—Basel. 2020; 10 (15):41. doi: 10.3390/app10155208. [ CrossRef ] [ Google Scholar ]
• Al-Omari M, Rawashdeh M, Qutaishat F, Alshira'H M, Ababneh N. An intelligent tree-based intrusion detection model for cyber security. Journal of Network and Systems Management. 2021; 29 (2):18. doi: 10.1007/s10922-021-09591-y. [ CrossRef ] [ Google Scholar ]
• Alabdallah A, Awad M. Using weighted Support Vector Machine to address the imbalanced classes problem of Intrusion Detection System. KSII Transactions on Internet and Information Systems. 2018; 12 (10):5143–5158. doi: 10.3837/tiis.2018.10.027. [ CrossRef ] [ Google Scholar ]
• Alazab M, Alazab M, Shalaginov A, Mesleh A, Awajan A. Intelligent mobile malware detection using permission requests and API calls. Future Generation Computer Systems—the International Journal of eScience. 2020; 107 :509–521. doi: 10.1016/j.future.2020.02.002. [ CrossRef ] [ Google Scholar ]
• Albahar MA, Al-Falluji RA, Binsawad M. An empirical comparison on malicious activity detection using different neural network-based models. IEEE Access. 2020; 8 :61549–61564. doi: 10.1109/ACCESS.2020.2984157. [ CrossRef ] [ Google Scholar ]
• AlEroud AF, Karabatis G. Queryable semantics to detect cyber-attacks: A flow-based detection approach. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2018; 48 (2):207–223. doi: 10.1109/TSMC.2016.2600405. [ CrossRef ] [ Google Scholar ]
• Algarni AM, Thayananthan V, Malaiya YK. Quantitative assessment of cybersecurity risks for mitigating data breaches in business systems. Applied Sciences (switzerland) 2021 doi: 10.3390/app11083678. [ CrossRef ] [ Google Scholar ]
• Alhowaide A, Alsmadi I, Tang J. Towards the design of real-time autonomous IoT NIDS. Cluster Computing—the Journal of Networks Software Tools and Applications. 2021 doi: 10.1007/s10586-021-03231-5. [ CrossRef ] [ Google Scholar ]
• Ali S, Li Y. Learning multilevel auto-encoders for DDoS attack detection in smart grid network. IEEE Access. 2019; 7 :108647–108659. doi: 10.1109/ACCESS.2019.2933304. [ CrossRef ] [ Google Scholar ]
• AlKadi O, Moustafa N, Turnbull B, Choo KKR. Mixture localization-based outliers models for securing data migration in cloud centers. IEEE Access. 2019; 7 :114607–114618. doi: 10.1109/ACCESS.2019.2935142. [ CrossRef ] [ Google Scholar ]
• Allianz. 2021. Allianz Risk Barometer. https://www.agcs.allianz.com/content/dam/onemarketing/agcs/agcs/reports/Allianz-Risk-Barometer-2021.pdf . Accessed 15 May 2021.
• Almiani Muder, AbuGhazleh Alia, Al-Rahayfeh Amer, Atiewi Saleh, Razaque Abdul. Deep recurrent neural network for IoT intrusion detection system. Simulation Modelling Practice and Theory. 2020; 101 :102031. doi: 10.1016/j.simpat.2019.102031. [ CrossRef ] [ Google Scholar ]
• Alsaedi A, Moustafa N, Tari Z, Mahmood A, Anwar A. TON_IoT telemetry dataset: A new generation dataset of IoT and IIoT for data-driven intrusion detection systems. IEEE Access. 2020; 8 :165130–165150. doi: 10.1109/access.2020.3022862. [ CrossRef ] [ Google Scholar ]
• Alsamiri J, Alsubhi K. Internet of Things cyber attacks detection using machine learning. International Journal of Advanced Computer Science and Applications. 2019; 10 (12):627–634. doi: 10.14569/IJACSA.2019.0101280. [ CrossRef ] [ Google Scholar ]
• Alsharafat W. Applying artificial neural network and eXtended classifier system for network intrusion detection. International Arab Journal of Information Technology. 2013; 10 (3):230–238. [ Google Scholar ]
• Amin RW, Sevil HE, Kocak S, Francia G, III, Hoover P. The spatial analysis of the malicious uniform resource locators (URLs): 2016 dataset case study. Information (switzerland) 2021; 12 (1):1–18. doi: 10.3390/info12010002. [ CrossRef ] [ Google Scholar ]
• Arcuri MC, Gai LZ, Ielasi F, Ventisette E. Cyber attacks on hospitality sector: Stock market reaction. Journal of Hospitality and Tourism Technology. 2020; 11 (2):277–290. doi: 10.1108/jhtt-05-2019-0080. [ CrossRef ] [ Google Scholar ]
• Arp Daniel, Spreitzenbarth Michael, Hubner Malte, Rieck Konrad, et al. Drebin: Effective and explainable detection of android malware in your pocket. NDSS Conference. 2014; 14 :23–26. [ Google Scholar ]
• Ashtiani M, Azgomi MA. A distributed simulation framework for modeling cyber attacks and the evaluation of security measures. Simulation—Transactions of the Society for Modeling and Simulation International. 2014; 90 (9):1071–1102. doi: 10.1177/0037549714540221. [ CrossRef ] [ Google Scholar ]
• Atefinia R, Ahmadi M. Network intrusion detection using multi-architectural modular deep neural network. Journal of Supercomputing. 2021; 77 (4):3571–3593. doi: 10.1007/s11227-020-03410-y. [ CrossRef ] [ Google Scholar ]
• Avila R, Khoury R, Khoury R, Petrillo F. Use of security logs for data leak detection: A systematic literature review. Security and Communication Networks. 2021; 2021 :29. doi: 10.1155/2021/6615899. [ CrossRef ] [ Google Scholar ]
• Azeez NA, Ayemobola TJ, Misra S, Maskeliunas R, Damasevicius R. Network Intrusion Detection with a Hashing Based Apriori Algorithm Using Hadoop MapReduce. Computers. 2019; 8 (4):15. doi: 10.3390/computers8040086. [ CrossRef ] [ Google Scholar ]
• Bakdash JZ, Hutchinson S, Zaroukian EG, Marusich LR, Thirumuruganathan S, Sample C, Hoffman B, Das G. Malware in the future forecasting of analyst detection of cyber events. Journal of Cybersecurity. 2018 doi: 10.1093/cybsec/tyy007. [ CrossRef ] [ Google Scholar ]
• Barletta VS, Caivano D, Nannavecchia A, Scalera M. Intrusion detection for in-vehicle communication networks: An unsupervised Kohonen SOM approach. Future Internet. 2020 doi: 10.3390/FI12070119. [ CrossRef ] [ Google Scholar ]
• Barzegar M, Shajari M. Attack scenario reconstruction using intrusion semantics. Expert Systems with Applications. 2018; 108 :119–133. doi: 10.1016/j.eswa.2018.04.030. [ CrossRef ] [ Google Scholar ]
• Bessy-Roland Yannick, Boumezoued Alexandre, Hillairet Caroline. Multivariate Hawkes process for cyber insurance. Annals of Actuarial Science. 2021; 15 (1):14–39. doi: 10.1017/S1748499520000093. [ CrossRef ] [ Google Scholar ]
• Bhardwaj A, Mangat V, Vig R. Hyperband tuned deep neural network with well posed stacked sparse AutoEncoder for detection of DDoS attacks in cloud. IEEE Access. 2020; 8 :181916–181929. doi: 10.1109/ACCESS.2020.3028690. [ CrossRef ] [ Google Scholar ]
• Bhati BS, Rai CS, Balamurugan B, Al-Turjman F. An intrusion detection scheme based on the ensemble of discriminant classifiers. Computers & Electrical Engineering. 2020; 86 :9. doi: 10.1016/j.compeleceng.2020.106742. [ CrossRef ] [ Google Scholar ]
• Bhattacharya S, Krishnan SSR, Maddikunta PKR, Kaluri R, Singh S, Gadekallu TR, Alazab M, Tariq U. A novel PCA-firefly based XGBoost classification model for intrusion detection in networks using GPU. Electronics. 2020; 9 (2):16. doi: 10.3390/electronics9020219. [ CrossRef ] [ Google Scholar ]
• Bibi I, Akhunzada A, Malik J, Iqbal J, Musaddiq A, Kim S. A dynamic DL-driven architecture to combat sophisticated android malware. IEEE Access. 2020; 8 :129600–129612. doi: 10.1109/ACCESS.2020.3009819. [ CrossRef ] [ Google Scholar ]
• Biener C, Eling M, Wirfs JH. Insurability of cyber risk: An empirical analysis. Geneva Papers on Risk and Insurance: Issues and Practice. 2015; 40 (1):131–158. doi: 10.1057/gpp.2014.19. [ CrossRef ] [ Google Scholar ]
• Binbusayyis A, Vaiyapuri T. Identifying and benchmarking key features for cyber intrusion detection: An ensemble approach. IEEE Access. 2019; 7 :106495–106513. doi: 10.1109/ACCESS.2019.2929487. [ CrossRef ] [ Google Scholar ]
• Biswas R, Roy S. Botnet traffic identification using neural networks. Multimedia Tools and Applications. 2021 doi: 10.1007/s11042-021-10765-8. [ CrossRef ] [ Google Scholar ]
• Bouyeddou B, Harrou F, Kadri B, Sun Y. Detecting network cyber-attacks using an integrated statistical approach. Cluster Computing—the Journal of Networks Software Tools and Applications. 2021; 24 (2):1435–1453. doi: 10.1007/s10586-020-03203-1. [ CrossRef ] [ Google Scholar ]
• Bozkir AS, Aydos M. LogoSENSE: A companion HOG based logo detection scheme for phishing web page and E-mail brand recognition. Computers & Security. 2020; 95 :18. doi: 10.1016/j.cose.2020.101855. [ CrossRef ] [ Google Scholar ]
• Brower, D., and M. McCormick. 2021. Colonial pipeline resumes operations following ransomware attack. Financial Times .
• Cai H, Zhang F, Levi A. An unsupervised method for detecting shilling attacks in recommender systems by mining item relationship and identifying target items. The Computer Journal. 2019; 62 (4):579–597. doi: 10.1093/comjnl/bxy124. [ CrossRef ] [ Google Scholar ]
• Cebula, J.J., M.E. Popeck, and L.R. Young. 2014. A Taxonomy of Operational Cyber Security Risks Version 2 .
• Chadza T, Kyriakopoulos KG, Lambotharan S. Learning to learn sequential network attacks using hidden Markov models. IEEE Access. 2020; 8 :134480–134497. doi: 10.1109/ACCESS.2020.3011293. [ CrossRef ] [ Google Scholar ]
• Chatterjee S, Thekdi S. An iterative learning and inference approach to managing dynamic cyber vulnerabilities of complex systems. Reliability Engineering and System Safety. 2020 doi: 10.1016/j.ress.2019.106664. [ CrossRef ] [ Google Scholar ]
• Chattopadhyay M, Sen R, Gupta S. A comprehensive review and meta-analysis on applications of machine learning techniques in intrusion detection. Australasian Journal of Information Systems. 2018; 22 :27. doi: 10.3127/ajis.v22i0.1667. [ CrossRef ] [ Google Scholar ]
• Chen HS, Fiscus J. The inhospitable vulnerability: A need for cybersecurity risk assessment in the hospitality industry. Journal of Hospitality and Tourism Technology. 2018; 9 (2):223–234. doi: 10.1108/JHTT-07-2017-0044. [ CrossRef ] [ Google Scholar ]
• Chhabra GS, Singh VP, Singh M. Cyber forensics framework for big data analytics in IoT environment using machine learning. Multimedia Tools and Applications. 2020; 79 (23–24):15881–15900. doi: 10.1007/s11042-018-6338-1. [ CrossRef ] [ Google Scholar ]
• Chiba Z, Abghour N, Moussaid K, Elomri A, Rida M. Intelligent approach to build a Deep Neural Network based IDS for cloud environment using combination of machine learning algorithms. Computers and Security. 2019; 86 :291–317. doi: 10.1016/j.cose.2019.06.013. [ CrossRef ] [ Google Scholar ]
• Choras M, Kozik R. Machine learning techniques applied to detect cyber attacks on web applications. Logic Journal of the IGPL. 2015; 23 (1):45–56. doi: 10.1093/jigpal/jzu038. [ CrossRef ] [ Google Scholar ]
• Chowdhury Sudipta, Khanzadeh Mojtaba, Akula Ravi, Zhang Fangyan, Zhang Song, Medal Hugh, Marufuzzaman Mohammad, Bian Linkan. Botnet detection using graph-based feature clustering. Journal of Big Data. 2017; 4 (1):14. doi: 10.1186/s40537-017-0074-7. [ CrossRef ] [ Google Scholar ]
• Cost Of A Cyber Incident: Systematic Review And Cross-Validation, Cybersecurity & Infrastructure Agency , 1, https://www.cisa.gov/sites/default/files/publications/CISA-OCE_Cost_of_Cyber_Incidents_Study-FINAL_508.pdf (2020).
• D'Hooge L, Wauters T, Volckaert B, De Turck F. Classification hardness for supervised learners on 20 years of intrusion detection data. IEEE Access. 2019; 7 :167455–167469. doi: 10.1109/access.2019.2953451. [ CrossRef ] [ Google Scholar ]
• Damasevicius R, Venckauskas A, Grigaliunas S, Toldinas J, Morkevicius N, Aleliunas T, Smuikys P. LITNET-2020: An annotated real-world network flow dataset for network intrusion detection. Electronics. 2020; 9 (5):23. doi: 10.3390/electronics9050800. [ CrossRef ] [ Google Scholar ]
• Giovanni De, Domenico Arturo Leccadito, Pirra Marco. On the determinants of data breaches: A cointegration analysis. Decisions in Economics and Finance. 2020 doi: 10.1007/s10203-020-00301-y. [ CrossRef ] [ Google Scholar ]
• Deng Lianbing, Li Daming, Yao Xiang, Wang Haoxiang. Retracted Article: Mobile network intrusion detection for IoT system based on transfer learning algorithm. Cluster Computing. 2019; 22 (4):9889–9904. doi: 10.1007/s10586-018-1847-2. [ CrossRef ] [ Google Scholar ]
• Donkal G, Verma GK. A multimodal fusion based framework to reinforce IDS for securing Big Data environment using Spark. Journal of Information Security and Applications. 2018; 43 :1–11. doi: 10.1016/j.jisa.2018.10.001. [ CrossRef ] [ Google Scholar ]
• Dunn C, Moustafa N, Turnbull B. Robustness evaluations of sustainable machine learning models against data Poisoning attacks in the Internet of Things. Sustainability. 2020; 12 (16):17. doi: 10.3390/su12166434. [ CrossRef ] [ Google Scholar ]
• Dwivedi S, Vardhan M, Tripathi S. Multi-parallel adaptive grasshopper optimization technique for detecting anonymous attacks in wireless networks. Wireless Personal Communications. 2021 doi: 10.1007/s11277-021-08368-5. [ CrossRef ] [ Google Scholar ]
• Dyson, B. 2020. COVID-19 crisis could be ‘watershed’ for cyber insurance, says Swiss Re exec. https://www.spglobal.com/marketintelligence/en/news-insights/latest-news-headlines/covid-19-crisis-could-be-watershed-for-cyber-insurance-says-swiss-re-exec-59197154 . Accessed 7 May 2020.
• EIOPA. 2018. Understanding cyber insurance—a structured dialogue with insurance companies. https://www.eiopa.europa.eu/sites/default/files/publications/reports/eiopa_understanding_cyber_insurance.pdf . Accessed 28 May 2018
• Elijah AV, Abdullah A, JhanJhi NZ, Supramaniam M, Abdullateef OB. Ensemble and deep-learning methods for two-class and multi-attack anomaly intrusion detection: An empirical study. International Journal of Advanced Computer Science and Applications. 2019; 10 (9):520–528. doi: 10.14569/IJACSA.2019.0100969. [ CrossRef ] [ Google Scholar ]
• Eling M, Jung K. Copula approaches for modeling cross-sectional dependence of data breach losses. Insurance Mathematics & Economics. 2018; 82 :167–180. doi: 10.1016/j.insmatheco.2018.07.003. [ CrossRef ] [ Google Scholar ]
• Eling M, Schnell W. What do we know about cyber risk and cyber risk insurance? Journal of Risk Finance. 2016; 17 (5):474–491. doi: 10.1108/jrf-09-2016-0122. [ CrossRef ] [ Google Scholar ]
• Eling M, Wirfs J. What are the actual costs of cyber risk events? European Journal of Operational Research. 2019; 272 (3):1109–1119. doi: 10.1016/j.ejor.2018.07.021. [ CrossRef ] [ Google Scholar ]
• Eling Martin. Cyber risk research in business and actuarial science. European Actuarial Journal. 2020; 10 (2):303–333. doi: 10.1007/s13385-020-00250-1. [ CrossRef ] [ Google Scholar ]
• Elmasry W, Akbulut A, Zaim AH. Empirical study on multiclass classification-based network intrusion detection. Computational Intelligence. 2019; 35 (4):919–954. doi: 10.1111/coin.12220. [ CrossRef ] [ Google Scholar ]
• Elsaid Shaimaa Ahmed, Albatati Nouf Saleh. An optimized collaborative intrusion detection system for wireless sensor networks. Soft Computing. 2020; 24 (16):12553–12567. doi: 10.1007/s00500-020-04695-0. [ CrossRef ] [ Google Scholar ]
• Estepa R, Díaz-Verdejo JE, Estepa A, Madinabeitia G. How much training data is enough? A case study for HTTP anomaly-based intrusion detection. IEEE Access. 2020; 8 :44410–44425. doi: 10.1109/ACCESS.2020.2977591. [ CrossRef ] [ Google Scholar ]
• European Council. 2021. Cybersecurity: how the EU tackles cyber threats. https://www.consilium.europa.eu/en/policies/cybersecurity/ . Accessed 10 May 2021
• Falco Gregory, Eling Martin, Jablanski Danielle, Weber Matthias, Miller Virginia, Gordon Lawrence A, Wang Shaun Shuxun, Schmit Joan, Thomas Russell, Elvedi Mauro, Maillart Thomas, Donavan Emy, Dejung Simon, Durand Eric, Nutter Franklin, Scheffer Uzi, Arazi Gil, Ohana Gilbert, Lin Herbert. Cyber risk research impeded by disciplinary barriers. Science (american Association for the Advancement of Science) 2019; 366 (6469):1066–1069. doi: 10.1126/science.aaz4795. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fan ZJ, Tan ZP, Tan CX, Li X. An improved integrated prediction method of cyber security situation based on spatial-time analysis. Journal of Internet Technology. 2018; 19 (6):1789–1800. doi: 10.3966/160792642018111906015. [ CrossRef ] [ Google Scholar ]
• Fang ZJ, Xu MC, Xu SH, Hu TZ. A framework for predicting data breach risk: Leveraging dependence to cope with sparsity. IEEE Transactions on Information Forensics and Security. 2021; 16 :2186–2201. doi: 10.1109/tifs.2021.3051804. [ CrossRef ] [ Google Scholar ]
• Farkas S, Lopez O, Thomas M. Cyber claim analysis using Generalized Pareto regression trees with applications to insurance. Insurance: Mathematics and Economics. 2021; 98 :92–105. doi: 10.1016/j.insmatheco.2021.02.009. [ CrossRef ] [ Google Scholar ]
• Farsi H, Fanian A, Taghiyarrenani Z. A novel online state-based anomaly detection system for process control networks. International Journal of Critical Infrastructure Protection. 2019; 27 :11. doi: 10.1016/j.ijcip.2019.100323. [ CrossRef ] [ Google Scholar ]
• Ferrag MA, Maglaras L, Moschoyiannis S, Janicke H. Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications. 2020; 50 :19. doi: 10.1016/j.jisa.2019.102419. [ CrossRef ] [ Google Scholar ]
• Field, M. 2018. WannaCry cyber attack cost the NHS £92m as 19,000 appointments cancelled. https://www.telegraph.co.uk/technology/2018/10/11/wannacry-cyber-attack-cost-nhs-92m-19000-appointments-cancelled/ . Accessed 9 May 2018.
• FitchRatings. 2021. U.S. Cyber Insurance Market Update (Spike in Claims Leads to Decline in 2020 Underwriting Performance). https://www.fitchratings.com/research/insurance/us-cyber-insurance-market-update-spike-in-claims-leads-to-decline-in-2020-underwriting-performance-26-05-2021 .
• Fossaceca JM, Mazzuchi TA, Sarkani S. MARK-ELM: Application of a novel Multiple Kernel Learning framework for improving the robustness of network intrusion detection. Expert Systems with Applications. 2015; 42 (8):4062–4080. doi: 10.1016/j.eswa.2014.12.040. [ CrossRef ] [ Google Scholar ]
• Franke Ulrik, Brynielsson Joel. Cyber situational awareness – A systematic review of the literature. Computers &amp; Security. 2014; 46 :18–31. doi: 10.1016/j.cose.2014.06.008. [ CrossRef ] [ Google Scholar ]
• Freeha Khan, Hwan Kim Jung, Lars Mathiassen, Robin Moore. Data breach management: An integrated risk model. Information &amp; Management. 2021; 58 (1):103392. doi: 10.1016/j.im.2020.103392. [ CrossRef ] [ Google Scholar ]
• Ganeshan R, Rodrigues Paul. Crow-AFL: Crow based adaptive fractional lion optimization approach for the intrusion detection. Wireless Personal Communications. 2020; 111 (4):2065–2089. doi: 10.1007/s11277-019-06972-0. [ CrossRef ] [ Google Scholar ]
• GAO. 2021. CYBER INSURANCE—Insurers and policyholders face challenges in an evolving market. https://www.gao.gov/assets/gao-21-477.pdf . Accessed 16 May 2021.
• Garber, J. 2021. Colonial Pipeline fiasco foreshadows impact of Biden energy policy. https://www.foxbusiness.com/markets/colonial-pipeline-fiasco-foreshadows-impact-of-biden-energy-policy . Accessed 4 May 2021.
• Gauthama Raman MR, Somu Nivethitha, Jagarapu Sahruday, Manghnani Tina, Selvam Thirumaran, Krithivasan Kannan, Shankar Sriram VS. An efficient intrusion detection technique based on support vector machine and improved binary gravitational search algorithm. Artificial Intelligence Review. 2020; 53 (5):3255–3286. doi: 10.1007/s10462-019-09762-z. [ CrossRef ] [ Google Scholar ]
• Gavel S, Raghuvanshi AS, Tiwari S. Distributed intrusion detection scheme using dual-axis dimensionality reduction for Internet of things (IoT) Journal of Supercomputing. 2021 doi: 10.1007/s11227-021-03697-5. [ CrossRef ] [ Google Scholar ]
• GDPR.EU. 2021. FAQ. https://gdpr.eu/faq/ . Accessed 10 May 2021.
• Georgescu TM, Iancu B, Zurini M. Named-entity-recognition-based automated system for diagnosing cybersecurity situations in IoT networks. Sensors (switzerland) 2019 doi: 10.3390/s19153380. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Giudici Paolo, Raffinetti Emanuela. Cyber risk ordering with rank-based statistical models. AStA Advances in Statistical Analysis. 2020 doi: 10.1007/s10182-020-00387-0. [ CrossRef ] [ Google Scholar ]
• Goh, J., S. Adepu, K.N. Junejo, and A. Mathur. 2016. A dataset to support research in the design of secure water treatment systems. In CRITIS.
• Gong XY, Lu JL, Zhou YF, Qiu H, He R. Model uncertainty based annotation error fixing for web attack detection. Journal of Signal Processing Systems for Signal Image and Video Technology. 2021; 93 (2–3):187–199. doi: 10.1007/s11265-019-01494-1. [ CrossRef ] [ Google Scholar ]
• Goode Sigi, Hoehle Hartmut, Venkatesh Viswanath, Brown Susan A. USER compensation as a data breach recovery action: An investigation of the sony playstation network breach. MIS Quarterly. 2017; 41 (3):703–727. doi: 10.25300/MISQ/2017/41.3.03. [ CrossRef ] [ Google Scholar ]
• Guo H, Huang S, Huang C, Pan Z, Zhang M, Shi F. File entropy signal analysis combined with wavelet decomposition for malware classification. IEEE Access. 2020; 8 :158961–158971. doi: 10.1109/ACCESS.2020.3020330. [ CrossRef ] [ Google Scholar ]
• Habib Maria, Aljarah Ibrahim, Faris Hossam. A Modified multi-objective particle swarm optimizer-based Lévy flight: An approach toward intrusion detection in Internet of Things. Arabian Journal for Science and Engineering. 2020; 45 (8):6081–6108. doi: 10.1007/s13369-020-04476-9. [ CrossRef ] [ Google Scholar ]
• Hajj S, El Sibai R, Abdo JB, Demerjian J, Makhoul A, Guyeux C. Anomaly-based intrusion detection systems: The requirements, methods, measurements, and datasets. Transactions on Emerging Telecommunications Technologies. 2021; 32 (4):36. doi: 10.1002/ett.4240. [ CrossRef ] [ Google Scholar ]
• Heartfield R, Loukas G, Bezemskij A, Panaousis E. Self-configurable cyber-physical intrusion detection for smart homes using reinforcement learning. IEEE Transactions on Information Forensics and Security. 2021; 16 :1720–1735. doi: 10.1109/tifs.2020.3042049. [ CrossRef ] [ Google Scholar ]
• Hemo, B., T. Gafni, K. Cohen, and Q. Zhao. 2020. Searching for anomalies over composite hypotheses. IEEE Transactions on Signal Processing 68: 1181–1196. 10.1109/TSP.2020.2971438
• Hindy H, Brosset D, Bayne E, Seeam AK, Tachtatzis C, Atkinson R, Bellekens X. A taxonomy of network threats and the effect of current datasets on intrusion detection systems. IEEE Access. 2020; 8 :104650–104675. doi: 10.1109/ACCESS.2020.3000179. [ CrossRef ] [ Google Scholar ]
• Hong W, Huang D, Chen C, Lee J. Towards accurate and efficient classification of power system contingencies and cyber-attacks using recurrent neural networks. IEEE Access. 2020; 8 :123297–123309. doi: 10.1109/ACCESS.2020.3007609. [ CrossRef ] [ Google Scholar ]
• Husák Martin, Zádník M, Bartos V, Sokol P. Dataset of intrusion detection alerts from a sharing platform. Data in Brief. 2020; 33 :106530. doi: 10.1016/j.dib.2020.106530. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• IBM Security. 2020. Cost of a Data breach Report. https://www.capita.com/sites/g/files/nginej291/files/2020-08/Ponemon-Global-Cost-of-Data-Breach-Study-2020.pdf . Accessed 19 May 2021.
• IEEE. 2021. IEEE Quick Facts. https://www.ieee.org/about/at-a-glance.html . Accessed 11 May 2021.
• Firat Ilhan, Kilincer Ertam Fatih, Abdulkadir Sengur. Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Computer Networks. 2021; 188 :107840. doi: 10.1016/j.comnet.2021.107840. [ CrossRef ] [ Google Scholar ]
• Jaber AN, Ul Rehman S. FCM-SVM based intrusion detection system for cloud computing environment. Cluster Computing—the Journal of Networks Software Tools and Applications. 2020; 23 (4):3221–3231. doi: 10.1007/s10586-020-03082-6. [ CrossRef ] [ Google Scholar ]
• Jacobs, J., S. Romanosky, B. Edwards, M. Roytman, and I. Adjerid. 2019. Exploit prediction scoring system (epss). arXiv:1908.04856
• Jacobsen Annika, de Miranda Ricardo, Azevedo Nick Juty, Batista Dominique, Coles Simon, Cornet Ronald, Courtot Mélanie, Crosas Mercè, Dumontier Michel, Evelo Chris T, Goble Carole, Guizzardi Giancarlo, Hansen Karsten Kryger, Hasnain Ali, Hettne Kristina, Heringa Jaap, Hooft Rob W.W., Imming Melanie, Jeffery Keith G, Kaliyaperumal Rajaram, Kersloot Martijn G, Kirkpatrick Christine R, Kuhn Tobias, Labastida Ignasi, Magagna Barbara, McQuilton Peter, Meyers Natalie, Montesanti Annalisa, van Reisen Mirjam, Rocca-Serra Philippe, Pergl Robert, Sansone Susanna-Assunta, da Silva Luiz Olavo Bonino, Santos Juliane Schneider, Strawn George, Thompson Mark, Waagmeester Andra, Weigel Tobias, Wilkinson Mark D, Willighagen Egon L, Wittenburg Peter, Roos Marco, Mons Barend, Schultes Erik. FAIR principles: Interpretations and implementation considerations. Data Intelligence. 2020; 2 (1–2):10–29. doi: 10.1162/dint_r_00024. [ CrossRef ] [ Google Scholar ]
• Jahromi AN, Hashemi S, Dehghantanha A, Parizi RM, Choo KKR. An enhanced stacked LSTM method with no random initialization for malware threat hunting in safety and time-critical systems. IEEE Transactions on Emerging Topics in Computational Intelligence. 2020; 4 (5):630–640. doi: 10.1109/TETCI.2019.2910243. [ CrossRef ] [ Google Scholar ]
• Jang S, Li S, Sung Y. FastText-based local feature visualization algorithm for merged image-based malware classification framework for cyber security and cyber defense. Mathematics. 2020; 8 (3):13. doi: 10.3390/math8030460. [ CrossRef ] [ Google Scholar ]
• Javeed D, Gao TH, Khan MT. SDN-enabled hybrid DL-driven framework for the detection of emerging cyber threats in IoT. Electronics. 2021; 10 (8):16. doi: 10.3390/electronics10080918. [ CrossRef ] [ Google Scholar ]
• Johnson P, Gorton D, Lagerstrom R, Ekstedt M. Time between vulnerability disclosures: A measure of software product vulnerability. Computers & Security. 2016; 62 :278–295. doi: 10.1016/j.cose.2016.08.004. [ CrossRef ] [ Google Scholar ]
• Johnson P, Lagerström R, Ekstedt M, Franke U. Can the common vulnerability scoring system be trusted? A Bayesian analysis. IEEE Transactions on Dependable and Secure Computing. 2018; 15 (6):1002–1015. doi: 10.1109/TDSC.2016.2644614. [ CrossRef ] [ Google Scholar ]
• Junger Marianne, Wang Victoria, Schlömer Marleen. Fraud against businesses both online and offline: Crime scripts, business characteristics, efforts, and benefits. Crime Science. 2020; 9 (1):13. doi: 10.1186/s40163-020-00119-4. [ CrossRef ] [ Google Scholar ]
• Kalutarage Harsha Kumara, Nguyen Hoang Nga, Shaikh Siraj Ahmed. Towards a threat assessment framework for apps collusion. Telecommunication Systems. 2017; 66 (3):417–430. doi: 10.1007/s11235-017-0296-1. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kamarudin MH, Maple C, Watson T, Safa NS. A LogitBoost-based algorithm for detecting known and unknown web attacks. IEEE Access. 2017; 5 :26190–26200. doi: 10.1109/ACCESS.2017.2766844. [ CrossRef ] [ Google Scholar ]
• Kasongo SM, Sun YX. A deep learning method with wrapper based feature extraction for wireless intrusion detection system. Computers & Security. 2020; 92 :15. doi: 10.1016/j.cose.2020.101752. [ CrossRef ] [ Google Scholar ]
• Keserwani Pankaj Kumar, Govil Mahesh Chandra, Pilli Emmanuel S, Govil Prajjval. A smart anomaly-based intrusion detection system for the Internet of Things (IoT) network using GWO–PSO–RF model. Journal of Reliable Intelligent Environments. 2021; 7 (1):3–21. doi: 10.1007/s40860-020-00126-x. [ CrossRef ] [ Google Scholar ]
• Keshk M, Sitnikova E, Moustafa N, Hu J, Khalil I. An integrated framework for privacy-preserving based anomaly detection for cyber-physical systems. IEEE Transactions on Sustainable Computing. 2021; 6 (1):66–79. doi: 10.1109/TSUSC.2019.2906657. [ CrossRef ] [ Google Scholar ]
• Khan IA, Pi DC, Bhatia AK, Khan N, Haider W, Wahab A. Generating realistic IoT-based IDS dataset centred on fuzzy qualitative modelling for cyber-physical systems. Electronics Letters. 2020; 56 (9):441–443. doi: 10.1049/el.2019.4158. [ CrossRef ] [ Google Scholar ]
• Khraisat A, Gondal I, Vamplew P, Kamruzzaman J, Alazab A. Hybrid intrusion detection system based on the stacking ensemble of C5 decision tree classifier and one class support vector machine. Electronics. 2020; 9 (1):18. doi: 10.3390/electronics9010173. [ CrossRef ] [ Google Scholar ]
• Khraisat Ansam, Gondal Iqbal, Vamplew Peter, Kamruzzaman Joarder. Survey of intrusion detection systems: Techniques, datasets and challenges. Cybersecurity. 2019; 2 (1):20. doi: 10.1186/s42400-019-0038-7. [ CrossRef ] [ Google Scholar ]
• Kilincer IF, Ertam F, Sengur A. Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Computer Networks. 2021; 188 :16. doi: 10.1016/j.comnet.2021.107840. [ CrossRef ] [ Google Scholar ]
• Kim D, Kim HK. Automated dataset generation system for collaborative research of cyber threat analysis. Security and Communication Networks. 2019; 2019 :10. doi: 10.1155/2019/6268476. [ CrossRef ] [ Google Scholar ]
• Kim Gyeongmin, Lee Chanhee, Jo Jaechoon, Lim Heuiseok. Automatic extraction of named entities of cyber threats using a deep Bi-LSTM-CRF network. International Journal of Machine Learning and Cybernetics. 2020; 11 (10):2341–2355. doi: 10.1007/s13042-020-01122-6. [ CrossRef ] [ Google Scholar ]
• Kirubavathi G, Anitha R. Botnet detection via mining of traffic flow characteristics. Computers & Electrical Engineering. 2016; 50 :91–101. doi: 10.1016/j.compeleceng.2016.01.012. [ CrossRef ] [ Google Scholar ]
• Kiwia D, Dehghantanha A, Choo KKR, Slaughter J. A cyber kill chain based taxonomy of banking Trojans for evolutionary computational intelligence. Journal of Computational Science. 2018; 27 :394–409. doi: 10.1016/j.jocs.2017.10.020. [ CrossRef ] [ Google Scholar ]
• Koroniotis N, Moustafa N, Sitnikova E. A new network forensic framework based on deep learning for Internet of Things networks: A particle deep framework. Future Generation Computer Systems. 2020; 110 :91–106. doi: 10.1016/j.future.2020.03.042. [ CrossRef ] [ Google Scholar ]
• Kruse Clemens Scott, Frederick Benjamin, Jacobson Taylor, Kyle Monticone D. Cybersecurity in healthcare: A systematic review of modern threats and trends. Technology and Health Care. 2017; 25 (1):1–10. doi: 10.3233/THC-161263. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kshetri N. The economics of cyber-insurance. IT Professional. 2018; 20 (6):9–14. doi: 10.1109/MITP.2018.2874210. [ CrossRef ] [ Google Scholar ]
• Kumar R, Kumar P, Tripathi R, Gupta GP, Gadekallu TR, Srivastava G. SP2F: A secured privacy-preserving framework for smart agricultural Unmanned Aerial Vehicles. Computer Networks. 2021 doi: 10.1016/j.comnet.2021.107819. [ CrossRef ] [ Google Scholar ]
• Kumar R, Tripathi R. DBTP2SF: A deep blockchain-based trustworthy privacy-preserving secured framework in industrial internet of things systems. Transactions on Emerging Telecommunications Technologies. 2021; 32 (4):27. doi: 10.1002/ett.4222. [ CrossRef ] [ Google Scholar ]
• Laso PM, Brosset D, Puentes J. Dataset of anomalies and malicious acts in a cyber-physical subsystem. Data in Brief. 2017; 14 :186–191. doi: 10.1016/j.dib.2017.07.038. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lee J, Kim J, Kim I, Han K. Cyber threat detection based on artificial neural networks using event profiles. IEEE Access. 2019; 7 :165607–165626. doi: 10.1109/ACCESS.2019.2953095. [ CrossRef ] [ Google Scholar ]
• Lee SJ, Yoo PD, Asyhari AT, Jhi Y, Chermak L, Yeun CY, Taha K. IMPACT: Impersonation attack detection via edge computing using deep Autoencoder and feature abstraction. IEEE Access. 2020; 8 :65520–65529. doi: 10.1109/ACCESS.2020.2985089. [ CrossRef ] [ Google Scholar ]
• Leong Yin-Yee, Chen Yen-Chih. Cyber risk cost and management in IoT devices-linked health insurance. The Geneva Papers on Risk and Insurance—Issues and Practice. 2020; 45 (4):737–759. doi: 10.1057/s41288-020-00169-4. [ CrossRef ] [ Google Scholar ]
• Levi, M. 2017. Assessing the trends, scale and nature of economic cybercrimes: overview and Issues: In Cybercrimes, cybercriminals and their policing, in crime, law and social change. Crime, Law and Social Change 67 (1): 3–20. 10.1007/s10611-016-9645-3.
• Li C, Mills K, Niu D, Zhu R, Zhang H, Kinawi H. Android malware detection based on factorization machine. IEEE Access. 2019; 7 :184008–184019. doi: 10.1109/ACCESS.2019.2958927. [ CrossRef ] [ Google Scholar ]
• Li DQ, Li QM. Adversarial deep ensemble: evasion attacks and defenses for malware detection. IEEE Transactions on Information Forensics and Security. 2020; 15 :3886–3900. doi: 10.1109/tifs.2020.3003571. [ CrossRef ] [ Google Scholar ]
• Li DQ, Li QM, Ye YF, Xu SH. A framework for enhancing deep neural networks against adversarial malware. IEEE Transactions on Network Science and Engineering. 2021; 8 (1):736–750. doi: 10.1109/tnse.2021.3051354. [ CrossRef ] [ Google Scholar ]
• Li RH, Zhang C, Feng C, Zhang X, Tang CJ. Locating vulnerability in binaries using deep neural networks. IEEE Access. 2019; 7 :134660–134676. doi: 10.1109/access.2019.2942043. [ CrossRef ] [ Google Scholar ]
• Li X, Xu M, Vijayakumar P, Kumar N, Liu X. Detection of low-frequency and multi-stage attacks in industrial Internet of Things. IEEE Transactions on Vehicular Technology. 2020; 69 (8):8820–8831. doi: 10.1109/TVT.2020.2995133. [ CrossRef ] [ Google Scholar ]
• Liu HY, Lang B. Machine learning and deep learning methods for intrusion detection systems: A survey. Applied Sciences—Basel. 2019; 9 (20):28. doi: 10.3390/app9204396. [ CrossRef ] [ Google Scholar ]
• Lopez-Martin M, Carro B, Sanchez-Esguevillas A. Application of deep reinforcement learning to intrusion detection for supervised problems. Expert Systems with Applications. 2020 doi: 10.1016/j.eswa.2019.112963. [ CrossRef ] [ Google Scholar ]
• Loukas G, Gan D, Vuong Tuan. A review of cyber threats and defence approaches in emergency management. Future Internet. 2013; 5 :205–236. doi: 10.3390/fi5020205. [ CrossRef ] [ Google Scholar ]
• Luo CC, Su S, Sun YB, Tan QJ, Han M, Tian ZH. A convolution-based system for malicious URLs detection. CMC—Computers Materials Continua. 2020; 62 (1):399–411. doi: 10.32604/cmc.2020.06507. [ CrossRef ] [ Google Scholar ]
• Mahbooba B, Timilsina M, Sahal R, Serrano M. Explainable artificial intelligence (XAI) to enhance trust management in intrusion detection systems using decision tree model. Complexity. 2021; 2021 :11. doi: 10.1155/2021/6634811. [ CrossRef ] [ Google Scholar ]
• Mahdavifar S, Ghorbani AA. DeNNeS: Deep embedded neural network expert system for detecting cyber attacks. Neural Computing & Applications. 2020; 32 (18):14753–14780. doi: 10.1007/s00521-020-04830-w. [ CrossRef ] [ Google Scholar ]
• Mahfouz A, Abuhussein A, Venugopal D, Shiva S. Ensemble classifiers for network intrusion detection using a novel network attack dataset. Future Internet. 2020; 12 (11):1–19. doi: 10.3390/fi12110180. [ CrossRef ] [ Google Scholar ]
• Maleks Smith, Z., E. Lostri, and J.A. Lewis. 2020. The hidden costs of cybercrime. https://www.mcafee.com/enterprise/en-us/assets/reports/rp-hidden-costs-of-cybercrime.pdf . Accessed 16 May 2021.
• Malik J, Akhunzada A, Bibi I, Imran M, Musaddiq A, Kim SW. Hybrid deep learning: An efficient reconnaissance and surveillance detection mechanism in SDN. IEEE Access. 2020; 8 :134695–134706. doi: 10.1109/ACCESS.2020.3009849. [ CrossRef ] [ Google Scholar ]
• Manimurugan S. IoT-Fog-Cloud model for anomaly detection using improved Naive Bayes and principal component analysis. Journal of Ambient Intelligence and Humanized Computing. 2020 doi: 10.1007/s12652-020-02723-3. [ CrossRef ] [ Google Scholar ]
• Martin A, Lara-Cabrera R, Camacho D. Android malware detection through hybrid features fusion and ensemble classifiers: The AndroPyTool framework and the OmniDroid dataset. Information Fusion. 2019; 52 :128–142. doi: 10.1016/j.inffus.2018.12.006. [ CrossRef ] [ Google Scholar ]
• Mauro MD, Galatro G, Liotta A. Experimental review of neural-based approaches for network intrusion management. IEEE Transactions on Network and Service Management. 2020; 17 (4):2480–2495. doi: 10.1109/TNSM.2020.3024225. [ CrossRef ] [ Google Scholar ]
• McLeod A, Dolezel D. Cyber-analytics: Modeling factors associated with healthcare data breaches. Decision Support Systems. 2018; 108 :57–68. doi: 10.1016/j.dss.2018.02.007. [ CrossRef ] [ Google Scholar ]
• Meira J, Andrade R, Praca I, Carneiro J, Bolon-Canedo V, Alonso-Betanzos A, Marreiros G. Performance evaluation of unsupervised techniques in cyber-attack anomaly detection. Journal of Ambient Intelligence and Humanized Computing. 2020; 11 (11):4477–4489. doi: 10.1007/s12652-019-01417-9. [ CrossRef ] [ Google Scholar ]
• Miao Y, Ma J, Liu X, Weng J, Li H, Li H. Lightweight fine-grained search over encrypted data in Fog computing. IEEE Transactions on Services Computing. 2019; 12 (5):772–785. doi: 10.1109/TSC.2018.2823309. [ CrossRef ] [ Google Scholar ]
• Miller, C., and C. Valasek. 2015. Remote exploitation of an unaltered passenger vehicle. Black Hat USA 2015 (S 91).
• Mireles JD, Ficke E, Cho JH, Hurley P, Xu SH. Metrics towards measuring cyber agility. IEEE Transactions on Information Forensics and Security. 2019; 14 (12):3217–3232. doi: 10.1109/tifs.2019.2912551. [ CrossRef ] [ Google Scholar ]
• Mishra N, Pandya S. Internet of Things applications, security challenges, attacks, intrusion detection, and future visions: A systematic review. IEEE Access. 2021 doi: 10.1109/ACCESS.2021.3073408. [ CrossRef ] [ Google Scholar ]
• Monshizadeh M, Khatri V, Atli BG, Kantola R, Yan Z. Performance evaluation of a combined anomaly detection platform. IEEE Access. 2019; 7 :100964–100978. doi: 10.1109/ACCESS.2019.2930832. [ CrossRef ] [ Google Scholar ]
• Moreno VC, Reniers G, Salzano E, Cozzani V. Analysis of physical and cyber security-related events in the chemical and process industry. Process Safety and Environmental Protection. 2018; 116 :621–631. doi: 10.1016/j.psep.2018.03.026. [ CrossRef ] [ Google Scholar ]
• Moro ED. Towards an economic cyber loss index for parametric cover based on IT security indicator: A preliminary analysis. Risks. 2020 doi: 10.3390/risks8020045. [ CrossRef ] [ Google Scholar ]
• Moustafa N, Adi E, Turnbull B, Hu J. A new threat intelligence scheme for safeguarding industry 4.0 systems. IEEE Access. 2018; 6 :32910–32924. doi: 10.1109/ACCESS.2018.2844794. [ CrossRef ] [ Google Scholar ]
• Moustakidis S, Karlsson P. A novel feature extraction methodology using Siamese convolutional neural networks for intrusion detection. Cybersecurity. 2020 doi: 10.1186/s42400-020-00056-4. [ CrossRef ] [ Google Scholar ]
• Mukhopadhyay Arunabha, Chatterjee Samir, Bagchi Kallol K, Kirs Peteer J, Shukla Girja K. Cyber Risk Assessment and Mitigation (CRAM) framework using Logit and Probit models for cyber insurance. Information Systems Frontiers. 2019; 21 (5):997–1018. doi: 10.1007/s10796-017-9808-5. [ CrossRef ] [ Google Scholar ]
• Murphey, H. 2021a. Biden signs executive order to strengthen US cyber security. https://www.ft.com/content/4d808359-b504-4014-85f6-68e7a2851bf1?accessToken=zwAAAXl0_ifgkc9NgINZtQRAFNOF9mjnooUb8Q.MEYCIQDw46SFWsMn1iyuz3kvgAmn6mxc0rIVfw10Lg1ovJSfJwIhAK2X2URzfSqHwIS7ddRCvSt2nGC2DcdoiDTG49-4TeEt&sharetype=gift?token=fbcd6323-1ecf-4fc3-b136-b5b0dd6a8756 . Accessed 7 May 2021.
• Murphey, H. 2021b. Millions of connected devices have security flaws, study shows. https://www.ft.com/content/0bf92003-926d-4dee-87d7-b01f7c3e9621?accessToken=zwAAAXnA7f2Ikc8L-SADkm1N7tOH17AffD6WIQ.MEQCIDjBuROvhmYV0Mx3iB0cEV7m5oND1uaCICxJu0mzxM0PAiBam98q9zfHiTB6hKGr1gGl0Azt85yazdpX9K5sI8se3Q&sharetype=gift?token=2538218d-77d9-4dd3-9649-3cb556a34e51 . Accessed 6 May 2021.
• Murugesan V, Shalinie M, Yang MH. Design and analysis of hybrid single packet IP traceback scheme. IET Networks. 2018; 7 (3):141–151. doi: 10.1049/iet-net.2017.0115. [ CrossRef ] [ Google Scholar ]
• Mwitondi KS, Zargari SA. An iterative multiple sampling method for intrusion detection. Information Security Journal. 2018; 27 (4):230–239. doi: 10.1080/19393555.2018.1539790. [ CrossRef ] [ Google Scholar ]
• Neto NN, Madnick S, De Paula AMG, Borges NM. Developing a global data breach database and the challenges encountered. ACM Journal of Data and Information Quality. 2021; 13 (1):33. doi: 10.1145/3439873. [ CrossRef ] [ Google Scholar ]
• Nurse, J.R.C., L. Axon, A. Erola, I. Agrafiotis, M. Goldsmith, and S. Creese. 2020. The data that drives cyber insurance: A study into the underwriting and claims processes. In 2020 International conference on cyber situational awareness, data analytics and assessment (CyberSA), 15–19 June 2020.
• Oliveira N, Praca I, Maia E, Sousa O. Intelligent cyber attack detection and classification for network-based intrusion detection systems. Applied Sciences—Basel. 2021; 11 (4):21. doi: 10.3390/app11041674. [ CrossRef ] [ Google Scholar ]
• Page Matthew J, McKenzie Joanne E, Bossuyt Patrick M, Boutron Isabelle, Hoffmann Tammy C, Mulrow Cynthia D, Shamseer Larissa, Tetzlaff Jennifer M, Akl Elie A, Brennan Sue E, Chou Roger, Glanville Julie, Grimshaw Jeremy M, Hróbjartsson Asbjørn, Lalu Manoj M, Li Tianjing, Loder Elizabeth W, Mayo-Wilson Evan, McDonald Steve, McGuinness Luke A, Stewart Lesley A, Thomas James, Tricco Andrea C, Welch Vivian A, Whiting Penny, Moher David. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews. 2021; 10 (1):89. doi: 10.1186/s13643-021-01626-4. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pajouh HH, Javidan R, Khayami R, Dehghantanha A, Choo KR. A two-layer dimension reduction and two-tier classification model for anomaly-based intrusion detection in IoT backbone networks. IEEE Transactions on Emerging Topics in Computing. 2019; 7 (2):314–323. doi: 10.1109/TETC.2016.2633228. [ CrossRef ] [ Google Scholar ]
• Parra GD, Rad P, Choo KKR, Beebe N. Detecting Internet of Things attacks using distributed deep learning. Journal of Network and Computer Applications. 2020; 163 :13. doi: 10.1016/j.jnca.2020.102662. [ CrossRef ] [ Google Scholar ]
• Paté-Cornell ME, Kuypers M, Smith M, Keller P. Cyber risk management for critical infrastructure: A risk analysis model and three case studies. Risk Analysis. 2018; 38 (2):226–241. doi: 10.1111/risa.12844. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pooser, D.M., M.J. Browne, and O. Arkhangelska. 2018. Growth in the perception of cyber risk: evidence from U.S. P&C Insurers. The Geneva Papers on Risk and Insurance—Issues and Practice 43 (2): 208–223. 10.1057/s41288-017-0077-9.
• Pu, G., L. Wang, J. Shen, and F. Dong. 2021. A hybrid unsupervised clustering-based anomaly detection method. Tsinghua Science and Technology 26 (2): 146–153. 10.26599/TST.2019.9010051.
• Qiu J, Luo W, Pan L, Tai Y, Zhang J, Xiang Y. Predicting the impact of android malicious samples via machine learning. IEEE Access. 2019; 7 :66304–66316. doi: 10.1109/ACCESS.2019.2914311. [ CrossRef ] [ Google Scholar ]
• Qu X, Yang L, Guo K, Sun M, Ma L, Feng T, Ren S, Li K, Ma X. Direct batch growth hierarchical self-organizing mapping based on statistics for efficient network intrusion detection. IEEE Access. 2020; 8 :42251–42260. doi: 10.1109/ACCESS.2020.2976810. [ CrossRef ] [ Google Scholar ]
• Shafiur Rahman, Md, Sajal Halder Md, Uddin Ashraf, Acharjee Uzzal Kumar. An efficient hybrid system for anomaly detection in social networks. Cybersecurity. 2021; 4 (1):10. doi: 10.1186/s42400-021-00074-w. [ CrossRef ] [ Google Scholar ]
• Ramaiah M, Chandrasekaran V, Ravi V, Kumar N. An intrusion detection system using optimized deep neural network architecture. Transactions on Emerging Telecommunications Technologies. 2021; 32 (4):17. doi: 10.1002/ett.4221. [ CrossRef ] [ Google Scholar ]
• Raman, M.R.G., K. Kannan, S.K. Pal, and V.S.S. Sriram. 2016. Rough set-hypergraph-based feature selection approach for intrusion detection systems. Defence Science Journal 66 (6): 612–617. 10.14429/dsj.66.10802.
• Rathore, S., J.H. Park. 2018. Semi-supervised learning based distributed attack detection framework for IoT. Applied Soft Computing 72: 79–89. 10.1016/j.asoc.2018.05.049.
• Romanosky Sasha, Ablon Lillian, Kuehn Andreas, Jones Therese. Content analysis of cyber insurance policies: How do carriers price cyber risk? Journal of Cybersecurity (oxford) 2019; 5 (1):tyz002. [ Google Scholar ]
• Sarabi A, Naghizadeh P, Liu Y, Liu M. Risky business: Fine-grained data breach prediction using business profiles. Journal of Cybersecurity. 2016; 2 (1):15–28. doi: 10.1093/cybsec/tyw004. [ CrossRef ] [ Google Scholar ]
• Sardi Alberto, Rizzi Alessandro, Sorano Enrico, Guerrieri Anna. Cyber risk in health facilities: A systematic literature review. Sustainability. 2021; 12 (17):7002. doi: 10.3390/su12177002. [ CrossRef ] [ Google Scholar ]
• Sarker Iqbal H, Kayes ASM, Badsha Shahriar, Alqahtani Hamed, Watters Paul, Ng Alex. Cybersecurity data science: An overview from machine learning perspective. Journal of Big Data. 2020; 7 (1):41. doi: 10.1186/s40537-020-00318-5. [ CrossRef ] [ Google Scholar ]
• Scopus. 2021. Factsheet. https://www.elsevier.com/__data/assets/pdf_file/0017/114533/Scopus_GlobalResearch_Factsheet2019_FINAL_WEB.pdf . Accessed 11 May 2021.
• Sentuna A, Alsadoon A, Prasad PWC, Saadeh M, Alsadoon OH. A novel Enhanced Naïve Bayes Posterior Probability (ENBPP) using machine learning: Cyber threat analysis. Neural Processing Letters. 2021; 53 (1):177–209. doi: 10.1007/s11063-020-10381-x. [ CrossRef ] [ Google Scholar ]
• Shaukat K, Luo SH, Varadharajan V, Hameed IA, Chen S, Liu DX, Li JM. Performance comparison and current challenges of using machine learning techniques in cybersecurity. Energies. 2020; 13 (10):27. doi: 10.3390/en13102509. [ CrossRef ] [ Google Scholar ]
• Sheehan B, Murphy F, Mullins M, Ryan C. Connected and autonomous vehicles: A cyber-risk classification framework. Transportation Research Part a: Policy and Practice. 2019; 124 :523–536. doi: 10.1016/j.tra.2018.06.033. [ CrossRef ] [ Google Scholar ]
• Sheehan Barry, Murphy Finbarr, Kia Arash N, Kiely Ronan. A quantitative bow-tie cyber risk classification and assessment framework. Journal of Risk Research. 2021; 24 (12):1619–1638. doi: 10.1080/13669877.2021.1900337. [ CrossRef ] [ Google Scholar ]
• Shlomo A, Kalech M, Moskovitch R. Temporal pattern-based malicious activity detection in SCADA systems. Computers & Security. 2021; 102 :17. doi: 10.1016/j.cose.2020.102153. [ CrossRef ] [ Google Scholar ]
• Singh KJ, De T. Efficient classification of DDoS attacks using an ensemble feature selection algorithm. Journal of Intelligent Systems. 2020; 29 (1):71–83. doi: 10.1515/jisys-2017-0472. [ CrossRef ] [ Google Scholar ]
• Skrjanc I, Ozawa S, Ban T, Dovzan D. Large-scale cyber attacks monitoring using Evolving Cauchy Possibilistic Clustering. Applied Soft Computing. 2018; 62 :592–601. doi: 10.1016/j.asoc.2017.11.008. [ CrossRef ] [ Google Scholar ]
• Smart, W. 2018. Lessons learned review of the WannaCry Ransomware Cyber Attack. https://www.england.nhs.uk/wp-content/uploads/2018/02/lessons-learned-review-wannacry-ransomware-cyber-attack-cio-review.pdf . Accessed 7 May 2021.
• Sornette D, Maillart T, Kröger W. Exploring the limits of safety analysis in complex technological systems. International Journal of Disaster Risk Reduction. 2013; 6 :59–66. doi: 10.1016/j.ijdrr.2013.04.002. [ CrossRef ] [ Google Scholar ]
• Sovacool Benjamin K. The costs of failure: A preliminary assessment of major energy accidents, 1907–2007. Energy Policy. 2008; 36 (5):1802–1820. doi: 10.1016/j.enpol.2008.01.040. [ CrossRef ] [ Google Scholar ]
• SpringerLink. 2021. Journal Search. https://rd.springer.com/search?facet-content-type=%22Journal%22 . Accessed 11 May 2021.
• Stojanovic B, Hofer-Schmitz K, Kleb U. APT datasets and attack modeling for automated detection methods: A review. Computers & Security. 2020; 92 :19. doi: 10.1016/j.cose.2020.101734. [ CrossRef ] [ Google Scholar ]
• Subroto A, Apriyana A. Cyber risk prediction through social media big data analytics and statistical machine learning. Journal of Big Data. 2019 doi: 10.1186/s40537-019-0216-1. [ CrossRef ] [ Google Scholar ]
• Tan Z, Jamdagni A, He X, Nanda P, Liu RP, Hu J. Detection of denial-of-service attacks based on computer vision techniques. IEEE Transactions on Computers. 2015; 64 (9):2519–2533. doi: 10.1109/TC.2014.2375218. [ CrossRef ] [ Google Scholar ]
• Tidy, J. 2021. Irish cyber-attack: Hackers bail out Irish health service for free. https://www.bbc.com/news/world-europe-57197688 . Accessed 6 May 2021.
• Tuncer T, Ertam F, Dogan S. Automated malware recognition method based on local neighborhood binary pattern. Multimedia Tools and Applications. 2020; 79 (37–38):27815–27832. doi: 10.1007/s11042-020-09376-6. [ CrossRef ] [ Google Scholar ]
• Uhm Y, Pak W. Service-aware two-level partitioning for machine learning-based network intrusion detection with high performance and high scalability. IEEE Access. 2021; 9 :6608–6622. doi: 10.1109/ACCESS.2020.3048900. [ CrossRef ] [ Google Scholar ]
• Ulven JB, Wangen G. A systematic review of cybersecurity risks in higher education. Future Internet. 2021; 13 (2):1–40. doi: 10.3390/fi13020039. [ CrossRef ] [ Google Scholar ]
• Vaccari I, Chiola G, Aiello M, Mongelli M, Cambiaso E. MQTTset, a new dataset for machine learning techniques on MQTT. Sensors. 2020; 20 (22):17. doi: 10.3390/s20226578. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Valeriano B, Maness RC. The dynamics of cyber conflict between rival antagonists, 2001–11. Journal of Peace Research. 2014; 51 (3):347–360. doi: 10.1177/0022343313518940. [ CrossRef ] [ Google Scholar ]
• Varghese JE, Muniyal B. An Efficient IDS framework for DDoS attacks in SDN environment. IEEE Access. 2021; 9 :69680–69699. doi: 10.1109/ACCESS.2021.3078065. [ CrossRef ] [ Google Scholar ]
• Varsha M. V., Vinod P., Dhanya K. A. Identification of malicious android app using manifest and opcode features. Journal of Computer Virology and Hacking Techniques. 2017; 13 (2):125–138. doi: 10.1007/s11416-016-0277-z. [ CrossRef ] [ Google Scholar ]
• Velliangiri S, Pandey HM. Fuzzy-Taylor-elephant herd optimization inspired Deep Belief Network for DDoS attack detection and comparison with state-of-the-arts algorithms. Future Generation Computer Systems—the International Journal of Escience. 2020; 110 :80–90. doi: 10.1016/j.future.2020.03.049. [ CrossRef ] [ Google Scholar ]
• Verma A, Ranga V. Machine learning based intrusion detection systems for IoT applications. Wireless Personal Communications. 2020; 111 (4):2287–2310. doi: 10.1007/s11277-019-06986-8. [ CrossRef ] [ Google Scholar ]
• Vidros S, Kolias C, Kambourakis G, Akoglu L. Automatic detection of online recruitment frauds: Characteristics, methods, and a public dataset. Future Internet. 2017; 9 (1):19. doi: 10.3390/fi9010006. [ CrossRef ] [ Google Scholar ]
• Vinayakumar R, Alazab M, Soman KP, Poornachandran P, Al-Nemrat A, Venkatraman S. Deep learning approach for intelligent intrusion detection system. IEEE Access. 2019; 7 :41525–41550. doi: 10.1109/access.2019.2895334. [ CrossRef ] [ Google Scholar ]
• Walker-Roberts S, Hammoudeh M, Aldabbas O, Aydin M, Dehghantanha A. Threats on the horizon: Understanding security threats in the era of cyber-physical systems. Journal of Supercomputing. 2020; 76 (4):2643–2664. doi: 10.1007/s11227-019-03028-9. [ CrossRef ] [ Google Scholar ]
• Web of Science. 2021. Web of Science: Science Citation Index Expanded. https://clarivate.com/webofsciencegroup/solutions/webofscience-scie/ . Accessed 11 May 2021.
• World Economic Forum. 2020. WEF Global Risk Report. http://www3.weforum.org/docs/WEF_Global_Risk_Report_2020.pdf . Accessed 13 May 2020.
• Xin Y, Kong L, Liu Z, Chen Y, Li Y, Zhu H, Gao M, Hou H, Wang C. Machine learning and deep learning methods for cybersecurity. IEEE Access. 2018; 6 :35365–35381. doi: 10.1109/ACCESS.2018.2836950. [ CrossRef ] [ Google Scholar ]
• Xu, C., J. Zhang, K. Chang, and C. Long. 2013. Uncovering collusive spammers in Chinese review websites. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management.
• Yang J, Li T, Liang G, He W, Zhao Y. A Simple recurrent unit model based intrusion detection system with DCGAN. IEEE Access. 2019; 7 :83286–83296. doi: 10.1109/ACCESS.2019.2922692. [ CrossRef ] [ Google Scholar ]
• Yuan BG, Wang JF, Liu D, Guo W, Wu P, Bao XH. Byte-level malware classification based on Markov images and deep learning. Computers & Security. 2020; 92 :12. doi: 10.1016/j.cose.2020.101740. [ CrossRef ] [ Google Scholar ]
• Zhang S, Ou XM, Caragea D. Predicting cyber risks through national vulnerability database. Information Security Journal. 2015; 24 (4–6):194–206. doi: 10.1080/19393555.2015.1111961. [ CrossRef ] [ Google Scholar ]
• Zhang Ying, Li Peisong, Wang Xinheng. Intrusion detection for IoT based on improved genetic algorithm and deep belief network. IEEE Access. 2019; 7 :31711–31722. doi: 10.1109/ACCESS.2019.2903723. [ CrossRef ] [ Google Scholar ]
• Zheng, Muwei, Hannah Robbins, Zimo Chai, Prakash Thapa, and Tyler Moore. 2018. Cybersecurity research datasets: taxonomy and empirical analysis. In 11th {USENIX} workshop on cyber security experimentation and test ({CSET} 18).
• Zhou X, Liang W, Shimizu S, Ma J, Jin Q. Siamese neural network based few-shot learning for anomaly detection in industrial cyber-physical systems. IEEE Transactions on Industrial Informatics. 2021; 17 (8):5790–5798. doi: 10.1109/TII.2020.3047675. [ CrossRef ] [ Google Scholar ]
• Zhou YY, Cheng G, Jiang SQ, Dai M. Building an efficient intrusion detection system based on feature selection and ensemble classifier. Computer Networks. 2020; 174 :17. doi: 10.1016/j.comnet.2020.107247. [ CrossRef ] [ Google Scholar ]

• Search Menu
• Sign in through your institution
• Advance articles
• Editor's Choice
• Author Guidelines
• Submission Site
• Open Access
• About International Data Privacy Law
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

The rise of cybersecurity and its impact on data protection.

Editor-in-Chief.

Managing Editor.

• Article contents
• Figures & tables
• Supplementary Data

Christopher Kuner, Dan Jerker B. Svantesson, Fred H. Cate, Orla Lynskey, Christopher Millard, The rise of cybersecurity and its impact on data protection, International Data Privacy Law , Volume 7, Issue 2, May 2017, Pages 73–75, https://doi.org/10.1093/idpl/ipx009

• Permissions Icon Permissions

Cybersecurity is attracting more attention than ever—not just in headlines, but among policymakers, industry leaders, academics, and the public. Successful cyberattacks are becoming more frequent and threatening as adversaries become more determined, more sophisticated, and more likely to be connected with a nation state. No one and nothing seems safe. The May WannaCry ransomware attack affected more than 300,000 computers in 150 countries. The presidential elections in France and the United States (U.S.) have been the subject of major attacks, followed by strategically timed disclosures. Yahoo, in the midst of its sale to Verizon, reported that information of approximately 1.5 billion user accounts had been stolen. In the United States (U.S.), the NSA and the CIA appear to be haemorrhaging top secret documents apparently stolen by insiders, while the U.S. Office of Personnel Management was unable to protect 21.5 million records on government employees and contractors holding security clearances.

Part of the escalating attention to cybersecurity is the result of society’s growing reliance on digital systems to control important infrastructure, such as cars, airplanes, utilities, supply chains, and industrial systems. In 2010, for example, the U.S. and Israel reportedly cooperated in the development and use of Stuxnet, a software program that destroyed centrifuges critical to Iran’s nuclear weapons program by inferring with their control systems. Hackers used cyberattacks to temporarily shutter three power distribution companies in western Ukraine and operations at a Venezuelan oil unloading facility. In 2014, cyberattacks on a German iron plant caused widespread damage. In 2015, thieves stole $81 million by exploiting weak security at the Central Bank of Bangladesh to persuade the network that controls international transfers of money between banks to transfer the money from the Federal Reserve Bank of New York to the thieves’ accounts. The following year, the Mirai botnet exploited vulnerabilities in the Internet of Things devices to overwhelm the Dyn domain server, causing major Internet platforms and services to be unavailable in the U.S. and Europe. Enterprising security researchers have hacked insulin pumps, drones in flight, and cars on the road.

It is no wonder that cybersecurity is attracting more attention, but such attention raises important issues for personal privacy and the data protection tools we use to protect it. The relationship between security and data privacy has always been complicated. Privacy depends absolutely on security. No obligation to provide privacy, whether entered into voluntarily or compelled by law, will be meaningful if the data to be protected are accessed or stolen by unauthorized third parties. As a result, all modern data protection principles include an obligation to protect security as well. For example, the influential 1980 Guidelines on the Protection of Privacy and Transborder Flows of Personal Data , adopted by the Committee of Ministers of the Organization for Economic Cooperation and Development (OECD) in 1980, included the Security Safeguards Principle as one of the eight foundational principles of data protection: ‘Personal data should be protected by reasonable security safeguards against such risks as loss or unauthorised access, destruction, use, modification or disclosure of data.’ This principle was retained in the 2013 revision of the Guidelines (the OECD Privacy Framework), and supplemented by additional security-related language covering data breaches. And security has been recognized in every significant codification of data protection law since then, including the EU Data Protection Directive, the U.S. Federal Trade Commission’s fair information practice principles, the APEC Privacy Framework, and the EU General Data Protection Regulation.

Data privacy and cybersecurity are often advanced by common tools, such as encryption, data minimization, and limits on collecting, retaining, and transferring personal data. In short, what is good for privacy is often good for security as well.

But this is not always the case. Despite the foundational importance of information security for modern data protection and the considerable overlap between many tools for protecting privacy and security, privacy and security are often in tension as well. Many measures employed to enhance cybersecurity pose a risk to privacy. For example, proposals to enhance cybersecurity by requiring identity verification, reducing online anonymity, and sharing potentially personal information about cyberattacks all pose risks for personal privacy. This tension is more than theoretical: concerns about the privacy impact of proposed data sharing legislation in the U.S. led to widespread protests online and offline, delayed its passage for more than four years, and resulted in a substantially weakened final law.

The commitment of an ever-increasing portion of scarce resources in industry and government to addressing cybersecurity challenges threatens to diminish investment in data protection. This is not just a matter of money. Institutions only have so much bandwidth, and as more time, attention, and resources are focused on enhancing security, privacy runs the risk of being shortchanged.

Data protection officials and practitioners often face a Hobson’s choice of leaving information security (and the resources that go with it) to others or adding information security to their portfolios, at the risk of diminishing their attention to privacy.

Historically, when security and privacy priorities have competed head-on, privacy is lost. We see evidence of this following major terrorist attacks, when national governments consider and, in many cases, adopt private-restrictive measures based on the premise that it is necessary to sacrifice a little privacy in the cause of greater security. This bargain rarely proves productive, yet we run the risk of repeating it in the context of measures designed to enhance cybersecurity.

Privacy is deeply rooted in human rights principles and law; cybersecurity historically has not been. A greater focus on cybersecurity runs the risk of diminishing the individual and human rights components of data protection law.

Many data protection professionals in industry and government have historically lacked training or experience in computer science or other technologies. Fortunately, this is beginning to change. However, pressure to focus more attention on cybersecurity issues runs the risk of concentrating too much on technology and neglecting other important skills, to the detriment of both privacy and security.

By drawing attention to the challenges of information governance broadly, the growing focus on cybersecurity may lead to increased funding and other resources for privacy work as well. This is especially true because security is so integral to privacy and public acceptance of new security measures often depends, at least in part, on the degree to which those measures protect privacy.

Enhanced attention to information security, and especially the sense of urgency with which these threats must be addressed, may lead not only to more attention being given to privacy as well, but also to greater insistence that data protection tools, like cybersecurity tools, adapt and change more readily to the challenges of the 21st century. Data protection law has rarely been thought nimble; pressure to deal with cybersecurity may help change that.

The importance of technological skills for cybersecurity professionals may intensify the movement towards more data protection professionals trained in technologies as well. At the same time, the broader range of disciplines traditionally applied to privacy may help facilitate a much-needed expansion of cybersecurity competencies as well. After all, the vast majority of successful cyberattacks involves human or institutional failures, so greater attention to human and institutional behaviour, training, incentives, and risk management is key to enhancing cybersecurity, being applied to privacy.

The human rights foundations of data protection law could benefit efforts to improve cybersecurity as well. For years, many institutions calculated the ‘cost’ of information security breaches only in terms of the losses suffered by the institution. A greater understanding that information security, as a component of data protection, is not just a financial obligation, but a human rights obligation might contribute to a broader accounting of the harms that may be caused by breaches and the range of parties who may be injured.

Civilization needs better protection for cybersecurity—far better than we have seen to date–urgently, but it also needs better data protection. The significance of the possible effects on data protection—both positive and negative—of the increased attention being paid to cybersecurity suggests that privacy professionals in government, industry, civil society, and academia should, at a minimum, be paying close attention to the emergence of cybersecurity. Even better would be to think constructively and proactively about how to take advantage of this important development to ensure that people everywhere enjoy strong, effective protections for their privacy and for the security of their data.

Author notes

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 2044-4001
• Print ISSN 2044-3994
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

The Digital World Is Changing Rapidly. Your Cybersecurity Needs to Keep Up.

by Andrew Burt

Summary .   

In 2022 alone, a total of 4,100 publicly disclosed data breaches occurred, comprising some 22 billion records that were exposed. All this despite the fact that organizations around the world spent a record-breaking $150 billion on cybersecurity in 2021. Software itself is changing, too. The rise of artificial intelligence in general, and generative AI in particular, is fundamentally altering the way companies use software. The increasing use of AI is, in turn, making software’s attack surfaces more complicated and software itself more vulnerable. How, then, should companies go about securing their software and data? What companies aim to achieve from their security programs must evolve, just as the way that companies’ use of data and software has evolved. It is past time for their cybersecurity efforts to change. This article covers three such changes that companies can make to adapt to the growing insecurities of the digital world.

What is the point of cybersecurity?

Partner Center

Home / Resources / ISACA Journal / Issues / 2022 / Volume 3 / Better Cybersecurity Awareness Through Research

Better cybersecurity awareness through research.

In the last few years, information security professionals have faced tremendous challenges. Just in 2021, there were more than two billion malware attacks and trillions of intrusion attempts. 1 Ransomware attacks alone have increased by 151 percent compared with 2020. 2 In fall of 2020, Cybersecurity Ventures estimated worldwide cybercrime costs would reach US$6 trillion annually by the end of 2021, ransomware damage costs would rise to US$20 billion, and an enterprise would fall victim to a ransomware attack every 11 seconds during the year. 3 The European Union introduced 474 separate enforcement actions for EU General Data Protection Regulations (GDPR) violations, starting from the time enforcement of the GDPR began in 2018 through December 2020, with fines totaling US$312.4 million. 4

Cybercrime is projected to worsen due to the rapid changes resulting from the COVID-19 pandemic. Thousands of organizations allowed employees to continue working from home throughout 2021, and there are indications that many will permit hybrid work indefinitely. Remote work opens a Pandora’s box of issues for organizations trying to maintain some semblance of security. Cybersecurity firm Malwarebytes reported in an August 2020 survey that remote workers caused security breaches in 20 percent of the organizations it surveyed. 5 Although the need for security awareness training for remote employees is pressing, many organizations have been finding it more difficult to implement than providing established training in a centralized workplace.

Importance of Awareness and Training

One study conducted with participation from more than 5,000 organizations around the world discovered that organizations are becoming more aware of the role of their employees play in information security incidents. Survey data collected as part of the study reported that 52 percent of organizations indicated employees were their biggest weakness in IT security, with their actions putting the business and the organizational information security strategy at risk. Forty-three percent of the organizations polled considered deployment of more sophisticated software an effective way to safeguard themselves against evolving threats ( figure 1 ). Offering staff training was the second most popular method for safeguarding organizations according to the survey, closely followed by increasing internal IT or IT security staff. 6

Similar to those findings, Verizon’s 2021 Data Breach Investigations Report states that nearly 85 percent of incidents and data breaches from 2020 were attributable to human error. 7  Additional data from the field comes from the Willis Towers Watson Cyber Claims Analysis Report , 8  which reveals that clients filed close to 1,200 data breach claims in nearly 50 countries from 2013 to December 2019. The report identifies human error such as employees clicking on links in phishing emails or replying to spoofed emails as the most common root causes of breaches (figure 2) . The costliest events were typically those where the threat actor impersonated a chief executive officer (CEO) or senior manager. The most frequently employed social engineering tactic was impersonation of a vendor or supplier. All these things could easily be prevented through employee education and training, the report concludes. 9

According to a white paper from Osterman Research, employees who received cybersecurity training demonstrated a significantly improved ability to recognize potential threats, earning the respect of their organization’s security teams. 10 By applying a model that Osterman developed to data acquired through a survey of 230 individuals in North American organizations, the researchers concluded that smaller organizations could achieve a return on investment (ROI) of nearly 70 percent and larger organizations could achieve an ROI of 500 percent, on average, by implementing security awareness training. 11

Deficiencies in Current Learning Techniques

The importance of cybersecurity awareness is underscored by reports of incidents attributed to careless human behavior and lack of training, which continue to rise at alarming rates, despite commitments from small and large organizations to increase staffing in information security support groups and expand cybersecurity technology budgets. Many organizations either underestimate the effort needed to educate a workforce or do not realize that their current cybersecurity training approaches are ineffective.

KnowBe4’s 2021 State of Privacy and Security Awareness Report notes that a large percentage of surveyed employees did not feel confident that they could identify a social engineering attack, recognize the warning signs that their computers were infected with malware or describe to their senior management the security risk associated with employees working from home. 12 Government, healthcare and education employees were the least aware of various social engineering threats.

Much of the current literature and research on improving cybersecurity awareness training is focused on how to develop an effective program or how to identify the components missing from a program. This is a great start, but it is not enough. For example, the 2021 SANS Security Awareness Report: Managing Human Cyber Risk 13 identifies which needs to prioritize while building an effective program, such as having several full-time employees focused on changing behavior, providing job titles commensurate with responsibilities, ensuring leadership support, fostering collaboration among departments and engaging people with specialized communication skills to strike the right balance by not being too technical or lengthy with organizational messaging.

Conspicuously missing from many current reports, including the ones already mentioned, are inquiries into whether both the training material content and its delivery are inherently flawed. Something is lacking in the current environment. Could it be related to the technique, or lack of it, in delivering cyberawareness material within organizations? Are some approaches more effective? Do people learn, absorb and remember better when material is presented a certain way?

For example, the SANS Security Awareness Maturity Model (figure 3) gives organizations the ability to compare and contrast the maturity level of their security awareness program and helps them focus on areas that need improvement. 14  However, the model could be made even more helpful if it included references to how organizations can apply research on how humans learn and the most effective methods of content delivery. Organizations that do this should be rated higher on the maturity scale.

Many organizations either underestimate the effort needed to educate a workforce or do not realize that their current cybersecurity training approaches are ineffective.

Models and frameworks are a great start, but developing a structured awareness program with tools for metrics monitoring only solves part of the puzzle. Organizations also need research-based information on how to create inspiring content, along with techniques for delivering it effectively.

Better Ways to Learn

Review of the literature on how humans learn and retain information reveals interesting techniques and practices that are applicable to cybersecurity awareness and training campaigns. Organizations looking to maximize return on investment when it comes to cyberawareness would do well to take a closer look at adopting some of the following into their own learning systems.

Distributed vs. Massed Practice There may be benefits to moving away from training assignments that offer a single, continuous training session. Offering an initial fact-sharing or concept-learning session followed by periodic reviews may be a better approach. Research indicates that providing the same information after the initial session in smaller chunks and at a carefully chosen frequency reinforces learning. 15  Short, spaced-out study sessions lead to meaningful learning, whereas cram sessions often lead to nothing deeper than memorization.

In a 2019 interview, a researcher at Dartmouth College, Hanover, New Hampshire, USA, said that studying information or practicing a task just once is not good enough. 16 For permanent learning, the timing of the review or practice of the information is critical. Distributed practice refers to studying the material to be learned at a specified time after the original learning event. Massed practice, on the other hand, refers to study sessions that happen right after the original learning event.

Research shows that distributed lessons improved elementary school children’s ability to generalize their learning 17 and that when faced with unique situations, college students who participated in a spaced review following the primary lecture adapted better than students who just received a massed online review. 18

Conspicuously missing from many current reports… are inquiries into whether both the training material content and its delivery are inherently flawed.

Distributed reviews of the same information make the technique effective. It is not the same as spreading different chunks of material over several sessions or days. Organizations that comprehend the advantages of distributed practice over massed practice may choose to drop the idea of holding annual or quarterly cybersecurity awareness training events—that is, cram sessions—that take all day or multiple days, usually with exercise sessions on learning objectives at the end. A better approach may be to opt for short sessions that introduce the idea of a single cybersecurity concept, say phishing, followed by defined, specific breaks of days or weeks before the same concept is revisited in the form of multiple follow-up sessions or a set of spaced-out exercises emailed to the participants. The examples in figure 4 illustrate the differences between the two approaches.

Massed practice, or the boot camp approach, may work to some extent for achieving a short-term goal such as passing an exam. However, for employees to achieve long-term retention of learned concepts—something that is of vital importance to organizations when it comes to cybersecurity—distributed practice is a superior method of learning.

Reconsolidation Tweaking a distributed practice approach to include memory reconsolidation can make training more effective. Although distributed practice involves presenting the same information, a small alteration of facts or measures reinforces the training. The key is to combine distributed practice with subtle changes in the follow-ups.

Making slight changes to the study material or task during practice sessions may help trainees master a skill much more quickly than they would without alteration. The results of a study by researchers at Johns Hopkins University, Baltimore, Maryland, USA, lends credence to the theory of reconsolidation by showing that motor skills are strengthened when existing memories are recalled and modified with new knowledge. 19 The researchers found that the gains in performance such as speedier and more accurate task completion nearly doubled in the experimental group given an altered second session, compared to a group that repeated the same task without any change. The researchers concluded that a trainee learns more and learns faster by practicing a subtly altered version of a task than by practicing the same thing multiple times in a row. However, the changes in the training must be subtle because if the modification renders the task noticeably different, trainees do not realize the desired gain.

The science behind reconsolidation is still subject to debate, but results so far offer a glimpse of the possibilities for using it to strengthen learning. Consolidation refers to how the human brain learns new material. Retrieving that material after first exposure but before the learner has had a chance to forget it reactivates the learning process. It theoretically gives the learner an opportunity to weaken or strengthen memory retention. It appears possible to disrupt or impair retention by providing conflicting or incorrect information after an initial learning event or to strengthen it by providing correct information with slight updates. The updates might help to close gaps in the initial learning experience, thereby strengthening it.

For example, when using the distributed practice approach to teach employees about phishing, it might be beneficial to use the principles of reconsolidation in the follow-up sessions by subtly changing the phishing scenarios in the exercises. Each social engineering red flag ( figure 5 ) in the set might be used to introduce a slight variation to the initial learning session to reinforce the main learning subject of phishing.

The Value of Case Studies

Nothing raises cybersecurity awareness more effectively than showing the aftermath of real breaches in targeted enterprises and highlighting how vulnerable all organizations are to cyberthreats. One well-known and effective training technique is the use of case studies in problem-based learning (PBL) scenarios. The results of multiple studies indicate that PBL has a significant positive effect on students’ skill development and knowledge retention. 20 The US National Center for Case Study Teaching in Science polled more than a hundred university faculty members who had been trained to use case studies and found that more than 90 percent reported that students who were taught using sample cases learned new ways to think about issues and took an active part in the learning process. 21

Key characteristics of a good discussion case include being concise; being somewhat controversial to grab attention, but maintaining balance and not getting carried away; having memorable characters act out the case study with dialogue; ensuring that the subject material is relevant to the learners; presenting a dilemma to be solved; being contemporary rather than purely historical; using real rather than fabricated scenarios and having clear learning objectives.

Additional research expands on the qualities that make a good case study, 22 including being pertinent to the class and learning objectives, 23 being connected to theory and practice  24 , 25 , 26 and telling a story containing some form of ambiguity. 27

Information security officers struggling to sell internal leadership and stakeholders on abstract concepts such as segregation of duties (SoD), change management and other internal IT controls might find that case studies based on events in the news are effective tools to get their message across.

For example, a complete case study on the Bernard Madoff financial scam—complete with short news video clips and government press releases on IT personnel being charged with crimes—helped illustrate complex topics in an easy-to-understand format for the leadership at a not-for-profit and get its buy-in for implementing organizationwide checks and balances ( figure 6 ).

Learning From Incidents and Accidents Industrial accidents have been studied and analyzed over many decades, and learning from these incidents that inevitably occur in large systems—especially ones that pose a danger to human safety, such as those that occur with chemicals and other hazardous materials—has always been a top priority. By establishing a framework for learning from incidents, an organization can reduce risk and minimize loss and, thus, become a more reliable organization over time. Learning from incidents and accidents fosters a culture of continuous organizational improvement that will reduce incident severity and risk of disaster. Organizations that do not learn from past errors are doomed to repeat them, for example:

• The US National Aeronautics and Space Administration (NASA) lost two space shuttles, the Challenger in 1986 and the Columbia in 2003. The Columbia Accident Investigation Board noted that “[T]he causes of the institutional failure responsible for Challenger have not been fixed. Second, the Board strongly believes that if these persistent, systemic flaws are not resolved, the scene is set for another accident.” 28
• Failure to learn was among the causes for the Deep Water Horizon disaster. British Petroleum had experienced several major incidents before— specifically the Grangemouth refinery incident in Scotland, the Texas City refinery explosion in the US state of Texas, and the Prudhoe Bay leaks in the US state of Alaska. 29

Incidents, however, do not always have to end in disaster. 30 A system can be put in place to control their severity. It is important to recognize that, in most cases, a disaster results from a chain of events going undetected rather than from a standalone, spontaneous event. If an effective learning system could detect the incident, the chain of events could be broken and a disaster could be prevented.

By establishing a framework for learning from incidents, an organization can reduce risk and minimize loss and, thus, become a more reliable organization over time.

In addition to implementing systems that enable learning from incidents, technology organizations should investigate the use of mitigating systems, including a sort of kill switch that could potentially prevent an incident from becoming a disaster. One IT organization that suffered a series of ransomware events used data security software to study the data from the incidents and determined that all the attacks originated from end-user workstations and spread to network shares before showing up on assets of higher value. 31 A mitigating system using the data security platform was soon put in place so that ransomware activity detected at the end-user’s virtual local area network (VLAN) would automatically disable the user account the questionable activity was operating under and close ports to other parts of the network. Other examples of mitigating systems are automatic blocking of remote Internet Protocol (IP) addresses based on scanning activity detected by a security information and event management (SIEM) tool and automatic disabling of user accounts flagged as exhibiting unusual activity by other security tools.

In IT, especially cybersecurity, distilling the data collected from past incidents and accidents into actionable, effective training remains a challenge.

Organizations other than typical industrial factories have already adopted techniques to prevent incidents from descending into chaos and disaster. In the mid-1980s, researchers at the University of California, Berkeley, USA, began taking a closer look at why some organizations, despite their complex and risk-prone environments, continued to succeed in avoiding major safety incidents. 32 The term high reliability organizations (HRO) soon evolved to refer to this category of organizations. Researchers discovered that HROs use several tools and initiatives to learn from safety incidents, some of which could be adapted for use in healthcare, for example. 33 The researchers conducted a systematic review to identify effective learning tools that multidisciplinary teams in healthcare could adapt and use following a patient safety incident. IT-reliant organizations are on the cusp of facing major disasters; the attempted poisoning of city water in Oldsmar, Florida, USA, 34 and the Colonial Gas pipeline ransomware event are early warning signs. 35 If IT does not invest in learning systems championed by industries other than IT over the years, catastrophes will become inevitable.

Sometimes lessons are not learned. This can happen for various reasons, including the handling of information within and between organizations in a way that discourages dissemination of lessons learned from emergencies, training and educational programs, placing an unhealthy emphasis on what to learn rather than how to learn and ingrained organizational cultures preventing learning. 36

Recommendations for avoiding these traps include creating an official policy for identifying and learning lessons, developing techniques to identify and learn lessons from mock exercises and major emergencies, and engaging in an ongoing process of cross-training so that diverse teams can fully develop a broad understanding of how other teams think and operate under pressure. Data from the incident learning system can be applied in future training sessions. 37

Safety-conscious industrial organizations, often under the watchful eye of regulators, have long prioritized collecting data about accidents and feeding them into learning systems that are then converted to simulations used in training. Although the typical IT organization may not have as many situations that could affect human safety as other industry sectors, IT in general is known for having a culture of investigating failures and conducting root-cause analyses. However, in IT, especially cybersecurity, distilling the data collected from past incidents and accidents into actionable, effective training remains a challenge.

For example, one organization responded to the challenge by converting root-cause analysis of its actual cybersecurity incidents into animated training simulations that were then emailed out as quarterly or annual retrospectives (figure 7) . IT personnel received a more detailed tear-down of the incidents using the same animation medium.

Despite investing in training, technology and technical know-how and expanding their budgets to support ever-increasing information security operational costs, organizations are still falling victim to cyberattacks every day. These attacks show no sign of slowing down. Checkpoint research reported 900 weekly attacks per organization in 2021, a 50 percent jump compared with 2020. 38

Organization leaders need to realize that cybersecurity awareness is not just about training nontechnical employees about phishing and online scams and then arming them with better security practices. Cybersecurity awareness also plays a vital role in changing an organization’s culture for the better by changing certain behaviors (e.g., leaky change management practices; arming people with the data to make better decisions; providing the confidence and means to challenge unethical behavior, such as a senior manager falsifying disaster-recovery test results). The server left unpatched, the files left unprotected and the critical security feature, which took nine months to roll out, reflect organizational cultures that are in dire need of reform.

To create lasting change, organizations must not only build the components of an effective cyberawareness program, but also improve the quality of their content and the mechanisms for its delivery. Informed by research concerning how humans learn, organizations can adopt the most effective techniques to aid the developers of their training systems. With the right tools, developers can tailor content that improves employees’ learning speed and information retention and help employees adapt quickly to changing environments and situations, even during times of heavy workload and high pressure.

Author’s Note

The information and views expressed in this article are those of the author and do not constitute any official position, policy or pronouncement of his employer.

1 SonicWall, 2021  SonicWall Cyber Threat Report , USA, 2021,  https://www.sonicwall.com/resources/white-papers/2021-sonicwall-cyber-threat-report/ 2 Ibid. 3 Morgan, S.; “Cybercrime to Cost the World $10.5 Trillion Annually by 2025,”  Cybercrime Magazine , 13 November 2020, https://cybersecurityventures.com/hackerpocalypse-original-cybercrime-report-2016/ 4 KnowBe4 , 2021 State of Privacy and Security Awareness Report , USA, 2021, https://www.knowbe4.com/hubfs/2021-State-of-Privacy-Security-Awareness-Report-Research_EN-US.pdf 5 Malwarebytes, Enduring From Home: COVID-19’s Impact on Business Security , USA, 2020, https://www.malwarebytes.com/resources/files/2020/08/malwarebytes_enduringfromhome_report_final.pdf 6 Kaspersky, “The Human Factor in IT Security: How Employees Are Making Businesses Vulnerable From Within,” Kaspersky Daily, https://www.kaspersky.com/blog/the-human-factor-in-it-security/ 7 Verizon, 2021 Data Breach Investigations Report , USA, 2021, https://enterprise.verizon.com/content/verizon-enterprise/us/en/index/resources/reports/2021-data-breach-investigations-report.pdf 8 Willis Towers Watson, Cyber Claims Analysis Report , United Kingdom, 2020, https://www.wtwco.com/en-NZ/Insights/2020/07/cyber-claims-analysis-report 9 Ibid. 10 Osterman Research, Inc., The ROI of Security Awareness Training , USA, August 2019, https://www.mimecast.com/resources/analyst-reports/osterman-research---the-roi-of-security-awareness-training/ 11 Ibid. 12 Op cit KnowBe4 13 DeBeaubien, ; L. Spitzner; H. Xu; N. Zhang; 2021 SANS Security Awareness Report: Managing Human Cyber Risk , USA, 2021, https://www.sans.org/security-awareness-training/resources/reports/sareport-2021/ 14 SANS, “Measuring Program Maturity,” https://www.sans.org/security-awareness-training/resources/maturity-model/ 15 Carpenter, K.; N. J. Cepeda; D. Rohrer;H. K. Kang; H. Pashler; “Using Spacing to Enhance Diverse Forms of Learning: Review of Recent Research and Implications for Instruction,” Educational Psychology Review , vol. 24, iss. 3, http://www.jstor.org/stable/43546797 16 Francisco, A.; “Ask the Cognitive Scientist: Distributed Practice,” Digital Promise, 8 May 2019, https://digitalpromise.org/2019/05/08/ask-the-cognitive-scientist-distributed-practice/ 17 Vlach, H.; C. Sandhofer; “Distributing Learning Over Time: The Spacing Effect in Children’s Acquisition and Generalization of Science Concepts,” Child Development , 22 May 2012, https://ncbi.nlm.nih.gov/pmc/articles/PMC3399982/ncbi.nlm.nih.gov/pmc/articles/PMC3399982/ 18 Kapler, ; T. Weston; M. Wiseheart; “Spacing in a Simulated Undergraduate Classroom: Long-Term Benefits for Factual and Higher-Level Learning,” Learning and Instruction , April 2015, https://www.sciencedirect.com/science/article/abs/pii/S0959475214001042?via%3Dihub 19 Wymbs, N.; A. Bastian; P. Celnik; “Motor Skills Are Strengthened Through Reconsolidation,” Current Biology , 8 February 2016, https://www.sciencedirect.com/science/article/pii/S0960982215015146 20 Herreid, ; “Using Case Studies to Teach Science,” American Institute of Biological Sciences, 2005, https://files.eric.ed.gov/fulltext/ED501359.pdf 21 Herreid, ; “Case Studies in Science–A Novel Method of Science Education,” Journal of College Science Teaching , February 1994,  https://eric.ed.gov/?id=EJ487069 22 Anderson, ; “Teaching Developmental Theory With Interrupted Video Case Studies,” Journal of the Scholarship of Teaching and Learning , December 2019, https://scholarworks.iu.edu/journals/index.php/josotl/article/view/25385/3711 23 McFarlane, D.; “Guidelines for Using Case Studies in the Teaching-Learning Process,” College Quarterly , Winter 2015, https://files.eric.ed.gov/fulltext/EJ1070008.pdf 24 Anderson, B.; S. Bradshaw; J. Banning; Using Interrupted Video Case Studies to Teach Developmental Theory: A Pilot Study , Gauisus, 2016, https://sotl.illinoisstate.edu/downloads/gauisus/AndersonVolume4.pdf 25  Penn, ; C. Currie; K. Hoad; F. O’Brien; “The Use of Case Studies in OR Teaching,” Higher Education Pedagogies, 8 March 2016, www.tandfonline.com/doi/full/10.1080/23752696.2015.1134201 26 Prud’homme-Généreux, A.; “Case Study: Formulating Questions That Address Student Misconceptions in a Case Study,” Journal of College Science Teaching , March 2017, https://eric.ed.gov/?id=EJ1136640 27 Boston University Center for Teaching and Learning, Massachusetts, USA, “Using Case Studies to Teach,” https://www.bu.edu/ctl/teaching-resources/using-case-studies-to-teach/ 28 Columbia Accident Investigation Board, Report Volume I, USA, August 2003, http://s3.amazonaws.com/akamai.netstorage/nasa-global/CAIB/CAIB_lowres_full.pdf 29 Dechy, N.; J. Rousseau; F. Jeffroy; “Learning Lessons From Accidents With a Human and Organisational Factors Perspective: Deficiencies and Failures of Operating Experience Feedback Systems,” EUROSAFE Forum 2011, researchgate.net/publication/233997934 30 Cooke, ; T. Rohleder; “Learning From Incidents: From Normal Accidents to High Reliability,” System Dynamics Review , September 2006, https://onlinelibrary.wiley.com/doi/10.1002/sdr.338 31 Varonis, Varonis Case Study: City of San Diego, USA, https://info.varonis.com/hubfs/docs/case_studies/en/Varonis_Case_Study_San_Diego.pdf 32 Roberts, K. H.; “HRO Has Prominent History,” Anesthesia Patient Safety Foundation Newslette r, 18, iss. 1, Spring 2003, https://www.apsf.org/article/hro-has-prominent-history/ 33 Serou, ; L. Sahota; A. Husband; S. Forrest; Slight; S. Slight; “Learning From Safety Incidents in High-Reliability Organizations: A Systematic Review of Learning Tools That Could Be Adapted and Used in Healthcare,” International Journal for Quality in Health Care , 17 March 2021, https://academic.oup.com/intqhc/article/33/1/mzab046/6174559 34 Staff, “‘A Matter of National Security: FBI, Secret Service Investigate After Hacker Tried to Poison a Florida City’s Water With Lye,” USA Today , 9 February 2021, https://www.usatoday.com/story/news/nation/2021/02/09/oldsmar-florida- water-hacker-lye-sodium-hydroxide/4444387001/ 35 Turton, W.; K. Mehrotra; “Hackers Breached Colonial Pipeline Using Compromised Password,” Bloomberg, 4 June 2021, https://www.bloomberg.com/news/articles/2021-06-04/hackers-breached-colonial- pipeline-using-compromised-password 36 Coles, E; “Learning the Lessons From Major Incidents: A Short Review of the Literature,” Emergency Planning College , June 2014 37 Ibid . 38 Check Point Research, “Check Point Research: Cyber Attacks Increased 50 Percent Year Over Year,” 2022, https://blog.checkpoint.com/2022/01/10/check-point-research-cyber- attacks-increased-50-year-over-year/

Ranjit Bhaskar, CISA, CISM, CISSP

Is a senior security architect at Texas Windstorm Insurance Association (TWIA). Bhaskar has 25 years of experience in enterprise architecture and is the author of the op-ed, “A Cybersecurity Culture Score.” He can be reached via LinkedIn at https://www.linkedin.com/in/ranjit-bhaskar-467877218 .

Explainable deep learning approach for advanced persistent threats (APTs) detection in cybersecurity: a review

• Open access
• Published: 18 September 2024
• Volume 57 , article number  297 , ( 2024 )

Cite this article

You have full access to this open access article

• Noor Hazlina Abdul Mutalib 1 ,
• Aznul Qalid Md Sabri 1 ,
• Ainuddin Wahid Abdul Wahab 2 ,
• Erma Rahayu Mohd Faizal Abdullah 1 &
• Nouar AlDahoul 3  

In recent years, Advanced Persistent Threat (APT) attacks on network systems have increased through sophisticated fraud tactics. Traditional Intrusion Detection Systems (IDSs) suffer from low detection accuracy, high false-positive rates, and difficulty identifying unknown attacks such as remote-to-local (R2L) and user-to-root (U2R) attacks. This paper addresses these challenges by providing a foundational discussion of APTs and the limitations of existing detection methods. It then pivots to explore the novel integration of deep learning techniques and Explainable Artificial Intelligence (XAI) to improve APT detection. This paper aims to fill the gaps in the current research by providing a thorough analysis of how XAI methods, such as Shapley Additive Explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME), can make black-box models more transparent and interpretable. The objective is to demonstrate the necessity of explainability in APT detection and propose solutions that enhance the trustworthiness and effectiveness of these models. It offers a critical analysis of existing approaches, highlights their strengths and limitations, and identifies open issues that require further research. This paper also suggests future research directions to combat evolving threats, paving the way for more effective and reliable cybersecurity solutions. Overall, this paper emphasizes the importance of explainability in enhancing the performance and trustworthiness of cybersecurity systems.

Explore related subjects

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

Advanced persistent threats (APTs) represent a significant cybersecurity challenge in the digital era. (Hasan et al. 2023 ). In this study, we explore the integration of Explainable Artificial Intelligence (XAI) with deep learning models to enhance the detection of Advanced Persistent Threats (APTs). APTs are prolonged and targeted cyberattacks in which an intruder gains access to a network and remains undetected for an extended period. Unlike traditional attacks, APTs aim to steal sensitive data over time rather than causing immediate damage. APT attacks are conducted by highly skilled adversaries with vast resources and target organizations through IT network systems for long-term access, without being discovered. APT attacks target critical infrastructure, such as financial institutions, government agencies, energy companies, shipping and transportation companies, industrial companies, and food services, which can disrupt critical infrastructure and cause massive damage. APT attacks have major impacts on countries, organizations, and individuals, leading to severe financial losses, reputational damage, legal liability, lost productivity, business continuity issues, and disruption of critical operations (Salim et al. 2023 ). For example , the Stuxnet worm, attributed to state-sponsored actors, targeted Iran’s nuclear facilities and caused significant operational disruptions (Ahmad et al. 2024 ). Another notable example is the SolarWinds attack in 2020, which involved compromising the Orion software platform (Alkhadra et al. 2021 ). This attack affected numerous government agencies and private sector companies, leading to widespread data breaches and significant financial losses. The estimated cost of the SolarWinds breach is still being assessed, but the vulnerability of even the most secure networks to sophisticated APTs has been highlighted.

The National Institute of Standards and Technology (NIST) defines APTs as sophisticated, resourceful adversaries that use multiple attack methods, such as cyber-attacks, physical intrusions, and deception to infiltrate an organization’s infrastructure​​ (de Abreu et al. 2020 ). APTs are particularly challenging to detect due to their stealth and persistence, blending with legitimate network traffic and activities (Salim et al. 2023 ). Unlike typical malware, which executes its payload quickly, APTs focus on long-term espionage or data theft, making detection more difficult​​ (Jabar and Mahinderjit Singh 2022 ).

(Source: Symantec)

APT attack distribution by Leafminer group.

In August 2018, Symantec Footnote 1 reported APT attacks by the Leafminer group, known as RASPITE, in the Middle Eastern region since 2017 Footnote 2 . Figure  1 illustrates the distribution of APT attacks by the Leafminer group, across various industrial sectors. Government agencies were one of the primary targets, representing 17% of all attacks. Financial institutions were equally targeted, accounting for 17% of all attacks. The Ponemon Institute, as analyzed by IBM Security, reported that the average cost of an APT attack and data breach in 2023 was $4.35 million Footnote 3 . Large organizations experienced an average loss of $6.93 million per incident. This distribution emphasizes the need for robust cybersecurity measures across diverse industries to protect against sophisticated threats, such as those posed by the Leafminer group. In 2020, $945 billion was lost due to cyber incidents, and another $145 billion was spent on cybersecurity. These costs have surged by more than 50% since 2018, when approximately $600 billion was allocated to mitigate cybercrime (B. Ballard, 2021).

In summary, APT attack detection requires specialized techniques that go beyond the traditional detection methods. These specialized techniques must address the unique challenges posed by APTs, including their stealth, persistence, targeted nature, sophistication, advanced evasion techniques, and focus on data exfiltration. Detecting APTs often involves leveraging advanced deep-learning methods, continuous monitoring, and the use of XAI to ensure transparency and trust in the detection process (Schwalbe and Finzel 2023 ).

1.1 Paper organization

The organization structure of the review

Figure  2 provides the organization structure for this review. It outlines the main sections and subsections of the paper and provides a clear roadmap of the content and topics covered. In Sect. 2, we introduce the background of Advanced Persistent Threats (APTs), including the lifecycle and characteristics of these cybersecurity threats. Section 3 outlines the research methods, detailing the search strategy and eligibility criteria for selecting relevant literature. Section 4 delves into related work, discussing the classification of APT detection, various deep learning models used, and the limitations of existing detection methods. Section 5 presents the methodology, including a comparative analysis of the deep learning and XAI approaches, the need for XAI, and relevant case studies. Section 6 focuses on explainable AI in cybersecurity, explores explanation methods, integrates XAI techniques in deep learning, and addresses the current issues surrounding black box models. Section 7 discusses the findings, providing key considerations of XAI, challenges, and recommendations for future research. Finally, Sect. 8 concludes the review and suggests future research directions at the intersection of XAI and APT detection.

1.2 Motivation

The increasing sophistication of APT attacks and the limitations of existing IDS drive the key motivation for this review. Traditional IDS methods struggle with low detection accuracy, high false-positive rates, and the inability to detect unknown or early-stage attacks. For example, signature-based IDS can detect only known threats, making them ineffective against novel APTs (Sarker et al. 2024 ). Furthermore, these methods have difficulty detecting unknown or early-stage attacks because of their reliance on signature-based detection mechanisms, which fail to recognize novel threats.

Additionally, comprehensive studies detailing the current state of interpretability in published research and the application of state-of-the-art XAI models in the cybersecurity domain are lacking (Saeed and Omlin 2023 ). Without interpretability, cybersecurity experts cannot fully trust or understand the decisions made by AI models, which is crucial for accurately identifying false positives and negatives and for adapting to new attack vectors (Brown et al. 2022 ).

This review examines a critical gap in the literature on the implementation of deep learning techniques with XAI for APT detection. Table  1 provides a summary of various studies on APT detection methods, highlighting the objectives, motivations, and key findings of each study. This comparative analysis helps to identify gaps and challenges in existing methods, providing a foundation for enhancing state-of-the-art APT detection.

By addressing these gaps, our study aims to enhance the effectiveness and transparency of APT detection systems, thereby advancing the state-of-the-art in cybersecurity.

1.3 Contribution

In this review, we address the challenges of APT detection by integrating deep learning techniques with XAI methods. The following section discusses the key contributions of our research.

Our research expands the state-of-the-art in cybersecurity by providing robust, scalable, and interpretable detection systems that can effectively combat sophisticated APT attacks.

We evaluated robust APT detection frameworks that improve detection accuracy and scalability while providing clear, interpretable insights into the model’s decision-making process. This approach ensures that cybersecurity experts can respond more effectively to threats and understand how and why specific decisions are made.

We highlight the limitations of existing detection systems, such as low detection accuracy and high false-positive rates. Our study demonstrates how integrating deep learning with XAI can address these challenges, paving the way for more robust and real-time detection capabilities.

We include case studies to demonstrate the application of XAI in combating APTs. These scenarios illustrate the potential impact of implementing XAI techniques in real-world settings, providing a comprehensive view of how these advanced methods can transform APT detection and response.

We examine various issues in the cybersecurity area and propose an improved method of APT detection on the basis of the specific attributes of such attacks. This tailored approach enhances detection capabilities and addresses the unique challenges posed by APTs.

In summary, our research provides significant contributions to the field of cybersecurity by integrating XAI APT detection systems. By addressing the current limitations and demonstrating the practical application of our methods, we pave the way for more robust, scalable, and transparent cybersecurity solutions. Our work enhances the overall resilience of network systems to APTs, ensuring better protection and response capabilities against sophisticated APT attacks.

The scope of this paper includes a comprehensive review of various deep learning techniques utilized for APT detection, the application of XAI methods to improve model interpretability and trustworthiness, and a critical analysis of existing approaches. By examining the intersection of deep learning and XAI, this paper aims to provide valuable insights and pave the way for more effective and reliable cybersecurity solutions. Our approach aligns with the need for explainability in AI models to ensure their practical application and trustworthiness.

Having established the importance of APT detection and the potential of XAI, we delve into the background study to provide a foundation for our research.

2 Background

In this section, we provide an overview of the background studies relevant to APT detection. Our focus is on the various methods that have been explored in the literature, such as deep learning and XAI approaches. This review aims to place our research within the wider scope of existing research and highlight the contributions we make to advancing the state-of-the-art in APT detection.

Deep learning approaches have gained significant attention for their ability to automatically learn complex patterns from large datasets. Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM) networks have shown promising results when applied to APT detection. These models capture temporal and spatial dependencies in data, making them suitable for detecting sophisticated attack patterns. However, the black-box nature of deep learning models poses challenges in interpretability and transparency, which are critical for cybersecurity applications. To address the interpretability challenge, XAI techniques have been developed. XAI aims to make AI models’ decision-making processes transparent and understandable to humans. Techniques such as SHAP and LIME help elucidate the contributions of individual features to a model’s predictions, enhancing trust and facilitating collaboration between human experts and AI systems.

APTs pose a unique characteristic cyber-attack problem. They target specific victims and use tricks to hide. Skilled, well-funded attackers use various TTPs to evade detection (Sharma et al. 2023 ). APTs are managed by highly skilled and resourceful adversaries, often backed by nation-states or organized crime groups (Lemay et al. 2018 ). These threat actors have clear goals, such as espionage, sabotage, and damage, and spend much of their time succeeding (Ahmad et al. 2019 ). For example, the Lazarus Group established a method for compromising common software on the Internet with Trojans. These advanced attackers use social engineering techniques to inject malicious software into a target system. This allows them to attack the supply chain via untrustworthy sources (Villalón-Huerta et al. 2022 ).

The detection of APT attacks presents several distinct challenges compared with malware. The ways in which APT attack detection differs include the following:

Stealth and persistence: APTs remain undetected for extended periods by blending with legitimate network traffic and activities (Bierwirth et al. 2024 ) (Moustafa & Slay, 2016 ). Unlike typical malware, which may execute its payload quickly, APTs focus on long-term espionage or data theft, making detection more difficult.

Targeted nature: APTs are highly targeted and tailored to specific organizations or individuals, rendering generic security solutions less effective. Additionally, APT tactics, techniques, and procedures (TTPs) are continuously evolving (Ahmad et al. 2019 ) requiring deep learning models to be frequently retrained to detect new threats.

Sophistication and complexity: APT attacks involve multiple stages, such as initial access, lateral movement (Fang et al., 2022 ), data exfiltration, and persistence. This multistep approach necessitates sophisticated detection mechanisms to identify and correlate various stages.

Advanced evasion techniques: APTs use advanced evasion techniques,  such as encryption, obfuscation, and polymorphism, to avoid detection by traditional methods (Shenderovitz & Nissim, 2024 ). They may also use legitimate system tools for malicious activities, complicating detection.

Focus on data exfiltration: While malware might aim to cause immediate damage or disruption, APTs primarily aim to steal sensitive information, posing significant detection challenges (Sharma et al. 2023 a).

Nation-State Involvement: Many APTs are linked to nation-state actors conducting espionage or intelligence gathering, leveraging vast resources and advanced capabilities (Holt et al., 2023) (Bierwirth et al. 2024 ).

Mitigation Strategies: Addressing the threat of APTs requires a multilayered approach (Mohamed, 2023 ) that includes proactive network monitoring, continuous vulnerability assessments, user awareness training, and robust incident response plans.

2.1 APT attack

APTs follow a well-defined lifecycle that consists of several stages. Initially, attackers infiltrate the target system using methods such as phishing, spear-phishing, or exploiting vulnerabilities in web applications. Once inside, they establish a foothold by installing malware to maintain persistence within the system. Subsequently, they escalate privileges by exploiting system vulnerabilities to gain deeper access. The attackers then perform internal reconnaissance, moving laterally within the network to map infrastructure and identify critical assets. In the next stage, attackers extract sensitive data and send it to the external servers they control. Finally, attackers maintain persistence using advanced evasion techniques to remain undetected and retain long-term access to compromised systems (Bierwirth et al. 2024 ).

APTs are characterized by their sophisticated and prolonged attack campaigns, which follow a well-defined lifecycle. The APT attack life cycle can be divided into six phases, intelligence gathering, point of entry, command and control (C&C), lateral movement, data of interest and the external server. The initial phase of an APT attack is reconnaissance or intelligence gathering, in which the attackers gather information about their target organization such as network infrastructure and identify potential vulnerabilities. This phase involves passive information gathering techniques and social media analysis (Bodstrom & Hamalainen, 2019 ). Once the attackers have identified potential entry points, they move to the establish phase, where they prepare customer malware to exploit the target environment. If successful, the attackers establish a foothold in the compromised systems, installing backdoors and remote access tools (RATs) to maintain persistence. From this point, the APT actors proceed to the command and control (C&C) phase, establishing secure communication channels with their malicious infrastructure to receive updates and exfiltrate data. With a persistent presence in the target network, the attackers move to the lateral movement phase, where they seek to escalate privileges and gain access to additional systems and resources. The final stages of the APT lifecycle involve the actual achievement of the attackers’ objectives, such as data of interest, data exfiltration, sabotage, or establishing long-term access for future attacks (Chen et al., 2018 ).

In summary, APTs represent a sophisticated and persistent threat to organizations worldwide. Their ability to remain undetected and adapt to new defenses makes them particularly challenging to address. Recent high-profile cases, such as the SolarWinds (Alkhadra et al. 2021 ) and Hafnium attacks, highlight the critical need for advanced detection methods. This paper aims to explore the integration of deep learning techniques with Explainable AI (XAI) to enhance APT detection and provide more transparent, interpretable solutions for cybersecurity experts.

3 Research methods

In this section, we outline the research questions and search strategy that guided our study. Next, we explain the research process.

3.1 Research questions

We established a set of research questions to address important aspects of APT detection. These questions will guide the development of an effective detection system. They are designed to explore the performance, limitations, and potential enhancements of APT detection systems. Table  2 provides description of the research questions as follows:

This section outlines the key research questions that drive our investigation into APT detection. By addressing these questions, we aim to uncover the strengths and weaknesses of current systems, explore the benefits of integrating XAI, and identify the unique challenges in applying these advanced techniques.

3.2 Search strategy and eligibility criteria

In this review, we conducted a rigorous search strategy and defined clear eligibility criteria for the selection of relevant papers. We searched multiple academic databases, including IEEE Xplore, ACM Digital Library, ScienceDirect, Springer Link, Web of Science, and Google Scholar. These databases were chosen for their relevance to research in computer science, cybersecurity, and artificial intelligence. We used specific search terms to find studies related to APT detection mechanisms. Boolean expressions such as (“Advanced Persistent Threat” OR “APT”) AND (“Deep Learning AND “Cybersecurity”) AND (“Explainable Artificial Intelligence” OR “XAI”) were used to combine search terms.

Our review aimed to include papers presenting novel methods, particularly those involving deep learning-based APT detection, the application of deep learning, and XAI techniques, along with empirical results. The inclusion criteria required papers to be published in English in peer-reviewed journals, conference proceedings, or book chapters from 2018 to 2024. We excluded papers without abstracts, without access to full content, or not written in English.

Overview of the search strategy and eligibility criteria

We reviewed selected articles that met our inclusion criteria, focusing on those that provided insights into the integration of XAI with APT detection. Through this process, we collected 100 articles deemed the most relevant to our study. Figure  3 provides an overview of our search strategy and eligibility criteria. To refine the search results and relevant papers, we applied the following inclusion criteria:

The paper must be written in English and published in peer-reviewed journals, conference proceedings, or book chapters.

The study must focus on applying deep learning techniques specifically for detecting APT attacks.

The study must involve applying XAI methods in cybersecurity, specifically for APT detection.

The study must provide empirical results, such as experiments or case studies, which demonstrate the effectiveness of the proposed approach in real-world or simulated scenarios.

The papers must have been published between 2018 to 2024, reflecting recent advancements in deep learning and XAI research.

This section outlines the rigorous search strategy and well-defined eligibility criteria used to select relevant papers for this review. The search was conducted across several major academic databases using specific terms related to APT detection, deep learning, and XAI.

4 Related work

In this section, we review the existing literature on APT detection methods, focusing on both traditional and advanced approaches. We organize the discussion to provide a clearer understanding of the strengths and weaknesses of each approach relative to our study. Current IDS and security measures face significant limitations when dealing with APTs.

4.1 The classification of APT detection approaches

Classifying APT detection approaches on the basis of data sources such as Network Traffic Analysis, Host-based Analysis, and Log and Event Analysis allows for a more targeted and efficient detection strategy. A study by (Do Xuan et al. 2020 ) analyzed network traffic into IP-based network flows, reconstructed IP information from these flows, and used deep learning models to extract features to distinguish APT attack IPs from other IPs. They introduced a combined deep learning model using Bidirectional Long Short-Term Memory (BiLSTM) and Graph Convolutional Networks (GCN). Meanwhile, host-based analysis focuses on the data and activities occurring on individual hosts or endpoints within a network.(Chen et al. 2024 ) proposed a hybrid Network Intrusion Detection System (NIDS) that combines host-based intrusion detection (HIDS) and network-based detection to enhance the detection of APTs and other network intrusions. Log and Event Analysis involves collecting and analyzing logs and events generated by various systems, applications, and devices within an organization. (Wang et al. 2022 ) presented a novel method for reconstructing APTs in large-scale networks to improve attack forensics and traceability. The proposed method addresses these issues with a low transmission cost and does not require raw data from terminal devices, making it suitable for extensive networks with numerous terminal devices. Classifying APT detection approaches based on explainability involves distinguishing between black-box models and XAI models. The black-box models will be explained in Sect. 5.4. XAI models offer transparent and interpretable insights into the detection process. These models aim to maintain a high level of accuracy while ensuring that the reasoning behind their predictions is clear and understandable to cybersecurity experts. This transparency increases trust in the detection system and facilitates a faster and more effective response to identified threats.

4.2 Deep learning models

A taxonomy of AI/XAI based methods for cybersecurity modeling. Adapted this figure from (Sarker et al. 2024 )

Machine learning (ML) is a key component of data science, and is important in terms of its flexibility, scalability, and adaptability to new challenges. This article explores ML applications in cybersecurity, including phishing detection, network intrusion detection, spam detection in social networks, smart meter energy consumption profiling, and security concerns inherent in ML techniques. To understand this, we have added Fig.  4 , which shows taxonomies based on specific themes and deep learning (DL) methods. This structured approach will not only add more organization to our review but also increase its reference value by providing a clearer framework. This study emphasizes the methodology of collecting large datasets, extracting relevant features, and training ML models using supervised learning algorithms to achieve high accuracy and low false positive rates.

(Manoharan et al. 2023 ) addressed the challenge of detecting insider threats, which pose significant cybersecurity risks. This study evaluated various supervised machine learning algorithms on a balanced dataset using the same feature extraction method and investigated the impact of hyperparameter tuning and different conditions on imbalanced datasets. Using the publicly available CERT r4.2 dataset, the results showed that Random Forest achieved the best accuracy and F1-score of 95.9%, outperforming existing methods such as the DNN, LSTM Autoencoder, and User Behavior Analysis. (Raju et al. 2021 ) explored various ML models, including decision trees and support vector machines, to detect APT activities. Although these models offered improved detection capabilities over traditional methods, they still struggled with interpretability. Cybersecurity experts often find it difficult to understand the reasoning behind ML predictions, which hampers their trust and usability. (Stojanović et al. 2020 ) proposed the use of ensemble methods to combine multiple ML models, thereby enhancing detection accuracy. However, this approach increases computational complexity and may not be suitable for real-time applications. Our study leverages the strengths of ML while addressing its weaknesses by using deep learning and XAI to provide more interpretable and scalable solutions.

Deep learning is a widely used technique for APT detection, offering several advantages over traditional machine learning techniques. These advantages include the ability to process and analyze unstructured data, such as text, images, and network traffic, as well as high-dimensional data, which are common in cybersecurity domains. Unlike traditional methods, which often rely on predefined rules and signatures, deep learning models can automatically learn complex patterns and features from large datasets. This capability allows for more accurate and robust detection of sophisticated cyber threats, making deep learning an essential tool in the evolving landscape of cybersecurity (Alzubaidi et al. 2021 ).

(Mittal et al. 2023 ) conducted a systematic review on deep learning for detecting Distributed Denial of Service (DDoS) attacks, analyzing literature from multiple sources. They categorize findings into five key areas: deep learning approaches, methodologies, datasets, preprocessing strategies, and research gaps. This study evaluates existing methods, highlights strengths and weaknesses, and identifies gaps in current research, suggesting future directions. This review offers a comprehensive overview, is organized for clarity, and emphasizes the relevance of deep learning in addressing the evolving threat of DDoS attacks. Despite its thoroughness, the study’s reliance on existing literature and the rapid evolution of attack strategies may limit its timeliness and practical application.

4.2.1 Convolutional neural networks (CNNs)

Among the various types of deep learning techniques, Convolutional Neural Networks (CNNs) excel at handling high-dimensional data and automatically learn and extract relevant spatial features from raw data. This capability makes them particularly effective in identifying complex patterns associated with APT activities. In computer vision, researchers use CNNs to recognize APTs by extracting features from APT data (Teuwen and Moriakov 2020 ). The CNN architecture includes five block layers: convolutional layers, pooling layers, channel max-pooling layers, rectified linear unit (ReLU) layers, fully connected layers, and a SoftMax loss function (Alzubaidi et al. 2021 ). However, CNNs have limitations in converting one-dimensional traffic flows into two-dimensional flows without considering the spatial correlations between the traffic flows. (Alzubaidi et al. 2021 )

(Jayapradha et al. 2024 ) proposed an innovative intrusion detection system (IDS) that leverages deep learning algorithms to enhance phishing detection. Specifically, CNNs automatically extract sophisticated features from raw input data, whereas RNNs effectively model sequential data to recognize phishing patterns over time. This IDS adapts in real-time to new phishing variants through back propagation-based model optimization, significantly improving detection accuracy compared with traditional rule-based methods. By utilizing the KDD-CUP99 dataset for training, the study demonstrates a robust defense against evolving phishing threats. Consequently, the system enables a proactive incident response and strengthens overall network security. Despite potential challenges in terms of computational complexity and data dependency, the proposed system offers substantial improvements in protecting networks and users from sophisticated phishing attacks.

(Patel et al. 2024 ) proposed an advanced malware detection system that leverages RNNs and CNNs to enhance cybersecurity vigilance. The study highlights a pioneering capability by extending detection to unconventional formats such as GIFs and images, addressing emerging threats that exploit multimedia channels. This approach demonstrates the adaptability and comprehensiveness of the deep learning-based system. Traditional cybersecurity methods struggle to keep pace with dynamic cyber adversaries, but this research leverages deep learning’s ability to recognize complex patterns within vast datasets. By integrating RNNs and CNNs, this study aims to improve the accuracy of threat identification and the system’s resilience against emerging and previously unseen cyber threats. This study focuses on broadening the detection scope to multimedia formats and enhancing the system’s adaptability and reliability. The strengths of this study lie in its innovative approach and comprehensive coverage of diverse data formats, whereas its weaknesses include potential computational complexity and reliance on high-quality datasets for training.

(Tadesse and Choi 2024 ) proposed a novel intrusion detection system (IDS) that converts raw datasets into image datasets using the Short-Term Fourier Transform (STFT) to enhance pattern recognition. This system uses a lightweight convolutional neural network (CNN) to classify denial of service (DoS) and distributed denial of service (DDoS) attacks. Evaluated on both modern (CSE-CIC-IDS2018) and legacy (NSLKDD) datasets, the proposed methods achieve high accuracy and low false alarm rates, demonstrating high specificity and sensitivity. The study highlights the model’s excellent generalizability and avoidance of overfitting across different datasets, emphasizing the effectiveness of the dataset conversion methodology.

(Najar & S., 2024 ) proposed an innovative feature selection approach to develop a robust and reliable intrusion detection system (IDS) for detecting and classifying Distributed Denial-of-Service (DDoS) attack types. Using the CICDDoS2019 benchmark dataset, the model demonstrates high performance, achieving 96.82% accuracy, 96.82% recall, 96.76% precision, and a 96.50% F1 score. It also provides rapid prediction times, identifies attacks in just 0.189 milliseconds, and outperforms existing methods and baseline models. The contributions include a novel feature selection approach, effective preprocessing techniques, and comprehensive evaluation using benchmark datasets. The strengths of this study are its high detection accuracy, rapid response, balanced performance metrics, and innovative methodology. However, weaknesses include challenges with imbalanced data handling, computational complexity, dependency on dataset quality, generalizability to other attack types, and implementation challenges.

(Korium et al. 2024 ) proposed an advanced IDS tailored for the Internet of Vehicles environment, leveraging CNNs to detect both traditional and new cyber-attacks effectively. The novel network architecture of the model at the data processing layer and the use of the synthetic minority oversampling technique significantly enhanced the detection accuracy and speed, achieving a notable 94% detection rate using the AWID dataset. The strengths of this study are its high detection accuracy, improved data processing through synthetic oversampling, innovative network architecture, and comprehensive evaluation with the AWID dataset. The model’s performance is heavily dependent on the AWID dataset, and its scalability may be limited by the computational complexity introduced by CNNs and data processing techniques.

(Sun 2024 ) proposed a novel intrusion detection system (IDS) that combines data preprocessing with four deep neural network models: Convolutional Neural Networks (CNN), Bi-directional Long Short-Term Memory (BiLSTM), Bidirectional Gate Recurrent Unit (BiGRU), and Attention mechanism to identify network attacks accurately. Evaluated using the NSL-KDD dataset, the models utilize preprocessing techniques and particle swarm optimization for feature selection, with hyperparameter tuning via the BO-TPE algorithm. The study’s contributions include a comprehensive IDS framework, the introduction of multiple DNN architectures, and effective feature selection and class imbalance handling, resulting in high detection accuracy rates of 0.999158 in binary classification and 0.999091 in multiclass classification. The strengths of this study include its thorough approach, high accuracy, and advanced optimization techniques, whereas its weaknesses include dataset dependency, computational complexity, generalizability concerns, implementation challenges, and the need for further real-world application exploration.

(Yin et al. 2024 ) introduced a novel multi-scale Convolutional Neural Networks (CNN) and bidirectional Long-short Term Memory (bi-LSTM) arbitration dense network model (MSCBL-ADN) for detecting LDDoS attacks effectively with limited datasets and reduced time consumption. The MSCBL-ADN model integrates a CNN for spatial feature extraction, bi-LSTM for temporal relationship extraction, an arbitration network to re-weigh feature importance, and a 2-block dense connection network for final classification. The experimental results on the ISCX-2016-SlowDos dataset demonstrate that the MSCBL-ADN model significantly improves detection accuracy and time performance compared to state-of-the-art models.

(Ersavas et al. 2024 ) explored the potential of Convolutional Neural Networks (CNNs) beyond traditional image processing, highlighting their ability to analyze high-dimensional datasets by transforming them into pseudo-images. Despite the rise of newer architectures such as Transformers, CNNs remain crucial in various applications, including Generative AI. The authors introduce DeepMapper, a pipeline that enables the analysis of complex datasets without intermediate filtering or dimension reduction, thus preserving data integrity and detecting small variations typically considered noise. They demonstrated that DeepMapper can efficiently and accurately identify subtle perturbations in large datasets with numerous features, highlighting its superiority in speed and comparable accuracy to existing methods.

Mendonça et al. (2023) proposed a new IDS using a hierarchical tree-CNN algorithm with soft-root-sign (SRS) activation (Daoud et al. 2023 ) to detect attacks in infiltrations, Distributed Denial-of-Service (DDoS), brute force, and web attacks. The authors reported that their proposed system could detect DDoS attacks with high accuracy and a reduced execution time of approximately 36%. Furthermore, the results showed a significant increase in the average detection accuracy of 98% when all the analyzed attacks were considered. This indicates that Tree-CNN performs better because it is less complex, requires less processing time, and consumes fewer computing resources than other current machine-learning-based IDSs do.

On the other hand, Zhu & Zu, 2022 ) implemented a fully convolutional neural network (FCNN) classifier architecture that substitutes the linear layers and correlated activation functions from prevalent CNN classifiers. They trained the FCNN classifier using SoftMax loss. The main benefits of this architecture are its simplicity and flexibility. However, it does not consider channel information. They can use SoftMax loss directly to train the network by reconstructing the number of output channels. The predefined best distribution (POD loss) of the latent features was used to improve the recognition rate performance with SoftMax loss.

Network Intrusion Detection Systems (NIDS) are essential for detecting malicious activities in modern networks. However, class imbalance in intrusion detection datasets hinders the performance of classifiers in minority classes. To address this, (Zhang et al. 2020 ) proposed a novel class imbalance processing technique called the SGM, which combines the Synthetic Minority Over-Sampling Technique (SMOTE) and under-sampling using Gaussian Mixture Model (GMM). They developed a flow-based intrusion detection model, SGM-CNN, which integrates imbalance processing with a convolutional neural network (CNN). They evaluated the performance on the UNSW-NB15 and CICIDS2017 datasets, and reported high detection rates of 99.74% for binary classification, 96.54% for multiclass classification for the UNSW-NB15 dataset, and 99.85% for 15-class classification for the CICIDS2017 dataset. The authors claimed that SGM-CNN effectively addresses the challenge of imbalanced intrusion detection and outperforms the existing state-of-the-art methods.

(Tian, 2020) proposed a combination approach, the CNN algorithm, to learn the deep features of an image using CNNs and RNNs in parallel. The authors subsequently created a ShortCut3-ResNet residual module. In their study, they demonstrated that the convolutional neural network algorithm can identify various features of images, optimize the accuracy of feature extraction, and improve the ability of the convolutional neural network to recognize images.

CNNs have shown significant promise in enhancing cybersecurity measures through various innovative approaches. These studies highlight the adaptability of CNNs in recognizing complex patterns, handling diverse data formats, and integrating with other deep learning models to improve detection accuracy and resilience against sophisticated cyber threats. Despite challenges such as computational complexity and data dependency, CNN-based systems offer substantial improvements in protecting networks and users from evolving threats.

4.2.2 Recurrent neural networks (RNNs)

Recurrent Neural Networks (RNNs) are specifically designed to handle sequential data, making them particularly suitable for analyzing time-series data in network traffic. By incorporating internal memory states, RNNs can effectively process input sequences of varying lengths, capturing temporal dependencies within the data (DiPietro and Hager 2020 ). This capability allows RNNs and their advanced variants, such as Long Short-Term Memory (LSTM) networks, to model the sequential nature of APT attack campaigns (Yuan et al. 2017 ). RNNs have proven effective for APT detection because they can learn from event sequences over time and predict future actions based on past observations. This makes RNNs an invaluable tool for identifying and mitigating sophisticated cyber threats (Galli et al. 2024 ).

In the current landscape of complex cyber threats, network security is of utmost importance. (Kumaresan et al. 2024 ) evaluated the efficacy of Recurrent Neural Networks (RNNs) in network anomaly detection, comparing them with traditional methods such as statistical techniques and simple neural networks. Using an extensive dataset of normal and malicious network traffic, the authors demonstrate the potential of RNNs to detect anomalies by utilizing sequential dependencies in the data. They investigated various RNN architectures, hyperparameter settings, and feature representations to improve detection performance. The paper also addresses challenges such as model interpretability, scalability, and computational resource demands, and proposes ways to increase the resilience of RNN-based systems against malicious interference.

(Yang et al. 2024 ) introduced the Hypergraph Recurrent Neural Network (HRNN), a novel intrusion detection method that leverages hypergraph structures and recurrent networks. The HRNN represents flow data as hypergraph structures to enhance information representation and incorporates a recurrent module to extract temporal features. This design integrates rich spatial and temporal semantics, significantly improving anomaly detection capabilities. Evaluations on several publicly available datasets demonstrate that the HRNN outperforms other state-of-the-art methods, demonstrating its superior performance in detecting network anomalies. However, the increased complexity due to the use of hypergraph structures and recurrent networks may impact computational efficiency and scalability. Furthermore, the performance of the HRNN may heavily depend on the quality and characteristics of the datasets used for training and evaluation, potentially requiring fine-tuning for different data forms and limiting its transferability. Integrating HRNN with existing network infrastructure might also present technical challenges.

(Saravanan et al. 2023 ) proposed a Blockchain-based African Buffalo (BbAB) scheme with a Recurrent Neural Network (RNN) to enhance an IDS. The method encrypts normal and malware user datasets using Identity Based Encryption (IBE) and securely stores them in a blockchain within a cloud environment. The RNN detects intrusions, using African buffalo optimization for continuous monitoring. The approach achieves 99.87% accuracy and 99.92% recall, demonstrating robust detection capabilities. While the method improves security and monitoring, it presents challenges such as high computational complexity and significant resource requirements. Overall, the model enhances IDS effectiveness in cloud environments.

(Pahuja and Ojha 2024 ) addressed the growing threat of network attacks by deploying deep learning techniques, including Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU), to detect and mitigate Denial-of-Service (DoS) attacks. These models analyze sequential time-series data to identify patterns linked to DoS attacks. Among the techniques evaluated, LSTM achieved the highest accuracy of 92.3%, demonstrating superior capability in classifying attack traffic. This approach aims to reduce system unavailability and potential losses caused by DoS attacks, although it faces challenges in computational complexity and scalability.

(Sakthipriya et al. 2024 ) proposed enhancing IoT network security by reducing data dimensionality for efficient attack classification on memory-constrained devices. They used a Conditional Adversarial Auto Encoder (CAAE) to generate realistic botnet traffic and extract deep features, combined with a Dilated and Cascaded Recurrent Neural Network (DC-RNN) for accurate classification. The model achieves 96% accuracy, 98% precision, 97% F1-score, and 96% recall. This approach addresses deep learning implementation challenges in IoT environments, outperforming conventional methods. However, it faces issues with computational complexity, data dependency, scalability, generalizability, and implementation challenges in existing IoT infrastructures.

(Hasan et al. 2023 ) proposed an effective identification model for APT attacks by boosting-based machine learning methods combined with XAI to enhance prediction and provide actionable insights. The model, which achieves a high weighted F1 score of 0.97 with XGBoost, effectively predicts APT attacks and utilizes SHAP to make predictions understandable and actionable for cybersecurity stakeholders. While the approach demonstrates high detection accuracy and offers a promising framework for future research, it faces challenges such as computational complexity, data dependency, generalizability, implementation difficulties, and resource intensity, potentially impacting scalability and practical application in resource-constrained environments.

Furthermore, Lo et al. ( 2023 ) introduced the XG-BoT GNN and used GNNExplainer on a botnet graph dataset. Their model demonstrated impressive precision between 99.23% and 99.63%. The effectiveness of this approach relies heavily on the quality of the input data.

AlDahoul et al. ( 2021 ) developed a fusion model that combined two DNNs, trained using class-weight optimization. The main idea is to learn complex patterns from rare anomalies in the traffic data. The authors used Adam optimization algorithms with classes to train the DNNs. Their results showed that their method could achieve a higher accuracy in terms of the Fβ score and the false alarm rate when their proposed model compared to conventional single DNNs. Furthermore, their experiment used the ZYELL real-world datasetand yielded promising results.

These studies demonstrate the versatility and effectiveness of RNNs and their variants in enhancing cybersecurity measures, particularly in detecting and mitigating sophisticated cyber threats. Despite challenges such as computational complexity and data dependency, RNN-based systems offer substantial improvements in network security.

4.2.3 Autoencoders

Autoencoders (AEs) are used for anomaly detection to represent normal behavior and subsequently identify deviations from this learned behavior. These neural network models are trained on datasets comprising normal activity, enabling them to compress and reconstruct input data accurately. When presented with anomalous data, autoencoders produce higher reconstruction errors, thus flagging the anomalies. They have shown significant promise in detecting APTs by highlighting unusual activities within the network traffic, such as unexpected patterns that differ from established norms, thereby providing a robust mechanism for identifying potential security breaches.

(Yashwanth et al. 2024 ) proposed a novel approach by combining Auto-encoders with Multi-Layer Perceptron (MLP). The study evaluates three algorithms: Auto-encoders, Auto-encoders with MLP, and CNNs, and the results demonstrate their effectiveness in detecting network intrusions. This study highlights the significance of using diverse machine learning techniques for effective anomaly detection, pattern recognition in complex network traffic, and handling imbalanced data, although challenges in computational complexity, resource requirements, and implementation in existing IDS infrastructure exist. Table  3 summarizes the deep learning-based methods used in cybersecurity.

4.3 Limitations of existing detection methods

The application of deep learning for APT detection presents several challenges. Traditional IDSs typically employ predefined signatures of known threats, which fail to detect new, unknown, or evolving APT tactics that do not match existing signatures. Anomalybased detection systems generate numerous false positives by flagging benign activities as potential threats, leading to alert fatigue and overwhelming cybersecurity experts, thereby reducing the overall effectiveness of the security operations center (SOC). Detecting early-stage attacks, such as initial compromise and establishing a foothold, is difficult because these activities often involve subtle actions that are difficult to distinguish from normal behavior. Traditional methods lack the ability to detect early indicators of compromise. APTs employ sophisticated evasion techniques, including encryption, polymorphism, and the use of legitimate system tools for malicious purposes, making traditional IDSs challenging. Furthermore, conventional methods often lack the ability to contextualize alerts within a broader network environment, failing to correlate seemingly unrelated events that together indicate an ongoing APT attack.

The limitations of the existing detection methods emphasize the need for advanced techniques that can effectively identify and mitigate APTs. Deep learning is a promising approach because of its ability to analyze large volumes of data and identify complex patterns indicative of APT activities. For example, using Convolutional Neural Networks (CNNs) to detect anomalies in network traffic data can uncover subtle changes indicative of a potential threat that traditional methods might miss. XAI can highlight the specific features of the data that led to this detection. However, the black-box nature of deep-learning models poses challenges in terms of interpretability and trust, making it difficult for cybersecurity experts to understand and act upon the model’s predictions (Hassija et al., 2024 ). XAI enhances the transparency and interpretability of deep learning models, enabling cybersecurity experts to gain insight into the model’s decision-making process. This not only improves the effectiveness of APT detection, but also fosters trust and reliability in automated security systems.

By focusing on the unique characteristics of APTs and addressing the specific limitations of current methods, this review aims to improve state-of-the-art APT detection and provide practical solutions for enhancing cybersecurity defenses. Cybersecurity experts must understand the explanations behind the model’s predictions.

4.4 Datasets

Overview of datasets used in APT detection

In this section, we delve into datasets curated for APT detection. Figure  5 shows an overview of the most widely used methods in this field. In cybersecurity, benchmark datasets play a critical role in APT attacks (Agrawal et al. 2024 ).

We categorize the datasets based on their characteristics, such as the type of data, network traffic, system logs, presence of labeled attack scenarios, and extent of real-world applicability. This analysis helps to understand the strengths and limitations of each dataset, guiding researchers in selecting the most appropriate data for their studies. However, publicly available datasets that capture the behavior of APT attacks are lacking (Khraisat et al. 2019 ). This limitation hinders the development of effective APT detection models. Some existing datasets for analyzing APT attacks, such as DARPA1998, NSK-KDD 2009, UNSW-NB15, CICIDS2017, and ZYELL, focus on general network intrusion detection. However, they do not specifically target APT attacks.

DARPA1998 is the first dataset collection of attack traces from the internet. MIT Lincoln compiled and distributed it under DARPA and ARFL to evaluate network intrusion detection. It has seven weeks of training data and two weeks of testing data containing 38 attacks from four diverse groups: DoS, U2R, R2L, and probe (Homoliak et al. 2020 ). Another widely used dataset is the NSL-KDD (2009), which is a revised version of the KDD99 dataset. This revision reduces classifier bias and provides better detection rates (V C et al., 2023 ). Researchers commonly use the NSL-KDD for evaluation. This is an improved version of the original KDD-Cup99 benchmark. It considers 148,514 network traffic items across 41 features and five main attack types (Yang et al. 2021 ). The dataset includes denial-of-service attacks, remote access breaches, R2L, U2R, and probes from 77,054 benign samples (Barnard et al. 2022 ). The ISCX-IDS2012 dataset contains 2,381,532 data samples and is based on profiles that include intrusion details.

The UNSW-NB15 dataset was created at the Australian Center for Cyber Security (ACCS) Cyber RangeLab, using the IXIA Perfect Storm tool. It exhibits both benign and malicious attacks. The dataset included 49 features, with a single-class label indicating the connection property of each data instance (Moustafa and Slay 2016 ). The dataset contains nine types of attacks: analysis, fuzzers, generic, exploits, DoS, backdoors, reconnaissance, worms, and shellcodes. Another dataset, CICIDS2017, was created based on a large-scale cybersecurity research project that collected information from more than three million computers globally beginning in April 2016. The CICIDS2017 comprises 78 features, 168,186 normal samples, and 2,180 attack samples (Patil et al. 2022 ).

The ZYELL dataset was generated to detect network anomalies using real-world network traffic data obtained from the ZYELL security system (L. Chen et al., 2021 ). The dataset comprises both benign and malicious network traffic with a proportion of anomalies of approximately 1%. It also targets two main types of attacks: probing and DoS (AlDahoul et al. 2021 ). It has 22 features, including connection duration and inbound/outbound traffic counts in bytes, and is stored as csv files (L. Chen et al., 2021 ). The dataset includes 9,241,463 training samples and 13,290,530 testing samples. However, creating an effective APT detection system using network datasets from different sources and organizations can be challenging because of privacy concerns and the lack of a universal dataset format.

In conclusion, a lack of comprehensive datasets remains a major challenge in the research and development of deep-learning models for APT detection (Karim et al. 2024 ). Collaborative efforts among researchers, industry partners, and government agencies are needed to create large-scale, diverse, and labeled datasets that can support the advancement of deep learning models in this domain. This will help the deep learning techniques in this field move forward. While previous studies have demonstrated the promise of deep learning and XAI in cybersecurity, there remains a need for integrated approaches that enhance both detection accuracy and interpretability.

5 Methodology

We outline the methodology used in our study on how XAI techniques are integrated with deep learning models. The methodology included the following key steps; data collection, data preprocessing, deep learning models, black box models, XAI models, and model explanations. Figure  6 provides an overview of the APT detection enhancement process, illustrating the various stages from data collection to model explanation. This framework aims to provide a clear understanding of the integration of different models and techniques for effective and concise APT detection.

Proposed XAI pipeline for APT detection

5.1 Data collection

We gathered data from various sources, including network logs, system events, and user activities. The latest dataset from the Symantec Virtual Conference (SVC) 2021, SCVIC-APT-2021 (Liu et al. 2022 ) were utilized due to their comprehensive logs of simulated APT attacks and normal network traffic.

5.2 Data preprocessing

To ensure the quality and consistency of the data, the following preprocessing steps were performed:

Data cleaning involves identifying and removing irrelevant or redundant data points to improve the quality of the dataset (Sakthipriya et al. 2024 ) (Ridzuan and Zainon 2019 ). Irrelevant data points, such as incomplete logs or noise generated by benign activities, do not contribute to APT detection. Redundant data points are duplicate records that can skew the analysis.

Normalization ensures that data values are standardized to a common scale, which is crucial for the performance of machine learning models (Davis et al. 2020 ). Different features in the dataset may have varying scales, and normalization helps bring them to a comparable range (Siddiqi and Pak 2021 ).

Feature engineering involves identifying and extracting relevant features that can distinguish between normal and malicious activities (Abbas et al. 2023 ). This process enhances the model’s ability to detect APTs by providing it with informative and discriminative attributes (Abu Bakar et al. 2023 ).

5.3 Deep learning models

Deep learning models have significantly advanced the field of cybersecurity, particularly in detecting advanced persistent threats (APTs). We utilized several deep learning models, each suited to different types of data:

Convolutional Neural Networks (CNNs) are widely employed for their ability to automatically learn and extract spatial features from raw data, making them effective for high-dimensional data (Taye 2023 ). CNNs apply convolutional layers to input data, use filters to detect spatial hierarchies and identify various patterns and structures, which is particularly useful for datasets such as images or network traffic logs (Ersavas et al. 2024 ). CNNs automatically extract spatial features through layers of convolution and pooling. This hierarchical feature extraction process enables CNNs to construct increasingly abstract representations of the input data, from low-level details to high-level concepts (Geng and Niu 2024 ). Leveraging their strengths in feature extraction and pattern recognition, CNNs effectively analyze patterns in network traffic, such as abnormal packet flows or unusual access behaviors, allowing them to detect complex and subtle anomalies that signify APTs. By using their powerful feature extraction and pattern recognition capabilities, CNNs detect these intricate patterns and enhance the detection of sophisticated cyber threats.

Recurrent Neural Networks (RNNs) are another crucial type of deep learning model, that are adept at capturing temporal patterns in network traffic, such as time-series data, thereby enhancing the detection of APTs (Hewamalage et al. 2021 ). RNNs are specialized for sequential data, making them suitable for analyzing time-series data such as network traffic logs (Das et al. 2023 ). RNNs have internal memory states that enable them to retain information about previous inputs while processing current ones, effectively capturing temporal dependencies. Long Short-Term Memory (LSTM) networks, a type of RNN, are designed to handle long-term dependencies by using gates to control the flow of information, preventing the vanishing gradient problem commonly encountered in standard RNNs (Smagulova and James 2020 ). RNNs and LSTMs can model the sequential nature of network activities, such as user login sessions or data transfer patterns over time (Al-Selwi et al. 2024 ). This helps in identifying unusual sequences of events that could indicate an ongoing APT attack. For example, RNN-based IDSs can detect APTs by identifying actions that deviate from typical behavior, such as accessing sensitive files at odd hours, which could indicate a compromised account (Keshk et al. 2023 ).

Autoencoders (AEs) are also extensively utilized for anomaly detection by learning to represent normal behavior and subsequently identifying deviations from this learned behavior, thus enabling the detection of potential threats (Schneider et al. 2022 ). Autoencoders are effective for anomaly detection in network traffic, as they can identify unusual patterns that deviate from the learned normal behavior (Hdaib et al. 2024 ). This makes them useful for detecting subtle and rare event characteristic of APTs (Salim et al. 2023 ). For example, an autoencoder trained on normal network traffic can flag unusual login times or data access rates as anomalies, helping security analysts identify and respond to potential threats.

5.3.1 Transformer-based approach

Transformer architectures, which were originally developed for natural language processing, have been utilized for APT detection. This model can focus on important parts of the input sequence, thus enhancing the detection of relevant events and activities within large volumes of data. In recent advancements, (Zhang et al. 2023 ) proposed an intrusion detection method that leverages the strengths of both the Transformer and LSTM models. This combination results in robust intrusion detection with high accuracy and efficient processing capabilities. However, the complexity of integrating both models may pose challenges for real-time processing. (Ullah et al. 2023 ) developed IDS-INT, a system utilizing transformer-based transfer learning specifically designed for imbalanced network traffic. This method significantly improves the detection rates for minority classes, thereby addressing a common issue in network security. Nonetheless, the requirement for large volumes of labeled data presents a notable limitation, particularly in scenarios with imbalanced datasets. (Y.Liu and Wu 2023 ) focused on enhancing the standard transformer model to increase the intrusion detection performance. Their improvements resulted in significant accuracy gains; however, the computational intensity of the enhanced model may hinder real-time detection applications.(Y. Wang and Li 2023 ) proposed an anomaly-detection method for time-series data based on transformer reconstruction. This approach offers accurate and timely anomaly detection. However, managing high-dimensional time-series data remains challenging, which may impact overall performance.

(Ullah et al. 2022 ) introduced an explainable system using transformer-based transfer learning combined with multi-model visual representation. This approach enhances interpretability and accuracy; conversely, reliance on visual representation can result in increased computational overhead. (Z. Zhang and Wang 2022 ) proposed an efficient intrusion detection model that integrates CNNs with transformer models. This hybrid approach achieved high accuracy with reduced computational costs. Nevertheless, the complexity introduced by combining the CNN and Transformer models can complicate training and maintenance processes. (Huang et al., n.d. )applied a Transformer with TSGL for Named Entity Recognition (NER) in the cyber threat intelligence domain. This method improved the accuracy of the information extraction. The limitation is that this application may be restricted to the cyber-threat intelligence domain and may not be generalizable to other areas.

5.3.2 Training and validation

The Data were split into training and validation sets. The models were trained on the training set, and their performance was validated on the validation set. Techniques such as cross-validation were used to ensure robust evaluation. These studies demonstrate the potential of transformer-based models for enhancing the effectiveness and interpretability of IDSs. Each method has unique strengths, while also presenting specific challenges that need to be addressed for broader application in real-world scenarios. ​.

5.4 Black box model

Understanding the decision-making processes of AI is important for organizations. Monitoring AI models and ensuring accountability is essential rather than blindly trusting them. XAI aids in machine learning (ML), deep learning, and neural network algorithms. Many ML models are often viewed as black boxes (Chennam et al., 2023 ), as neural networks are extremely difficult to interpret. Additionally, biases in AI models and performance drift due to different production data pose significant risks. Therefore, organizations need to continuously monitor and manage the model to promote AI explainability, build trust; and reduce compliance, legal, security, and reputational risk.

5.5 XAI models

XAI models are crucial for interpreting the predictions of deep learning models used in APT detection. Techniques such as LIME, SHAP, and LRP provide insights into the model’s decision-making process, enhancing transparency and trustworthiness. The details of this process are presented below.

5.5.1 Local interpretable model-agnostic explanations (LIME)

LIME creates local approximations of the model by perturbing the input data and fitting a simple, interpretable model to these perturbed instances, helping explain individual predictions by highlighting the specific features that influence the decision. LIME is widely used for interpreting local predictions (Hasan et al. 2023 ) by learning local linear approximations (Ibrahim et al. 2023 ). This algorithm provides local explanations of individual predictions (Volkov and Averkin 2023 ).

5.5.2 Shapley additive explanations (SHAP)

SHAP uses Shapley values from cooperative game theory to indicate the contribution of each feature to the model’s predictions, providing a consistent measure of feature importance (Hasan et al. 2023 ). This technique determines which features are most influential in detecting APT activities and offers global explanations, thus helping cybersecurity experts understand the model’s decision-making process. For example, SHAP can highlight unusual login times or data transfer volumes that might indicate an APT attack (Lundberg and Lee 2017 ). By assigning contribution values to each feature, their importance in decision making can be demonstrated (Band et al. 2023 ). This method, which relies on Shapley’s principles from cooperative game theory, has been effective in various areas, including IDSs.

Specifically, SHAP proposes the following three methods:

kernelSHAP: A model-agnostic version of the algorithm that operates only with the inputs and outputs of the function (Remman et al. 2021 ).

treeSHAP: Computes the Shapley values on tree-based classification models (Wallsberger et al. 2022 ).

deepSHAP: Optimizes the computation for neural network architectures (Meister et al. 2021 ).

By utilizing these methods, SHAP provides a comprehensive and interpretable framework for understanding and explaining model predictions.

5.5.3 Layer-wise relevance propagation (LRP)

The LRP computes relevance scores for individual input features by backpropagating the relevance of the class output node in a layer-wise fashion down to the input layer (Bach et al. 2015 ). This propagation adheres to a strict conservation property, ensuring that the relevance received by any neuron is redistributed equally. In CNNs, the LRP carries information about the relevance of the output class from higher layers back to the input pixel space layer by layer (Gu et al. 2019 ). By assigning relevance scores to input features, the LRP aims to interpret their contributions to model decisions. This technique requires access to the neural network’s structure because it performs a backward pass through the network to compute relevance scores(Montavon et al. 2019 ). This method enhances the interpretability of neural networks by providing insight into the importance of individual features in the decision-making process of the model.

By integrating deep learning models with XAI techniques, we aimed to improve the detection accuracy and interpretability of APT detection systems. This approach ensured that cybersecurity experts could respond effectively to threats and understand the rationale behind specific decisions made by the models.

6 Explainable AI in cybersecurity

XAI is a type of artificial intelligence (AI) that focuses on the study and development of techniques to enable machines to behave in smart ways that humans can understand (Gunning, 2019). The goal is to create models that are easier to understand and more accurate. This allows people to trust and control their next-generation AI partners (Capuano et al. 2022a ). The use of XAI techniques has gained significant attention in cybersecurity, especially for APT detection (Pawlicki et al. 2024 ).

XAI aims to provide transparency and explainability (Saeed and Omlin 2023 ). This shows how APT detection models make decisions so that cybersecurity experts can understand and trust the model’s prediction (Haque et al. 2023 ). Moreover, XAI systems should respond to feedback from cybersecurity experts or decision-makers and adjust their actions accordingly. By incorporating XAI methods in APT detection systems, organizations can improve their defense strategies, enhance operational efficiency, and increase transparency and accountability. However, current XAI-related work in cybersecurity still lacks sufficient comparisons of different XAI algorithms in terms of metrics such as model specificity, scope, and methodology (Yang et al. 2023 ). Improving the coverage of detection system studies that leverage modern interpretability techniques will reinforce the technical depth of APT detection.

Timeline of XAI review papers published from 2018 - 2024

Figure  7 illustrates the timeline of the significant XAI review papers published from 2018 to 2024. This timeline provides a historical perspective on the development and evolution of XAI techniques, highlighting key contributions and advancements in the field. The exploration of XAI has garnered significant interest across various domains, particularly in enhancing the interpretability and transparency of complex machine learning models. This review delves into the contributions of researchers in advancing the field of XAI.

(Reis et al. 2019 ) developed a framework for explainable machine learning aimed at detecting fake news, focusing on both interpretability and accuracy. Their approach highlighted the necessity of transparent AI models to mitigate misinformation. (Xu et al. 2019 ) provided a brief survey of the history, research areas, approaches, and challenges of XAI, offering a foundational understanding of the field’s evolution.

Insider threats pose significant cybersecurity challenges that common security solutions do not adequately address. (Homoliak et al. 2020 ) proposed a structural taxonomy and novel categorization to organize and clarify insider threat incidents and defense solutions. They utilized a grounded theory method for rigorous literature review, categorizing incidents, datasets, analysis, simulations, and defense solutions. Their taxonomy builds on existing frameworks and the 5W1H information gathering method. This survey aimed to enhance insider threat research by providing a structured taxonomy, an overview of publicly available datasets, references to existing case studies and frameworks, and a discussion on trends and future research directions.

(Speith 2022 ) explored deep learning-based XAI concepts, proposed a taxonomy method, and created a timeline of key XAI studies. They demonstrated the need for structured approaches to navigate the rapidly growing field of explainable artificial intelligence (XAI). The recent surge in publications related to XAI has created a daunting challenge for those seeking to start or stay current with the latest developments. (Giudici and Raffinetti 2021 )introduced the Shapley-Lorenz method for XAI, which focuses on fairness and transparency in AI decisions. Their work emphasized ethical considerations in AI model explanations. (Fouladgar and Främling 2020 ) reviewed both the practical and theoretical aspects of XAI and proposed a framework for integrating XAI into various applications. They bridge the gap between theory and practice, highlighting the versatility of XAI methods. (Kuhn et al. 2020 ) discussed combinatorial methods for enhancing the interpretability of AI models and provided insights into advanced techniques for model interpretation. (Angelov et al. 2021 ) presented an analytical review of XAI methods, critically evaluating their strengths and weaknesses. They offered a nuanced perspective on the effectiveness of different XAI techniques in various contexts. (Linardatos et al. 2021 ) reviewed machine learning interpretability methods, categorized them and discussed their applications across different domains. Their work emphasized the importance of methodological categorization in understanding XAI techniques.

(Sharma et al. 2022 ) explored the application of XAI techniques in cybersecurity, and proposed methods to enhance transparency and trustworthiness. They address the critical need for interpretable AI models in securing digital infrastructure. (Zhang et al. 2022 )surveyed current literature on XAI techniques in cybersecurity, emphasizing the need for transparent and accountable models. They propose a clear roadmap for future XAI research in this domain. provided an overview of various XAI methods, summarizing their applications and effectiveness. This concise overview serves as a valuable reference for researchers and practitioners. (Capuano et al. 2022b )surveyed XAI techniques in cybersecurity, and proposed approaches to enhance interpretability and effectiveness. They highlighted the evolving landscape of XAI applications for securing digital environments.

(Rjoub et al. 2023 ) reviewed XAI techniques in cybersecurity, and proposed approaches for enhancing the interpretability of AI models. Their work focused on making AI systems more transparent and understandable. (Band et al. 2023 ) conducted a systematic review of interpretability methods in medical health applications, and proposed a framework for XAI in healthcare. They demonstrated the critical role of interpretability in clinical decision making. (Hassija et al. 2024b ) provided a meticulous review and comprehensive analysis of state-of-the-art XAI models to address their complexity and lack of interpretability.. They offer insight into overcoming the challenges associated with complex AI systems. (Shams Khoozani et al. 2024 ) discussed the challenges, innovations, and future directions of concept-supported XAI, navigating the landscape of XAI research and application.

These contributions collectively advance the field of XAI by addressing key challenges, proposing innovative methods, and highlighting practical applications of XAI across various domains. This review highlights the importance of continued research and development on XAI to ensure the deployment of trustworthy and interpretable AI systems.

6.1 Explanation methods

Before delving deeper into the potential use of XAI for APT detection, we first briefly review XAI techniques. We investigated the most prominent, state-of-the-art XAI techniques. We analyzed the explanations provided by LIME, SHAP (Galli et al. 2024 ) LRP (Bach et al. 2015 ), and the attention mechanism. XAI techniques represent a diverse range of approaches. LIME focuses on local model explanations through perturbations of input data,  whereas SHAP uses a game-theoretical approach to allocate contributions to each feature. The LRP propagates relevance through neural network layers, and attention mechanisms highlight the input features attended to by the model.

LIME and SHAP offer model-agnostic explainability, making them applicable to a wide range of models, including neural networks. Decision trees and rule-based models are inherently interpretable. (Sarker et al. 2024 ) proposed a decision tree-based model for intrusion detection, which allows for easy interpretation of detection rules.

The LRP specifically addresses layer-wise relevance in neural networks, and attention mechanisms are commonly used in neural networks for sequence data. LIME and SHAP have gained widespread acceptance in the research community due to their versatility and effectiveness in describing complex models. The LRP is renowned for its application in neural networks, and attention mechanisms have become standard in natural language processing and computer vision tasks.

6.2 Integration of XAI techniques in deep learning models

This paper thoroughly explores how XAI techniques, such as feature attribution and saliency maps, enhance deep learning models for APT detection (Lo et al. 2023 ). Specifically, it demonstrates how these approaches identify critical factors in the model’s decision-making process. By incorporating SHAP, LIME, and LRP into the deep learning framework, this paper offers a comprehensive approach to improve interpretability. This integration clarifies influential factors and helps identify potential threats, thereby increasing the efficacy and transparency of cybersecurity measures.

This paper uses feature attribution methods to determine the importance of each factor in the model’s predictions and employs saliency maps to visualize critical regions in network traffic data (Gevaert et al. 2024 ). By utilizing these XAI approaches, the interpretability of deep learning models can be enhanced. This transparency allows cybersecurity experts to understand and trust the model’s predictions, making it easier to validate and fine-tune the detection systems (Sharma et al. 2022 ). Furthermore, integrating XAI helps pinpoint the factors most indicative of APT activity, improving detection accuracy and enabling quicker response times by focusing on the most relevant data aspects.

The paper includes practical examples and case studies demonstrating the application of XAI in real-world scenarios, illustrating how its use leads to more effective and transparent APT detection systems.

6.3 Comparative analysis of XAI- based APT detection

In recent years, the integration of XAI techniques with deep learning models has gained significant attention in the field of APT detection. This subsection provides a comprehensive overview of recent advancements from 2020 to 2024, highlighting various approaches and their effectiveness in enhancing cybersecurity measures.

(Singh et al. 2024 ) introduced SFC-NIDS, a sustainable and explainable network intrusion detection approach that analyzes VM traffic at the hypervisor level. It uses a gradient descent-based flow filtering mechanism, auto-encoders to reconstruct traffic features, and a 1D-CNN to detect malicious flows. When validated with hypervisor traffic artifacts and the KDD99 dataset, it achieves 98.9% and 99.97% accuracy, respectively. The strengths of these methods include high accuracy, adaptability, and explainability, but they are difficult to implement and specific to hypervisor environments, requiring further validation with various datasets.

(Khan et al. 2024 ) presented a novel security model for biomedical data collection and transmission, addressing privacy, security, and reputation concerns in medical networks. They introduced a threat-vector database based on the dynamic behaviors of smart healthcare systems and designed an improved SRU network to mitigate fading gradient issues and enhance the learning process by reducing computational costs. This approach is parallelizable and computationally efficient, dynamically adjusting the number of participating clients to reduce the communication overhead. Additionally, the model enhances the understanding of security experts by visualizing the decision process and explaining feature relevance. Compared with existing methods, this security model thoroughly analyzes and detects severe security threats with high accuracy, reduces overhead, lowers computational costs, and enhances the privacy of biomedical data. However, the potential complexity in implementing the novel security model across diverse healthcare networks and the need for further validation and testing in real-world healthcare environments to ensure robustness and effectiveness are noted as weaknesses.

Building on the idea of integrating explainability into AI-based cybersecurity solutions. (Galli et al. 2024 ) proposed a framework that integrates XAI methodologies within AI-based malware detection processes to address the lack of interpretability in traditional machine learning (ML) and deep learning (DL) approaches. The framework incorporates four XAI methods, such as SHAP, LIME, LRP, and Attention mechanisms. The key strengths of the proposed system include the integration of multiple XAI methods to improve model interpretability, thorough evaluation across diverse datasets, insightful comparisons of the LSTM and GRU models, and enhanced real-world applicability due to improved explainability. However, the study also faces challenges such as increased computational complexity, the need for further validation across different types of malware and cybersecurity contexts, and potential trade-offs between model explainability and performance.

Similarly, (Hasan et al. 2023 ) developed a highly effective model for identifying APTs using boosting-based machine learning methods, with XGBoost achieving an impressive weighted F1 score of 0.97. They enhance model interpretability by integrating SHAP, providing actionable insights for stakeholders. The key strengths of their work include exceptional predictive accuracy and enhanced transparency through XAI. However, potential challenges include computational complexity and the need for broader validation across various cyber threats. This approach underscores the promise of boosting-based XAI models in cybersecurity.

Expanding on the use of XAI in cybersecurity, (Zolanvari et al. 2023 ) employed TRUST XAI using multimodal Gaussian distributions and the TRUST Explainer technique. They evaluated their model on multiple datasets, including WUSTL-IIoT, NSL-KDD, and UNSW, and achieved 98% accuracy. A limitation of this study is that they did not address the computational resources necessary to implement their models.

(Khan et al. 2022 ) proposed a novel explainable deep learning framework for cyber threat discovery in Industrial IoT (IIoT) networks, addressing the critical need for data integrity and accuracy. They used an autoencoder-based detection system with convolutional and recurrent networks, employing a two-step sliding window technique to enhance feature extraction from raw time series data. Fully connected networks classify and explain attack events using temporal and spatial features. Empirical results demonstrated robust performance, with the framework outperforming contemporary methods. However, its complexity might limit scalability, and further exploration is needed for generalizability, real-time adaptation, and explainability depth.

(Khan et al. 2024 ) focused on bidirectional simple recurrent units (SRUs) and developed the XSRU-IoMT method, which was tested on the ToN_IoT dataset. Their model achieved 98% accuracy, but was validated on only a single dataset, limiting its generalizability. (Zhou et al. 2022 ) used an M&M Decision Tree model with prime implicant explanation techniques on various DDoS attack datasets, and achieved perfect recall and F1 scores. This method, which is based on an artificial immune system, may not be suitable for all intrusion-detection scenarios.

Another approach by (Patil et al. 2022 ) implemented an ensemble method using voting classifiers and LIME for interpretability on the CICIDS2017 dataset, achieving 96.25% accuracy. Reliance on a black box model may limit its transparency. (Barnard et al. 2022 ) combined extreme gradient boosting (XGBoost) with autoencoders (AEs) and utilized SHAP for interpretability, achieving 93% accuracy on the NSL-KDD dataset. However, this framework was assessed using only this dataset.

(Houda et al. 2022 ) applied deep neural networks (DNNs) with RuleFit, LIME, and SHAP on the NSL-KDD and UNSW NB-15 datasets, achieving 88% accuracy. However, their framework evaluation on only two datasets may not be generalizable to other datasets. Le et al. (2022) used ensemble trees (DT and RF classifiers) and SHAP on NF-BoT-IoT v2 and NF-ToN-IoT-v2 datasets, and achieved  100% accuracy. However, the method’s long training time might render it unsuitable for real-time IDS systems.

(Kuppa and Le-Khac 2021 ) explored autoencoders with counterfactual explanations on the Leaked Password Dataset, achieving 96.7% accuracy. Their study did not cover the broader aspects of machine learning or AI beyond cybersecurity. (Liu et al. 2021 ) proposed FAIXID using Boolean Rule Column Generation (BRCG) with data cleaning methods on real-world datasets, which achieved 87% accuracy. In this study, they did not provide detailed information on the size and diversity of the datasets used for the evaluation. Wali et al. ( 2021 ) utilized a primary Random Forest Classifier (RFC) with SHAP, achieving 98.5–100% accuracy on the Hop Skip Jump Attack and CICIDS datasets. The proposed framework requires significant computational resources.

(Antwarg et al. 2021 ) employed autoencoders with kernel SHAP on the KDD Cup 1999 and Credit Card datasets, focusing on explaining anomalies with a mean MSE of 0.0006. However, this method may not be applicable to APTs. Wang et al. ( 2020 ) used one-vs-all and multiclass classifiers with SHAP on the NSL-KDD dataset, achieving accuracies between 80.3% and 80.6%. The proposed framework was assessed using an outdated NSL-KDD dataset, which limited its relevance. Table  4 summarizes the XAI-based methods employed in cybersecurity.

This review of XAI-based APT detection methods highlights the diversity and advancements in the field while also identifying specific shortcomings that need to be addressed. These insights pave the way for future research to develop more robust, interpretable, and effective APT detection systems.

6.4 Case studies

This section presents various case studies that demonstrate the application of XAI techniques for combating APTs. These scenarios provide a comprehensive overview of how XAI can enhance the detection and mitigation of APTs.

Figure  8 compares different attack scenarios with the corresponding XAI techniques applied to improve detection and interpretation. The figure shows how techniques such as SHAP, LIME, LRP, and attention mechanisms enhance the interpretability of models in various attack scenarios such as phishing, insider threats, DoS attacks, and malware.

Attack scenarios vs. XAI techniques

To address various attack scenarios, recent studies have applied XAI techniques to improve the detection and interpretation of cyber threats. (Adebowale et al. 2023 ) proposed integrating CNN and LSTM models with LIME to enhance the email classification model for phishing attacks. This hybrid XAI approach effectively addresses the challenges posed by large datasets and significantly improves classifier prediction performance.

(Homoliak et al. 2020 ) utilized SHAP to explain the global behavior of a detection model. Their study emphasized the diverse nature of insider threats and demonstrated how SHAP can identify common patterns and unique contributing factors across different clusters, such as fraud, IP theft, and sabotage. For DoS attacks, (Hariharan et al. 2021 ) applied Permutation Importance, SHAP, LIME, and Contextual Importance and Utility algorithms to provide a unified measure of feature importance and contribution to model predictions. Their case study revealed key insights into the impact of various features on IDS prediction performance, highlighting the most influential features for detecting DoS attacks.

A downloader, a type of Trojan horse, downloads and installs malicious software without user consent. The explanation provided by the LRP highlights the Image 4 call, which attempts to copy files from one part of the file system to another, as discussed by (Galli et al. 2024 ). Finally, for malware attacks, (Galli et al. 2024 ) integrated the SHAP, LIME, LRP, and Attention mechanisms with robust deep learning models (e.g., LSTM and GRU) trained on labeled datasets. Their findings demonstrated that these XAI techniques significantly enhance the interpretability of malware detection models, ensuring that they remain effective and understandable in real-world applications.

Beyond cybersecurity, XAI has significant applications in other domains, such as the following:

Healthcare providers have observed an increase in phishing attempts and suspicious activity in their networks. Given the sensitivity of patient data, they sought to improve their security measures by using an advanced detection system that combines deep learning and XAI techniques (Kalutharage et al. 2024 ).The data used include email traffic data collected from the providers’ email servers, encompassing metadata and content analysis, as well as network logs from firewalls and intrusion detection systems (IDSs) that capture network activity and possible intrusions. CNNs can analyze email traffic and detect phishing attempts (McGinley and Monroy 2021 ), whereas RNNs can analyze sequential network logs for signs of APT activity. XAI techniques, such as SHAP, explain the phishing detection of CNN models, while LIME provides interpretability for network activity anomalies. The explanations provided by SHAP and LIME helped security teams identify specific email campaigns targeting employees and unusual data streams that indicated an APT in progress. This facilitated immediate corrective actions, including email filtering and network segmentation (Jia et al. 2021 ).

A major financial institution experienced unusual network activity, suggesting a potential APT attack. The institution’s security team decided to use an integrated approach that combines deep learning models and XAI techniques to identify and mitigate the threat. The data used included network traffic data collected from the institution’s internal network, encompassing packet headers, payloads, and flow data over a six-month period. Additionally, system and application logs from critical servers and endpoints were analyzed, providing insights into user activities and system changes. SHAP values indicated that unusual login times and connections to known malicious IP addresses were significant factors in these detections. The LIME explanations for specific alerts revealed that certain endpoints repeatedly communicated with suspicious external servers, suggesting a potential APT foothold. These insights allowed the security team to prioritize these alerts and initiate a targeted investigation. The integrated approach not only detected the APT early but also provided clear explanations that helped the security team understand the attack vector and take swift action. As a result, the financial institution was able to isolate the affected endpoints, mitigate the threat, and prevent data exfiltration.

These use cases demonstrate how XAI techniques can be applied to various aspects of APT detection, from email classification and insider threat detection to network anomaly detection and malware identification. By examining these case studies, cybersecurity experts can better understand the practical applications of XAI and see how they can enhance their ability to investigate and mitigate APT attacks. These examples also highlight the potential benefits of XAI in real-world APT detection scenarios.

6.5 Role of XAI in APT detection

To effectively integrate XAI with deep learning for APT detection, several enhancements are necessary to address the unique challenges posed by sophisticated threats. (1) Handling sparse data is critical because traditional methods struggle with sparse datasets. Integrating XAI with anomaly-detection techniques, such as autoencoders, can help identify and explain anomalies in sparse data. (2) The high-dimensional nature of APT data requires dimensionality reduction techniques, such as Principal Component Analysis (PCA) or t-distributed Stochastic Neighbor Embedding (t-SNE), to simplify data complexity while retaining essential features, thereby improving interpretability. Moreover, APTs are characterized by evolving tactics, techniques, and procedures (TTPs). To adapt to these changes, XAI techniques must be dynamic and capable of real-time updating through continuous learning frameworks. This allows models to refine their understanding of new attack patterns and provide explanations that highlight these adaptations. (3) In terms of high-dimensional space, advanced feature importance techniques, such as Integrated Gradients, should be used to attribute importance across different layers of neural networks, offering a deeper understanding of model decisions. Finally, the implementation of a real-time feedback loop, in which cybersecurity experts can interact with the XAI system to refine explanations is crucial. This can be achieved using reinforcement learning to adapt models based on analyst feedback, ensuring that explanations remain accurate and relevant (Kute et al. 2021 ). To address sparse data, high-dimensional space, evolving tactics, and the need for real-time adaptation, this review aims to provide a more robust framework for explaining AI models used in APT detection, thereby advancing state-of-the-art models in cybersecurity.

In the discussion section, we explore the implications of these findings and how they contribute to advancing APT detection. Specifically, we discuss how the enhanced interpretability provided by XAI techniques can lead to more effective threat mitigation and greater trust in AI-driven security solutions.

7 Discussion and recommendations

This section delves into the need for explainability, challenges, and future directions in APT detection using deep learning and XAI. Traditional anomaly detection techniques are effective for known threats but struggle with new, evolving threats and large-scale data. Recurrent Neural Networks (RNNs) show promise in detecting complex and sequential patterns in network traffic, but they face challenges in interpretability and computational demands. XAI offers potential solutions by making model decisions transparent and understandable, adopting trust and collaboration between human experts and AI systems. This integration aims to increase the effectiveness and robustness of APT detection systems.

7.1 Key considerations for XAI

7.1.1 the need for explainability.

XAI is crucial in making deep learning models interpretable and transparent, especially in cybersecurity where trust and reasoning are essential. Various techniques have been developed to enhance model interpretability. Feature importance methods, such as permutation importance and SHAP, determine the significance of each feature in the decision-making process (Hariharan et al. 2021 ). Partial Dependence Plots (PDPs) visualize the marginal effect of a feature on the predicted outcome (Ding et al. 2021 ). Activation Maximization identifies inputs that maximize the activation of specific neurons, revealing what each neuron “wants to see” (Zeltner et al. 2021 ). Saliency Maps highlight parts of the input most relevant to the model’s decision (Brunke et al. 2020 ). LIME explains the predictions of any classifier by learning a simple model locally around the prediction (Houda et al. 2022 ). Layer-wise Relevance Propagation (LRP) backpropagates the output through the network to assign a “relevance score” to each input feature (Seibold et al. 2021 ). These techniques collectively enhance the transparency and trustworthiness of AI models, facilitating their adoption in critical cybersecurity applications.

7.1.2 Need for trust

Trust is essential in cybersecurity, and XAI methods enhance this by providing clear, understandable explanations of AI model predictions. The SHAP and LIME techniques help experts determine which features influence decisions, fostering confidence in AI systems [1]. Transparent models reduce perceived risk and ensure fair, accurate operations by identifying and correcting biases (Saeed and Omlin 2023 ). This increased trust leads to better decision-making and faster response times, which are crucial in mitigating APTs. By integrating XAI, cybersecurity experts can confidently use AI systems, resulting in more robust and effective threat detection and response strategies (Sourati et al. 2023 ).

7.1.3 Need for reasoning

XAI enables cybersecurity experts to understand and validate model reasoning. This leads to better human-AI collaboration. This makes the detection of APT threats more effective. When security teams understand how the model works, they can better mitigate the potential vulnerabilities to adversarial attacks. This can enhance the robustness of the APT detection system more effectively. By seeing how the model makes decisions, security teams can identify and fix errors more efficiently. This helps improve the APT detection system.

XAI enables cybersecurity experts to understand and validate AI model reasoning, enhancing human-AI collaboration and APT detection (Sarker et al. 2024 ). By revealing how models arrive at their conclusions, XAI helps identify and mitigate vulnerabilities to adversarial attacks, improving system robustness. Understanding model decisions allows security teams to correct errors efficiently, maintaining detection accuracy (Došilović et al. 2018 ). XAI also fosters a shared understanding between AI systems and human operators, promoting the trust and effective use of AI in cybersecurity, ultimately leading to better detection and response strategies. The next subsection addresses the current issues associated with black-box models.

7.2 Current issues surrounding black-box models

One problem with using deep learning models for APT detection is their black box behavior, which makes it difficult to interpret and explain their decisions. (Kute et al. 2021 ). For example, when dealing with APT incidents, cybersecurity experts must understand why a particular event is deemed malicious. The lack of transparency in black-box models can lead to several issues. Cybersecurity experts may not trust a model’s predictions without understanding the reasoning, especially when false positives or false negatives can have severe consequences. (Galli et al. 2024 ). When a model makes an incorrect prediction, identifying the root cause and making the necessary adjustments becomes challenging without insight into the model’s decision-making process (Samek et al., 2017). During a security incident, organizations may face difficulties in explaining and justifying actions based on the model’s predictions, potentially leading to legal and regulatory issues.

Additionally, black-box models are vulnerable to adversarial examples (Jia et al. 2024 ), which are carefully crafted inputs that deceive a model and lead to incorrect predictions. Without a clear understanding of how the model works internally, detecting and mitigating such attacks is more difficult (Ali et al. 2023 ).Therefore, promoting AI explainability is imperative to ensure the reliability and robustness of AI systems in critical applications such as cybersecurity.

Black box models pose significant challenges because of their opaque nature, making it difficult to interpret and explain their decisions. This lack of transparency can lead to issues such as biases, performance drifting, and vulnerability to adversarial attacks. In cybersecurity, the inability to understand and justify model predictions can undermine trust, complicate incident response, and lead to legal and regulatory challenges. XAI is essential for enhancing model transparency, building trust, and mitigating various risks, ensuring the reliability and robustness of AI systems in critical applications.

7.3 Research challenges and recommendations

Research challenges for APT detection

Figure  9 shows that APT detection is technically challenging for several reasons. To address the challenges identified in APT detection using deep learning and XAI, several practical solutions are recommended.

Highly imbalanced datasets can hinder the performance of AI models in detecting minority class events. We address this issue using data augmentation techniques, such as the Synthetic Minority Oversampling Technique (SMOTE), to generate synthetic samples for the minority class(Y. Liu and Wu 2023 ). Employing unsupervised or semi-supervised learning techniques specifically designed to handle imbalanced data can also be effective (Zhang et al. 2020 ), (Kute et al. 2021 )

The rapidly evolving threat landscape requires the continuous adaptation of detection models. To address this problem, continual learning approaches allow models to adapt to new data without forgetting previously learned information. Automated pipelines for data collection, model retraining, and deployment ensure that the detection system remains updated with the latest threat information (Khoozani et al., 2024 ). APT tactics, techniques, and procedures (TTPs) are continuously changing, making it difficult for static models to remain effective. By integrating XAI with deep learning models, we enhance understanding and adapt to new attack patterns. This integration improves the robustness of the system against evolving threats, ensuring that it remains effective in detecting and mitigating APTs.

Comprehensive datasets for effective APT detection are lacking. This problem can be addressed through collaborative efforts among academia, industry, and government agencies to create large-scale, diverse, and labeled datasets. Promoting data-sharing initiatives encourages organizations to share anonymized threat data for research purposes, thereby enriching the pool of available datasets (Ferrag et al. 2020 ).

Integrating XAI techniques into APT detection systems is challenging because of the complexity and scale of operational environments. (Kuhn et al. 2020 ) developed modular XAI frameworks that can be easily integrated into existing cybersecurity infrastructures. In addition, providing training and resources for cybersecurity experts to use XAI tools effectively enhances their practical applicability.

Deep-learning models are often criticized for their difficulty in interpretation and lack of transparency (Liang et al. 2021 ). XAI techniques, such as SHAP and LIME, highlight the features that most influence model predictions and provide visual explanations that are easy to interpret. AI models must provide clear, understandable explanations for their decisions to build trust and enable cybersecurity experts to make informed decisions (Galli et al. 2024 ).

Black-box models can be tricked by adversarial examples, which can lead to incorrect predictions. Understanding the internal operation of the model is crucial for detecting and mitigating such attacks. The development of XAI systems that reveal the decision-making process of the model can help identify and counter adversarial inputs effectively (Kenny et al. 2021 ).

Our discussion highlights the strengths and potential of integrating XAI with deep learning for APT detection. The enhanced interpretability provided by XAI techniques allows cybersecurity experts to understand and trust model predictions, leading to more effective threat mitigation. In conclusion, we summarize our contributions and suggest directions for future research, emphasizing the need for continued innovation in this critical area of cybersecurity.

8 Conclusions and future research directions

Future research directions for APT research

Our research addresses the challenges in APT detection by integrating XAI with deep learning models. As illustrated in Fig.  10 , future research directions for APT detection focus on several key areas. First, examining the potential of deep feature extraction methods is essential for enhancing detection capabilities. Second, integrating XAI techniques with deep learning models is crucial, as it offers transparent and effective explanations for APT detection. Additionally, optimizing automatically extracted features can enhance detection accuracy, reduce human bias, and improve resource efficiency. Furthermore, augmenting existing datasets by generating synthetic examples of APT activities and benign behaviors can enrich the data available for training. Moreover, it is important to evaluate detection models using comprehensive evaluation metrics and scenario-based security testing to ensure robust performance. Finally, maintaining privacy and security is essential when robust detection models that safeguard data privacy are constructed, thus ensuring the overall integrity of the system.

This review provides a comprehensive overview of state-of-the-art models for dvanced persistent threats (APTs) detection. Through extensive analysis, we found that deep learning-based techniques, especially those incorporating XAI methods are increasingly prevalent. These approaches address significant challenges in traditional IDSs, such as low detection accuracy, high false-positive rates, and difficulties in detecting unknown or early-stage attacks. By incorporating XAI techniques such as SHAP, cybersecurity experts can understand the contribution of each feature to the model’s prediction, such as unusual login times or unexpected IP addresses, allowing for quicker verification of alerts and reduced false positives. Moreover, the transparency provided by XAI models enhances trust and accountability in detection systems. Users and stakeholders can audit and validate the decisions made by the models, ensuring that the detection process is transparent and reliable. Empirical evidence from recent studies further supports the effectiveness of integrating XAI with deep learning, demonstrating improvements in detection accuracy and a significant reduction in false positives. This not only validates our approach but also underscores its practical benefits in real-world applications.

Our review also highlights the strengths and weaknesses of various APT detection methods across different datasets, techniques, and experimental results. By identifying gaps and challenges in current methodologies, we propose integrating XAI with deep learning models to advance the state-of-the-art. This integration not only improves detection accuracy and scalability but also provides clear, interpretable insights into the model’s decision-making process, ensuring effective threat response. The impact of our findings is substantial for cybersecurity. By enhancing the transparency and interpretability of APT detection systems, we can enable more effective and timely responses to sophisticated threats. This research contributes to the development of advanced security measures that adapt to evolving cyber attacker tactics.

In conclusion, our research paves the way for more robust, scalable, and interpretable APT detection systems that effectively combat sophisticated APT attacks. By addressing both technical and interpretability challenges, we advance the state-of-the-art in cybersecurity and enhance network resilience to APTs.

Data availability

No datasets were generated or analysed during the current study.

https://symantec-enterprise-blogs.security.com/blogs/threat-intelligence/leafminer-espionage-middle-east .

https://attack.mitre.org/groups/G0099/ .

https://www.ibm.com/reports/data-breach .

Abbas G, Farooq U, Singh P, Khurana SS, Singh P (2023) Feature Engineering and Ensemble Learning-based classification of VPN and Non-VPN-Based Network traffic over temporal features. SN Comput Sci 4(5):546. https://doi.org/10.1007/s42979-023-01944-5

Article   Google Scholar  

Abu Bakar R, Huang X, Javed MS, Hussain S, Majeed MF (2023) An Intelligent Agent-based detection system for DDoS attacks using automatic feature extraction and selection. SENSORS 23(6). https://doi.org/10.3390/s23063333

Adebowale MA, Lwin KT, Hossain MA (2023) Intelligent phishing detection scheme using deep learning algorithms. J Enterp Inform Manage 36(3):747–766. https://doi.org/10.1108/JEIM-01-2020-0036

Agrawal, G.; Kaur, A.; Myneni, S. A Review of Generative Models in Generating Synthetic Attack Data for Cybersecurity. Electronics 2024, 13, 322. https://doi.org/10.3390/electronics13020322

Ahmad A, Webb J, Desouza KC, Boorman J (2019) Strategically-motivated advanced persistent threat: definition, process, tactics and a disinformation model of counterattack. COMPUTERS Secur 86:402–418. https://doi.org/10.1016/j.cose.2019.07.001

Ahmad HB, Gao H, Latif N, Aziiz A, Auraangzeb M, Khan MT (2024) Adversarial Machine Learning for Detecting Advanced Threats Inspired by StuxNet in Critical Infrastructure Networks. 2024 12th International Symposium on Digital Forensics and Security (ISDFS) , 1–7. https://doi.org/10.1109/ISDFS60797.2024.10527326

AlDahoul N, Karim A, H., Ba Wazir AS (2021) Model fusion of deep neural networks for anomaly detection. J Big Data 8(1):1–18

Ali S, Abuhmed T, El-Sappagh S, Muhammad K, Alonso-Moral JM, Confalonieri R, Guidotti R, Del Ser J, Díaz-Rodríguez N, Herrera F (2023) Explainable Artificial Intelligence (XAI): what we know and what is left to attain Trustworthy Artificial Intelligence. Inform Fusion 99:101805. https://doi.org/10.1016/j.inffus.2023.101805

Alkhadra R, Abuzaid J, AlShammari M, Mohammad N (2021) Solar Winds Hack: In-Depth Analysis and Countermeasures. 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT) , 1–7. https://doi.org/10.1109/ICCCNT51525.2021.9579611

Al-Selwi SM, Hassan MF, Abdulkadir SJ, Muneer A, Sumiea EH, Alqushaibi A, Ragab MG (2024) RNN-LSTM: from applications to modeling techniques and beyond—systematic review. J King Saud Univ - Comput Inform Sci 36(5):102068. https://doi.org/10.1016/j.jksuci.2024.102068

Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8(1):53. https://doi.org/10.1186/s40537-021-00444-8

Angelov PP, Soares EA, Jiang R, Arnold NI, Atkinson PM (2021) Explainable artificial intelligence: an analytical review. Wiley Interdisciplinary Reviews: Data Min Knowl Discovery 11(5). https://doi.org/10.1002/widm.1424

Antwarg L, Miller RM, Shapira B, Rokach L (2021) Explaining anomalies detected by autoencoders using Shapley Additive explanations. Expert Syst Appl 186:115736. https://doi.org/10.1016/j.eswa.2021.115736

Bach S, Binder A, Montavon G, Klauschen F, Müller K-R, Samek W (2015) On pixel-wise explanations for non-linear classifier decisions by Layer-wise Relevance Propagation. PLoS ONE 10(7):e0130140. https://doi.org/10.1371/journal.pone.0130140

Ballard (2021) Cybercrime apparently cost the world over $1 trillion in 2020. https://www.techradar.com/news/cybercrime-cost-the-world-over-dollar1-trillion-in-2020

Band S, Yarahmadi S, Hsu A, Biyari C-C, Sookhak M, Ameri M, Dehzangi R, Chronopoulos I, A. T., Liang H-W (2023) Application of explainable artificial intelligence in medical health: a systematic review of interpretability methods. Inf Med Unlocked 40:101286. https://doi.org/10.1016/j.imu.2023.101286

Barnard P, Marchetti N, DaSilva LA (2022) Robust Network Intrusion Detection through Explainable Artificial Intelligence (XAI). IEEE Netw Lett 4(3):167–171. https://doi.org/10.1109/LNET.2022.3186589

Bierwirth T, Pfützner S, Schopp M, Steininger C (2024) Design and evaluation of Advanced Persistent threat scenarios for Cyber ranges. IEEE Access 12:72458–72472. https://doi.org/10.1109/ACCESS.2024.3402744

Bodström T, Hämäläinen T (2019) A novel deep learning stack for APT detection. Appl Sci 9:1055. https://doi.org/10.3390/app9061055

Brown D, Cianfarani G, Vlajic N (2022) Real World snapshot of trends in IoT device and Protocol Deployment: IEEE CNS 22 poster. 2022 IEEE Conf Commun Netw Secur (CNS) 1(2). https://doi.org/10.1109/CNS56114.2022.9947257

Brunke L, Agrawal P, George N (2020) Evaluating input perturbation methods for interpreting CNNs and Saliency Map Comparison. In: Bartoli A, Fusiello A (eds) Computer vision – ECCV 2020 Workshops. Springer International Publishing, pp 120–134

Capuano N, Fenza G, Loia V, Stanzione C (2022a) Explainable Artificial Intelligence in CyberSecurity: a Survey. IEEE Access 10:93575–93600. https://doi.org/10.1109/ACCESS.2022.3204171

Capuano N, Fenza G, Loia V, Stanzione C (2022b) Explainable Artificial Intelligence in CyberSecurity: a Survey. IEEE ACCESS 10:93575–93600. https://doi.org/10.1109/ACCESS.2022.3204171

Chen J, Su C, Yeh KH, Yung M (2018) Special issue on advanced persistent threat. Future Gener Comput Syst 79(Part 1):243–246. https://doi.org/10.1016/j.future.2017.11.005

Chen L, Weng S, Peng C, Shuai H, Cheng W (2021) ZYELL-NCTU NetTraffic-1.0: a large-scale dataset for real-world network anomaly detection. In: 2021 IEEE international conference on consumer electronics-Taiwan (ICCE-TW), pp 1-2

Chennam KK, Mudrakola S, Maheswari VU, Aluvalu R, Rao KG (2023) Black box models for eXplainable artificial intelligence. In: Mehta M, Palade V, Chatterjee I (eds) Explainable AI: foundations, methodologies and applications. Intelligent systems reference library, vol 232. Springer, Cham. https://doi.org/10.1007/978-3-031-12807-3_1

Chen Z, Simsek M, Kantarci B, Bagheri M, Djukic P (2024) Machine learning-enabled hybrid intrusion detection system with host data transformation and an advanced two-stage classifier. Comput Netw 250:110576. https://doi.org/10.1016/j.comnet.2024.110576

Daoud M, Dahmani Y, Bendaoud M, Ouared A, Ahmed H (2023) Convolutional neural network-based high-precision and speed detection system on CIDDS-001. Data Knowl Eng 144:102130. https://doi.org/10.1016/j.datak.2022.102130

Das S, Tariq A, Santos T, Kantareddy SS, Banerjee I (2023) Recurrent Neural Networks (RNNs): Architectures, Training Tricks, and Introduction to Influential Research. In O. Colliot (Ed.), Machine Learning for Brain Disorders (pp. 117–138). Springer US. https://doi.org/10.1007/978-1-0716-3195-9_4

Davis A, Gill S, Wong R, Tayeb S (2020) Feature Selection for Deep Neural Networks in Cyber Security Applications. 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) , 1–7. https://doi.org/10.1109/IEMTRONICS51293.2020.9216403

de Abreu SF, Kendzierskyj S, Jahankhani H (2020) Attack Vectors and Advanced Persistent Threats. In H. Jahankhani, S. Kendzierskyj, N. Chelvachandran, & J. Ibarra (Eds.), Cyber Defence in the Age of AI, Smart Societies and Augmented Humanity (pp. 267–288). Springer International Publishing. https://doi.org/10.1007/978-3-030-35746-7_13

Ding R, Yin W, Cheng G, Chen Y, Wang J, Wang R, Rui Z, Li J, Liu J (2021) Boosting the optimization of membrane electrode assembly in proton exchange membrane fuel cells guided by explainable artificial intelligence. Energy AI 5:100098. https://doi.org/10.1016/j.egyai.2021.100098

DiPietro R, Hager GD (2020) Chapter 21 - Deep learning: RNNs and LSTM. In S. K. Zhou, D. Rueckert, & G. Fichtinger (Eds.), Handbook of Medical Image Computing and Computer Assisted Intervention (pp. 503–519). Academic Press. https://doi.org/10.1016/B978-0-12-816176-0.00026-0

Došilović FK, Brčić M, Hlupić N (2018) Explainable artificial intelligence: A survey. 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) , 210–215. https://doi.org/10.23919/MIPRO.2018.8400040

Do Xuan C, Dao MH, Nguyen HD (2020) APT attack detection is based on flow network analysis techniques using deep learning. J Intell Fuzzy Syst 39(3):4785–4801. https://doi.org/10.3233/JIFS-200694

Ersavas T, Smith MA, Mattick JS (2024) Novel applications of Convolutional Neural Networks in the age of transformers. Sci Rep 14(1):10000. https://doi.org/10.1038/s41598-024-60709-z

Fang Y, Wang C, Fang Z, Huang C (2022) LMTracker: Lateral movement path detection based on heterogeneous graph embedding. Neurocomputing 474:37-47. https://doi.org/10.1016/j.neucom.2021.12.026

Ferrag MA, Maglaras L, Moschoyiannis S, Janicke H (2020) Deep learning for cyber security intrusion detection: approaches, datasets, and comparative study. J Inform Secur Appl 50:102419. https://doi.org/10.1016/j.jisa.2019.102419

Fouladgar N, Främling K (2020) XAI-P-T: A Brief Review of Explainable Artificial Intelligence from Practice to Theory

Galli A, La Gatta V, Moscato V, Postiglione M, Sperlì G (2024) Explainability in AI-based behavioral Malware Detection systems. Computers Secur 103842. https://doi.org/10.1016/j.cose.2024.103842

Geng L, Niu B (2024) APSSF: adaptive CNN pruning based on structural similarity of filters. Int J Comput Intell Syst 17(1):129. https://doi.org/10.1007/s44196-024-00518-4

Gevaert A, Rousseau A-J, Becker T, Valkenborg D, De Bie T, Saeys Y (2024) Evaluating feature attribution methods in the image domain. Mach Learn. https://doi.org/10.1007/s10994-024-06550-x

Article   MathSciNet   Google Scholar  

Giudici P, Raffinetti E (2021) Shapley-Lorenz eXplainable Artificial Intelligence. Expert Syst Appl 167(October 2020):114104. https://doi.org/10.1016/j.eswa.2020.114104

Gu J, Yang Y, Tresp V (2019) Understanding individual decisions of CNNs via Contrastive Backpropagation. In: Jawahar CV, Li H, Mori G, Schindler K (eds) Computer vision – ACCV 2018. Springer International Publishing, pp 119–134

Haque AKMB, Islam AKMN, Mikalef P (2023) Explainable Artificial Intelligence (XAI) from a user perspective: a synthesis of prior literature and problematizing avenues for future research. Technol Forecast Soc Chang 186:122120. https://doi.org/10.1016/j.techfore.2022.122120

Hariharan S, Velicheti A, Anagha AS, Thomas C, Balakrishnan N (2021) Explainable Artificial Intelligence in Cybersecurity: A Brief Review. 2021 4th International Conference on Security and Privacy (ISEA-ISAP) , 1–12. https://doi.org/10.1109/ISEA-ISAP54304.2021.9689765

Hasan M, Islam MU, Uddin J (2023) Advanced persistent threat identification with boosting and explainable AI. SN Comput Sci 4:1–9

Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, Scardapane S, Spinelli I, Mahmud M, Hussain A (2024a) Interpreting Black-Box models: a review on explainable Artificial Intelligence. Cogn Comput 16(1):45–74. https://doi.org/10.1007/s12559-023-10179-8

Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, Scardapane S, Spinelli I, Mahmud M, Hussain A (2024b) Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. In Cognitive Computation (Vol. 16, Issue 1, pp. 45–74). Springer. https://doi.org/10.1007/s12559-023-10179-8

Hdaib M, Rajasegarar S, Pan L (2024) Quantum deep learning-based anomaly detection for enhanced network security. Quantum Mach Intell 6(1):26. https://doi.org/10.1007/s42484-024-00163-2

Hewamalage H, Bergmeir C, Bandara K (2021) Recurrent neural networks for Time Series forecasting: current status and future directions. Int J Forecast 37(1):388–427. https://doi.org/10.1016/j.ijforecast.2020.06.008

Holt, T. J., Griffith, M., Turner, N., Greene-Colozzi, E., Chermak, S., & Freilich, J. D. (2023). Assessing nation-state-sponsored cyberattacks using aspects of Situational Crime Prevention. Criminology & Public Policy, 22, 825–848. https://doi.org/10.1111/1745-9133.12646

Homoliak I, Toffalini F, Guarnizo J, Elovici Y, Ochoa M (2020) Insight into insiders and IT. ACM-CSUR 52(2):1–40. https://doi.org/10.1145/3303771

Houda ZA, El, Brik B, Khoukhi L (2022) Why should I trust your IDS? An Explainable Deep Learning Framework for Intrusion Detection systems in Internet of things networks. IEEE Open J Commun Soc 3:1164–1176. https://doi.org/10.1109/OJCOMS.2022.3188750

Huang YH, Su M, Xu YT, Liu T (n.d.) NER in Cyber threat intelligence domain using transformer with TSGL. J CIRCUITS Syst COMPUTERS. https://doi.org/10.1142/S0218126623502018

Ibrahim SM, Ansari SS, Hasan SD (2023) Towards white box modeling of compressive strength of sustainable ternary cement concrete using explainable artificial intelligence (XAI). Appl Soft Comput 149:110997. https://doi.org/10.1016/j.asoc.2023.110997

Jabar T, Mahinderjit Singh M (2022) Exploration of Mobile device behavior for Mitigating Advanced Persistent threats (APT): a systematic literature review and conceptual Framework. Sensors 22(13):4662. https://doi.org/10.3390/s22134662

Jayapradha J, Vineethkumar S, Vigneshwaran R, Ramprasath A (2024) Intrusion detection system for Phising Detection Using Convolution Neural Network. Educational Administration: Theory Pract 30(5):5565–5575. https://doi.org/10.53555/kuey.v30i5.3823

Jia W, Liu Z, Zhang H, Yu R, Li L (2024) Towards Score-Based Black-Box Adversarial Examples Attack in Real World. In Y. Pei, H. S. Ma, Y.-W. Chan, & H.-Y. Jeong (Eds.), Proceedings of Innovative Computing 2024, Vol. 4 (pp. 211–216). Springer Nature Singapore

Jia Y, McDermid J, Lawton T, Habli I (2021) The role of Explainability in Assuring Safety of Machine Learning in Healthcare. May , 1–30. http://arxiv.org/abs/2109.00520

Kalutharage CS, Liu X, Chrysoulas C, Bamgboye O (2024) Utilizing the Ensemble Learning and XAI for Performance Improvements in IoT Network Attack Detection. In S. Katsikas, H. Abie, S. Ranise, L. Verderame, E. Cambiaso, R. Ugarelli, I. Praça, W. Li, W. Meng, S. Furnell, B. Katt, S. Pirbhulal, A. Shukla, M. Ianni, M. Dalla Preda, K.-K. R. Choo, M. Pupo Correia, A. Abhishta, G. Sileno, … N. Yanai (Eds.), Computer Security. ESORICS 2023 International Workshops (pp. 125–139). Springer Nature Switzerland

Karim SS, Afzal M, Iqbal W, Abri D, Al (2024) Advanced Persistent threat (APT) and intrusion detection evaluation dataset for linux systems 2024. Data Brief 54:110290. https://doi.org/10.1016/j.dib.2024.110290

Kenny EM, Ford C, Quinn M, Keane MT (2021) Explaining black-box classifiers using post-hoc explanations-by-example: the effect of explanations and error-rates in XAI user studies. Artif Intell 294:103459. https://doi.org/10.1016/j.artint.2021.103459

Keshk M, Koroniotis N, Pham N, Moustafa N, Turnbull B, Zomaya AY (2023) An explainable deep learning-enabled intrusion detection framework in IoT networks. Inf Sci 639:119000. https://doi.org/10.1016/j.ins.2023.119000

Khan IA, Moustafa N, Pi D, Sallam KM, Zomaya AY, Li B (2022) A New Explainable Deep Learning Framework for Cyber threat Discovery in Industrial IoT Networks. IEEE Internet Things J 9(13):11604–11613. https://doi.org/10.1109/JIOT.2021.3130156

Khan IA, Moustafa N, Razzak I, Tanveer M, Pi D, Pan Y, Ali BS (2022b) XSRU-IoMT: explainable simple recurrent units for threat detection in internet of medical things networks. Future Generation Comput Syst 127:181–193. https://doi.org/10.1016/j.future.2021.09.010

Khan IA, Razzak I, Pi D, Zia U, Kamal S, Hussain Y (2024) A Novel Collaborative SRU Network with dynamic behaviour aggregation, reduced communication overhead and explainable features. IEEE J Biomedical Health Inf 28(6):3228–3235. https://doi.org/10.1109/JBHI.2024.3352013

Khraisat A, Gondal I, Vamplew P, Kamruzzaman J (2019) Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity 2(1):20. https://doi.org/10.1186/s42400-019-0038-7

Korium MS, Saber M, Beattie A, Narayanan A, Sahoo S, Nardelli PHJ (2024) Intrusion detection system for cyberattacks in the internet of vehicles environment. Ad Hoc Netw 153:103330. https://doi.org/10.1016/j.adhoc.2023.103330

Kuhn DR, Kacker RN, Lei Y, Simos DE (2020) Combinatorial Methods for Explainable AI. 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) , 167–170. https://doi.org/10.1109/ICSTW50294.2020.00037

Kumaresan SJ, Senthilkumar C, Kongkham D, B, B. B., Nirmala P (2024) Investigating the Effectiveness of Recurrent Neural Networks for Network Anomaly Detection. 2024 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE) , 1–5. https://doi.org/10.1109/IITCEE59897.2024.10467790

Kuppa A, Le-Khac NA (2021) Adversarial XAI methods in Cybersecurity. IEEE Trans Inf Forensics Secur 16:4924–4938. https://doi.org/10.1109/TIFS.2021.3117075

Kute DV, Pradhan B, Shukla N, Alamri A (2021) Deep learning and explainable Artificial Intelligence techniques Applied for detecting money Laundering-A critical review. IEEE Access 9:82300–82317. https://doi.org/10.1109/ACCESS.2021.3086230

Lee JS, Chen YC, Chew CJ, Chen CL, Huynh TN, Kuo CW (2022) CoNN-IDS: Intrusion detection system based on collaborative neural networks and agile training. Comput Secur 122:102908. https://doi.org/10.1016/j.cose.2022.102908

Lemay A, Calvet J, Menet F, Fernandez JM (2018) Survey of publicly available reports on advanced persistent threat actors. Computers Secur 72:26–59. https://doi.org/10.1016/j.cose.2017.08.005

Le T-T-H, Kim H, Kang H, Kim H (2022) Classification and explanation for intrusion detection system based on ensemble trees and SHAP method. Sensors 22:1154. https://doi.org/10.3390/s22031154

Liang Y, Li S, Yan C, Li M, Jiang C (2021) Explaining the black-box model: a survey of local interpretation methods for deep neural networks. Neurocomputing 419:168–182. https://doi.org/10.1016/j.neucom.2020.08.011

Linardatos P, Papastefanopoulos V, Kotsiantis S (2021) Explainable AI: a review of machine learning interpretability methods. Entropy 23(1). https://doi.org/10.3390/e23010018

Liu H, Zhong C, Alnusair A, Islam SR (2021) FAIXID: a Framework for enhancing AI explainability of intrusion detection results using data cleaning techniques. J Netw Syst Manage 29(4):1–30. https://doi.org/10.1007/s10922-021-09606-8

Liu J, Shen Y, Simsek M, Kantarci B, Mouftah HT, Bagheri M, Djukic P (2022) A new realistic benchmark for Advanced Persistent threats in Network Traffic. IEEE Netw Lett 4:1. https://doi.org/10.1109/LNET.2022.3185553

Liu Y, Wu L (2023) Intrusion detection model based on Improved Transformer. Appl Sci 13(10). https://doi.org/10.3390/app13106251

Lo WW, Kulatilleke G, Sarhan M, Layeghy S, Portmann M (2023) XG-BoT: an explainable deep graph neural network for botnet detection and forensics. Internet Things 22:100747. https://doi.org/10.1016/j.iot.2023.100747

Lundberg S, Lee S-I (2017) A Unified Approach to Interpreting Model Predictions . http://arxiv.org/abs/1705.07874

Manoharan P, Yin J, Wang H, Zhang Y, Ye W (2023) Insider threat detection using supervised machine learning algorithms. Telecommunication Syst. https://doi.org/10.1007/s11235-023-01085-3

McGinley C, Monroy SAS (2021) Convolutional Neural Network Optimization for Phishing email classification. 2021 IEEE Int Conf Big Data (Big Data) 5609–5613. https://doi.org/10.1109/BigData52589.2021.9671531

Meister S, Wermes M, Stüve J, Groves RM (2021) Investigations on explainable Artificial Intelligence methods for the deep learning classification of fibre layup defect in the automated composite manufacturing. Compos Part B: Eng 224(May):109160. https://doi.org/10.1016/j.compositesb.2021.109160

Mendonça RV, Teodoro AA, Rosa RL, Saadi M, Melgarejo DC, Nardelli PH, Rodríguez DZ (2021) Intrusion detection system based on fast hierarchical deep convolutional neural network. IEEE Access 96:1024–61034 https://doi.org/10.1109/ACCESS.2021.3074664

Mittal M, Kumar K, Behal S (2023) Deep learning approaches for detecting DDoS attacks: a systematic review. Soft Comput 27(18):13039–13075. https://doi.org/10.1007/s00500-021-06608-1

Mohamed, N. (2023). Current trends in AI and ML for cybersecurity: a state-of-the-art survey. Cogent Eng 10(2). https://doi.org/10.1080/23311916.2023.2272358

Montavon G, Binder A, Lapuschkin S, Samek W, Müller K-R (2019) Layer-Wise Relevance Propagation: An Overview. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 193–209). https://doi.org/10.1007/978-3-030-28954-6_10

Moustafa N, Slay J (2016) The evaluation of Network Anomaly Detection systems: statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set. Inform Secur Journal: Global Perspective 25(1–3):18–31. https://doi.org/10.1080/19393555.2015.1125974

Najar AA,S., M. N (2024) A robust DDoS intrusion detection system using convolutional neural network. Comput Electr Eng 117:109277. https://doi.org/10.1016/j.compeleceng.2024.109277

Pahuja V, Ojha SS (2024) DeepDeter: Strengthening Cybersecurity Against DoS Attacks with Deep Learning. 2024 2nd International Conference on Device Intelligence, Computing and Communication Technologies (DICCT) , 1–6. https://doi.org/10.1109/DICCT61038.2024.10533167

Patel D, Rajesh T, Balamurugan G (2024) Enhancing Cybersecurity Vigilance with Deep Learning for Malware Detection. 2024 10th International Conference on Communication and Signal Processing (ICCSP) , 1005–1010. https://doi.org/10.1109/ICCSP60870.2024.10544228

Patil S, Varadarajan V, Mazhar SM, Sahibzada A, Ahmed N, Sinha O, Kumar S, Shaw K, Kotecha K (2022) Explainable Artificial Intelligence for Intrusion Detection System. Electronics 11(19). https://doi.org/10.3390/electronics11193079

Pawlicki M, Pawlicka A, Kozik R, Choraś M (2024) Advanced insights through systematic analysis: mapping future research directions and opportunities for xAI in deep learning and artificial intelligence used in cybersecurity. Neurocomputing 590:127759. https://doi.org/10.1016/j.neucom.2024.127759

Raju AD, Abualhaol IY, Giagone RS, Zhou Y, Huang S (2021) A Survey on Cross-architectural IoT Malware threat Hunting. IEEE Access 9:91686–91709. https://doi.org/10.1109/access.2021.3091427

Reis JCS, Correia A, Murai F, Veloso A, Benevenuto F (2019) Explainable machine learning for fake news detection. WebSci 2019 - Proc 11th ACM Conf Web Sci 17–26. https://doi.org/10.1145/3292522.3326027

Remman SB, Strümke I, Lekkas AM (2021) Causal versus Marginal Shapley Values for Robotic Lever Manipulation Controlled using Deep Reinforcement Learning . http://arxiv.org/abs/2111.02936

Ridzuan F, Zainon WMN (2019) A review on data cleansing methods for Big Data. Procedia Comput Sci 161:731–738. https://doi.org/10.1016/j.procs.2019.11.177

Rjoub G, Bentahar J, Abdel Wahab O, Mizouni R, Song A, Cohen R, Otrok H, Mourad A (2023) A Survey on Explainable Artificial Intelligence for Cybersecurity. IEEE Trans Netw Serv Manage 20(4):5115–5140. https://doi.org/10.1109/TNSM.2023.3282740

Saeed W, Omlin C (2023) Explainable AI (XAI): a systematic meta-survey of current challenges and future opportunities. Knowl Based Syst 263:110273. https://doi.org/10.1016/j.knosys.2023.110273

Sakthipriya N, Govindasamy V, Akila V (2024) Security-aware IoT botnet attack detection framework using dilated and cascaded deep learning mechanism with conditional adversarial autoencoder-based features. Peer-to-Peer Netw Appl 17(3):1467–1485. https://doi.org/10.1007/s12083-024-01657-3

Salim DT, Singh MM, Keikhosrokiani P (2023) A systematic literature review for APT detection and effective Cyber situational awareness (ECSA) conceptual model. Heliyon 9(7):e17156. https://doi.org/10.1016/j.heliyon.2023.e17156

Samek W, Wiegand T, Müller KR (2017) Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296

Saravanan V, Madiajagan M, Shaik M, Sanju P, Rehman T, Pattanaik B (2023) IoT-based blockchain intrusion detection using optimized recurrent neural network. Multimedia Tools Appl 83:1–22. https://doi.org/10.1007/s11042-023-16662-6

Sarker IH, Janicke H, Mohsin A, Gill A, Maglaras L (2024) Explainable AI for cybersecurity automation, intelligence and trustworthiness in digital twin: methods, taxonomy, challenges and prospects. ICT Express. https://doi.org/10.1016/j.icte.2024.05.007

Schneider S, Antensteiner D, Soukup D, Scheutz M (2022) Autoencoders - A Comparative Analysis in the Realm of Anomaly Detection. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , 1985–1991. https://doi.org/10.1109/CVPRW56347.2022.00216

Schwalbe G, Finzel B (2023) A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Mining and Knowledge Discovery . https://doi.org/10.1007/s10618-022-00867-8

Seibold C, Hilsmann A, Eisert P (2021) Focused LRP: Explainable AI for Face Morphing Attack Detection. Proceedings – 2021 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2021 , 88–96. https://doi.org/10.1109/WACVW52041.2021.00014

Shams Khoozani Z, Sabri AQM, Seng WC, Seera M, Eg KY (2024) Navigating the landscape of concept-supported XAI: challenges, innovations, and future directions. Multimedia Tools Appl. https://doi.org/10.1007/s11042-023-17666-y

Sharma A, Gupta BB, Singh AK, Saraswat VK (2023) Advanced Persistent threats (APT): evolution, anatomy, attribution and countermeasures. J Ambient Intell Humaniz Comput 14(7):9355–9381. https://doi.org/10.1007/s12652-023-04603-y

Sharma DK, Mishra J, Singh A, Govil R, Srivastava G, Lin JC-W (2022) Explainable Artificial Intelligence for Cybersecurity. Comput Electr Eng 103:108356. https://doi.org/10.1016/j.compeleceng.2022.108356

Shenderovitz G, Nissim N (2024) Bon-APT: Detection, attribution, and explainability of APT malware using temporal segmentation of API calls. Comput Secur 142:103862 https://doi.org/10.1016/j.cose.2024.103862

Siddiqi MA, Pak W (2021) An Agile Approach to identify single and hybrid normalization for Enhancing Machine Learning-Based Network Intrusion Detection. IEEE Access 9:137494–137513. https://doi.org/10.1109/ACCESS.2021.3118361

Singh A, Mishra P, Vinod P, Gaur A, Conti M (2024) SFC-NIDS: a sustainable and explainable flow filtering based concept drift-driven security approach for network introspection. Cluster Comput. https://doi.org/10.1007/s10586-024-04444-0

Smagulova K, James AP (2020) Overview of Long Short-Term Memory Neural Networks. In A. P. James (Ed.), Deep Learning Classifiers with Memristive Networks: Theory and Applications (pp. 139–153). Springer International Publishing. https://doi.org/10.1007/978-3-030-14524-8_11

Sourati Z, Prasanna Venkatesh VP, Deshpande D, Rawlani H, Ilievski F, Sandlin H-Â, Mermoud A (2023) Robust and explainable identification of logical fallacies in natural language arguments. Knowl Based Syst 266:110418. https://doi.org/10.1016/j.knosys.2023.110418

Speith T (2022) A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , 2239–2250. https://doi.org/10.1145/3531146.3534639

Stojanović B, Hofer-Schmitz K, Kleb U (2020) APT datasets and attack modeling for automated detection methods: a review. Computers Secur 92:19. https://doi.org/10.1016/j.cose.2020.101734

Stojanovic B, Hofer-Schmitz K, Kleb U (2020) APT datasets and attack modeling for automated detection methods: A review. COMPUTERS & SECURITY , 92 . https://doi.org/10.1016/j.cose.2020.101734

Sun BXL, X. M. C. L. Z. D (2024) Strengthening Network Security: deep learning models for intrusion detection with optimized feature subset and effective Imbalance Handling. Computers Mater Continua 78(2):1995–2022. https://doi.org/10.32604/cmc.2023.046478

Tadesse YE, Choi Y-J (2024) Pattern augmented lightweight convolutional neural network for intrusion detection system. Electronics 13(5). https://doi.org/10.3390/electronics13050932

Taye MM (2023) Theoretical understanding of convolutional neural network: concepts, architectures, applications, future directions. Computation 11(3). https://doi.org/10.3390/computation11030052

Teuwen J, Moriakov N (2020) Chapter 20 - Convolutional neural networks. In S. K. Zhou, D. Rueckert, & G. Fichtinger (Eds.), Handbook of Medical Image Computing and Computer Assisted Intervention (pp. 481–501). Academic Press. https://doi.org/10.1016/B978-0-12-816176-0.00025-9

Tian Y (2020) Artificial intelligence image recognition method based on convolutional neural network algorithm. IEEE Access 81:25731–125744. https://doi.org/10.1109/Access.6287639

Ullah F, Alsirhani A, Alshahrani MM, Alomari A, Naeem H, Shah SA (2022) Explainable Malware Detection System using transformers-based transfer learning and Multi-model Visual representation. SENSORS 22(18). https://doi.org/10.3390/s22186766

Ullah F, Ullah S, Srivastava G, Lin JC-W (2023) IDS-INT: intrusion detection system using transformer-based transfer learning for imbalanced network traffic. Digit Commun Networks. https://doi.org/10.1016/j.dcan.2023.03.008

Vajipayajula S (2023) Comparative Analysis of Deep Learning and Machine Learning models for Network Intrusion Detection. 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT) , 1–13. https://doi.org/10.1109/ICCCNT56998.2023.10308108

Villalón-Huerta A, Marco-Gisbert H, Ripoll-Ripoll I (2022) A taxonomy for threat actors’ persistence techniques. Computers Secur 121:102855. https://doi.org/10.1016/j.cose.2022.102855

Volkov EN, Averkin AN (2023) Possibilities of Explainable Artificial Intelligence for Glaucoma Detection Using the LIME Method as an Example. 2023 XXVI International Conference on Soft Computing and Measurements (SCM) , 130–133. https://doi.org/10.1109/SCM58628.2023.10159038

Wali S, Khan I (2021) Explainable AI and random forest based reliable intrusion detection system

Wallsberger R, Eberhardt TD, Bartlau P-A, Dörnte ML, Schröter TL, Matzka S (2022) Explainable Artificial Intelligence for a high dimensional condition monitoring application using the SHAP Method. 2022 5th International Conference on Artificial Intelligence for Industries (AI4I) , 68–72. https://doi.org/10.1109/AI4I54798.2022.00024

Wang M, Zheng K, Yang Y, Wang X (2020) An Explainable Machine Learning Framework for Intrusion Detection systems. IEEE Access 8:73127–73141. https://doi.org/10.1109/ACCESS.2020.2988359

Wang YF, Guo YB, Fang C (2022) An end-to-end method for advanced persistent threats reconstruction in large-scale networks based on alert and log correlation. J Inform Secur Appl 71. https://doi.org/10.1016/j.jisa.2022.103373

Wang Y, Li J (2023) Anomaly Detection Method for Time Series Data Based on Transformer Reconstruction. Proceedings of the 2023 12th International Conference on Informatics, Environment, Energy and Applications , 58–63. https://doi.org/10.1145/3594692.3594702

Xu F, Uszkoreit H, Du Y, Fan W, Zhao D, Zhu J (2019). Explainable AI: A brief survey on history, research areas, approaches and challenges. In: Tang J, Kan MY, Zhao D, Li S, Zan H (eds) Natural language processing and Chinese computing. NLPCC 2019. Lecture notes in computer science, vol 11839. Springer, Cham. https://doi.org/10.1007/978-3-030-32236-6_51

Yang H, Zeng R, Xu G, Zhang L (2021) A network security situation assessment method based on adversarial deep learning. Appl Soft Comput 102:107096. https://doi.org/10.1016/j.asoc.2021.107096

Yang W, Wei Y, Wei H, Chen Y, Huang G, Li X, Li R, Yao N, Wang X, Gu X, Amin MB, Kang B (2023) Survey on explainable AI: from approaches, limitations and Applications aspects. Human-Centric Intell Syst 3(3):161–188. https://doi.org/10.1007/s44230-023-00038-y

Yang Z, Ma Z, Zhao W, Li L, Gu F (2024) HRNN: Hypergraph Recurrent Neural Network for Network Intrusion Detection. J Grid Comput 22(2):52. https://doi.org/10.1007/s10723-024-09767-1

Yashwanth T, Ashwini K, Chaithanya GS, Tabassum A (2024) Network Intrusion Detection using Auto-encoder Neural Networks and MLP. 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE) , 1–6. https://doi.org/10.1109/ICDCECE60827.2024.10548660

Yin X, Fang W, Liu Z, Liu D (2024) A novel multi-scale CNN and Bi-LSTM arbitration dense network model for low-rate DDoS attack detection. Sci Rep 14(1):5111. https://doi.org/10.1038/s41598-024-55814-y

Yuan X, Li C, Li X (2017) DeepDefense: Identifying DDoS Attack via Deep Learning. 2017 IEEE International Conference on Smart Computing (SMARTCOMP) , 1–8. https://doi.org/10.1109/SMARTCOMP.2017.7946998

Zeltner D, Schmid B, Csiszár G, Csiszár O (2021) Squashing activation functions in benchmark tests: towards a more eXplainable Artificial Intelligence using continuous-valued logic. Knowl Based Syst 218:106779. https://doi.org/10.1016/j.knosys.2021.106779

Zhang H, Huang L, Wu CQ, Li Z (2020) An effective convolutional neural network based on SMOTE and Gaussian mixture model for intrusion detection in imbalanced dataset. Comput Netw 177:107315. https://doi.org/10.1016/j.comnet.2020.107315

Zhang Z, Hamadi H, Al, Damiani E, Yeun CY, Taher F (2022) Explainable Artificial Intelligence Applications in Cyber Security: state-of-the-art in Research. IEEE Access 10:93104–93139. https://doi.org/10.1109/ACCESS.2022.3204051

Zhang Z, Si X, Li L, Gao Y, Li X, Yuan J, Xing G (2023) An Intrusion Detection Method Based on Transformer-LSTM Model. 2023 3rd International Conference on Neural Networks, Information and Communication Engineering (NNICE) , 352–355. https://doi.org/10.1109/NNICE58320.2023.10105733

Zhang Z, Wang L (2022) An Efficient Intrusion Detection Model Based on Convolutional Neural Network and Transformer. 2021 Ninth International Conference on Advanced Cloud and Big Data (CBD) , 248–254. https://doi.org/10.1109/CBD54617.2021.00050

Zhou Q, Li R, Xu L, Nallanathan A, Yang J, Fu A (2022) Towards Explainable Meta-Learning for DDoS Detection . http://arxiv.org/abs/2204.02255

Zhu Q, Zu X (2022) Fully convolutional neural network structure and its loss function for image classification. IEEE Access 10:35541–35549. https://doi.org/10.1109/ACCESS.2022.3163849

Zolanvari M, Yang Z, Khan K, Jain R, Meskin N (2023) TRUST XAI: model-agnostic explanations for AI with a case study on IIoT Security. IEEE Internet Things J 10(4):2967–2978. https://doi.org/10.1109/JIOT.2021.3122019

Download references

This work is funded by the Ministry of Higher Education, Malaysia (JPT(BKPI)1000/016/018/25(58)) through Malaysia Big Data Research Excellence Consortium (BiDaREC), via the research grant managed by Universiti Malaya (Grant No.: KKP002-2021).

Author information

Authors and affiliations.

Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, University Malaya, Kuala Lumpur, Wilayah Persekutuan Kuala Lumpur, 50603, Malaysia

Noor Hazlina Abdul Mutalib, Aznul Qalid Md Sabri & Erma Rahayu Mohd Faizal Abdullah

Department of Computer System & Technology, Faculty of Computer Science and Information Technology, University Malaya, Wilayah Persekutuan Kuala Lumpur, Kuala Lumpur, 50603, Malaysia

Ainuddin Wahid Abdul Wahab

Department of Computer Science, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates

Nouar AlDahoul

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, N.H.A.M, A.Q.M. and N.A.; methodology, N.H.A.M and A.Q.M; original draft preparation and writing of the article, N.H.A.M; review, mentoring and proofreading, A.Q.M, A.W.A.W, E.R.M.F.A and N.A. All authors have read and agreed to submit this article.

Corresponding author

Correspondence to Aznul Qalid Md Sabri .

Ethics declarations

Conflict of interest.

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Mutalib, N.H.A., Sabri, A.Q.M., Wahab, A.W.A. et al. Explainable deep learning approach for advanced persistent threats (APTs) detection in cybersecurity: a review. Artif Intell Rev 57 , 297 (2024). https://doi.org/10.1007/s10462-024-10890-4

Download citation

Accepted : 25 July 2024

Published : 18 September 2024

DOI : https://doi.org/10.1007/s10462-024-10890-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Advanced persistent threats (APTs)
• Explainable artificial intelligence (XAI)
• Interpretability
• Deep learning
• Black-box models
• Cybersecurity
• Find a journal
• Publish with us
• Track your research

Let's Start Treating Cyber Security Like it Matters

That means a real investigatory board for cyber incidents, not the hamstrung one we’ve got now.

Bruce Schneier

Tarah Wheeler

Bruce Schneier and Tarah Wheeler argue that the Cyber Safety Review Board (CSRB) ought to provide more concrete standards for cyber safety and its jurisprudence. "We need and deserve more than one-off anecdotes about how one company didn’t do security well and should do it better in future.  Let’s start treating cybersecurity like the equivalent of public safety and get some real lessons learned."

Read more on Defense One .

You might also like

• community Women Journalists Face Increased Online Violence
• community ‘TECH DOESN’T JUST STAY AT THE BORDER’: PETRA MOLNAR ON SURVEILLANCE’S LONG REACH
• community Biden Cannot Protect Privacy or Defend Democracy by Expanding Surveillance Powers

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Artificial Intelligence
• Generative AI
• Business Operations
• IT Leadership
• Application Security
• Business Continuity
• Cloud Security
• Critical Infrastructure
• Identity and Access Management
• Network Security
• Physical Security
• Risk Management
• Security Infrastructure
• Vulnerabilities
• Software Development
• Enterprise Buyer’s Guides
• United States
• United Kingdom
• Newsletters
• Foundry Careers
• Terms of Service
• Privacy Policy
• Cookie Policy
• Member Preferences
• About AdChoices
• E-commerce Links
• Your California Privacy Rights

Our Network

• Computerworld
• Network World

Entro Security Labs Releases Non-Human Identities Research Security Advisory

Analysis of millions of real-world NHI secrets by Entro Security Labs reveals widespread, significant risks, emphasizes need for improved Secrets Management security practices   

Entro Security , pioneer of the award-winning Non-Human Identity (NHI) and Secrets Management platform, today released its research report, “ 2025 State of Non-Human Identities and Secrets in Cybersecurity .” The Entro Security Lab found that 97% of NHIs have excessive privileges increasing unauthorized access and broadening the attack surface, and 92% of organizations are exposing NHIs to third parties, also resulting in unauthorized access if third-party security practices are not aligned with organizational standards. Surprisingly, 44% of tokens are exposed in the wild, being sent or stored over platforms like Teams, Jira tickets, Confluence pages, code commits and more. Such practices put sensitive information at serious risk of being intercepted and exposed–the root of all secrets and non-human identity breaches. 

Entro Security Labs’ research reveals alarming trends in the handling of both human and NHIs, with significant misconfigurations and risks prevalent across organizations. Key findings include: 

• For each human identity, there are an average of 92 non-human identities. An overwhelming number of non-human identities increases the complexity of identity management and the potential for security vulnerabilities 
• 91% of former employee tokens remain active, leaving organizations vulnerable to potential security breaches 
• 50% of organizations are onboarding new vaults without proper security approval which can introduce vulnerabilities and misconfigurations from the outset 
• 73% of vaults are misconfigured, also leading to unauthorized access and exposure of sensitive data and compromised systems 
• 60% of NHIs are being overused, with the same NHI being utilized by more than one application, increasing the risk of a single point of failure and widespread compromise if exposed 
• 62% of all secrets are duplicated and stored in multiple locations, causing unnecessary redundancy and increasing the risk of accidental exposure 
• 71% of non-human identities are not rotated within the recommended time frames, increasing the risk of compromise over time 

Additional findings are discussed in the report and reveal a critical need for organizations to reassess their NHIs and secrets management practices.  

Data from this report has been collected using a mixed-methods approach, integrating quantitative data analysis with qualitative insights derived from industry observations. The quantitative component focuses on statistical analysis of security incidents and vulnerabilities, while the qualitative aspect provides context and interpretation of these findings within the broader cybersecurity landscape. The data sources include proprietary data from Entro’s cybersecurity infrastructure, secondary data from publicly available industry reports and survey data from IT and security professionals. 

Entro’s complete  research report on non-human identities  is available on their website. 

To learn more or schedule a demo, please visit  https://entro.security/demo/ .  

About Entro Security  

An award-winning pioneer platform, Entro Security provides Non-Human Identity Lifecycle Management, Secrets Security and Non-Human Identity Detection and Response. Unlike traditional methods that reactively scan for exposed secrets, Entro integrates seamlessly within an organization’s existing vaults, and secret creation and exposure locations, offering a single pane of glass to securely use and manage non-human identities and secrets at scale. Headquartered in Boston and backed by top cybersecurity VCs, Entro was named a Cool Vendor by Gartner, Venafi’s Most Promising Machine Identity startup and is a 2023 Globee Awards Winner for Startup Achievement of the Year. For more information, please visit  https://www.entro.security . 

Senior Account Executive

Hannah Sather

Montner Tech PR

[email protected]

Related content

Spycloud unveils massive scale of identity exposure due to infostealers, highlighting need for advanced cybersecurity measures, adaptive shield showcases new itdr platform for saas at black hat usa, infinidat revolutionizes enterprise cyber storage protection to reduce ransomware and malware threat windows, memcyco report reveals only 6% of brands can protect their customers from digital impersonation fraud, from our editors straight to your inbox, show me more, how cybersecurity red teams can boost backup protections.

Australian cops bust underworld app through compromised software updates

Personhood: Cybersecurity’s next great authentication battle as AI improves

CSO Executive Sessions: Guardians of the Games - How to keep the Olympics and other major events cyber safe

CSO Executive Session India with Dr Susil Kumar Meher, Head Health IT, AIIMS (New Delhi)

CSO Executive Session India with Charanjit Bhatia, Head of Cybersecurity, COE, Bata Brands

CSO Executive Sessions: DocDoc’s Rubaiyyaat Aakbar on security technology

CSO Executive Sessions: Hong Kong Baptist University’s Allan Wong on security leadership

CSO Executive Sessions: EDOTCO’s Mohammad Firdaus Juhari on safeguarding critical infrastructure in the telecommunications industry

Sponsored Links

• Visibility, monitoring, analytics. See Cisco SD-WAN in a live demo.
• OpenText Financial Services Summit 2024 in New York City!

Cautious Optimism on OSTP Research Cybersecurity Requirements

The Office of Science and Technology Policy has released its final requirements for research security programs, which federal research funding agencies will have to apply to colleges and universities that average $50 million or more per year in federal research grants. The requirements include potentially positive guidelines for research cybersecurity at covered institutions.

In early 2023, the White House Office of Science and Technology Policy (OSTP) released its initial proposal for a "research security program standard requirement." All federal research funding agencies would have to apply the requirement to colleges and universities that receive more than $50 million per year in federal research funding. Footnote 1 The development of these comprehensive research security mandates stems from National Security Presidential Memorandum – 33 (NSPM-33), "Supported Research and Development National Security Policy." When finalized, the "standard requirement" would establish the basic parameters for the research security programs that covered institutions must have in place to continue competing for federal research grants.

Most of the proposed framework addresses research security issues such as faculty conflicts of interest and commitment and research talent recruitment programs of foreign governments. However, it also includes a research cybersecurity section that essentially would make the cybersecurity guidelines for Federal contract information (FCI) the standards for higher education research cybersecurity. As the Policy team discussed in our review of this issue last summer, EDUCAUSE member feedback indicated that the FCI basic safeguards do not fit well with higher education research environments because they are primarily intended for administrative contexts and data. Footnote 2 EDUCAUSE urged OSTP to revamp its proposed research security program guidance and focus on allowing institutions to pursue a risk management approach to research cybersecurity. Rather than the one-size-fits-all checklist model that the FCI guidelines would impose, a risk management approach would enable institutions to prioritize cybersecurity measures and resources based on national security risks associated with research areas and projects.

EDUCAUSE was not alone in asking OSTP to alter its course and base its research security program guidance on risk management. The Association of American Universities (AAU), the Association of Public and Land-grant Universities (APLU), and the Council on Governmental Relations (COGR) also stressed the need for a risk management emphasis in other areas of higher education research security. Fortunately, OSTP heard the combined input of our respective associations. Rather than rushing forward with research security program requirements that largely reflected those in its original proposal, OSTP took roughly one year to rethink its guidance before releasing the final version on July 9, 2024. The final research security program guidelines do not base research cybersecurity program requirements on the FCI safeguards. Instead, OSTP points to a pending report on higher education research cybersecurity from the National Institute of Standards and Technology (NIST).

As the first element of the standardized requirement, federal research agencies shall require institutions of higher education to certify that the institution will implement a cybersecurity program consistent with the cybersecurity resource for research institutions described in the CHIPS and Science Act, [18] within one year after the National Institute of Standards and Technology (NIST) of the Department of Commerce publishes that resource. Footnote 3

Footnote 18 in the memorandum (in brackets above) identifies the relevant NIST report as NIST Interagency Report (IR) 8481: Cybersecurity for Research: Findings and Possible Paths Forward , which is currently available in "Initial Public Draft" (IPD) form. The CHIPS and Science Act provision from which the report stems required NIST to explore the resources it could develop to better support research cybersecurity at higher education institutions. Footnote 4 NIST conducted substantial outreach to EDUCAUSE and its members in pursuing the project, leading to a draft that largely incorporates the recommendations of our research cybersecurity community. It is a welcome development to see OSTP cite the report as the governing reference for research cybersecurity under its research security program guidelines.

Although OSTP's reliance on a report that reflects substantial EDUCAUSE member input provides a basis for cautious optimism regarding how federal research agencies will implement research cybersecurity requirements, there is still room for agency compliance efforts to jump the rails. The OSTP memorandum does not explain or provide parameters for what constitutes "a cybersecurity program consistent with" the NIST report (emphasis added). Footnote 5 Given the overall tenor of the guidelines, which stress the importance of federal research agencies providing substantial flexibility and discretion to higher education institutions in establishing and maintaining research security programs, research agencies might reasonably develop policies and procedures that allow institutions to draw from the range of resources identified in the NIST report—as well as models and frameworks similar to them—in determining the basis of their programs. However, the lack of guidance on what "consistent with" means may leave space for agencies to mandate that their grantees implement specific frameworks or measures presented in the NIST report. Such a development could produce substantial risks for institutions and agencies alike, given that not all resources identified in the draft NIST report will necessarily lead to optimal—or even appropriate—outcomes in all higher education research contexts.

Our concern about the potential for agencies to mandate inappropriate requirements is exacerbated by the fact that the NIST report was not written for the purposes for which OSTP is applying it. As previously mentioned, the CHIPS and Science Act charged NIST with identifying ways the agency could better support higher education research cybersecurity. Given that task, the current draft of the report—not surprisingly—focuses on highlighting a variety of options that institutions might explore to advance their research cybersecurity posture. This focus does not exactly match how OSTP wants to use the report in its research security program guidelines. The advisory nature of the NIST report may lend itself to the institutional flexibility and discretion that the OSTP memo implies should be the basis of federal agency approaches to research (cyber)security. However, the report does not provide clear direction about what cybersecurity should look like for research security programs that comply with NSPM-33. Without a definitive framework, both research agencies and higher education institutions may struggle to determine what constitutes compliance.

Fortunately, EDUCAUSE members should not have to wait long to get a sense of whether federal agencies that fund research will either try to be highly prescriptive or allow covered institutions to choose what elements of the NIST report—or options similar to them—will form the basis of their research cybersecurity programs. The memo from OSTP states that agencies will have six months from the date the memo was published to provide OSTP and the Office of Management and Budget (OMB) with their proposed implementation plans for the research security program guidelines. Once those agency plans are submitted, colleges and universities should be able to better understand what agencies' compliance regimes might look like. Agencies will then have another six months to implement their policies and processes, with institutions getting up to eighteen months from that point to ensure that they have compliant research security programs. Footnote 6 Based on these time frames, we should see research agency implementation plans by early January 2025, with the final execution of those plans due by mid-2025. Institutions would then have to achieve compliance with the relevant agency policies and processes by around December 2026.

Remember, though, that OSTP provides a unique timeline for its research cybersecurity requirements. As stated above, institutions will have one year from the publication of the NIST final report to ensure that they have research cybersecurity programs that are "consistent with" the report. With that in mind, NIST could try to align the release of its final report with the timeline for institutional compliance with OSTP's research security program guidelines. In this case, the overall measures mandated by the OSTP guidelines would have to be in place by the end of 2026. However, nothing in the OSTP memo precludes NIST from starting the research cybersecurity clock much sooner by releasing its final report at some point later this year or in early 2025. At this juncture, we will have to wait for NIST to provide more information about its plans, which will most likely include making some adjustments between the draft and final versions to account for how research agencies and higher education institutions will have to make use of the final report for compliance purposes.

EDUCAUSE will continue to monitor developments in this space and look for opportunities to inform OSTP, NIST, and agency implementation efforts. In the interim, EDUCAUSE members should review the draft NIST report for reference points that align with their current institutional research cybersecurity program and for resources they might find useful in strengthening their research cybersecurity posture given NSPM-33 and the OSTP research security guidelines that derive from it.

• Arati Prabhakar, Memorandum for the Heads of Federal Research Agencies, "Guidelines for Research Security Programs at Covered Institutions," (Office of Science and Technology Policy, Executive Office of the President, July 9, 2024), 3. Jump back to footnote 1 in the text. ↩
• EDUCAUSE letter to Stacy Murphy, Deputy Chief Operations Officer/Security Officer, Office of Science and Technology Policy,  "Regarding Comment on Research Security Programs,"  June 5, 2023. Jump back to footnote 2 in the text. ↩
• Prabhakar, "Guidelines for Research Security Programs," 4. Jump back to footnote 3 in the text. ↩
• Jarret Cummings, "NIST Explores Developing Research Cybersecurity Resources for Higher Ed,"   EDUCAUSE Review , August 1, 2023. Jump back to footnote 4 in the text. ↩
• Prabhakar, "Guidelines for Research Security Programs," 4–5. Jump back to footnote 5 in the text. ↩
• Ibid., 9. Jump back to footnote 6 in the text. ↩

Jarret Cummings is Senior Advisor, Policy and Government Relations, at EDUCAUSE.

© 2024 EDUCAUSE. The content of this work is licensed under a Creative Commons BY-NC-ND 4.0 International License.

• Picus Security Validation Platform
• for Prevention Controls
• for Detection Controls
• Attack Surface Validation
• Cloud Security Validation
• Attack Path Validation
• Detection Rule Validation
• Integrations and Supported Technologies
• Request a Demo
• Start a Trial
• Breach and Attack Simulation
• Pen Testing Automation
• Continuous Threat Exposure Management
• MITRE ATT&CK
• Blue Report
• Actionable Threat Intelligence Report
• Emerging Threats
• Purple Academy
• Cybersecurity 101
• Case Studies
• Resource Library
• Press Releases
• Partner Program
• Technology Alliance Program

From Exposure Assessment to Management: The Power of Validation in CTEM

Suleyman Ozarslan, PhD | September 17, 2024

The Red Report 2024

Defend Against the Top 10 MITRE ATT&CK TTPs

As our reliance on digital systems grows, so does the complexity and sophistication of cyber threats. Since organizations aim to stay ahead of a potential breach, it becomes not only beneficial but highly essential to understand and address security exposures. If building a resilient cybersecurity posture is your goal, assessment and validation of exposures must be core parts of your exposure management strategy. This blog outlines how exposure assessment serves as the cornerstone of effective cybersecurity and why validation is indispensable in this process.

What is Exposure in Cybersecurity?

Exposure is the presence of any type of vulnerability, misconfiguration, or security gap in an organization's IT environment that may be exploited by any threat actor. These different types of exposures range from software vulnerabilities and missing patches to weak encryption and misconfigured security controls.

Think of exposures as the holes in your armor that can give room for unauthorized access, data breaches, or other types of cyberattacks. Proactively identifying and addressing these exposures is a major key to maintaining a robust security posture and minimizing the risk of successful attacks.

Understanding Exposure Assessment in Cybersecurity

Basically, exposure assessment in the cybersecurity domain is a systematic and continuous action taken towards identifying and quantifying exposures across an organization's IT landscape. Modern exposure assessment platforms (EAPs) consolidate vulnerability assessment and vulnerability prioritization technologies.  Such consolidation provides an effective way of uncovering the relevant attack surfaces and prioritizing vulnerabilities.

However, any truly effective approach would go beyond identification. Validation is a major follow-through that ascertains that identified vulnerabilities are manageable and that no critical issues get left behind. According to Gartner, organizations that use CVSS scores to prioritize exposures will not fully harness the potential benefits of EAPs. Organizations must use exposure validation to validate exposures and better understand the real risks they create.

The Role of Validation in Exposure Assessment

Exposure validation is the process of continuous and automated demonstration of the feasibility of various attack scenarios by using offensive security technologies such as Breach and Attack Simulation (BAS) and automated penetration testing . In addition to demonstrating the existence of exposures like exposure assessment platforms, exposure validation technologies also validate the exploitability of exposures and evaluate the effectiveness of existing defensive security controls and processes in mitigating and remediating these exposures.

Also referred to as adversarial exposure validation by Gartner, exposure validation processes and technologies focus on the most critical issues, ensuring more informed prioritization and remediation. Integration of validation in the exposure management process allows organizations to parse raw data from exposure assessment into actionable insight. According to 2024 Gartner® Hype Cycle™ for Security Operations , adversarial exposure validation filters "theoretical risks (e.g., list of high-priority issues) by highlighting only attacks that are demonstrated to work."

Exposure Validation as an Integral Part of a CTEM Program

Continuous Threat Exposure Management (CTEM) is a comprehensive process that continuously improves an organization's governance and operationalization of threat exposure. CTEM incorporates five critical phases in its operation: scoping, discovery, prioritization, validation, and mobilization. These phases respectively involve:

• scoping the threat exposures, 
• discovery of exposures (vulnerabilities and misconfigurations),
• prioritization of these exposures by risk and criticality
• validation of the exploitability of exposure, and 
• mobilization of necessary mitigations or remediations.

CTEM allows an organization to be in a more proactive, resilient cybersecurity posture, continuously assessing and addressing the exposures within these phases. Each phase is critical because efficient scoping and discovery provide a basis for understanding the threat landscape, while prioritization and validation ensure remediation efforts are effective and resource-efficient.

As more vulnerabilities are being uncovered in the discovery step of CTEM, it becomes much more important to validate the issues to understand their true business potential impact. As stated by Gartner in the 2024 Strategic Roadmap for Managing Threat Exposure , without validation, what is today identified as an "unmanageably large issue" will become an "impossible task ."  This means that what initially appears to be a large set of exposures could easily become an impossible task if not validated. Every exposure must be validated to make sure security teams are working on actual threats. Organizations, therefore, prioritize security efforts much better by thoroughly validating security exposures and ensuring that resources are channeled toward the most significant and validated threats.

Therefore, the exposure validation step in CTEM requires necessity rather than an option. Accordingly, effective exposure assessment must be matched by strong validation to ensure cybersecurity defenses are effective and resilient in an organization. Through continuous threat exposure management driven by rigorous due processes for assessment and validation, an organization can make its confident way through the complexities of the modern threat landscape.

Practical Applications

Consider the following real-world example to illustrate the need for complete exposure management:

"A financial services company with a complex IT environment conducts an exposure assessment and subsequently finds it has over 1,000 discrete vulnerabilities within its network. The size of this number is thus too big to prioritize for remediation. By the use of adversarial exposure validation, the company simulates attack scenarios possible, determining that exploitation of 90% of these vulnerabilities are prevented by security controls such as NGFW, IPS, EDR, and WAF, and the remaining 100 vulnerabilities are immediately exploitable and present a high risk to critical assets such as customer databases and/or payment systems.

With this critical information, the company prioritizes remediation on the 100 high-risk vulnerabilities in such a way that the time and resources invested in managing their exposures are greatly reduced. By remediating these critical issues first, the company improves its overall security posture and mitigates the risk of a potentially devastating data breach."

This case is a good example of how validation is important in making raw data informative. Since the vulnerabilities can be validated whether they are exploitative or not, this helps organizations put efforts in such a way that the most important ones get addressed at the beginning.

How Picus Empowers Your CTEM Strategy

At Picus, we are aware that the stakes have never been higher in protecting your organization's digital environment. The full-spectrum approach to Continuous Threat Exposure Management ensures for us -and you- that it is not only possible to identify possible vulnerabilities but to actually confirm their exploitability - a way to ensure your security teams focus on what truly counts.

Picus provides you with the Picus Security Validation Platform that includes leading-class Breach and Attack Simulation and automated penetration testing products. These will enable you to continuously assess and validate your security posture against the most recent threat vectors and actors. Our solutions integrate into your existing security stack seamlessly, equipping you with the actionable insights required to prioritize and remediate effectively.

Take Your Cybersecurity Strategy to the Next Level

Schedule your demo to take the first step toward a truly holistic and validated exposure management strategy and to learn how our solutions will help you transform your organization's approach to exposure management.

Share this:

The Role of Adversarial Exposure Validation in CTEM

Uncovering Critical Defensive Gaps with Automated Penetration Testing Software

2024 Gartner® Hype Cycle™ for Security Operations

Blue Report 2024 Reveals 40% of Environments Exposed to Full Take Over

Choosing Which Vulnerabilities to Patch

Choosing Which Types of Attacks to Prevent

Choosing What to Log and What Alerts to Trigger

Choosing Between Preventing and Detecting Attacks

Get the latest insights delivered straight to your inbox.

Enhancing Cybersecurity in the Tobago’s Tourism Industry: A Zero Trust Architecture approach

Article sidebar, main article content.

As the government of Trinidad and Tobago prepare to digitally transform the country through the Ministry of Digital Transformation, Tobago’s Tourism specifically the Hotel industry will be the focus of this project as Tobago relies heavily on Tourism to boost the economy of the country. Cyber Security should be a major factor to take into consideration to ensure international engagement to the island. A strategic approach, such as Zero Trust, is essential to enhance cybersecurity and safeguard sensitive customer data, financial assets, and brand reputation. The Aim is to firstly identify the possible risks, then develop a strategic plan to alleviate those risks, followed by some recommendations to encourage the best possible methods/ techniques for dealing with cybersecurity within Tobago’s tourism industry. This project ’s Purpose is to strengthen the security of Tobago’s tourism industry by addressing any cyber security challenge which negatively affects the industry like remote working or the protection of customer sensitive data. With the implementation of the Zero Trust Architectural model, Tobago’s tourism industry stands a good chance of enhancing their security posture by maintaining customer trust and experiencing a reduction in cyber security attacks.

Article Details