Machine Learning For Energy Consumption Prediction IoT

Energy & Smart Grid · ML Project

Python · Data Preprocessing · Model Development · Evaluation

Project focus: prediction and forecasting using IoT sensor streams and timestamped device measurements.

D EEP L EARNING FOR A NOMALY D ETECTION : A S URVEY

Raghavendra Chalapathy Sanjay Chawla University of Sydney, Qatar Computing Research Institute (QCRI), HBKU Capital Markets Co-operative Research Centre (CMCRC) schawla@qf.org.qa rcha9612@uni.sydney.edu.au arXiv:1901.03407v2 [cs.LG] 23 Jan 2019

A BSTRACT Anomaly detection is an important problem that has been well-studied within diverse research areas and application domains. The aim of this survey is two-fold, firstly we present a structured and com- prehensive overview of research methods in deep learning-based anomaly detection. Furthermore, we review the adoption of these methods for anomaly across various application domains and assess their effectiveness. We have grouped state-of-the-art deep anomaly detection research techniques

into different categories based on the underlying assumptions and approach adopted. Within each category, we outline the basic anomaly detection technique, along with its variants and present key assumptions, to differentiate between normal and anomalous behavior. Besides, for each category, we also present the advantages and limitations and discuss the computational complexity of the tech- niques in real application domains. Finally, we outline open issues in research and challenges faced

while adopting deep anomaly detection techniques for real-world problems.

Keywords anomalies, outlier, novelty, deep learning

1 Introduction A common need when analyzing real-world data-sets is determining which instances stand out as being dissimilar to all others. Such instances are known as anomalies, and the goal of anomaly detection (also known as outlier detection) is to determine all such instances in a data-driven fashion (Chandola et al. [2007]). Anomalies can be caused by errors in the data but sometimes are indicative of a new, previously unknown, underlying process; Hawkins

[1980] defines an outlier as an observation that deviates so significantly from other observations as to arouse suspicion that it was generated by a different mechanism. In the broader field of machine learning, the recent years have witnessed a proliferation of deep neural networks, with unprecedented results across various application domains. Deep learning is a subset of machine learning that achieves good performance and flexibility by learning to represent

the data as a nested hierarchy of concepts within layers of the neural network. Deep learning outperforms the traditional machine learning as the scale of data increases as illustrated in Figure 1. In recent years, deep learning-based anomaly detection algorithms have become increasingly popular and have been applied for a diverse set of tasks as illustrated in and Marculescu [2015]). The aim of this survey is two-fold, firstly we present a structured and comprehensive review

of research methods in deep anomaly detection (DAD). Furthermore, we also discuss the adoption of DAD methods across various application domains and assess their effectiveness.

2 What are anomalies? Anomalies are also referred to as abnormalities, deviants, or outliers in the data mining and statistics literature (Ag- garwal [2013]). As illustrated in Figure 3, N1 and N2 are regions consisting of a majority of observations and hence considered as normal data instance regions, whereas the region O3 , and data points O1 and O2 are few data points which are located further away from the bulk of data points and hence are considered anomalies. arise due to several

(a) Video Surveillance, Image Analysis: Illegal Traffic detection Xie et al. [2017], (b) Health-care: Detecting Retinal Damage Schlegl et al. [2017] (c) Networks: Cyber-intrusion detection Javaid et al. [2016] (d) Sensor Networks: Internet of Things (IoT) big-data anomaly detection Mohammadi et al. [2017]

reasons, such as malicious actions, system failures, intentional fraud. These anomalies reveal exciting insights about the data and are often convey valuable information about data. Therefore, anomaly detection considered an essential step in various decision-making systems.

Novelty detection is the identification of a novel (new) or unobserved patterns in the data (Miljković [2010]). The novelties detected are not considered as anomalous data points; instead, they are been applied to the regular data model. A novelty score may be assigned for these previously unseen data points, using a decision threshold score (Pimentel et al. [2014]). The points which significantly deviate from this decision threshold may be considered as

anomalies or outliers. For instance, in Figure 4 the images of (white tigers) among regular tigers may be considered as a novelty, while the image of (horse, panther, lion, and cheetah) are considered as anomalies. The techniques used for anomaly detection are often used for novelty detection and vice versa.

4 Motivation and Challenges: Deep anomaly detection (DAD) techniques • Performance of traditional algorithms in detecting outliers is sub-optimal on the image (e.g. medical images) and sequence datasets since it fails to capture complex structures in the data. • Need for large-scale anomaly detection: As the volume of data increases let’s say to gigabytes then, it be- comes nearly impossible for the traditional methods to scale to such large scale data to find outliers.

• Deep anomaly detection (DAD) techniques learn hierarchical discriminative features from data. This au- tomatic feature learning capability eliminates the need of developing manual features by domain experts, therefore advocates to solve the problem end-to-end taking raw input data in domains such as text and speech recognition. • The boundary between normal and anomalous (erroneous) behavior is often not precisely defined in several data domains and is continually evolving. This lack of well-defined representative normal boundary poses

challenges for both conventional and deep learning-based algorithms.

1 —Our Survey, 2 —Kwon and Donghwoon Kwon et al. [2017], 5 —John and Derek Ball et al. [2017] 3 —Kiran and Thomas Kiran et al. [2018], 6 —Mohammadi and Al-Fuqaha Mohammadi et al. [2017] 4 —Adewumi and Andronicus Adewumi and Akinyelu [2017] 7 —Geert and Kooi et.al Litjens et al. [2017]. 1 2 3 4 5 6 7 Supervised X Unsupervised X Methods Hybrid Models X one-Class Neural Networks X

Fraud Detection X X Cyber-Intrusion Detection X X Medical Anomaly Detection X X Sensor Networks Anomaly Detection X X Applications Internet Of Things (IoT) Big-data Anomaly Detection X X Log-Anomaly Detection X Video Surveillance X X Industrial Damage Detection X

5 Related Work Despite the substantial advances made by deep learning methods in many machine learning problems, there is a relative scarcity of deep learning approaches for anomaly detection. Adewumi and Akinyelu [2017] provide a comprehen- sive survey of deep learning-based methods for fraud detection. A broad review of deep anomaly detection (DAD) techniques for cyber-intrusion detection is presented by Kwon et al. [2017]. An extensive review of using DAD tech-

niques in the medical domain is presented by Litjens et al. [2017]. An overview of DAD techniques for the Internet of

Things (IoT) and big-data anomaly detection is introduced by Mohammadi et al. [2017]. Sensor networks anomaly detection has been reviewed by Ball et al. [2017]. The state-of-the-art deep learning based methods for video anomaly detection along with various categories have been presented in Kiran et al. [2018]. Although there are some reviews in applying DAD techniques, there is a shortage of comparative analysis of deep learning architecture adopted for outlier

detection. For instance, a substantial amount of research on anomaly detection is conducted using deep autoencoders, but there is a lack of comprehensive survey of various deep architecture’s best suited for a given data-set and appli- cation domain. We hope that this survey bridges this gap and provides a comprehensive reference for researchers and engineers aspiring to leverage deep learning for anomaly detection. Table 1 shows the set of research methods and

application domains covered by our survey.

6 Our Contributions We follow the survey approach of (Chandola et al. [2007]) for deep anomaly detection (DAD). Our survey presents a detailed and structured overview of research and applications of DAD techniques. We summarize our main contribu- tions as follows: • Most of the existing surveys on DAD techniques either focus on a particular application domain or specific research area of interest (Kiran et al. [2018], Mohammadi et al. [2017], Litjens et al. [2017], Kwon et al.

[2017], Adewumi and Akinyelu [2017], Ball et al. [2017]). This review aims to provide a comprehensive out- line of state-of-the-art research in DAD techniques as well as several real-world applications these techniques is presented. • In recent years several new deep learning based anomaly detection techniques with greatly reduced computa- tional requirements have been developed. The purpose of this paper is to survey these techniques and classify

them into an organized schema for better understanding. We introduce two more sub-categories Hybrid mod- els (Erfani et al. [2016a])and one-class neural networks techniques (Chalapathy et al. [2018a]) as illustrated in Figure 5 based on the choice of training objective. For each category we discuss both the assumptions and techniques adopted for best performance. Furthermore, within each category, we also present the challenges, advantages, and disadvantages and provide an overview of the computational complexity of DAD methods.

7 Organization This chapter is organized by following structure described in Figure 5. In Section 8, we identify the various aspects that determine the formulation of the problem and highlight the richness and complexity associated with anomaly detection. We introduce and define two types of models: contextual and collective or group anomalies. In Section 9, we briefly describe the different application domains to which deep learning-based anomaly detection has been applied. In

subsequent sections, we provide a categorization of deep learning-based techniques based on the research area to which they belong. Based on training objectives employed and availability of labels deep learning-based anomaly detection

Type of Data Examples DAD model architecture Video,Speech Sequential Protein Sequence,Time Series CNN, RNN, LSTM Text (Natural language) Non- Image,Sensor Sequential Other (data) CNN, AE and its variants proposed in literature. CNN: Convolution Neural Networks, LSTM : Long Short Term Memory Networks AE: Autoencoders.

techniques can be categorized into supervised (Section 10.1), unsupervised (Section 10.5), hybrid (Section 10.3), and one-class neural network (Section 10.4). For each category of techniques we also discuss their computational complexity for training and testing phases. In Section 8.4 we discuss the point, contextual, and collective (group) deep learning-based anomaly detection techniques. We present some discussion of the limitations and relative performance

of various existing techniques in Section 12. Section 13 contains concluding remarks.

8 Different aspects of deep learning-based anomaly detection. This section identifies and discusses the different aspects of deep learning-based anomaly detection.

The choice of a deep neural network architecture in deep anomaly detection methods primarily depends on the nature of input data. Input data can be broadly classified into sequential (eg, voice, text, music, time series, protein sequences) or non-sequential data (eg, images, other data). Table 2 illustrates the nature of input data and deep model architectures used in anomaly detection. Additionally input data depending on the number of features (or attributes) can be further

classified into either low or high-dimensional data. DAD techniques have been to learn complex hierarchical feature relations within high-dimensional raw input data (LeCun et al. [2015]). The number of layers used in DAD techniques is driven by input data dimension, deeper networks are shown to produce better performance on high dimensional data. Later on, in Section 10 various models considered for outlier detection are reviewed at depth.

8.2 Based on Availability of labels

Labels indicate whether a chosen data instance is normal or an outlier. Anomalies are rare entities hence it is challeng- ing to obtain their labels. Furthermore, anomalous behavior may change over time, for instance, the nature of anomaly had changed so significantly and that it remained unnoticed at Maroochy water treatment plant, for a long time which resulted in leakage of 150 million liters of untreated sewerage to local waterways (Ramotsoela et al. [2018]).

Deep anomaly detection (DAD) models can be broadly classified into three categories based on the extent of availabil- ity of labels. (1) Supervised deep anomaly detection. (2) Semi-supervised deep anomaly detection. (3) Unsupervised deep anomaly detection.

8.2.1 Supervised deep anomaly detection Supervised deep anomaly detection involves training a deep supervised binary or multi-class classifier, using labels of both normal and anomalous data instances. For instance supervised DAD models, formulated as multi-class classifier aids in detecting rare brands, prohibited drug name mention and fraudulent health-care transactions (Chalapathy et al. [2016a,b]). Despite the improved performance of supervised DAD methods, these methods are not as popular as

semi-supervised or unsupervised methods, owing to the lack of availability of labeled training samples. Moreover, the performance of deep supervised classifier used an anomaly detector is sub-optimal due to class imbalance (the total number of positive class instances are far more than the total number of negative class of data). Therefore we do not consider the review of supervised DAD methods in this survey.

8.2.2 Semi-supervised deep anomaly detection The labels of normal instances are far more easy to obtain than anomalies, as a result, semi-supervised DAD techniques are more widely adopted, these techniques leverage existing labels of single (normally positive class) to separate

outliers. One common way of using deep autoencoders in anomaly detection is to train them in a semi-supervised way on data samples with no anomalies. With sufficient training samples, of normal class autoencoders would produce low reconstruction errors for normal instances, over unusual events (Wulsin et al. [2010], Nadeem et al. [2016], Song et al. [2017]). We consider a detailed review of these methods in Section 10.2.

8.2.3 Unsupervised deep anomaly detection Unsupervised deep anomaly detection techniques detect outliers solely based on intrinsic properties of the data in- stances. Unsupervised DAD techniques are used in automatic labeling of unlabelled data samples since labeled data is very hard to obtain (Patterson and Gibson [2017]). Variants of Unsupervised DAD models (Tuor et al. [2017]) are shown to outperform traditional methods such as principal component analysis (PCA) (Wold et al. [1987]), support

vector machine (SVM) Cortes and Vapnik [1995] and Isolation Forest (Liu et al. [2008]) techniques in applications domains such as health and cyber-security. Autoencoders are the core of all Unsupervised DAD models. These models assume a high prevalence of normal instances than abnormal data instances failing which would result in high false positive rate. Additionally unsupervised learning algorithms such as restricted Boltzmann machine (RBM) (Sutskever

et al. [2009]), deep Boltzmann machine (DBM), deep belief network (DBN) (Salakhutdinov and Larochelle [2010]), generalized denoising autoencoders (Vincent et al. [2008]) , recurrent neural network (RNN) (Rodriguez et al. [1999]) Long short term memory networks (Lample et al. [2016]) which are used to detect outliers are discussed in detail in Section 11.7.

8.3 Based on the training objective

In this survey we introduce two new categories of deep anomaly detection (DAD) techniques based on training objec- tives employed 1) Deep hybrid models (DHM). 2) One class neural networks (OC-NN).

8.3.1 Deep Hybrid Models (DHM) Deep hybrid models for anomaly detection use deep neural networks mainly autoencoders as feature extractors, the features learned within the hidden representations of autoencoders are input to traditional anomaly detection algo- rithms such as one-class SVM (OC-SVM) to detect outliers (Andrews et al. [2016a]). Figure 7 illustrates the deep hybrid model architecture used for anomaly detection. Following the success of transfer learning to obtain rich rep-

resentative features from models pre-trained on large data-sets, hybrid models have also employed these pre-trained transfer learning models as feature extractors with great success (Pan et al. [2010]). A variant of hybrid model was proposed by Ergen et al. [2017] which considers joint training of feature extractor along-with OC-SVM (or SVDD) objective to maximize the detection performance. A notable shortcoming of these hybrid approaches is the lack of

trainable objective customized for anomaly detection, hence these models fail to extract rich differential features to detect outliers. In order to overcome this limitation customized objective for anomaly detection such as Deep one-class classification (Ruff et al. [2018a]) and One class neural networks (Chalapathy et al. [2018a]) is introduced.

8.3.2 One-Class Neural Networks (OC-NN) One class neural network (OC-NN) Chalapathy et al. [2018a] methods are inspired by kernel-based one-class clas- sification which combines the ability of deep networks to extract a progressively rich representation of data with the one-class objective of creating a tight envelope around normal data. The OC-NN approach breaks new ground for the following crucial reason: data representation in the hidden layer is driven by the OC-NN objective and is thus

customized for anomaly detection. This is a departure from other approaches which use a hybrid approach of learn- ing deep features using an autoencoder and then feeding the features into a separate anomaly detection method like one-class SVM (OC-SVM). The details of training and evaluation of one class neural networks is discussed in Sec- tion 10.4. Another variant of one class neural network architecture Deep Support Vector Data Description (Deep

SVDD) (Ruff et al. [2018a]) trains deep neural network to extract common factors of variation by closely mapping the normal data instances to the center of sphere, is shown to produce performance improvements on MNIST (LeCun et al. [2010]) and CIFAR-10 (Krizhevsky and Hinton [2009]) datasets.

Anomalies can be broadly classified into three types: point anomalies, contextual anomalies and collective anomalies. Deep anomaly detection (DAD) methods have been shown to detect all three types of anomalies with great success.

8.4.1 Point Anomalies The majority of work in literature focuses on point anomalies. Point anomalies often represent an irregularity or deviation that happens randomly and may have no particular interpretation. For instance, in Figure 10 a credit card transaction with high expenditure recorded at Monaco restaurant seems a point anomaly since it significantly deviates from the rest of the transactions. Several real world applications, considering point anomaly detection, are reviewed

8.4.2 Contextual Anomaly Detection A contextual anomaly is also known as the conditional anomaly is a data instance that could be considered as anoma- lous in some specific context (Song et al. [2007]). Contextual anomaly is identified by considering both contextual and behavioural features. The contextual features, normally used are time and space. While the behavioral features may be

a pattern of spending money, the occurrence of system log events or any feature used to describe the normal behavior. just before June; this value is not indicative of a normal value found during this time. Figure 9b illustrates using deep Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber [1997]) based model to identify anomalous system log events (Du et al. [2017]) in a given context (e.g event 53 is detected as being out of context).

(a) Temperature data Hayes and Capretz (b) System logs Du et al. [2017]. [2015].

8.4.3 Collective or Group Anomaly Detection.

Anomalous collections of individual data points are known as collective or group anomalies, wherein each of the in- dividual points in isolation appears as normal data instances while observed in a group exhibit unusual characteristics. For example, consider an illustration of a fraudulent credit card transaction, in the log data shown in Figure 10, if a single transaction of ”MISC” would have occurred, it might probably not seem as anomalous. The following group of

transactions of valued at $75 certainly seems to be a candidate for collective or group anomaly. Group anomaly de- tection (GAD) with an emphasis on irregular group distributions (e.g., irregular mixtures of image pixels are detected using a variant of autoencoder model (Chalapathy et al. [2018b], Bontemps et al. [2016], Araya et al. [2016], Zhuang et al. [2017]).

8.5 Output of DAD Techniques

A critical aspect for anomaly detection methods is the way in which the anomalies are detected. Generally, the outputs produced by anomaly detection methods are either anomaly score or binary labels.

Anomaly score describes the level of outlierness for each data point. The data instances may be ranked according to anomalous score, and a domain-specific threshold (commonly known as decision score) will be selected by subject matter expert to identify the anomalies. In general, decision scores reveal more information than binary labels. For instance, in Deep SVDD approach the decision score is the measure of the distance of data point from the center of

the sphere, the data points which are farther away from the center are considered anomalous (Ruff et al. [2018b]).

CNN: Convolution Neural Networks, LSTM : Long Short Term Memory Networks GRU: Gated Recurrent Unit, DNN : Deep Neural Networks SPN: Sum Product Networks Techniques Model Architecture Section References Discriminative LSTM , CNN-LSTM-GRU, Section 11.7, 11.6, 11.1 Kim et al. [2016],Chawla et al. DNN [2018],Chen et al. [2018],Sohi et al. [2018],Vinayakumar et al. [2017] Hybrid GAN Section 10.3 Aghakhani et al. [2018], Li et al.

[2018] Generative AE, SPN, Section 11.8, 11.3 Gao et al. [2014],Peharz et al. [2018],Umer et al. [2018]

8.5.2 Labels: Instead of assigning scores, some techniques may assign a category label as normal or anomalous to each data instance. Unsupervised anomaly detection techniques using autoencoders measure the magnitude of the residual vector (i,e reconstruction error) for obtaining anomaly scores, later on, the reconstruction errors are either ranked or thresholded by domain experts to label data instances.

9 Applications of Deep Anomaly Detection In this section, we discuss several applications of deep anomaly detection. For each application domain, we discuss the following four aspects: —the notion of an anomaly; —nature of the data; —challenges associated with detecting anomalies; —existing deep anomaly detection techniques.

The intrusion detection system (IDS) refers to identifying malicious activity in a computer-related system (Phoha [2002]). IDS may be deployed at single computers known as Host Intrusion Detection (HIDS) to large networks Network Intrusion Detection (NIDS). The classification of deep anomaly detection techniques for intrusion detection is in Figure 11. IDS depending on detection method are classified into signature-based or anomaly based. Using signature-based IDS is not efficient to detect new attacks, for which no specific signature pattern is available, hence

anomaly based detection methods are more popular. In this survey, we focus on deep anomaly detection (DAD) methods and architectures employed in intrusion detection. 9.1.1 Host-Based Intrusion Detection Systems (HIDS): Such systems are installed software programs which monitors a single host or computer for malicious activity or policy violations by listening to system calls or events occurring within that host (Vigna and Kruegel [2005]). The system

call logs could be generated by programs or by user interaction resulting in logs as shown in Figure 9b. Malicious interactions lead to the execution of these system calls in different sequences. HIDS may also monitor the state of a system, its stored information, in Random Access Memory (RAM), in the file system, log files or elsewhere for a valid sequence. Deep anomaly detection (DAD) techniques applied for HIDS are required to handle the variable length

and sequential nature of data. The DAD techniques have to either model the sequence data or compute the similarity between sequences. Some of the success-full DAD techniques for HIDS is illustrated in Table 3.

9.1.2 Network Intrusion Detection Systems (NIDS): NIDS systems deal with monitoring the entire network for suspicious traffic by examining each and every network packet. Owing to real-time streaming behavior, the nature of data is synonymous to big data with high volume, velocity, variety. The network data also has a temporal aspect associated with it. Some of the success-full DAD techniques for NIDS is illustrated in Table 4 . This survey also lists the data-sets used for evaluating the DAD intrusion detection

methods in Table 5. A challenge faced by DAD techniques in intrusion detection is that the nature of anomalies keeps changing over time as the intruders adapt their network attacks to evade the existing intrusion detection solutions.

CNN: Convolution Neural Networks, LSTM : Long Short Term Memory Networks RNN: Recurrent Neural Networks, RBM : Restricted Boltzmann Machines DCA: Dilated Convolution Autoencoders, DBN : Deep Belief Network AE: Autoencoders, SAE: Stacked Autoencoders GAN: Generative Adversarial Networks, CVAE : Convolutional Variational Autoencoder. Techniques Model Architecture Section References Generative DCA, SAE, RBM, DBN, Section 11.6, 11.8, 11.1, 11.5 Yu et al. [2017],Thing [2017], Zolotukhin et al.

CVAE [2016], Cordero et al. [2016],Alrawashdeh and Purdy [2016],Tang et al. [2016],Lopez-Martin et al. [2017],Al-Qatf et al. [2018],Mirsky et al. [2018],Aygun and Yavuz [2017] Hybrid GAN Section 10.3 Lin et al. [2018],Yin et al. [2018], Ring et al. [2018], Latah [2018],Intrator et al. [2018],Mat- subara et al. [2018], Nicolau et al. [2016] ,Rigaki

[2017]. Discriminative RNN , LSTM ,CNN Section 11.7, 11.6 Yu et al. [2017], Malaiya et al. [2018] Kwon et al. [2018],Gao et al. [2014],Staudemeyer [2015],Naseer et al. [2018]

DataSet IDS Description Type References CTU-UNB NIDS CTU-UNB ucs [2017] dataset con- Hexadecimal Yu et al. [2017] sists of various botnet traffics from CTU-13 dataset [20] and normal traffics from the UNB ISCX IDS 2012 dataset Shiravi et al. [2012] Contagio-CTU-UNB NIDS Contagio-CTU-UNB dataset con- Text Yu et al. [2017].

sists of six types of network traffic data. Adam et al. [2008] NSL-KDD 1 NIDS The NSL-KDD data set is a refined Text Yin et al. [2017], Javaid et al. version of its predecessor KDD-99 [2016], Tang et al. [2016], Yousefi- data set. ucs [2017] Azar et al. [2017], Mohammadi and Namadchian [2017], Lopez-Martin et al. [2017] DARPA KDD- CUP NIDS DARPA KDD Stolfo et al. [2000] Text Alrawashdeh and Purdy [2016]

99 The competition task was to build , Van et al. [2017], Mohammadi and a network intrusion detector, a pre- Namadchian [2017] dictive model capable of distin- guishing between “bad” connec- tions, called intrusions or attacks, and “good” normal connections. MAWI NIDS The MAWI Fontugne et al. [2010] Text Cordero et al. [2016]

dataset consists of network traf- fic capturedfrom backbone links between Japan and USA. Every daysince 2007 Realistic Global NIDS RGCE jam [2009] contains realis- Text Zolotukhin et al. [2016] Cyber Environment tic Internet Service Providers (ISPs) (RGCE) and numerous different web ser- vices as in the real Internet. ADFA-LD HIDS The ADFA Linux Dataset (ADFA- Text Kim et al. [2016], Chawla et al.

LD). This dataset provides a con- [2018] temporary Linux dataset for evalua- tion by traditional HIDS Creech and Hu [2014] UNM-LPR HIDS Consists of system calls to evalute Text Kim et al. [2016] HIDS system University [2012] Infected PDF sam- HIDS Consists of set of Infected PDF Text Chen et al. [2018] ples samples, which are used to monitor

Fraud is a deliberate act of deception to access valuable resources (Abdallah et al. [2016]). The Pricewaterhouse- Coopers (PwC) global economic crime survey of 2018 (Lavion [2018], Zhao [2013]) found that half of the 7,200 companies they surveyed had experienced fraud of some nature. Fraud detection refers to the detection of unlawful activities across various industries, illustrated in 12.

Fraud in telecommunications, insurance ( health, automobile, etc) claims, banking ( tax return claims, credit card transactions etc) represent significant problems in both governments and private businesses. Detecting and preventing fraud is not a simple task since fraud is an adaptive crime. Many traditional machine learning algorithms have been applied successfully in fraud detection (Sorournejad et al. [2016]). The challenge associated with detecting fraud is

that it requires real-time detection and prevention. This section focuses on deep anomaly detection (DAD) techniques for fraud detection.

AE: Autoencoders, LSTM : Long Short Term Memory Networks RBM: Restricted Botlzmann Machines, DNN : Deep Neural Networks GRU: Gated Recurrent Unit, RNN: Recurrent Neural Networks CNN: Convolutional Neural Networks,VAE: Variational Autoencoders GAN: Generative Adversarial Networks Technique Used Section References AE Section 11.8 Schreyer et al. [2017], Wedge et al. [2017] , Paula et al. [2016], Ren- ström and Holmsten [2018], Kazemi and Zarrabi [2017], Zheng et al.

[2018a], Pumsirirat and Yan [2018] RBM Section 11.1 Pumsirirat and Yan [2018] DBN Section 11.1 Seeja and Zareapoor [2014] VAE Section 11.5 Sweers et al. [2018] GAN Section 11.5 Fiore et al. [2017], Choi and Jang [2018] DNN Section 11.1 Dorronsoro et al. [1997], Gómez et al. [2018] LSTM,RNN,GRU Section 11.7 Wiese and Omlin [2009], Jurgovsky et al. [2018], Heryadi and Warnars

[2017], Ando et al. [2016], Wang et al. [2017a], Alowais and Soon [2012], Amarasinghe et al. [2018a], Abroyan [2017a], Lp et al. [2018] CNN Section 11.6 Shen et al. [2007], Chouiekh and Haj [2018], Abroyan [2017b], Fu et al. [2016], Lu [2017], Wang et al. [2018a], Abroyan [2017a] , Zhang et al. [2018a]

CNN: convolution neural networks,DBN: Deep Belief Networks SAE: Stacked Autoencoders, DNN : Deep neural networks GAN: Generative Adversarial Networks Technique Used Section References CNN Section 11.6 Chouiekh and Haj [2018] SAE, DBN Section 11.8, 11.1 Alsheikh et al. [2016], Badhe [2017] DNN Section 11.1 Akhter and Ahamad [2012], Jain

[2017] GAN Section 11.5 Zheng et al. [2018b]

9.2.1 Banking fraud Credit card has become a popular payment method in online shopping for goods and services. Credit card fraud involves theft of a payment card details, and use it as a fraudulent source of funds in a transaction. Many techniques for credit card fraud detection have been presented in the last few years (Zhou et al. [2018], Suganya and Kamalraj [2015]). We will briefly review some of DAD techniques as shown in Table 6. The challenge in credit card fraud

detection is that frauds have no consistent patterns. The typical approach in credit card fraud detection is to maintain a usage profile for each user and monitor the user profiles to detect any deviations. Since there are billions of credit card users this technique of user profile approach is not very scalable. Owing to the inherent scalable nature of DAD techniques techniques are gaining broad spread adoption in credit card fraud detection.

9.2.2 Mobile cellular network fraud In recent times, mobile cellular networks have witnessed rapid deployment and evolution supporting billions of users and a vastly diverse array of mobile devices. Due to this broad adoption and low mobile cellular service rates, mobile cellular networks is now faced with frauds such as voice scams targeted to steal customer private information, and messaging related scams to extort money from customers. Detecting such fraud is of paramount interest and not an

easy task due to volume and velocity of the mobile cellular network. Traditional machine learning methods with static feature engineering techniques fail to adapt to the nature of evolving fraud. Table 7 lists DAD techniques for mobile cellular network fraud detection.

9.2.3 Insurance fraud Several traditional machine learning methods have been applied successfully to detect fraud in insurance claims (Joudaki et al. [2015], Roy and George [2017]). The traditional approach for fraud detection is based on features

DBN: Deep Belief Networks, DNN : Deep Neural Networks CNN: Convolutional Neural Networks,VAE: Variational Autoencoders GAN: Generative Adversarial Networks DBN Section 11.1 Viaene et al. [2005] VAE Section 11.5 Fajardo et al. [2018] GAN Section 11.5 Fiore et al. [2017], Choi and Jang [2018] DNN Section 11.1 Keung et al. [2009] CNN Section 11.6 Shen et al. [2007], Zhang

RBM: Restricted Botlzmann Machines, GAN: Generative Adversarial Networks Technique Used Section References RBM Section 11.1 Lasaga and Santhana [2018] GAN Section 11.5 Ghasedi Dizaji et al. [2018], Fin- layson et al. [2018] CNN Section 11.6 Esteva et al. [2017]

which are fraud indicators. The challenge with these traditional approaches is that the need for manual expertise to extract robust features. Another challenge is insurance fraud detection is the that the incidence of frauds is far less than the total number of claims, and also each fraud is unique in its way. In order to overcome these limitations several DAD techniques are proposed which are illustrated in Table 8.

9.2.4 Healthcare fraud Healthcare is an integral component in people’s lives, waste, abuse, and fraud drive up costs in healthcare by tens of billions of dollars each year. Healthcare insurance claims fraud is a significant contributor to increased healthcare costs, but its impact can be mitigated through fraud detection. Several machine learning models have been used effectively in health care insurance fraud (Bauder and Khoshgoftaar [2017]). Table 9 presents an overview of DAD

methods for health-care fraud identification.

Malware, short for Malicious Software. In order to protect legitimate users from malware, machine learning based efficient malware detection methods are proposed (Ye et al. [2017]). In classical machine learning methods, the process of malware detection is usually divided into two stages: feature extraction and classification/clustering. The performance of traditional malware detection approaches critically depend on the extracted features and the methods

for classification/clustering. The challenge associated in malware detection problems is the sheer scale of data, for instance considering data as bytes a specific sequence classification problem could be of the order of two million time steps. Furthermore, the malware is very adaptive in nature, wherein the attackers would use advanced techniques to hide the malicious behavior. Some DAD techniques which address these challenges effectively and detect malware

9.4 Medical Anomaly Detection

Several studies have been conducted to understand the theoretical and practical applications of deep learning in medi- cal and bio-informatics (Min et al. [2017], Cao et al. [2018a], Zhao et al. [2016], Khan and Yairi [2018]). Finding rare events (anomalies) in areas such as medical image analysis, clinical electroencephalography (EEG) records, enable to diagnose and provide preventive treatments for a variety of medical conditions. Deep learning based architectures are

employed with great success to detect medical anomalies as illustrated in Table 11. The vast amount of imbalanced data in medical domain presents significant challenges to detect outliers. Additionally deep learning techniques for long have been considered as black-box techniques. Even though deep learning models produce outstanding perfor- mance, these models lack interpret-ability. In recent times models with good interpret-ability are proposed and shown

to produce state-of-the-art performance (Gugulothu et al., Amarasinghe et al. [2018b], Choi [2018]).

AE: Autoencoders, LSTM : Long Short Term Memory Networks RBM: Restricted Botlzmann Machines, DNN : Deep Neural Networks GRU: Gated Recurrent Unit, RNN: Recurrent Neural Networks CNN: Convolutional Neural Networks,VAE: Variational Autoencoders GAN: Generative Adversarial Networks,CNN-BiLSTM: CNN- Bidirectional LSTM Technique Used Section References AE Section 11.8 Yousefi-Azar et al. [2017], Hardy et al. [2016], Yousefi-Azar et al.

[2017], De Paola et al. [2018], Sewak et al. [2018], Kebede et al. [2017], De Paola et al. [2018], David and Netanyahu [2015] word2vec Section 11.4 Cakir and Dogdu [2018], Silva et al. [2018] CNN Section 11.6 Kolosnjaji et al. [2018], Suciu et al. [2018], Srisakaokul et al. [2018], Srisakaokul et al. [2018], King et al. [2018], Huang and Kao [2017], Guo et al. [2017], Abdelsalam et al. [2018], Raff et al. [2017], Karbab et al. [2018], Martinelli et al.

[2017], McLaughlin et al. [2017], Gibert et al. [2018], Kolosnjaji et al. [2017] DNN Section 11.1 Rosenberg et al. [2018], Wang et al. [2017b] DBN Section 11.1 David and Netanyahu [2015], YANG et al. [2016], Ding et al. [2016], Yuxin and Siyi [2017], Selvaganapathy et al. [2018], Yuxin and Siyi [2017], Hou et al. [2017] LSTM Section 11.7 Tobiyama et al. [2016], Hu and Tan [2017], Tobiyama et al. [2018]

, Passalis and Tefas CNN-BiLSTM Section Le et al. [2018], Wang et al. [2017b] 11.6, 11.7 GAN Section 11.5 Kim et al. [2018] Hybrid model(AE- Section 10.3 Wang et al. [2018b], Li et al. [2015] CNN),(AE-DBN) RNN Section 11.7 HaddadPajouh et al. [2018]

AE: Autoencoders, LSTM : Long Short Term Memory Networks GRU: Gated Recurrent Unit, RNN: Recurrent Neural Networks CNN: Convolutional Neural Networks,VAE: Variational Autoencoders GAN: Generative Adversarial Networks, KNN: K-nearest neighbours RBM: Restricted Boltzmann Machines. Technique Used Section References AE Section 11.8 Wang et al. [2016], Cowton et al. [2018], Sato et al. [2018] DBN Section 11.1 Turner et al. [2014], Sharma et al. [2016], Wulsin et al.

[2010], Ma et al. [2018], Zhang et al. [2016], Wulsin et al. [2011] , Wu et al. [2015a] RBM Section 11.1 Liao et al. [2016] VAE Section 11.5 Xu et al. [2018], Lu and Xu [2018] GAN Section 11.5 Ghasedi Dizaji et al. [2018], Chen and Konukoglu [2018] LSTM ,RNN,GRU Section 11.7 Yang and Gao [2018], Jagannatha and Yu [2016], Cowton et al. [2018], O’Shea et al. [2016], Latif et al. [2018], Zhang and Zou

[2018], Chauhan and Vig [2015], Gugulothu et al., Amarasinghe et al. [2018b] CNN Section 11.6 Schmidt-Erfurth et al. [2018], Esteva et al. [2017], Wang et al. [2016], Iakovidis et al. [2018] Hybrid( AE+ KNN) Section 11.6 Song et al. [2017]

CNN: Convolution Neural Networks, LSTM : Long Short Term Memory Networks AE: Autoencoders, DAE: Denoising Autoencoders SVM : Support Vector Machines., DNN : Deep Neural Network Technique Used Section References AE,DAE Section 11.8 Zhang et al. [2017], Castellini et al. [2017] CNN-LSTM Section 11.6, 11.7 Sun et al. [2018], Shu et al. [2017], Yang et al. [2018] DNN Section 11.1 Li et al. [2017a]

Hybrid Models Section 10.3 Wei [2017] (CNN-LSTM-SVM)

CNN: Convolution Neural Networks, LSTM : Long Short Term Memory Networks GRU: Gated Recurrent Unit, DNN : Deep Neural Networks AE: Autoencoders, DAE: Denoising Autoencoders Techniques Section References LSTM Section 11.7 Hochreiter and Schmidhuber [1997], Brown et al. [2018], Tuor et al. [2017], Das et al. [2018], Malhotra et al. [2015] AE Section 11.8 Du et al. [2017], Andrews et al. [2016a] , Sakurada and Yairi

[2014], Nolle et al. [2018a], Nolle et al. [2016] LSTM-AE Section 11.7, Grover [2018], Wolpher [2018] 11.8 RNN Section 11.7 Brown et al. [2018], Zhang et al. [2018b], Nanduri and Sherry [2016], Fengming et al. [2017] DAE Section 11.8 Marchi et al. [2015], Nolle et al. [2016] CNN Section 11.6 Lu et al. [2018], Yuan et al. [2018a], Racki et al. [2018], Zhou et al. [2016], Gorokhov et al. [2017], Liao et al. [2017], Cheng

et al. [2017], Zhang et al. [2018c]

9.5 Deep learning for Anomaly detection in Social Networks

In recent times, online social networks have become part and parcel of daily life. Anomalies in a social network are irregular often unlawful behavior pattern of individuals within a social network; such individuals may be identified as spammers, sexual predators, online fraudsters, fake users or rumor-mongers. Detecting these irregular patterns is of prime importance since if not detected, the act of such individuals can have a serious social impact. A survey of

traditional anomaly detection techniques and its challenges to detect anomalies in social networks is a well studied topic in literature (Liu and Chawla [2017], Savage et al. [2014], Anand et al. [2017], Yu et al. [2016], Cao et al. [2018b], Yu et al. [2016]). The heterogeneous and dynamic nature of data presents significant challenges to DAD techniques. Despite these challenges, several DAD techniques illustrated in Table 12 are shown outperform state-of-

Anomaly detection in log file aims to find text, which can indicate the reasons and the nature of the failure of a system. Most commonly, a domain-specific regular-expression is constructed from past experience which finds new faults by pattern matching. The limitation of such approaches is that newer messages of failures are easily are not detected (Memon [2008]). The unstructured and diversity in both format and semantics of log data pose significant

challenges to log anomaly detection. Anomaly detection techniques should adapt to the concurrent set of log data generated and detect outliers in real time. Following the success of deep neural networks in real time text analysis, several DAD techniques illustrated in Table 13 model the log data as a natural language sequence are shown very effective in detecting outliers.

9.7 Internet of things (IoT) Big Data Anomaly Detection

IoT is identified as a network of devices that are interconnected with soft-wares, servers, sensors and etc. In the field of the Internet of things (IoT), data generated by weather stations, Radio-frequency identification (RFID) tags, IT infrastructure components, and some other sensors are mostly time-series sequential data. Anomaly detection in these

AE: Autoencoders, LSTM : Long Short Term Memory Networks DBN : Deep Belief Networks. Techniques Section References AE Section 11.8 Luo and Nagarajany [2018], Mo- hammadi and Kwasinski [2018] DBN Section 11.1 Kakanakova and Stoyanov [2017] LSTM Section 11.7 Zhang et al. [2018d], Mudassar et al. [2018]

CNN: Convolution Neural Networks, LSTM : Long Short Term Memory Networks GRU: Gated Recurrent Unit, DNN : Deep Neural Networks AE: Autoencoders, DAE: Denoising Autoencoders, SVM: Support Vector Machines SDAE: Stacked Denoising Autoencoders, RNN : Recurrent Neural Networks. Techniques Section References LSTM Section 11.7 Inoue et al. [2017], Thi et al. [2017], Kravchik and Shab- tai [2018], Huang et al. [2018], Park et al. [2018a], Chang

et al. [2018] AE Section 11.8 Yuan and Jia [2015], Araya et al. [2017], Qu et al. [2017], Sakurada and Yairi [2014], Bhattad et al. [2018] DNN Section 11.1 Lodhi et al. [2017] CNN Section 11.6 Faghih-Roohi et al. [2016], Christiansen et al. [2016], Lee et al. [2016], Faghih-Roohi et al. [2016], Dong et al. [2016], Nanduri and Sherry [2016], Fuentes et al. [2017], Huang et al. [2018], Chang et al. [2018]

SDAE,DAE Section 11.8 Yan and Yu [2015], Luo and Zhong [2017], Dai et al. [2017] RNN Section 11.7 Banjanovic-Mehmedovic et al. [2017], Thi et al. [2017] Hybrid Models (DNN-SVM) Section 10.3 Inoue et al. [2017]

IoT networks identifies fraudulent, faulty behavior of these massive scales of interconnected devices. The challenges associated with outlier detection is that heterogeneous devices are interconnected which renders the system more complex. A thorough overview of using deep learning (DL), to facilitate analytics and learning in the IoT domain is presented by (Mohammadi et al. [2018]). Table 14 illustrates the DAD techniques employed IoT devices.

9.8 Industrial Anomalies Detection

Industrial systems consisting of wind turbines, power plants, high-temperature energy systems, storage devices and with rotating mechanical parts are exposed to enormous stress on a day-to-day basis. Damage to these type of systems not only causes economic loss but also a loss of reputation, therefore detecting and repairing them early is of utmost importance. Several machine learning techniques have been used to detect such damage in industrial systems (Ramot-

soela et al. [2018], Martı́ et al. [2015]). Several papers published utilizing deep learning models for detecting early industrial damage show great promise (Atha and Jahanshahi [2018], de Deijn [2018], Wang et al. [2018c]). Damages caused to equipment are rare events, thus detecting such events can be formulated as an outlier detection problem. The challenges associated with outlier detection in this domain is both volumes as well as the dynamic nature of data

since failure is caused due to a variety of factors. Some of the DAD techniques employed across various industries are illustrated in Table 15.

9.9 Anomaly Detection in Time Series

Data recorded continuously over duration is known as time series. Time series data can be broadly classified into univariate and multivariate time series. In case of univariate time series, only single variable (or feature) varies over time. For instance, the data collected from a temperature sensor within the room for each second is an uni-variate time series data. A multivariate time series consists several variables (or features) which change over time. An ac-

celerometer which produces three-dimensional data for every second one for each axis (x, y, z) is a perfect example of multivariate time series data. In the literature, types of anomalies in univariate and multivariate time series are cat- egorized into following groups: (1) Point Anomalies. 8.4.1 (2) Contextual Anomalies 8.4.2 (3) Collective Anomalies

CNN: Convolution Neural Networks, GAN: Generative Adversarial networks, DNN: Deep Neural Networks,AE: Autoencoders,DAE: Denoising Autoencoders, VAE: Variational Autoencoder,SDAE: Stacked Denoising Autoencoders, LSTM: Long Short Term Memory Networks, GRU: Gated Recurrent Unit RNN: Recurrent Neural Networks, RNN: Replicator Neural Networks Techniques Section References LSTM Section 11.7 Shipmon et al. [2017a],Hundman et al. [2018],Zhu and Laptev

[2017]Malhotra et al. [2015],Chauhan and Vig [2015],As- sendorp [2017] Ahmad et al. [2017],Malhotra et al. [2016a],Bontemps et al. [2016],Taylor et al. [2016],Cheng et al. [2016],Lo- ganathan et al. [2018],Chauhan and Vig [2015],Malhotra et al. [2015],Gorokhov et al. [2017], Munir et al. [2018] AE,LSTM- Section 11.8 Shipmon et al. [2017b], Malhotra et al. [2016b], AE,CNN-AE,GRU- Filonov et al. [2016], Sugimoto et al. [2018], Oh and Yun

Continued Discussion and Extended Analysis

D EEP L EARNING FOR A NOMALY D ETECTION : A S URVEY

while adopting deep anomaly detection techniques for real-world problems.

Keywords anomalies, outlier, novelty, deep learning

of research methods in deep anomaly detection (DAD). Furthermore, we also discuss the adoption of DAD methods across various application domains and assess their effectiveness.

challenges for both conventional and deep learning-based algorithms.

niques in the medical domain is presented by Litjens et al. [2017]. An overview of DAD techniques for the Internet of

application domains covered by our survey.

of various existing techniques in Section 12. Section 13 contains concluding remarks.

8 Different aspects of deep learning-based anomaly detection. This section identifies and discusses the different aspects of deep learning-based anomaly detection.

8.2 Based on Availability of labels

8.3 Based on the training objective

In this survey we introduce two new categories of deep anomaly detection (DAD) techniques based on training objec- tives employed 1) Deep hybrid models (DHM). 2) One class neural networks (OC-NN).

(a) Temperature data Hayes and Capretz (b) System logs Du et al. [2017]. [2015].

8.4.3 Collective or Group Anomaly Detection.

8.5 Output of DAD Techniques

A critical aspect for anomaly detection methods is the way in which the anomalies are detected. Generally, the outputs produced by anomaly detection methods are either anomaly score or binary labels.

the sphere, the data points which are farther away from the center are considered anomalous (Ruff et al. [2018b]).

[2018] Generative AE, SPN, Section 11.8, 11.3 Gao et al. [2014],Peharz et al. [2018],Umer et al. [2018]

[2017]. Discriminative RNN , LSTM ,CNN Section 11.7, 11.6 Yu et al. [2017], Malaiya et al. [2018] Kwon et al. [2018],Gao et al. [2014],Staudemeyer [2015],Naseer et al. [2018]

that it requires real-time detection and prevention. This section focuses on deep anomaly detection (DAD) techniques for fraud detection.

[2017] GAN Section 11.5 Zheng et al. [2018b]

methods for health-care fraud identification.

9.4 Medical Anomaly Detection

to produce state-of-the-art performance (Gugulothu et al., Amarasinghe et al. [2018b], Choi [2018]).

AE: Autoencoders, LSTM : Long Short Term Memory Networks RBM: Restricted Botlzmann Machines, DNN : Deep Neural Networks GRU: Gated Recurrent Unit, RNN: Recurrent Neural Networks CNN: Convolutional Neural Networks,VAE: Variational Autoencoders GAN: Generative Adversarial Networks,CNN-BiLSTM: CNN- Bidirectional LSTM Technique Used Section References AE Section 11.8 Yousefi-Azar et al. [2017], Hardy et al. [2016], Yousefi-Azar et al.

Hybrid Models Section 10.3 Wei [2017] (CNN-LSTM-SVM)

et al. [2017], Zhang et al. [2018c]

9.5 Deep learning for Anomaly detection in Social Networks

9.7 Internet of things (IoT) Big Data Anomaly Detection

9.8 Industrial Anomalies Detection

since failure is caused due to a variety of factors. Some of the DAD techniques employed across various industries are illustrated in Table 15.

9.9 Anomaly Detection in Time Series

FAQ

Python, Jupyter Notebook, Pandas, NumPy, Scikit-learn, Matplotlib, XGBoost / Random Forest. These tools support data preparation, visualization, model training and evaluation.

This is primarily a software/data-science project. It can be extended into a deployed prototype using a web dashboard, edge device, cloud service or application interface where appropriate.

Typical results include exploratory plots, preprocessing statistics, model comparison tables, confusion matrices or error metrics, prediction plots, feature importance and a final evaluation summary.

Machine Learning Project Workflow

A clean project structure for dataset preparation, modelling, validation and presentation.

Dataset
Preparation

CSV · Database · APIs

Exploratory
Analysis

EDA · Statistics

Feature
Engineering

Selection · Scaling

Model
Training

ML Algorithms

Validation &
Testing

Metrics · CV

Visualization
Dashboard

Plots · Reports

Explainable
AI

Importance · Errors

Deployment &
Documentation

API · Report · Viva