Book demo
02 February 2024

Beyond Shannon Entropy – Pushing the boundaries of anomaly detection

In today’s world of relentless cyber threats, it’s important that all backup and storage systems can detect unusual and suspicious activity in the data they’re protecting. 

To do this, most major storage vendors use a traditional mathematical concept called Shannon entropy to measure how unexpected, or random the behaviour of the data is. 

However, with innovative machine learning techniques, there are new, more intelligent ways to analyse data.  

Predatar Signal™ puts these techniques to work.

So let’s take a look at the limitations of Shannon Entropy, and the benefits of the new techniques that Predatar brings into play.

Introduction

Shannon entropy, conceptualised by engineer and mathematician Claude Shannon in 1949, is a fundamental metric in information theory.

Since its conception, Shannon Entropy has become the go-to method for detecting behavioural anomalies in large data sets.

Today, it’s the prevalent method used by all major storage technology vendors to underpin their anomaly detection capabilities.

While effective in measuring the current state of data, Shannon entropy lacks predictive capabilities and complex pattern recognition.

As data becomes more complex, and cyber threats evolve to avoid detection, we need more sophisticated analysis methods.

Predatar Signal is the intelligence engine under the hood of Predatar Recovery Assurance Technology.

It uses three different machine learning models (LSTM Autoencoder, LSTM Traditional, and ARIMA) to detect anomalies in storage systems. The result – faster, more precise threat identification that will continue to get smarter and more effective.

Predatar Signal: An Advanced Approach 

Predatar Signal integrates three advanced machine learning models, each contributing uniquely to anomaly detection: 

  1. LSTM Autoencoder: This model excels in identifying intricate patterns within backup data. Unlike Shannon entropy, it can detect subtle abnormalities that might otherwise go unnoticed.
  2. LSTM Traditional: Utilising historical data, this model forecasts future backup behaviours. It provides proactive anomaly detection, a feature absent in Shannon entropy’s approach.
  3. ARIMA: Specialising in time series forecasting, ARIMA offers in-depth statistical insights into backup trends, something beyond the scope of Shannon entropy. 

Comparative Analysis 

Complex Pattern Recognition 

  • Shannon Entropy: Limited to measuring unpredictability, lacks the sophistication to identify complex patterns. 
  • Predatar Signal: The LSTM Autoencoder’s advanced pattern recognition capabilities provide a more nuanced and thorough analysis of backup data. 

Predictive Capabilities 

  • Shannon Entropy: Offers no predictive insights, solely focusing on the current state of data. 
  • Predatar Signal: LSTM Traditional model predicts future anomalies, enabling pre-emptive measures for cyber resilience. 

Time Series Analysis 

  • Shannon Entropy: Does not provide detailed temporal analysis. 
  • Predatar Signal: The ARIMA model excels in this area, offering valuable insights into time-dependent data trends. 

Conclusion 

While Shannon Entropy remains a valuable tool for understanding data randomness, Predatar Signal offers a significantly more comprehensive and predictive analysis.  

Integrating LSTM Autoencoder, LSTM Traditional, and ARIMA models, marks a paradigm shift in backup system anomaly detection.  

This advanced approach is not just about detecting anomalies; it’s about predicting and preparing for them, thus reinforcing the pillars of cyber resilience in an increasingly complex data landscape. 

What’s next? 

Predatar empowers businesses to shrink their recovery gap and gives them confidence in their ability to mount a fast and effective recovery. 

Our Machine Learning-powered platform automates daily recovery testing, identifying issues and malware infiltration within your storage environment. 

Don’t let the recovery gap become your downfall. Take control, secure your data, and embrace recovery assurance with Predatar. 

Download our free e-book to learn more and bridge the gap to complete confidence. 

Learn more about
Predatar recovery assurance