Idle

// Cloud Sentinel — Large-Scale Cloud Systems

ML Pipeline
Control Center

Train · Evaluate · Compare — Three autoencoder models in one run.

Models
3
AE · PCA+AE · LSTM
Architecture
Dense
+ LSTM Autoencoder
Batch Size
5K
Memory-safe loader
Detection Rule
≥500
HTTP Status Code

📂 Dataset Configuration

⬆
Upload Dataset File Drag & drop or click — .parquet or .csv accepted
Supported formats: .parquet  ·  .csv  ·  .tsv
Required columns: aggregated_stats_value, location, kind, method, statusCode

Select a dataset above and click Run to start training all 3 models.

// Model Evaluation Results

Results Dashboard

Precision · Recall · F1 · ROC-AUC comparison across all models.

Run the pipeline first to see results.
◈

No results yet. Run the pipeline to generate output.

FINAL YEAR PROJECT · BS Cyber Security

SentinelGuard

// ML Engineer · Cloud Systems · Sentinel Guard

A comparative study of autoencoder-based cloud sentinel architectures applied to large-scale cloud observability data. This project benchmarks Dense Autoencoders, PCA-enhanced Autoencoders, and LSTM Autoencoders on real HTTP telemetry logs.

▶ Launch Pipeline 📊 View Dashboard ⌥ GitHub

// FYP-II · RESEARCH PROJECT

Cloud Sentinel for Large-Scale Cloud Systems

Cloud systems generate millions of telemetry events per hour. This project builds an end-to-end unsupervised cloud sentinel pipeline that identifies abnormal HTTP request patterns — system failures, errors, and degraded services — using reconstruction error from three distinct autoencoder architectures.

Autoencoder (No PCA)
Dense encoder-decoder trained directly on full preprocessed feature space.
PCA + Autoencoder
Dimensionality reduction via PCA to 5 components before AE reconstruction.
LSTM Autoencoder
Sequence-aware temporal reconstruction using bidirectional LSTM layers.
TensorFlow 2.x Keras scikit-learn pandas PyArrow Flask SSE LSTM PCA Parquet Batch Processing

// System Architecture

Data Flow · Training → Inference → Evaluation
ParquetRaw data
→
Batch Loader5,000 rows/batch
→
PreprocessorScale + OneHot
→
3 ModelsAE · PCA+AE · LSTM
→
MSE Thresholdμ + 2σ rule
→
Detection LabelsBinary classify
→
EvaluationF1 · AUC · Prec

// Expected Performance Metrics

0.94+
ROC-AUC
~92%
Accuracy
F1
Primary Metric
100
Max Batches

// Technical Stack

🧠
Deep Learning
TensorFlow, Keras, LSTM
📊
ML / Statistics
sklearn, PCA, StandardScaler
🐍
Python
NumPy, pandas, PyArrow
⚡
Flask Backend
SSE streaming, REST API
🗄️
Big Data
Parquet, batch processing
🎨
Frontend
HTML, CSS, Vanilla JS

// About This Project

Problem Statement

Modern cloud systems produce massive telemetry streams. Manual inspection is impossible. This project automates cloud sentinel using unsupervised ML — no labeled training data required.

Key Contribution

A systematic comparison of three autoencoder variants under identical conditions — same data, same preprocessing, same evaluation — yielding objective performance benchmarks on real cloud HTTP data.

Detection Definition

HTTP status codes ≥ 500 are treated as anomalies (server errors). This pseudo-labeling strategy enables evaluation without manual annotation of millions of events.

Scalability

Memory-safe batch processing allows the pipeline to handle datasets larger than available RAM. PyArrow Parquet streaming processes 5,000 rows at a time with full garbage collection.