Skip to content

Repository files navigation

Smart City IDS

LLM-Driven Intrusion Detection System for Edge-Enabled Smart Cities

License: MIT Python 3.10+ FastAPI Kubernetes Prometheus Grafana PostgreSQL Falco

UDST Capstone Project — Ali Suhail (60106420), Khaled Rahman (60104156), Abdullah Mahmoud (60300336) Supervisor: Dr. Dana Haj Hussein

A Kubernetes-native intrusion detection system for smart-city IoT infrastructure. It combines Falco (runtime) and Suricata (network) detection with LLM-powered alert analysis across five providers, governed by configurable response automation that can isolate pods, scale services, and create ThreatResponse CRDs — all observable through Prometheus, Grafana, and a built-in operator dashboard.

The IoT workloads are protocol-faithful software emulators (MQTT, Modbus, ONVIF, DALI/TALQ, FHIR) running inside the Kubernetes cluster.


Project Scale

Metric Count
Total lines of code 75 000+ (Python, Shell, JS, HTML, YAML, SQL, Markdown)
Python source 35 700+ lines across IDS backend, forwarders, operator, IoT emulators
Kubernetes manifests 18 YAML files deploying 31 pods across 8 namespaces
API endpoints 92 REST endpoints (FastAPI)
LLM providers integrated 5 (xAI Grok, OpenAI GPT-4o, Anthropic Claude, Google Gemini, Kimi)
IoT protocols 5 (MQTT, Modbus, ONVIF, DALI/TALQ, FHIR)
Smart city services 5 emulated services with full protocol stacks
Database tables 12 (PostgreSQL)
Prometheus metrics 40+ metric families with 11 alerting rules
Grafana dashboards 2 auto-provisioned (IEEE Capstone + Unified Operations)
Operational scripts 17 (deployment, attacks, scaling, LLM management, evaluation)
Documentation pages 20 technical documents
Docker images 7 custom-built
Git commits 110+

Architecture

 ┌─────────────────┐     ┌────────────────┐     ┌────────────────────────┐     ┌─────────────────┐
 │  IoT Workloads  │     │   Detection    │     │      IDS Backend       │     │    Response      │
 ├─────────────────┤     ├────────────────┤     ├────────────────────────┤     ├─────────────────┤
 │ traffic-camera  │────▶│                │────▶│  Alert Intake          │     │ Isolate Pod     │
 │ healthcare-api  │     │  Falco (eBPF)  │     │       │                │     │ Scale Service   │
 │ parking-system  │     │   syscall +    │     │  Rate Limit / Dedup    │────▶│ Block IP        │
 │ env-sensor      │     │   runtime      │     │       │                │     │ ThreatResponse  │
 │ street-lighting │     ├────────────────┤     │  LLM Analysis (5)      │     │   CRD           │
 │ mqtt-broker     │────▶│  Suricata      │────▶│       │                │     └─────────────────┘
 └─────────────────┘     │   network +    │     │  Governance Gate       │
                         │   protocol     │     │       │                │
                         └────────────────┘     │  PostgreSQL (12 tbl)   │
                                                └───────────┬────────────┘
                                                            │
                                                ┌───────────┴────────────┐
                                                │   Observability        │
                                                │  Prometheus ─▶ Grafana │
                                                │  Operator Dashboard    │
                                                │  Audit Logs            │
                                                └────────────────────────┘

Features

Category Details
Detection Falco (runtime/syscall via eBPF) + Suricata (network/protocol IDS)
LLM Analysis 5 providers (xAI Grok, OpenAI GPT-4o, Anthropic Claude, Google Gemini, Kimi) with failover, circuit breakers, and cost tracking
Governance Manual, assisted, and autonomous modes with approval workflows and safety gates
Automation Pod isolation, service scaling, network policies, ThreatResponse CRDs via K8s operator
IoT Emulation MQTT, Modbus, ONVIF, DALI/TALQ, FHIR — protocol-faithful services with state models
Dashboard Real-time alert feed, LLM status, governance queue, AI analyst chat, IoT fleet view
Monitoring Prometheus (40+ metrics, alerting rules) → Grafana (IEEE + ops dashboards)
Persistence PostgreSQL (12 tables): alerts, analysis, actions, audit, chat, IoT, LLM health
Evaluation Comparative multi-provider LLM scoring pipeline with artifact-backed results

Quick Start

Prerequisites: Linux (tested on Kali/Ubuntu), 4 GB+ RAM, sudo, curl, jq, git.

# 1. Configure at least one LLM provider key
echo 'ANTHROPIC_API_KEY=sk-ant-...' > .env            # or XAI, OPENAI, GEMINI, KIMI
bash scripts/apply-llm-env-to-k8s-secret.sh .env

# 2. Deploy the full stack (K3s, namespaces, Falco, Suricata, Prometheus, Grafana, IoT services)
sudo bash scripts/start-everything.sh
bash scripts/deploy-code.sh

# 3. Validate everything is up
bash scripts/readiness-check.sh

# 4. Open the operator dashboard
#    http://localhost:30800/ui  (login: admin / admin)

# 5. Run a live attack exercise
bash scripts/run-live-attacks.sh --mode protocol --duration 30 --verbose

For stable localhost access (survives Wi-Fi IP changes):

bash scripts/access-stack.sh start
# IDS API    →  http://localhost:8000
# Grafana    →  http://localhost:3000  (admin / admin)
# Prometheus →  http://localhost:9090

See docs/LLM_CONFIGURATION.md for all five supported providers.


Monitoring Stack

Prometheus and Grafana are deployed automatically by start-everything.sh and wired together out of the box.

Component Access Purpose
Prometheus localhost:9090 (port-forward) or NodePort 31106 Scrapes IDS API, Suricata forwarder, Falco forwarder, and IoT pods every 5 s. 30 alerting rules across 8 groups.
Grafana localhost:3000 (port-forward) or NodePort 30300 Two auto-provisioned dashboards: IEEE Capstone (6-row academic format) and Unified Operations (10-row full ops view).
IDS /metrics localhost:30800/metrics Prometheus text exposition — 40+ metric families covering alerts, LLM calls, governance, IoT, cost, circuit breakers.

How they connect:

IDS API (/metrics)  ──┐
Suricata forwarder  ──┤──▶  Prometheus (scrape config)  ──▶  Grafana (datasource: Prometheus)
Falco forwarder     ──┤           │
IoT pods (DNS SD)   ──┘     Alert rules (8 groups, 30 rules)

Dashboards are auto-provisioned from k8s-manifests/grafana-provisioning-dashboards.yaml which embeds the JSON from infrastructure/monitoring/. The ConfigMap is applied during start-everything.sh alongside Prometheus and Grafana.


Project Structure

Path Purpose
services/ids-api/ Core IDS backend (FastAPI), dashboard UI, LLM engines, K8s automation
services/forwarders/ Falco and Suricata alert forwarders
services/ids-operator/ Kubernetes operator watching ThreatResponse CRDs
smart-city-services/ IoT emulators: traffic camera, healthcare API, parking, env sensor, street lighting
iot-simulator/ MQTT-based IoT device simulator
k8s-manifests/ All Kubernetes manifests (services, monitoring, RBAC, network policies)
infrastructure/monitoring/ Grafana dashboard JSON (source files for provisioning ConfigMap)
infrastructure/database/ PostgreSQL schema migrations
scripts/ Deployment, validation, attack simulation, scaling, and LLM management (index)
tests/ Smoke tests, stability tests, unit tests
docs/ Technical documentation (index)
docker/ Dockerfiles for IDS API, forwarders, and smart-city services
artifacts/llm-eval/ LLM evaluation run data (CSV, JSON)

Documentation

Start here, in order:

# Document Description
1 QUICKSTART First deployment and first checks
2 ARCHITECTURE System layout and component boundaries
3 HOW_IT_WORKS End-to-end alert processing flow
4 OPERATIONS Day-to-day operations and recovery

Then go deeper:

Topic Document
API endpoints docs/API_REFERENCE.md
Deployment procedures docs/DEPLOYMENT.md
LLM provider setup docs/LLM_CONFIGURATION.md
LLM evaluation method docs/LLM_EVALUATION.md
Attack scenarios docs/ATTACK_SIMULATION_GUIDE.md
IoT device integration docs/IOT_INTEGRATION_SDK.md
Security model docs/SECURITY_MODEL.md
Metrics contract docs/METRICS_SPEC.md
Troubleshooting docs/TROUBLESHOOTING.md

Full index: docs/INDEX.md

Scripts

All operational scripts live in scripts/ and share a common library (scripts/lib/). See scripts/README.md for the full reference.

Category Scripts
Deploy start-everything.sh, deploy-code.sh, cleanup.sh
Validate readiness-check.sh, test-governance-modes.sh, eval-complete.py
Attack run-live-attacks.sh, live-pipeline-log.sh, tail-pipeline-pods.sh
Scale scale-iot.sh, scale-profile.sh, scalability-test.sh
LLM llm-manager.sh, apply-llm-env-to-k8s-secret.sh, llm-compare-report.py
Monitoring access-stack.sh (port-forward IDS + Grafana + Prometheus)

Tech Stack

Layer Technology
Orchestration K3s / Kubernetes
IDS Backend Python 3.10+, FastAPI, Uvicorn
Runtime Detection Falco (eBPF)
Network Detection Suricata
LLM Providers xAI Grok, OpenAI GPT-4o, Anthropic Claude, Google Gemini, Moonshot Kimi
Storage PostgreSQL (12 tables)
Metrics Prometheus (40+ metrics, 30 alert rules)
Dashboards Grafana (auto-provisioned), built-in operator dashboard
IoT Protocols MQTT (Mosquitto), Modbus, ONVIF, DALI/TALQ, FHIR
Containerization Docker, K3s crictl

License

MIT — Ali Suhail, Khaled Rahman, Abdullah Mahmoud

About

Research-grade Smart City IDS on Kubernetes: Falco + Suricata detection, LLM-assisted alert analysis with failover, governance-controlled response automation, and IoT protocol emulation.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages