NetLens: A Passive Network Traffic Analyser and Visibility Dashboard for a College Computer Lab

NetLens is a passive network-traffic analyser built for a college computer lab — the 60-machine lab where the lab in-charge today answers "why is the network slow?" by guessing. It listens on the lab's own network segment through a mirror port on the lab's managed switch, reassembles packets into flow records (who talked to whom, on which port, how many bytes), stores them in MySQL, and serves a Flask API with a React dashboard.

The dashboard answers the everyday questions: which machines are the top talkers this week, how the traffic splits across HTTP, HTTPS, DNS and SMB, which DNS names the lab resolves most often, and how many machines are actually idle. On the security side it computes two passive indicators — a port-scan pattern (one source sending SYNs to many distinct ports) and duplicate-IP / ARP-storm behaviour — and raises them as events to investigate, never as automatic blocking. NetLens is a visibility and measurement tool: it reads, it never intercepts, modifies or blocks a single packet.

Everything is captured header-only (no web content, no emails, no passwords), stored for a fixed 30-day retention window, and the whole deployment sits inside the college's own lab with written permission from the lab in-charge. Scapy, the industry-standard Python packet library, is used alongside the VTU-taught stack (MySQL from DBMS, React from the elective lab), which makes this a defensive-security project that stays fully within the syllabus and within ethical bounds.

A college computer lab of 60 machines has no systematic record of its own network activity. Slowdowns are investigated by guesswork, capacity planning (how many machines actually need internet during a lab session?) is done on intuition, and security-relevant behaviour — a machine scanning others, ARP anomalies, sudden DNS spikes — goes unnoticed because nothing is observing the segment. One-off Wireshark captures give minutes of visibility with no history and no per-machine view, while commercial network-management systems are far beyond a lab budget and over-featured for a single segment.

What is needed is a passive, privacy-respecting, low-cost analyser that runs permanently inside the lab's own network: it should convert raw packets into queryable flow history, expose the answers to everyday operational questions on a dashboard, flag scan-like behaviour as indicators for human review — and never intercept, alter or block traffic, and never capture content beyond packet headers.

Abstract

College computer labs run on shared infrastructure nobody measures. When a lab slows down, staff reboot the switch or blame a batch of students; when something suspicious happens, there is no record to look back at. NetLens is a passive traffic analyser that gives a single lab a permanent, queryable memory of its own network: flow records in MySQL, a Flask REST API and a React dashboard showing top talkers, protocol mix, DNS summary and two passive scan indicators, with 30-day retention and CSV/PDF reports.

Introduction

Our department has four labs of about 60 PCs each behind a managed switch. The lab in-charge must plan lab capacity, justify bandwidth requests and respond to security notices, but the only tools available are a one-off Wireshark capture (which shows minutes of traffic, not history) and the router's daily totals (which show nothing per machine). Nothing answers "which machine was talking to the internet at 2 pm last Thursday?" NetLens turns that question into a 5-second query.

Existing System

• Ad-hoc Wireshark runs on a technician's laptop: minutes of visibility, no storage, no history, no per-machine view.
• Router/ISP counters: totals only, no protocol or host breakdown, no export.
• Complaint-driven debugging: "the lab is slow" is investigated after the fact with no evidence.
• Limitations: no persistence, no per-machine accountability, security events noticed only when something breaks, and commercial network-management products cost far more than a lab budget allows.

Proposed System

• A capture service (Python + Scapy) running on one lab PC connected to a switch mirror port, assembling header-only packets into bidirectional flow records with idle timeouts.
• A MySQL store with a fixed 30-day retention job; no packet payloads are ever written to disk.
• Two passive indicators computed over rolling windows: (1) port-scan pattern — a source SYNning many distinct destination ports with few completions; (2) duplicate-IP / ARP-storm behaviour on the lab segment. Each becomes an event with severity and evidence, for a human to investigate.
• A Flask REST API and React dashboard: live counters, top talkers, protocol pie, DNS summary, event list with acknowledge workflow, and weekly CSV/PDF reports for the lab committee.

Feasibility Study

• Technical: Scapy, MySQL and Flask are open-source and documented; the capture PC needs one NIC on the mirror port and runs comfortably on a standard lab machine. A replay mode ingests public sample capture files so development and testing need no live network at all.
• Economic: ₹0 in licences. Hardware is one existing lab PC; the switch already supports port mirroring.
• Operational: The tool is read-only — it cannot affect lab traffic. Privacy is handled by design: headers only, hostnames aggregated, retention capped at 30 days, and access restricted to the lab in-charge, with a permission note signed before deployment.

In scope

• One lab network segment (60 machines) behind a managed switch with a mirror port; capture on the college's own systems only, with written permission.
• Header-only capture: Ethernet/IP/TCP/UDP/ARP/DNS headers; no payloads, no TLS content, no credentials.
• Flow records with 5-tuple, byte/packet counts and timestamps; 30-day retention with an automatic purge job.
• Two passive indicators (port-scan pattern, duplicate-IP/ARP-storm) as human-review events; acknowledge/close workflow.
• Dashboard (top talkers, protocol mix, DNS summary, idle-machine view), weekly reports as CSV and PDF.
• Offline replay of public sample pcaps for development and testing.

Out of scope (v1)

• Any inline function: blocking, rate-limiting,IPS-style prevention — NetLens is strictly observe-only.
• Internet-facing or wireless monitoring, monitor-mode Wi-Fi capture, or any network the college does not own.
• Payload inspection, DPI, or decryption of any kind.
• ML-based anomaly scoring (listed in future scope; v1 uses transparent thresholds).
• Multi-lab centralised deployment (v1 is one segment, one capture PC).

The project follows an incremental SDLC model across the 14 weeks of Major Project Phase-I, with each increment reviewed by the guide before the next begins. Requirements and permissions come first because a monitoring project is only ethical with explicit authorisation.

Testing approach: unit tests for flow assembly and each indicator rule; replay testing with public sample capture files (including scan-heavy samples) for repeatable, safe evaluation; a controlled localhost port-scan test on our own test laptop with the lab in-charge's knowledge; black-box test cases for every dashboard screen; and one clean week of live lab capture to measure false positives before fixing thresholds.

NetLens is a three-tier read-only pipeline: capture, storage, presentation.

1. Capture tier — a Python service on the lab PC uses Scapy to sniff the mirror-port interface, keeping only headers. It maintains an in-memory flow table (5-tuple + direction) flushed to MySQL on idle timeout or every 60 seconds, and an offline replay importer reads pcap files through the same code path, so replay and live capture produce identical records.
2. Storage tier — MySQL holds lab machines, flow records, security events (with JSON evidence) and capture sessions; a daily job deletes flow rows older than 30 days and logs the purge.
3. Presentation tier — a Flask REST API serves aggregated views (top talkers, protocol mix, DNS summary, events) to a React dashboard with two roles: lab in-charge (full access, acknowledge/close events) and viewer (read-only).

flowchart LR
  SW["Lab managed switch — mirror port"] -->|headers only| CAP["Capture service (Python + Scapy)"]
  PCAP["Public sample pcaps"] -->|replay mode| CAP
  CAP --> FT["In-memory flow table + indicator rules"]
  FT -->|flush / events| DB[("MySQL — flows, events, sessions")]
  DB -->|30-day purge| PURGE["Retention job"]
  DB --> API["Flask REST API"]
  API -->|role-based access| UI["React dashboard"]
  UI --> RPT["CSV / PDF weekly report"]

ER diagram

erDiagram
  CAPTURE_SESSION ||--o{ FLOW_RECORD : contains
  LAB_MACHINE ||--o{ FLOW_RECORD : participates
  FLOW_RECORD }o--|| LAB_MACHINE : "peer machine"
  LAB_MACHINE ||--o{ SECURITY_EVENT : "involved in"
  USER ||--o{ EVENT_UPDATE : "acknowledges"
  SECURITY_EVENT ||--o{ EVENT_UPDATE : "has"

  CAPTURE_SESSION {
    int id PK
    datetime startedAt
    string mode "live or replay"
    string interface
  }
  LAB_MACHINE {
    int id PK
    string ipAddress
    string macAddress
    string hostname
    string lab
  }
  FLOW_RECORD {
    bigint id PK
    int sessionId FK
    string srcIp
    string dstIp
    int srcPort
    int dstPort
    string protocol
    bigint bytesIn
    bigint bytesOut
    datetime firstSeen
    datetime lastSeen
  }
  SECURITY_EVENT {
    int id PK
    int machineId FK
    string kind "port-scan or arp-anomaly"
    string severity
    json evidence
    string status
    datetime raisedAt
  }
  EVENT_UPDATE {
    int id PK
    int eventId FK
    int userId FK
    string status
    string remark
  }
  USER {
    int id PK
    string email UK
    string passwordHash
    string role
  }

Data flow (context view)

flowchart TD
  SW(["Lab switch / mirror port"]) -- "packet headers" --> P1["1.0 Capture & assemble flows"]
  P1 <--> D1[("D1 Flow records")]
  P1 --> P2["2.0 Evaluate indicators"]
  D1 --> P2
  P2 --> D2[("D2 Security events")]
  D1 --> P3["3.0 Aggregate & report"]
  D2 --> P3
  P3 --> LC(["Lab in-charge / viewer"])
  LC -- "acknowledge / close" --> D2

Every arrow into NetLens is read-only from the network's point of view: the switch mirror port duplicates traffic to our NIC, and nothing we do can delay, drop or alter the packets the lab machines exchange.

• ML anomaly scoring: replace fixed thresholds with a scikit-learn model (from BCS602) trained on the clean-week baseline, reporting anomaly scores alongside rule events.
• NetFlow input: accept router-exported NetFlow v9 records as an alternative to raw capture, removing the mirror-port requirement.
• Multi-lab agents: one MySQL server aggregating several capture agents with per-lab dashboards.
• Grafana integration: export aggregates to Grafana so the department reuses existing dashboards.
• Indicator pack: SMB-version anomalies, unusual DNS name-length spikes, and beaconing (regular-interval flows) as additional review events.
• IPv6 support: dual-stack flow keys, since labs are beginning to see IPv6 router-advertisement traffic.