Dengue Outbreak Early-Warning and Surveillance Analytics System from NCVBDC Data (Python)

Dengue outbreaks in Indian cities follow the weather. Rain-filled containers breed Aedes aegypti mosquitoes, and reported cases surge a few weeks after sustained rain — a delay that gives a prepared health team a genuine head start. This project builds a weekly dengue early-warning prototype for a large Indian city (Chennai) entirely from public data, and turns that delay into a measurable alert lead time.

Three verifiable sources feed the study. The NCVBDC (National Centre for Vector-Borne Diseases Control, Ministry of Health and Family Welfare) publishes state/UT-wise dengue cases and deaths; the OpenDengue open dataset (CC BY 4.0) provides weekly India case counts at national and sub-national level; and the Open-Meteo historical weather API serves daily ERA5 reanalysis — temperature, rainfall and humidity — for any coordinate. The case series is aligned to epidemiological weeks and joined with climate features lagged 1–8 weeks, following the lag structure reported in the published literature for Indian states.

The modelling question is deliberately honest: can a simple, transparent model beat a seasonal-naive baseline at flagging outbreak weeks, under a strictly temporal train/validation/test split (no shuffled k-fold, no leakage)? Results are reported as ROC-AUC, precision, recall, F1 and practical lead time, and converted into an alert-threshold discussion written for municipal health officials: what threshold, how many weeks of warning, and what to do first.

This is an analysis project, not a packaged application — the deliverables are the pipeline scripts, the regenerated charts and an evaluation memo.

Health departments in dengue-endemic cities learn about an outbreak when hospitals report case clusters — typically two to four weeks after the weather conditions that triggered it. Surveillance data sits in situation reports and PDF tables, climate reanalysis sits on a public API, and nothing systematically joins the two into a forward-looking weekly signal. Meanwhile, published forecasting results are hard to trust: many use shuffled cross-validation that leaks future information into training, skip any baseline comparison, and stop at a model score without saying what a health official should actually do when it fires. There is a need for a small, honest, reproducible study that answers three questions on real Indian data: how many weeks of warning are achievable, at what precision and recall, and what alert rule would make that warning usable in daily municipal practice.

Abstract

Dengue is India's fastest-growing vector-borne disease, with NCVBDC recording well over a million reported cases during 2021–2025 and a peak in 2023. Municipal health teams tend to react to case surges rather than anticipate them, because the weather signal that precedes an outbreak is never routinely combined with surveillance data at the city level. This project builds a reproducible weekly early-warning prototype for Chennai that joins public dengue surveillance with reanalysis climate data, engineers climate-lag features, and measures how far in advance an outbreak week can be flagged with usable precision.

Introduction

The Aedes mosquito breeds in standing water, so weekly rainfall, temperature and humidity in preceding weeks are physically linked to case counts a few weeks later. Published studies for Indian states report lagged associations — rainfall and relative humidity near zero to a few weeks of lag, temperature effects stretching to one to three months. A student project cannot access confidential case-line data, but it does not need to: NCVBDC situation tables, the OpenDengue dataset and the Open-Meteo archive API together provide everything required at city-week resolution, free of cost.

Existing system vs proposed work

• Surveillance reaches the public as cumulative tables and PDF situation reports; nobody joins it with climate data routinely.
• City dashboards show the present; none offer a probabilistic view of the next 2–4 weeks.
• Many published forecasting results use shuffled cross-validation, which leaks the future into training and inflates accuracy; baselines are often missing.
• Proposed: a documented weekly pipeline — acquisition, cleaning, lag features, temporal split, baseline-versus-model comparison — with every figure regenerable from scripts and every source cited, ending in an actionable alert rule rather than a bare model score.

Feasibility study

• Data: all three sources are free; Open-Meteo needs no API key for non-commercial use, and OpenDengue is CC BY 4.0.
• Technical: pandas, NumPy and scikit-learn handle the entire pipeline on a student laptop; the joined dataset is only a few thousand rows.
• Economic: zero cost; everything runs offline after download.
• Operational: outputs are plain charts and a one-page alert memo that a health officer can act on; no production deployment is claimed.
• Ethical: only aggregate, publicly reported counts are used — no patient records and no personal identifiers are touched.

In scope

• One city (Chennai) with the option to swap in any other Indian city by changing coordinates and data extracts.
• Weekly resolution, using the longest window the case sources allow (OpenDengue weekly India data; NCVBDC state tables from 2021 onwards).
• Three free data sources: NCVBDC situation tables, OpenDengue (CC BY 4.0) and Open-Meteo ERA5 reanalysis.
• Climate-lag feature engineering, temporal validation, baseline-versus-model evaluation and an alert-threshold analysis.
• Aggregate counts only; no individual patient or hospital-identity data.

Out of scope (v1)

• Ward-level or street-level prediction — no public weekly data exists at that resolution.
• A deployed real-time system with automatic ingestion and SMS alerts; the study delivers the analytics and a design for deployment.
• Bed-capacity or hospital-logistics planning, and other vector-borne diseases beyond dengue.
• Collecting primary data from hospitals or the corporation; everything is secondary and public.

The study follows an adapted CRISP-DM cycle — business understanding, data understanding, data preparation, modelling, evaluation — compressed into a 14-week academic timeline and run as a research workflow rather than a software build.

Validation design: train on the earliest years, validate on 2023, and touch 2024–2025 only once at the end. Every transformer is fitted on training years alone, and a scripted guard asserts that no feature row can reference data from its own week or later. Model quality is reported only next to the seasonal-naive baseline — if the model cannot beat it on held-out years, the report says so.

The study is organised as a linear pipeline of small scripts over a local SQLite database, so each stage can be re-run and audited independently.

1. Ingestion layer — one downloader per source: a manual transcription helper for NCVBDC PDF tables (with the source file name logged), a CSV fetch for OpenDengue, and a requests client for the Open-Meteo archive endpoint.
2. Storage layer — SQLite holds raw and cleaned tables (cases, weather, weeks, features, predictions) so joins are SQL-auditable and the file can be submitted alongside the report.
3. Feature layer — pandas builds lagged climate variables and seasonality encodings on the epidemiological-week grid, with an automated leakage guard.
4. Model layer — scikit-learn pipelines fit baselines and classifiers; every split boundary is a date, never a random seed.
5. Evaluation layer — one script recomputes ROC-AUC, precision, recall, F1 and lead time for every entry in the results table.
6. Alert layer — translates predicted probabilities into a consecutive-weeks rule and renders the officer-facing memo and charts.

flowchart TD
  N["NCVBDC situation tables (PDF)"] -->|"transcribe + log source"| C["Clean weekly case series"]
  O["OpenDengue weekly CSV (CC BY 4.0)"] -->|download| C
  M["Open-Meteo archive API - ERA5 daily climate"] -->|JSON over HTTPS| W["Daily weather table"]
  C --> D[("SQLite: dengue.db")]
  W --> D
  D --> F["Feature builder: lags 1-8 weeks, rolling rainfall, seasonality"]
  F --> L["Leakage guard tests"]
  L --> S["Temporal split: train / validate / test"]
  S --> B["Baselines: seasonal naive, 4-week mean"]
  S --> G["Models: logistic regression, random forest, gradient boosting"]
  B --> E["Metrics: ROC-AUC, precision, recall, F1, lead time"]
  G --> E
  E --> T["Threshold analysis: precision-recall trade-off"]
  T --> A["Alert rule: 2 consecutive weeks above cut-off"]
  A --> R["Charts and one-page memo for health officials"]

Dataset schema

erDiagram
  WEEK_CASES ||--o{ WEEK_FEATURES : "joins on epi_week"
  WEEK_CLIMATE ||--o{ WEEK_FEATURES : "joins on epi_week"
  WEEK_FEATURES ||--o{ PREDICTION : "feeds"

  WEEK_CASES {
    int epi_week PK
    int year
    int week_no
    int cases
    int deaths
    string source
  }
  WEEK_CLIMATE {
    int epi_week PK
    float rain_mm_total
    float temp_max_mean
    float temp_min_mean
    float humidity_mean
  }
  WEEK_FEATURES {
    int epi_week PK
    float rain_lag1
    float rain_lag4
    float temp_lag8
    float sin_week
    float cos_week
    int outbreak_label
  }
  PREDICTION {
    int epi_week PK
    string model
    float prob_outbreak
    int alert_flag
  }

• Real-time ingestion: scheduled jobs that re-fetch OpenDengue and Open-Meteo weekly and refresh the alert memo automatically.
• Multi-city expansion: run the same pipeline across several Tamil Nadu cities and study whether shared climate drivers let one model serve all.
• ** richer features:** ENSO/Indian Ocean Dipole indices, soil moisture from ERA5-Land, and vector-density data if the corporation shares it.
• Calibrated probabilities: isotonic or Platt calibration so the threshold maps to a defensible alert frequency (for example, at most six alerts a year).
• Ward-level modelling if weekly ward data can be obtained through an academic MoU with the city health department.
• Deployment prototype: a small dashboard plus SMS/e-mail digest for corporation staff, designed in Chapter 6.