District Rainfall Anomaly & SPI Drought Early-Warning System for Telangana (Python ML + Power BI)

District Rainfall Anomaly & SPI Drought Early-Warning System is an advanced time-series project that converts the India Meteorological Department's 0.25° × 0.25° daily gridded rainfall dataset (1901–2024, published by IMD Pune) into operational drought intelligence for the 33 districts of Telangana — a state where a failed Northeast monsoon after a weak Southwest monsoon can push entire districts into agricultural distress within one season.

The pipeline parses the IMD binary/NetCDF grids with IMDLIB, aggregates daily rainfall to district-month series using a district shapefile overlay, computes each month's departure from the 30-year climatological normal, and derives the Standardized Precipitation Index (SPI) at 1-, 3- and 6-month scales by fitting the gamma distribution per district — the standard drought index used by the Indian Meteorological Department's own drought monitoring.

On top of that sit two models: a SARIMA forecaster (statsmodels) for next-month district rainfall with walk-forward validation, and a drought-onset classifier (scikit-learn HistGradientBoosting with class weighting) that predicts whether SPI-3 will fall below −1 next season — the moderate-drought threshold. Results live in MySQL and surface in a Power BI dashboard (CS308PC): district drought calendar, anomaly map, SPI trends and a monthly alert export that names the districts to watch.

The project fills CS705PC Project Stage-I (IV-1) with the literature survey, SPI implementation and baseline, and completes implementation, seminar and viva in Project Stage-II (IV-2, 11 credits).

Telangana districts swing between flood-worthy rainfall and multi-month dry spells, but drought response is organised on declarations that trail the deficit by weeks or months. District officials receive rainfall figures, yet not the two derived answers that trigger timely action: how abnormal the current accumulation is against the long-term normal, and how likely the coming season is to cross the moderate-drought threshold. Research systems that answer these questions depend on satellite soil moisture, reanalysis products and licensed data that a college team cannot procure, reproduce or defend.

There is a need for a reproducible, statistics-grounded early-warning pipeline built on a single canonical public dataset — IMD's gridded daily rainfall — that (a) aggregates 124 years of grids to current district boundaries, (b) computes monthly anomalies and SPI-1/3/6 by the textbook gamma-fit method, (c) forecasts next-month rainfall and classifies next-season drought onset under strict walk-forward validation, and (d) presents district-wise alerts in a dashboard a collectorate briefing can use, with every model's measured error on display.

Abstract

Drought arrives slowly but is declared late: by the time wilting shows up in government reports, the window for seed substitution, fodder planning and relief works has narrowed. This project builds a district-level early-warning system for Telangana entirely on public data — IMD's 124-year gridded daily rainfall — by computing monthly anomalies from the 30-year normal and the Standardized Precipitation Index at three accumulations, forecasting next-month rainfall with SARIMA, and classifying next-season drought onset with a gradient-boosting model. Outputs are published through MySQL and a Power BI dashboard designed for a collectorate briefing.

Introduction

Telangana's agriculture is rain-dependent across most of its cropped area, and its Deccan-plateau position makes it sensitive to monsoon breaks — long rainless spells inside an otherwise normal season. The IMD gridded rainfall product gives one of the world's longest uniform rainfall records (1901–2024) at a resolution fine enough to separate neighbouring districts. What is missing is a reproducible pipeline that turns those grids into the two numbers decision-makers actually use: how much is this month off the normal? and what is the probability the next season turns drought?

Existing System

• IMD's weekly and monthly rainfall reports give anomalies district-wise but no season-ahead drought probability.
• State disaster portals compile indicators only after the season is under way.
• Research drought models often mix satellite soil moisture and reanalysis data — powerful but hard for a B.Tech team to obtain, license and defend.
• Gap: a transparent, reproducible pipeline on one canonical public dataset, comparing classical statistics (gamma-fit SPI) and machine learning under one walk-forward protocol.

Proposed System

• Data engineering: IMDLIB downloads the 0.25° grids; a GeoPandas overlay with nearest-cell fallback aggregates daily rainfall to the 33 current districts; QA checks catch grid gaps and leap-day drift.
• Climatology & SPI: monthly normals (1991–2020) and percentage anomalies; SPI-1/3/6 via a gamma fit per district and month, with the fitted parameters validated against a reference implementation.
• Forecasting: SARIMA models per district for monthly rainfall, selected on AIC, validated walk-forward (train on years ≤ t, test t+1) with MAE and RMSE.
• Classification: a HistGradientBoosting classifier predicts whether next-season SPI-3 drops below −1, using seasonal lags and accumulated rainfall features; class weighting handles the rarity of drought onset, and PR-AUC plus recall are the reported metrics.
• Delivery: MySQL stores the series, indices and forecasts; Power BI renders the drought calendar, anomaly map and SPI trends; a script exports the monthly districts-to-watch alert list.

Feasibility Study

• Technical: IMDLIB handles the binary/NetCDF formats natively; the Telangana slice fits in laptop RAM; Docker (CS503PC) packages the pipeline so the examiners' machine runs the demo identically.
• Economic: zero — IMD's gridded data is free for research; the stack is open source; Power BI Desktop is free.
• Operational: the dashboard reads precomputed MySQL tables, so the monthly refresh is a cron-able script; the caveat — precipitation-only SPI monitoring ignores evapotranspiration and soil moisture — appears wherever the alert list does.

In scope

• IMD 0.25° gridded daily rainfall only (precipitation-driven drought monitoring).
• The 33 current districts of Telangana; district-level outputs, not mandal or farm level.
• SPI at 1-, 3- and 6-month scales; monthly anomaly percentages; 1991–2020 normals.
• SARIMA forecasting one month ahead; drought-onset classification one season ahead.
• MySQL storage, Power BI dashboard, monthly alert CSV, Docker packaging and a seminar deck.

Out of scope (v1)

• Temperature, evapotranspiration and soil-moisture inputs (SPEI and agricultural-drought indices) — future scope.
• Satellite NDVI or reanalysis datasets.
• Operational deployment to a government server; the deliverable is a demonstrable pipeline and dashboard.
• Mandal-level disaggregation and crop-specific drought sensitivity.
• Automatic SMS/WhatsApp alerting — the alert layer stops at CSV and dashboard.

The project follows the incremental (evolutionary) SDLC model with a statistical core: each increment ends in a runnable notebook, script or dashboard the guide can verify, which suits a two-stage university project where Stage-I already demands a working piece of analysis.

Validation protocol: every index and model change re-runs the same harness — SPI code against reference SPI values, forecasts against walk-forward MAE/RMSE, the classifier against per-year precision-recall curves; no number enters the report without harness output.

The system separates heavy offline computation from light online delivery. The offline pipeline (Python) downloads and aggregates IMD grids, computes indices and trains models, writing everything to MySQL; the online layer is Power BI reading MySQL directly, plus a cron-able alert exporter.

flowchart TD
  A["IMD Pune 0.25° gridded daily rainfall"] --> B["IMDLIB parser (.grd / NetCDF)"]
  B --> C["District aggregation (GeoPandas overlay)"]
  C --> D["MySQL: monthly rainfall series"]
  D --> E["Climatology & anomaly module"]
  D --> F["SPI module (gamma fit, 1/3/6-month)"]
  D --> G["SARIMA forecaster (statsmodels)"]
  F --> H["Drought-onset classifier (scikit-learn)"]
  E --> I["MySQL: indices, forecasts, model_runs"]
  F --> I
  G --> I
  H --> I
  I --> J["Power BI dashboard"]
  I --> K["Monthly alert CSV exporter"]

ER diagram

erDiagram
  DISTRICT ||--o{ MONTHLY_RAINFALL : "records"
  MONTHLY_RAINFALL ||--o{ SPI_VALUE : "derives"
  DISTRICT ||--o{ FORECAST : "receives"
  MODEL_RUN ||--o{ FORECAST : "produces"
  FORECAST ||--o{ ALERT : "triggers"
  DISTRICT {
    int district_id PK
    string district_name
    float centroid_lat
    float centroid_lon
  }
  MONTHLY_RAINFALL {
    int rain_id PK
    int district_id FK
    int year
    int month
    float total_mm
    float normal_mm
    float anomaly_pct
  }
  SPI_VALUE {
    int spi_id PK
    int rain_id FK
    int scale_months
    float spi
    string drought_class
  }
  MODEL_RUN {
    int run_id PK
    string model_name
    string params_json
    float mae
    float pr_auc
  }
  FORECAST {
    int forecast_id PK
    int district_id FK
    int run_id FK
    int target_year
    int target_month
    float predicted_mm
    float drought_probability
  }
  ALERT {
    int alert_id PK
    int forecast_id FK
    string severity
    date issued_on
  }

DFD level 0

flowchart LR
  IMD(["IMD Pune data server"]) -- "0.25° daily rainfall grids" --> SYS(("Drought Early-Warning<br/>System"))
  O(["District officer / collectorate staff"]) -- "month & district query" --> SYS
  SYS -- "anomaly, SPI, drought probability, alert CSV" --> O
  S(["Guide & seminar committee"]) -- "review requests" --> SYS
  SYS -- "validation tables, dashboard" --> S

A monthly refresh walk-through: the job downloads the latest grid month, recomputes district aggregates, updates SPI rows, re-scores forecasts, and the alert exporter writes the districts-to-watch CSV that Power BI's alert page renders with severity colours.

• Add temperature and evapotranspiration to compute SPEI, covering agricultural rather than only meteorological drought.
• Ingest NDVI from open satellite sources to cross-check drought flags with vegetation stress.
• Extend to the Krishna and Godavari basins across state borders with the same pipeline (change one shapefile).
• Replace the single best model with an ensemble of SARIMA, gradient boosting and a persistence baseline, weighted by recent walk-forward performance.
• Link alert lists to crop calendars so severity bands become action advisories per sowing window.
• A scheduled cloud job that refreshes the pipeline monthly and pushes the alert CSV to a public dashboard.