District-Level Crop Yield Forecasting & Sowing Advisory for Karnataka (Python ML + MySQL + React)

District-Level Crop Yield Forecasting & Sowing Advisory is a machine-learning project that predicts kharif-season yield (kg/hectare) for major Karnataka crops — ragi, jowar, maize, groundnut and red gram — one to three months before harvest, and turns each forecast into a plain-language advisory: safe to plan for normal yield, below-normal expected, or high risk — review alternate crop.

The data comes from the ICRISAT District Level Database (DLD), an open research dataset that carries season-wise crop area, production, yield, irrigation, rainfall and agricultural-credit variables for about 570 Indian districts going back to 1966. We build a clean district-year panel for 30 Karnataka districts, engineer agronomic features (monsoon rainfall and its lags, irrigated-area share, crop mix), and train baseline linear models against ridge/lasso regression and random-forest / gradient-boosting regressors in scikit-learn, validated with walk-forward (year-based) splits so no future year ever leaks into training.

Forecasts, model metadata and metrics are stored in MySQL, served through a small Express REST API, and shown on a React dashboard where an extension officer or student can pick a district and crop, compare this season's forecast with the five-year average, and export the advisory as CSV.

The project runs as BCS685 Major Project Phase-I (Sem 6) and continues into BCS786 Phase-II (Sem 7); the work split below gives each of the four members an individually examinable module.

Farmers and district agriculture officers in Karnataka decide sowing mix, input purchase and contingency plans in May–June, but the only district yield figures they can consult are at least a season old. Rainfall bulletins tell whether the monsoon has started, not what it will do to ragi in Mandya versus groundnut in Chitradurga. Research-grade yield models rely on satellite imagery and farm surveys that a student team cannot procure or defend.

Meanwhile the ICRISAT District Level Database — decades of district-wise area, production, yield, irrigation and rainfall — sits unused by the people making these decisions. There is a need for a transparent regression system that learns the historical relationship between season rainfall, irrigation and district-crop yield, forecasts the coming kharif yield per district and crop, classifies the outlook into plain advisory bands, and shows its own measured error so users know how much to trust it. The system must be reproducible, data-leakage-free and defensible in a university viva.

Abstract

Karnataka's farm economy swings with the monsoon: a ragi crop in Mandya and a groundnut crop in Anantapur-side Chitradurga respond very differently to the same rainfall season. District-level yield numbers exist — ICRISAT and the Ministry of Agriculture publish them — but they arrive months after harvest, and no simple tool tells a krishi-panchayat or extension officer before harvest whether the coming season looks normal or stressed. This project builds a reproducible regression pipeline that forecasts district-crop yield for the kharif season from historical area, production, irrigation and rainfall data, and packages it behind a small web dashboard.

Introduction

The ICRISAT District Level Database is a 74-dataset research collection covering roughly 571 districts across 20 states, with about 1,030 variables — season-wise area, production and yield per crop, irrigation, rainfall, night lights and credit. For Karnataka it gives an unbroken district-year record from 1966 to recent years. That is exactly the shape of data classical machine-learning regression needs: many rows (district × crop × year), a modest number of well-understood features, and one continuous target (yield). The project applies what the VTU 2022 scheme teaches in BCS602/BCSL606 — regression, ensembles, cross-validation, feature analysis — to a real, citable Indian dataset instead of a toy one.

Existing System

• Descriptive statistics: ICRISAT's own visualisation tool and AgriX/APY dashboards report what the yield was, never a forecast.
• Remote-sensing models in research papers (NDVI-based) are accurate but need satellite pipelines beyond a semester project.
• Krishi-meteorological advisories exist at state level but are not district-specific for rainfed crops and are not student-reproducible.
• Gap: a transparent, district-level, reproducible yield forecast with honest error bars, built only on tabular data a B.Tech team can actually handle and defend in a viva.

Proposed System

• A tidy district-year-crop panel (≈ 30 districts × 5 crops × 55 years ≈ 8,000 rows) assembled from ICRISAT DLD CSVs with documented cleaning rules for district reorganisations.
• Feature engineering: monsoon (Jun–Sep) rainfall and its 1- and 2-year lags, irrigated share of cropped area, cropping intensity, credit per hectare, and cropwise one-hot context.
• Models: ridge/lasso linear baselines, random forest and histogram gradient boosting; evaluated with walk-forward validation by year (train on years ≤ t, test on t+1) using MAPE and R².
• An advisory layer: forecast vs 5-year median → three bands (Normal / Below-normal / High risk), stored in MySQL and served through an Express API to a React dashboard with CSV export.

Feasibility Study

• Technical: all components are open source and run on a 8 GB RAM laptop; the largest CSV is a few hundred MB and pandas handles it comfortably.
• Economic: ₹0 — the dataset is open for research use, and hosting is local.
• Operational: the dashboard needs no training for a student or extension officer; the harder operational truth — that a forecast is a probability, not a promise — is stated on every screen.

In scope

• 30 Karnataka districts and 5 rainfed-dominant kharif crops (ragi, jowar, maize, groundnut, red gram) for years 1966–latest in the DLD.
• Tabular features only (area, production, irrigation, rainfall, credit) — no satellite or sensor data in v1.
• Yield forecast one season ahead at district × crop level; three-band advisory mapping.
• Walk-forward evaluation, per-crop error reports, model versioning in MySQL.
• React dashboard for desktop browsers and a CSV export matching the report tables.

Out of scope (v1)

• Farm-level or plot-level prediction (DLD is district-level).
• Satellite NDVI / soil-moisture ingestion — listed under future scope for Phase-II stretch.
• Rabi crops and horticulture crops with patchy DLD coverage.
• Automatic retraining or a production cloud deployment; the team runs the pipeline locally and demonstrates it live.
• Price or mandi-linkage advice (covered by a separate project family).

The project follows the iterative (incremental) SDLC model: data, model and app grow in reviewed increments so the guide always sees a runnable artefact. It is executed as Phase-I (BCS685, Sem 6) and Phase-II (BCS786, Sem 7) of the same project.

Validation protocol: every model change reruns the same walk-forward harness; a result is only reported if it appears in the harness output, never hand-computed.

The system is a pipeline plus a three-tier app. The offline pipeline (Python) ingests ICRISAT DLD exports, produces the feature panel, trains models and writes predictions into MySQL. The online part is a classic three-tier stack: React presentation tier, Express application tier with a thin validation layer, and MySQL as the data tier shared by both halves.

flowchart TD
  A["ICRISAT DLD CSV exports"] --> B["Cleaning & panel builder (pandas)"]
  B --> C["Feature engineering (lags, irrigation share)"]
  C --> D["Training harness (scikit-learn, walk-forward)"]
  D --> E["Model artefacts (joblib) + metrics"]
  E --> F["MySQL: panel, forecasts, model_runs"]
  F --> G["Express REST API"]
  G --> H["React advisory dashboard"]
  G --> I["CSV export for reports"]
  D --> J["EDA & backtest notebooks (Matplotlib)"]

ER diagram (MySQL feature store)

erDiagram
  DISTRICT ||--o{ OBSERVATION : "records per year"
  CROP ||--o{ OBSERVATION : "is observed as"
  OBSERVATION ||--o{ PREDICTION : "yields"
  MODEL_RUN ||--o{ PREDICTION : "produces"
  DISTRICT {
    int district_id PK
    string district_name
    int state_code
    date valid_from
  }
  CROP {
    int crop_id PK
    string crop_name
    string season
  }
  OBSERVATION {
    int obs_id PK
    int district_id FK
    int crop_id FK
    int year
    float yield_kg_ha
    float monsoon_rainfall_mm
    float irrigated_share
  }
  MODEL_RUN {
    int run_id PK
    string model_name
    string params_json
    float mape
    float r2
  }
  PREDICTION {
    int prediction_id PK
    int obs_id FK
    int run_id FK
    float predicted_yield
    string advisory_band
  }

DFD level 0

flowchart LR
  U(["Extension officer / student"]) -- "district, crop, season" --> SYS(("Crop Yield Advisory<br/>System"))
  SYS -- "forecast, advisory band, CSV" --> U
  D(["ICRISAT DLD export"]) -- "district-year-crop CSVs" --> SYS
  G(["Guide / examiner"]) -- "review requests" --> SYS
  SYS -- "metrics, backtest report" --> G

One request walk-through: the officer picks Mandya — ragi — kharif 2026; Express validates the query, reads the latest PREDICTION row for that OBSERVATION, joins the five-year median and returns the band; React renders the card, the comparison chart and the export button.

• Add satellite features (seasonal NDVI from MODIS via Google Earth Engine) for Phase-II stretch or an M.Tech continuation.
• Extend the panel beyond Karnataka (the DLD covers 20 states) and compare transfer-learning across states.
• Link yield forecasts to a mandi-price module for a revenue outlook, not just a production outlook.
• Replace the static band rule with a quantile-regression model that outputs a prediction interval.
• Automate yearly refresh: a scheduled job re-downloads the DLD update, rebuilds features and re-scores the season.
• Kannada-language advisory cards and an SMS digest for extension officers without regular internet.