District Court Cause-List Aggregator & Case-Status Tracker for eCourts Data (Python/Flask + MySQL)

District Court Cause-List Aggregator (DCLA) is a Python/Flask web application that turns the daily cause lists and case-status pages published on the eCourts portal into one searchable, local database for a single district court complex — the demo is configured for the district courts of Ranga Reddy district, Telangana. Litigants and advocates navigate the official portal court by court and date by date, and there is no way to be told when a hearing date changes.

DCLA runs in two parts. A scheduled fetcher (APScheduler) pulls the public cause-list and case-status pages for the configured courts a couple of times a day — politely, with rate limiting, retries with exponential backoff, an honest User-Agent and a SHA-256 content-hash check that skips pages already seen. A parser (BeautifulSoup) converts each page into structured rows — serial number, case type and number, parties, advocates, purpose of hearing — and quarantines rows it cannot parse, keeping the raw HTML so nothing is lost when a layout changes. The Flask front end offers search by case number, party or advocate, day-wise cause-list browsing, CSV export and a watchlist that flags hearing-date changes for tracked cases.

Python comes from the Osmania curriculum, and the scheduling, retry and consistency concerns come straight from the PC 751 Distributed Systems Lab; Flask, BeautifulSoup and APScheduler are industry-standard libraries used on top of that foundation. The system fetches only public pages at low volume and stores nothing beyond what the courts already print.

District-court litigants and small-practice advocates depend on daily cause lists to know where and when their matters appear, yet the official eCourts portal presents this public data one court and one date at a time, with search that requires exact case numbers and no way to learn that a hearing has been adjourned until the next list is published. Lawyers therefore keep private diaries of hearing dates, litigants depend on phone calls to the advocate's clerk, and a missed adjournment costs a day's wages and a wasted trip to the complex. The data needed to solve this already exists in public cause lists and case-status pages, but it is fragmented across dozens of per-court pages that change daily.

There is a need for a local aggregator that fetches those public pages on a schedule, parses them into a searchable database, tracks a watchlist of cases, detects when a hearing date or stage changes, and exposes all of it through a fast web interface — while fetching politely, tolerating layout changes and storing no more than the courts themselves publish.

Abstract

The eCourts programme has put cause lists, case status and orders of every district court online, but the information is organised portal-first: one court, one date, one page at a time. This project builds DCLA, an application that aggregates the public cause-list and case-status pages of one district court complex into a normalised MySQL database behind a Flask web app, adds a case watchlist with hearing-change detection, and runs the fetch–parse–store pipeline on a schedule with production-style reliability: retries, rate limiting, idempotent upserts and a quarantine for unparseable pages.

Introduction

A district court complex can have dozens of courts, each publishing a daily cause list — the day's schedule of cases with serial number, case details, parties and purpose of hearing. A litigant knows a case number or a party name; an advocate tracks twenty matters across courts. The official portal answers both needs, but one query at a time, and with no notification when an adjournment moves a hearing. All of the underlying data is public; what is missing is a local, searchable, history-aware view.

Existing System

• The eCourts services portal supports case-status search by CNR or party name and day-wise cause lists per court — but each lookup is manual, per court and per date.
• Advocates keep hearing dates in personal diaries copied from the portal; a missed adjournment means a wasted trip.
• Commercial legal-research services serve large firms, not small practices in a district town.
• Limitations: no combined search across a complex's courts, no per-case history of hearing movements, no alerting, no offline archive.

Proposed System

• A fetcher job that, twice a day, downloads the configured courts' cause lists (current and next working day) plus the status pages of watched cases, with rate limiting, timeouts, exponential-backoff retries and hash-based skip of unchanged pages.
• A parser built on BeautifulSoup with a configurable column map per layout; rows that fail validation go to a parse_error table with the raw HTML path for re-run — no silent data loss.
• Idempotent storage in MySQL: upserts keyed on (court, list date, serial number), with a full fetch log for provenance.
• A Flask app with search (case number, party, advocate), day-wise cause-list browsing, per-case history, CSV export and an admin screen to re-parse quarantined pages.
• A watchlist that records a hearing_change event whenever a tracked case's hearing date or stage moves.

Feasibility Study

• Technical: all libraries are open-source; the pipeline is deliberately tolerant — a layout change degrades to quarantined rows, not a crash — and parser tests run against saved HTML snapshots, so development and the viva demo never need the live site.
• Economic: zero licence cost; the college lab machine or a free-tier VM is enough.
• Operational/legal: only publicly printed information is collected, a few dozen requests per day, with an identifying User-Agent, robots.txt respected and raw pages cached for reproducibility. No document numbers beyond public case identifiers are stored.

In scope

• One district court complex (demo: Ranga Reddy district courts, Telangana), with court names and halls configurable in the database.
• Daily cause lists for the current and next working day for each configured court; case-status pages for watched cases only.
• Structured storage of serial number, case type/number/year, parties, advocates, purpose and stage as printed.
• Search (case number, party, advocate), day-wise browsing, per-case history, watchlist with hearing-change events, CSV export, admin screen for re-parsing.
• Offline mode driven by bundled HTML snapshots for development and demos.

Out of scope (v1)

• Bulk mirroring of the portal, orders/judgement PDFs, or high-court data.
• Automated SMS/WhatsApp/e-mail delivery of alerts (the alert queue is generated; sending is future work).
• Login for the general public; v1 has a single admin/staff account for the watchlist and admin screens.
• Any personal identifier beyond what the courts themselves print on the lists (no identity-document numbers are stored, ever).
• Predicting adjournments or any analytics beyond reporting.

The project follows the Incremental SDLC model with a pipeline-first order of work, so the riskiest part — parsing a real page layout — is proven early and the interface is built on stable data.

Testing approach: parser unit tests run against HTML snapshots captured once and versioned with the code (snapshot testing), so a portal layout change shows up as a failing test plus quarantined rows rather than silent corruption. Flask routes are integration-tested with pytest's test client and a local static server standing in for the portal; the watchlist job is tested by replaying two versions of the same list. Black-box test cases (ID, input, expected, actual, status) cover every screen.

DCLA has four components connected by a MySQL store:

1. Scheduler & fetcher — APScheduler cron jobs (early morning and evening) call the fetcher, which applies per-host rate limiting with jitter, connects with timeouts, retries with exponential backoff (max four attempts), identifies itself with a project User-Agent, honours robots.txt, and stores every response body under raw/ with its SHA-256 hash in fetch_log.
2. Parser — reads new raw pages, uses per-layout column maps to extract rows with BeautifulSoup, validates them (serial numbers sequential, case numbers matching the pattern for the complex), and either upserts rows into MySQL or writes a parse_error with the raw-file path.
3. MySQL 8 store — courts, cause lists, entries, tracked cases, hearing changes, parse errors and the fetch log, with unique keys that make every write an idempotent upsert.
4. Flask web app — Jinja templates for search, day-wise cause lists, case history, watchlist and admin re-parse; read paths hit MySQL indexes, and CSV export streams from a server-side cursor.

flowchart TD
  APS["APScheduler cron jobs (2x daily)"] --> F["Fetcher: rate limit, retries, User-Agent, robots.txt"]
  F -->|"GET public pages"| EC[("eCourts portal (public)")]
  F --> R[("raw pages + fetch_log + SHA-256")]
  R --> P["Parser: BeautifulSoup + column map"]
  P -->|"valid rows"| DB[("MySQL 8: cause lists & entries")]
  P -->|"unparseable"| Q[("parse_error quarantine")]
  DB --> W["Watchlist job: hearing-change detection"]
  W --> H[("hearing_change events")]
  DB --> FL["Flask app (Jinja views)"]
  H --> FL
  FL --> B["Browser: search, cause lists, watchlist, CSV"]

ER diagram

erDiagram
  COURT ||--o{ CAUSE_LIST : "publishes"
  CAUSE_LIST ||--|{ CAUSE_LIST_ENTRY : contains
  FETCH_LOG ||--o| CAUSE_LIST : "produced"
  CAUSE_LIST ||--o{ PARSE_ERROR : "may have"
  TRACKED_CASE ||--o{ HEARING_CHANGE : "accumulates"
  TRACKED_CASE ||--o{ CAUSE_LIST_ENTRY : "appears as"

  COURT {
    int id PK
    string name
    string hall
    string judge_designation
    string list_url_template
    boolean active
  }
  FETCH_LOG {
    int id PK
    string url
    int http_status
    int bytes
    string sha256
    datetime fetched_at
    int duration_ms
    string outcome
  }
  CAUSE_LIST {
    int id PK
    int court_id FK
    date list_date
    int fetch_id FK
    string source_sha256
    string status
  }
  CAUSE_LIST_ENTRY {
    int id PK
    int cause_list_id FK
    int serial_no
    string case_type
    string case_number
    int case_year
    string parties
    string advocates
    string purpose
    string stage
  }
  PARSE_ERROR {
    int id PK
    int cause_list_id FK
    string raw_path
    string message
    boolean resolved
  }
  TRACKED_CASE {
    int id PK
    string case_type
    string case_number
    int case_year
    string party_hint
    date last_hearing_date
    string last_stage
    date watch_since
  }
  HEARING_CHANGE {
    int id PK
    int tracked_case_id FK
    date old_date
    date new_date
    string old_stage
    string new_stage
    datetime detected_at
    boolean notified
  }

DFD — Level 0 (context diagram)

flowchart LR
  EC(["eCourts portal (public pages)"]) -- "cause lists, case status" --> SYS(("District Court<br/>Cause-List Aggregator"))
  A(["Staff / admin"]) -- "courts config, watchlist additions, re-parse" --> SYS
  SYS -- "search results, daily lists, change alerts, CSV" --> A
  L(["Litigants & advocates"]) -- "case number / party / advocate queries" --> SYS
  SYS -- "hearing dates & movements" --> L

• Alert delivery: e-mail and SMS dispatch from the existing alert queue, with digest mode for advocates tracking many matters.
• Telugu interface alongside English, since most litigants in the demo district prefer it.
• Multi-district configuration and a combined search across complexes, with per-district fetch budgets.
• Orders/judgement links: storing the portal's public document links for tracked cases (still fetching nothing in bulk).
• Full-text search using MySQL FULLTEXT or an SQLite FTS sidecar for typo-tolerant party-name search.
• Public read-only API with rate limiting so college apps and clinics can reuse the aggregated data responsibly.