Zero-Downtime Deployment Study: Blue-Green vs Rolling for a Small Web Service

Every college IT team running its own small services — a notices portal, a timetable API, a lab-booking helper — faces the same question: how do we release a new version without students seeing an error page? Two classic answers are blue-green deployment (run the new version beside the old and switch traffic atomically) and rolling deployment (replace instances one at a time). Blogs argue which is better; this project measures it.

We build a small stateless Node.js service serving student notices and timetable data — a workload a JNTUH-affiliated college actually runs — and a reproducible deployment rig: Docker containers, Nginx as reverse proxy and load balancer, and scripted pipelines that deploy each strategy with health checks and connection draining. Prometheus and Grafana (industry-standard monitoring tools, used as extras beyond the syllabus) record what users experience, while a load generator drives steady traffic.

Each strategy runs 10 times from an identical starting state, and we compute what matters: downtime, failed-request rate, p95 latency, deployment duration and rollback behaviour. The deliverable is not just a rig but an evidence-backed answer — which strategy suits a college service with modest memory but a hard rule that exam-day traffic never sees an error page — written up as an experimental report for the JNTUH R22 Stage-II seminar.

The work maps onto the R22 DevOps courses (CS503PC/CS505PC: containerisation, CI/CD) and CS409PC web technologies, and demonstrates measurement discipline beyond building an app: hypotheses, a controlled experiment, conclusions the panel can check against raw data.

College-run web services — notices, timetables, lab systems — are redeployed with a stop-start-restart cycle, and every redeployment is a visible outage for students; teams therefore avoid releasing during exam weeks precisely when fixes are most needed. Zero-downtime deployment strategies are well known, but the available guidance is scattered across vendor blogs and Kubernetes-centric documentation, and almost none of it is presented as a reproducible measurement on the single-server, open-source setups colleges actually have. As a result, a college IT team cannot answer a basic engineering question — which strategy gives fewer user-visible failures on our hardware for our traffic — before committing to one.

This project fills that gap: build a small, representative Node.js service and a scripted Docker/Nginx rig that deploys it by blue-green and rolling strategies with health checks and draining, measure downtime, error rate, p95 latency and switch time across repeated identical runs with Prometheus, and deliver an evidence-backed recommendation plus a rig that other teams can re-run and extend.

Abstract

Small web services in colleges are typically updated by stopping the old version and starting the new one — a window in which every user sees a failure. Zero-downtime techniques exist (blue-green, rolling, canary), but published comparisons are usually vendor-oriented or Kubernetes-only, not reproducible on a college's single server with open tools. This project constructs a controlled, measurable comparison of blue-green vs rolling deployment for a small Node.js service on Docker + Nginx, runs each strategy 10 times under identical load, and reports downtime, error rate, latency and switch time from Prometheus data. The outcome is both a reusable rig and a data-backed recommendation.

Introduction

The JNTUH R22 curriculum teaches DevOps — containerisation, CI/CD pipelines, monitoring — but deployment strategies are usually taught as diagrams, not as measured behaviour. Meanwhile the student-facing services a college runs genuinely cannot go down at particular times: results day, exam-schedule publication, counselling week. A deployment study turns an abstract syllabus topic into engineering evidence.

Background / Existing Practice

• Recreate deployment (stop, build, start): simplest, but every second of build time is user-visible downtime.
• Blue-green: two identical environments (Blue = live, Green = new); deploy to Green, health-check it, then switch the proxy to Green. Downtime approaches zero, rollback is a switch back, but memory usage doubles during the switch.
• Rolling: replace instances one at a time behind the load balancer, removing each old instance only after the new one passes health checks and drains connections. Uses N+1 resources instead of 2N, but in-flight requests can fail if draining is not done correctly.
• Published comparisons typically assume Kubernetes; few show a reproducible measurement method on commodity single-server setups, which is the gap this project addresses.

Proposed Work

• Build the reference service (stateless by design, so storage effects do not confound the deployment comparison) with a /health endpoint and Prometheus metrics.
• Script both pipelines on one machine: blue-green with an atomic Nginx upstream switch and reload; rolling with per-instance replace, health gate and connection drain.
• Drive identical traffic with a load generator during every run and capture per-run metrics: failed requests, refused connections, p95 latency, switch duration, peak memory.
• Run a 2-strategy × 10-run matrix (plus 5 rollback runs each), then analyse means and variance and write the Stage-II report with the rig released as a re-runnable artifact.

Feasibility Study

• Technical: everything runs on one 8 GB machine with Docker, Nginx, Node.js and Prometheus — all open source, all taught or listed in the R22 DevOps track. No cloud spend is required, which keeps the college lab reproducible.
• Economic: zero licence cost; the rig runs on an existing lab server and can be re-run each semester as a lab exercise.
• Operational: the pipelines are single scripts (deploy-bluegreen.sh, deploy-rolling.sh) with clear logs, so future batches can extend the study to canary releases without reverse-engineering the setup.

In scope

• One stateless Node.js service, containerised; single-host Docker + Nginx rig (the typical college setup).
• Two strategies fully implemented: blue-green and rolling, each with health checks, draining and rollback paths.
• Measurement of downtime, failed-request rate, p95 latency, switch duration and peak memory under a fixed load profile.
• 2 × 10 deployment runs plus 5 rollback runs per strategy, with raw metrics exported and plotted.

Out of scope (v1)

• Kubernetes and cloud load balancers — deliberately excluded to keep the rig reproducible on one lab machine.
• Canary releases and feature flags (listed as future work).
• Database schema migrations during deployment (the service is stateless by design; migration ordering is its own study).
• Multi-service or microservice deployments, autoscaling and multi-host networking.

The project is run as a controlled engineering experiment inside an incremental build plan — Stage-II of the JNTUH R22 project track (11 credits, semester IV-2, assessed at CIE 40 + SEE 60).

Hypotheses: H1 — blue-green produces zero user-visible failures but doubles memory during the switch; H2 — rolling without draining shows transient 502s, and draining removes most of them; H3 — switch duration, not strategy, dominates the error window on a single host.

Validity controls: identical starting state (fresh containers), same load profile and duration every run, same host and network, and a fixed release artifact — only the deployment strategy varies.

The rig is a single-host deployment topology in which everything — traffic, routing, metrics — is visible and measurable.

1. Traffic path: a load generator sends steady HTTP traffic to Nginx, which load-balances across the running application containers. During deployments only the upstream definition changes; the traffic path itself is never edited.
2. Application tier: the notices/timetable API runs as 2 (rolling) or 2×2 (blue-green) Docker containers, each exposing /health for readiness and /metrics for monitoring. The service is stateless by design so that data-tier effects cannot contaminate the deployment measurement.
3. Measurement tier: Prometheus scrapes Nginx and every app container every 5 seconds; Grafana visualises request rate, error rate and latency; a Bash/Node extraction script joins each run's window into one row of the results CSV.

flowchart LR
  LG[Load generator - fixed profile] -->|HTTP| NX[Nginx reverse proxy]
  NX --> B1[Blue app v1.0]
  NX --> B2[Blue app v1.0 - replica 2]
  NX -.->|switch| G1[Green app v1.1]
  NX -.->|switch| G2[Green app v1.1 - replica 2]
  B1 --> M[/metrics endpoint/]
  G1 --> M
  NX --> M
  M -->|scrape every 5 s| P[Prometheus]
  P --> GR[Grafana dashboards]
  P --> E[Per-run metric extractor]
  E --> CSV[(results.csv)]

Blue-green switch sequence

flowchart TD
  S0["1. Blue is live, serving 100%"] --> S1["2. Start Green with the new image"]
  S1 --> S2{"3. Green /health passes for 30 s?"}
  S2 -- no --> AB["Abort - Blue untouched, log failure"]
  S2 -- yes --> S3["4. Rewrite Nginx upstream to Green"]
  S3 --> S4["5. nginx -s reload - atomic worker refresh"]
  S4 --> S5["6. Drain Blue: stop new connections, wait for in-flight"]
  S5 --> S6{"7. Error rate normal for 60 s?"}
  S6 -- no --> RB["Rollback: upstream back to Blue + reload"]
  S6 -- yes --> S8["8. Mark release healthy, keep Blue as standby"]

Two design choices deserve the panel's attention: the Nginx reload is used instead of a restart so worker processes are refreshed without dropping listening sockets; and the drain window after every switch is what turns a technical success into a user-visible success — it is also the exact mechanism our hypotheses test.

• Canary strategy: route 10% of traffic to the new version, watch the error rate, and automate promotion or abort — completing the blue-green/rolling/canary triad.
• Automated rollback: a watcher that flips the upstream back when the post-switch error rate crosses a threshold, turning rollback from a script into a control loop.
• Database-migration ordering: add a stateful variant of the service and study expand/contract schema migrations during deployments.
• Multi-host rig with two VMs to measure how the strategies behave once the load balancer is no longer on the same host.
• Chaos injection: kill the new container mid-switch to compare how each strategy contains a genuinely broken release.
• CI integration: trigger the pipelines from a push to the main branch so every commit is measured, not just releases.