Back to all work
July 2018 — July 2021 · Bangalore, India

BigBasket

Senior Software Engineer

India's largest online grocery platform. Pioneered the monolith-to-microservice transition, built cross-database reporting at scale, and led data pipeline engineering for customer recommendations.

Team5 engineers + data scientists
Duration3 years
Stack
JavaPythonSparkAirflowRedshift
DomainE-commerce / Grocery
Projects

What I built here

01

"Stitch" Cross-Database Reporting System

Designed and developed Stitch, enabling cross-database join operations across MySQL, PostgreSQL, MS SQL, Redshift, and Athena — processing 20,000+ reports daily.

Challenge

Business data lived across 5+ database systems with no way to join or query across them. Teams relied on manual data extraction and Excel analysis.

Approach

Built a query federation layer that could join across MySQL, PostgreSQL, MS SQL, Redshift, and Athena, with scheduled report generation and distribution.

Impact

Processes 20,000+ reports daily, providing critical business metrics and improving operational efficiency for hubs and darkstores.

02

Analytics Data Pipeline

Led a team of 5 data scientists and engineers in building a data pipeline processing 3 million+ order items daily for customer basket-building recommendations.

Challenge

Recommendation data was stale and processing was slow, limiting the ability to personalize customer baskets effectively.

Approach

Airflow-orchestrated pipeline using pandas and Apache Spark, processing 3M+ order items daily to deliver transformed data for the recommendation engine.

Impact

Enabled real-time basket-building recommendations. Also contributed to MBQ service that reduced perishable waste by 40% and achieved 95% availability for 5,000+ SKUs.

03

Monolith to Microservices

Pioneered BigBasket's microservices transition, re-architecting core member APIs and improving performance by 64% in latency for login, sign-up, and address management.

Challenge

Monolithic architecture with tightly coupled domains, slow deployments, and cascading failures affecting all customer-facing features.

Approach

Extracted core member APIs (login, sign-up, address) into independent services, with proper API contracts and independent deployment pipelines.

Impact

64% latency reduction for core member operations. Established the pattern for subsequent service extractions across BigBasket.

Architecture

System design highlights

5 DB Sources
Stitch Federation
Report Engine
20K+ Daily Reports

Key decisions

  • Query federation over ETL-to-single-warehouse — faster to build, fresher data
  • Airflow + Spark for pipeline orchestration — team expertise and ecosystem maturity
  • Service extraction starting with highest-traffic endpoints (login, signup, address)

Lessons learned

  • Cross-database joins are powerful but query optimization is manual per database engine
  • Monolith extraction works best when you start with the most painful, highest-traffic paths
  • Data pipeline reliability depends on idempotent transformations and clear failure modes