Vehicle E-Commerce Analytics Platform
Production-grade analytics platform for an online vehicle marketplace with Snowflake, dbt, Dagster, and enterprise-grade data quality controls.
Timeline
Q1 2026
Role
Data Engineer
Team
Solo
Status
CompletedTechnology Stack
Key Challenges
- Designing a modular analytics model that scales from operational reporting to advanced KPI analysis.
- Ensuring data trust with layered quality checks across transformation and orchestration.
- Aligning orchestration, testing, and CI validation to support production-grade delivery.
Key Learnings
- Asset-driven orchestration with Dagster improves maintainability and operational visibility.
- Layered data modeling (staging -> intermediate -> marts) keeps business logic clear and reusable.
- Combining dbt tests, Elementary, and Great Expectations creates stronger confidence in decision-grade metrics.
Executive Summary
This project implements an end-to-end analytics platform for a vehicle e-commerce use case. The objective is to deliver trusted business metrics (revenue, retention, funnel performance, and product behavior) using a modern, testable, and observable data stack.
The solution follows Data Engineering best practices: modular ELT pipelines, dimensional modeling, event-driven orchestration, and CI-ready validation workflows.
Business Outcomes
- Reliable KPI coverage for revenue, orders, customer acquisition, retention, and churn.
- Clear ownership and traceability of transformation logic through dbt models and macros.
- Faster issue detection with automated observability and quality checks.
- Better readiness for analytics consumption in Power BI and Looker.
Architecture Overview
The platform is structured as a layered architecture:
- Ingestion with Airbyte from source APIs into Snowflake.
- Transformation with dbt Core across staging, intermediate, and marts layers.
- Orchestration with Dagster assets, sensors, and jobs.
- Data quality and observability using dbt tests, Elementary, and Great Expectations.
- Consumption via BI dashboards and analytics views.
Data Model Strategy
The dimensional model is designed to support both operational analytics and strategic reporting. It separates core entities from analytical marts and enables predictable query performance.
KPI and Analytics Scope
The analytics layer supports:
- Customer Lifetime Value (CLV)
- Cohort and retention analysis
- RFM segmentation
- Funnel analysis and cart abandonment
- Product affinity and cross-sell
- Daily revenue, orders, and acquisition metrics
Engineering Workflow
Development
- Build small, composable dbt models.
- Centralize reusable SQL logic with macros.
- Version historical changes with snapshots.
Quality and Validation
- Use dbt schema and data tests as baseline controls.
- Add Elementary monitoring for freshness, volume, and anomaly signals.
- Apply Great Expectations suites for high-impact datasets.
Delivery and CI/CD
- Execute validation pipelines with GitHub Actions.
- Run parse, test, and quality checks before merge.
- Keep deployment gates explicit and auditable.
Dashboard Example
What Makes This Project Production-Grade
- Strong separation of concerns across ingestion, modeling, orchestration, and consumption.
- Reproducible and test-first transformation workflows.
- Observable pipelines with actionable quality signals.
- Documentation-oriented structure for maintainability and handover.
Repository
- GitHub: marcellin-de/vehicle-ecommerce-analytics
- Recommended starting points:
dbt_project/models/marts/analytics/dbt_project/models/marts/core/orchestration/assets/docs/kpis_definitions.md