FOLIO search interface with Kafka pipeline actively scraping brands across 10 countries

FOLIO

May 2025Team project with Uzair Ahmed — roughly equal contribution

A real-time clothing search engine that scrapes product catalogues from 150+ brands across 10 countries, streams raw listings through Kafka, cleans and deduplicates them with Spark Structured Streaming, and serves results through a FastAPI backend to a React frontend. TF-IDF cosine similarity powers recommendations; a RandomForest ML discovery loop continuously finds new brands the system doesn't know about yet. Falls back to direct SQLite writes if Kafka is unavailable.

Screenshots

FOLIO results page showing 2144 listings across 43 brands with brand filter chips

Built with

  • Apache Kafka
  • Apache Spark (PySpark)
  • FastAPI
  • SQLite (FTS5)
  • React
  • Vite
  • Firebase Auth
  • Firestore
  • scikit-learn
  • Docker Compose
  • Python
All work