Technologies

React+Typescript, Java, GRPC, express.js

1 minute read

SuperBasket is a local-first shopping list application designed to work both offline and online, combining local data storage with cloud-based sharing and backup. This project is a practical application of concepts used in large-scale distributed systems, and it was highly inspired by the paper “Dynamo: Amazon’s Highly Available Key-Value Store”.

Architectural Overview

This project was built using a micro-service architecture to manage different tasks. Below is an overview of the architecture:

architecturl-overview

1. Frontend

  • UI: A simple and user-friendly interface for adding, removing, and editing items in the shopping list.
  • Offline-first: Works seamlessly offline, using IndexedDB to store data on each user’s device.
  • Per-user database: Each user has a dedicated local instance of IndexedDB to save their shopping list items.

2. Synchronization Service

  • Data Sync: Handles data synchronization between your local device and the cloud database, ensuring your shopping list is always up-to-date across devices.
  • CRDTs (Conflict-free Replicated Data Types): This Data Structure helps merge changes from different devices providing internal mechanisms to deal with conflicts, even when multiple users are updating the same list at the same time.

3. Database

  • Partitioned for efficiency: Uses Consistent Hashing to divide the data across multiple servers, ensuring faster access and better availability.
  • Fault tolerance: Includes features like Sloppy Quorums and Hinted Handoff to handle temporary server failures.
  • Version control: Uses Vector Clocks to track changes in data, allowing it to handle conflicts and maintain the correct version of your shopping list.
  • Efficient communication: Uses gRPC and Protocol Buffers for fast and efficient communication between different servers.

About CRDTs

CRDTs, or Conflict-free Replicated Data Types, are a key part of making SuperBasket work smoothly in a distributed system. They allow different users to update their shopping lists from different devices, even when offline, and merge the changes while dealing with the conflicts.

During development, we created a CRDT Library using TypeScript. It was a great learning experience, and I plan to write more about this topic in the future.

Want to know more? Click here

Team:

  • André Lima (Programmer)
  • Guilherme Almeida (Programmer)
  • Miguel Montes (Programmer)