• Digital Democracy: OpenStreetMap Without Servers [Part 2]: A peer-to-peer #osm database

    osm-p2p is a decentralized peer-to-peer database for storing and editing OpenStreetMap nodes, ways, and relations. It includes a node.js server that implements the core functionality of the OSM API, or it can be used completely in the browser using IndexedDB for persistent storage.Someone can make edits, replicate to a USB drive, bring the USB drive to another village, replicate, and then bring the USB drive back to replicate again. The replication copies data both ways, so after this procedure both villages will have the complete dataset.


    • Architecture

      osm-p2p implements a kappa architecture where all updates are written to an append-only log. This log is the “source of truth” that populates indexes called materialized views. These materialized views are only meant to answer questions faster than reading the whole log. If the views need to change in the future to accommodate different queries, they can be regenerated from the log without having the migrate the log schema.

      Each update is written to an append-only log provided by hyperlog, a module from the dat project. This log has indexes which provide different materialized views of the data.

      One view (hyperkv) provides a key/value interface mapping OSM ids to documents. Another view creates a spatial index to answer bounding box queries (hyperlog-kdb-index).

      During replication, each side requests the documents from the log that it doesn’t have and appends those documents to its own log. The indexes watch these inserts and buffer requests until the they are caught up with the latest known document in the log.