Single Point of Truth

Definitions should exist in one place only. We use continuous integration work flows to systematically integrate modules and deploy individual pieces. For example, the code examples and definitions in this documentation are automatically generated from tested code.

Privacy by Design, Privacy by Default, and Separation of Concerns principle

All of the above principles help reducing the risk that privacy of any user is compromised. For example, those who develop tools do not need to access original data generated by users (separation of concerns principle).

Open Source

Transparency is a key when working with user-generated content, and open source code means transparency. The documentation and all pieces of software used in this project are published as open source. This guarantees a sustainable, future proof development cycle.

Modular design

The core lbsn structure is described in a platform independent Protocol Buffers file. The Proto file can be used to compile and implement the proposed structure in any language such as Python, Java or C++.

This structure is tightly coupled with a relational data scheme (Postgres SQL) that is maintained separately, including a privacy-aware version that is used for demonstration purposes. The database is ready to use with several provided Docker containers that optionally include a PGadmin interface and a Jupyter Lab visualization backend.

There exist several tools that help migrating data and maintaining the structure. The python package lbsntransform allows bulk conversion and database update of large volumes of proprietary data (e.g. CSV, JSON) that was retrieved from various API endpoints (e.g. Twitter, Flickr).

A fullstack docker project generator is available that includes Postgres, pgadmin, a web api, a frontend and other features (such as Alembic database migrations).

This documentary illustrates the various use cases, definitions and applications. Contributions welcome.