Skip to content

The LBSN structure is in an early stage of development.

We invite anyone to contribute.

Generally, we aim for the following conventions and principles:

Single Point of Truth

Definitions should exist in one place only. We use continuous integration workflows to systematically integrate modules and deploy individual pieces. For example, the code examples and definitions in this documentation are automatically generated from tested code.

Privacy by Design, Privacy by Default, and Separation of Concerns principle

All of the above principles help reducing the risk that privacy of any user is compromised. For example, those who develop tools do not need to access original data generated by users (separation of concerns principle).

Open Source

Transparency is a key when working with user-generated content, and open source code means transparency. The documentation and all pieces of software used in this project are published as open source. This guarantees a sustainable, future proof development cycle.

Modular design

The core lbsn structure is described in a platform independent Protocol Buffers file. The Proto file can be used to compile and implement the proposed structure in any language such as Python, Java or C++.

This structure is tightly coupled with a relational datascheme (Postgres SQL) that is maintained separately, including a privacy-aware version that is used for demonstration purposes. The database is ready to use with several provided Docker containers that optionally include a PGadmin interface and a Jupyter Lab visualization backend.

There exist several tools that help migrating data and maintaining the structure. The python package lbsntransform allows bulk conversion and database update of large volumes of proprietary data (e.g. CSV, JSON) that was retrieved from various API endpoints (e.g. Twitter, Flickr).

A fullstack docker project generator is available that includes Postgres, pgadmin, a web api, a frontend and other features (such as Alembic database migrations).

This documentary illustrates the various use cases, definitions and applications. Contributions welcome.

Overview of Repositories and Tools

The following list provides an overview of tools and applications that make use of or are connected to the lbsn-structure.

Repository/Link      Version               Description
LBSN Base Structure:
RAW SQL version Relational PostgresSQL implementation of LBSN Structure
HLL SQL version A Privacy-aware version of RAW SQL, e.g. for visual analytics
Protocol Buffers version Defenition of LBSN Structure in Protocol Buffers, e.g. for use in RPC
Python Compiled pypi version A Python compiled version of Protobuf LBSN Structure
PHP Compiled packagist A PHP compiled version of Protobuf LBSN Structure
LBSN Docker Container:
RAW DB rawdb A ready to use Docker Container using the LBSN RAW SQL structure
HLL DB hlldb A ready to use Docker Container using the (privacy-aware) LBSN HLL SQL structure
pg-hll-empty version An empty Postgres Docker container with HLL extension installed, e.g. used in lbsntransform to anonymize data
jupyter lab A containerized version of Jupyter Lab, used for interactive visualization purposes, e.g. in the tutorial section
LBSN Tools:
lbsntransform PyPI version A python package for bulk data migration to and from lbsn structure (both Postgres and ProtoBuf specification)
fullstack-lbsn An easy to use shell-script to setup RAW DB and HLL DB Docker Container, including data export and import
LBSN API (theplink-docker):
Alembic Migrations Automatic migrations of live PostgresSQL with updates to the RAW LBSN Structure
Backend version Backend using Fastapi and LBSN Structure implemented as a SQLAlchemy model
Frontend version Frontend as a progressive web app based on VueJS and Mapbox GL JS, a prototype for local community discourse using the LBSN Structure

Last update: July 24, 2020