Importing lbsntransform as a package

For in-memory conversion, it is possible to import lbsntransform as a package:

import lbsntransform as lt

As a starting point, have a look at lbsntransform/, which includes the code that is invoked on command line use.

We plan to update this section with a Jupyter Lab example notebook.

Example tool "twitterparser"

An example program twitterparser that uses lbsntransform as a package is available here.

The tool demonstrates an (imaginary) custom processing pipeline for Twitter data, stored in ZIP files on a remote server. The twitterparser tool will (1) connect to this remote server via SSH, (2) pull zip files and extract the json files locally and (3) transfer json files to a remote lbsn-raw or lbsn-hll database using lbsntransform.

Use this tool as a template or starting point and modify __main__ to your needs.


lbsntransform is in an early stage of development.

You can contribute:

Structure of the project

The project structure follows the src-layout.

The packaging is organized as described in the setuptools declarative config (pyproject.toml)

Releases are made with python-semantic-release. At the moment, releases are triggered manually after certain progress is available. Preview release flow with:

semantic-release publish --verbosity=DEBUG --noop

Without --noop, semantic-release will do the following:

  1. Update changelog file.
  2. Run semantic-release version.
  3. Push changes to git.
  4. Run build_command (python -m build) and upload the distribution files to Pypi.
  5. Run semantic-release changelog and post to Gitlab/Github.
  6. Attach the files created by build_command to the release.

To trigger a test build:

pip install -q build
python -m build