Developers

Importing lbsntransform as a package

For in-memory conversion, it is possible to import lbsntransform as a package:

import lbsntransform as lt
lt.add_processed_records(
    record)
lt.store_lbsn_records()

As a starting point, have a look at lbsntransform/main.py, which includes the code that is invoked on command line use.

We plan to update this section with a Jupyter Lab example notebook.

Example tool "twitterparser"

An example program twitterparser that uses lbsntransform as a package is available here.

The tool demonstrates an (imaginary) custom processing pipeline for Twitter data, stored in ZIP files on a remote server. The twitterparser tool will (1) connect to this remote server via SSH, (2) pull zip files and extract the json files locally and (3) transfer json files to a remote lbsn-raw or lbsn-hll database using lbsntransform.

Use this tool as a template or starting point and modify __main__ to your needs.

Contribute

lbsntransform is in an early stage of development.

You can contribute: