What is Poschti?

Poschti is a shopping list for Android(Poschtizettel) with an optional server and web interface(Poschtiserver).
The server allows for distributed use of the shopping list.

It is also a learning project I expanded after a while to include more utility for myself and more learning.

This explanation is an overview, anyone is of course welcome to dive into the code for details.

• https://github.com/AndreasMeier12/Poschtizettel
• https://github.com/AndreasMeier12/Poschtiserver

Why?

I wasn’t happy with the shopping list I had on my phone and decided on a whim I could do something that I liked more.
When the big rona hit in 2020, I tackled the project. I used a lot less time to commute and building something took my mind off the pandemic for a few moments.

I formulated criteria I had for my app:

• Typo tolerance. Typing on a touch screen is hard.
• Organized by shop instead of by ingredient as I have seen with other shopping list apps
• No ads, no tracking beyond Android spying on me
• No online requirement

I decided to do the app with Kotlin because I wanted to give Kotlin a whirl as a learning exercise.

In short, this project was in the sweet spot of things that are achievable in a reasonable timeframe, useful to me and allow for learning new technologies.

So far it was a fairly typical “baby’s first CRUD” on Android.
The more I used my app, the more I realized I still was not happy with my workflow.
I often use my desktop to browse recipes and I had to type the list on my phone.
I found switching between my desktop and phone distracting. I prefer to type on my computer but I don’t carry it with me to the grocery store.
Therefore I wrote a server that allows me to sync states between the android app and a web interface.
I then had to worry about distributed state.

What are the architecture and design decisions?

There’s an android app (Poschtizettel), a server (Poschtiserver) and a web frontend.
The server state is updated using commands.
Commands are used to abstract the state changes and allow merging of updates between devices.

Server

The server consists of an SQLite database and a flask app. The flask app provides endpoints for android and browser use, as well as HTML for viewing.
The simplicity of the app .
Because there are not going to be multiple servers for my use, SQLite is more than enough.

The server handles merging.
The commands are sorted by their timestamp.
Then each command is applied.

• Create creates a new item
• Delete removes the item with the provided key
• Update replaces the item with the provided key
  The results are kept in a dict for easy access using their IDs as keys. Here is the code for item commands, lists work analogously but are simpler (there is no update command).

def merge(client, server=None):
    if server is None:
        server = []

    commands = sorted(client + server)
    res = OrderedDict()
    for command in commands:
        handle(command, res)
    return [x for x in res.values()]

def handle(command: Command, res):
    if command.type is CommandType.UPDATE:
        handle_update(command, res)
    if command.type is CommandType.CREATE:
        handle_create(command, res)
    if command.type is CommandType.DELETE:
        handle_delete(command, res)


def handle_create( a: Command, res: dict):

    b = copy.copy(a.get_item())
    res[b.id] = b


def handle_update(a: Command, res: dict):
    b: ShoppingItem = copy.copy(a.get_item())
    res[b.id] = b


def handle_delete(a: Command, res):
    if a.item.id in res:
        del res[a.get_id()]

The server also handles all authentication via password/ username or tokens.

Android

What’s in a command?

Each command has an ID to keep it unique.
Each list and shopping item has a long-lived ID.
I used randomized UUIDs for these purposes, the collision risks seemed small enough.
UUIDs allow easy creation of commands with a permanent ID.

A command contains a timestamp when it was issued to allow ordering.

A command contains state information about the shopping item in question

• name
• quantity
• shop
• is it done?

Server

The server uses the Flask framework for Python. I had used it before but I had not yet set up a project by myself.
The endpoints and routes used by the app and the browser-based versions are different because authentication is handled differently.

The business logic and the persistence layer use different data representations. I wrote the business logic first and the schema later.
That way, the business logic types don’t have to correspond to one particular way of representing them in the database.

Client

It’s an android app written in Kotlin. It consists of a number of fragments:

• overview of lists w/ delete
• adding lists
• view of a single list w/ items, marking done and deleting items
• adding items
• settings for syncing with token and the server URL

It allows for creation and deletion of lists, as well as creation, updating and deleting of items in lists.
It has features for syncing with the server.

Syncing

Merging between devices is done by sending the commands to the server.

The server orders the commands by time, then generates the state shown to the user.

What’s the Schema like ?

On the Server

A straightforward representation of the commands. Item Commands reference lists and item ids.
List commands reference list ids.

On the Android Client

The Android client has two sets of tables. One for lists and items as they are displayed on the phone.
The other contains commands that will be sent to the server on syncing.

Syncing:
The item tables are empty and refilled with the response from the server.
The command tables are truncated on successful syncing to save space and prevent duplication of commands on the server (although these would be easy to filter out).

Example: buying carrots

Starting with an empty list.
Items is empty.
Command is empty.

Step 1: creating the item
Items contains carrots.

Step 2: marking carrots done, i.e. you bought juicy carrots:
Items contains carrots, these are done
Command contains create carrots, update carrots

Step 3 Delete Carrots:
Items is empty
Commands contains create carrots, update carrots, delete carrots

Authentication

Creating an account from the web frontend is necessary before using the online features in the Android App.

Web-based

It’s account and password based. A user registers with an e-mail address and a password.
The passwords are hashed using Argon2ID.

Android

After creating an account, the user may generate tokens to authenticate using the android app.
The token is sent with each request.
The user may change the duration of validity for future tokes.
The tokens are sent with requests from the client.

The tokens used are JWT, these are encrypted.

Lessons Learned, Conclusion

Do I use it?

Heck yes!

What ideas were left by the wayside??

• Typescript, I hardly have a model in the browser. It was not worth the effort and would probably not have learned much.
• Editing the name, shops and quantities of items. This might be addressed in the future
• Using Conflict-free replicated data types. By using a central server for communication, I decided to make things easier for myself. I believe in trying simple solutions first.
• Any sort of in-depth representations of quantities. There are weights, volumes, packages, pieces. That’s probably important when developing an ERP but for my shopping list, plain text is enough.

What could break distributed use?

“Set to server”, “set to client” requests truncate the old logs on the server. I used them mainly for debugging.

Would distribution across more devices work?

I think so, yes. Although I have not yet tried it.

What did I learn? What are my impressions?

• Building an Android app
• Some Kotlin
  • having nullability as part of a type requiring explicit handling is good
  • interoperability with Java works and is nice, Java libraries are usable
  • the syntax is a whole lot cleaner than Java
• Setting up a flask app from scratch (having worked an existing flask project before)
• Containerizing my server
• Building a schema and using it with SQLAlchemy
• What’s in a JWT
• Abstracting updates in a way that allows merging from different sources
• One simple way of handling distributed CRUD
• The tutorial situation for Flask and Android apps
  • Flask Megatutorial was really helpful is probably the most time-efficient way to get started
  • I would prefer more documentation for Android. Most of it is in individual tutorials with a fairly narrow focus. A lot of the configuration is already set up, this could be part of the tutorial starting from one of the templates in Android Studio.

What would I change if I started over?

I might use a cross-platform framework for building the app, e.g. Xamarin. Then again, I don’t see myself switching phone operating systems.
More importantly, I would keep the server and the Android app in a single repository. That way, each commit represents a usable combination of both programs. Additionally, I would include a protocol version number in requests, allowing for easier interaction and backwards compatiblity.
An example would be moving from Update simply representing checking done to include editing. Old protocol versions would still be seen as only updating ‘done’.

Was it worth it?

Yes it was.