Quickstart

Reading messages

The hop client supports a python-based API for reading messages from a stream, as follows:

from hop import stream

with stream.open("kafka://hostname:port/topic", "r", format="json") as s:
    for idx, msg in s:
         print(msg)

This block will hang forever, listening to new messages and processing them as they arrive. By default, this will only process new messages since the connection was opened. The start_at option lets you control where in the stream you can start listening from. For example, if you’d like to listen to all messages stored in a topic, you can do:

with stream.open("kafka://hostname:port/topic", "r", format="json", start_at="latest") as s:
    for idx, msg in s:
         print(msg)

Writing messages

We can also publish messages to a topic, as follows:

from hop import stream

with stream.open("kafka://hostname:port/topic", "w", format="json") as s:
    s.write({"my": "message"})

Using the CLI

Publish a GCN

hop publish kafka://hostname:port/gcn mygcn.gcn3

An example RFC 822 formatted GCN circular (example.gcn3) is provided in tests/data.

Client configuration properties can be passed to hop publish via -X property=value or in a configuration file specified by -F <config-file>, mimicking the behavior of kafkacat. This can be used to connect to a Kafka broker with SSL authentication enabled, for example.

Consume a GCN

hop subscribe kafka://hostname:port/gcn mygcn.gcn3 -e

Configuration properties can be passed in a manner identical to hop publish above.