hop.io

class hop.io.Consumer(group_id, broker_addresses, topics, **kwargs)

An event stream opened for reading one or more topics. Instances of this class should be obtained from Stream.open().

close()

End all subscriptions and shut down.

mark_done(metadata)

Mark a message as fully-processed.

Parameters

metadata – A Metadata instance containing broker-specific metadata.

read(metadata=False, autocommit=True, **kwargs)

Read messages from a stream.

Parameters

• metadata – Whether to receive message metadata alongside messages.

• autocommit – Whether messages are automatically marked as handled via mark_done when the next message is yielded. Defaults to True.

• batch_size – The number of messages to request from Kafka at a time. Lower numbers can give lower latency, while higher numbers will be more efficient, but may add latency.

• batch_timeout – The period of time to wait to get a full batch of messages from Kafka. Similar to batch_size, lower numbers can reduce latency while higher numbers can be more efficient at the cost of greater latency. If specified, this argument should be a datetime.timedelta object.

class hop.io.Deserializer(value)

An enumeration.

class hop.io.Metadata(topic: str, partition: int, offset: int, timestamp: int, key: Union[str, bytes], _raw: cimpl.Message)

Broker-specific metadata that accompanies a consumed message.

class hop.io.Producer(broker_addresses, topic, **kwargs)

An event stream opened for writing to a topic. Instances of this class should be obtained from Stream.open().

close()

Wait for enqueued messages to be written and shut down.

write(message)

Write messages to a stream.

Parameters

message – The message to write.

class hop.io.Stream(auth=True, start_at=<ConsumerStartPosition.LATEST: 2>, persist=False)

Defines an event stream.

Sets up defaults used within the client so that when a stream connection is opened, it will use defaults specified here.

Parameters

• auth – A bool or Auth instance. Defaults to loading from auth.load_auth if set to True. To disable authentication, set to False.

• start_at – The message offset to start at in read mode. Defaults to LATEST.

• persist – Whether to listen to new messages forever or stop when EOS is received in read mode. Defaults to False.

open(url, mode='r', group_id=None)

Opens a connection to an event stream.

Parameters

• url – Sets the broker URL to connect to.

• mode – Read (‘r’) or write (‘w’) from the stream.

• group_id – The consumer group ID from which to read. Generated automatically if not specified.

Returns

An open connection to the client, either a Producer instance in write mode or a Consumer instance in read mode.

Raises

ValueError – If the mode is not set to read/write, if more than one topic is specified in write mode, or if more than one broker is specified