hop.io¶
-
class
hop.io.
Consumer
(group_id, broker_addresses, topics, **kwargs)[source]¶ An event stream opened for reading one or more topics. Instances of this class should be obtained from
Stream.open()
.-
mark_done
(metadata)[source]¶ Mark a message as fully-processed.
- Parameters
metadata – A Metadata instance containing broker-specific metadata.
-
read
(metadata=False, autocommit=True, **kwargs)[source]¶ Read messages from a stream.
- Parameters
metadata – Whether to receive message metadata alongside messages.
autocommit – Whether messages are automatically marked as handled via mark_done when the next message is yielded. Defaults to True.
batch_size – The number of messages to request from Kafka at a time. Lower numbers can give lower latency, while higher numbers will be more efficient, but may add latency.
batch_timeout – The period of time to wait to get a full batch of messages from Kafka. Similar to batch_size, lower numbers can reduce latency while higher numbers can be more efficient at the cost of greater latency. If specified, this argument should be a datetime.timedelta object.
-
-
class
hop.io.
Deserializer
(value)¶ An enumeration.
-
class
hop.io.
Metadata
(topic: str, partition: int, offset: int, timestamp: int, key: Union[str, bytes], _raw: cimpl.Message)[source]¶ Broker-specific metadata that accompanies a consumed message.
-
class
hop.io.
Producer
(broker_addresses, topic, **kwargs)[source]¶ An event stream opened for writing to a topic. Instances of this class should be obtained from
Stream.open()
.
-
class
hop.io.
Stream
(auth=True, start_at=<ConsumerStartPosition.LATEST: 2>, persist=False)[source]¶ Defines an event stream.
Sets up defaults used within the client so that when a stream connection is opened, it will use defaults specified here.
- Parameters
auth – A bool or
Auth
instance. Defaults to loading fromauth.load_auth
if set to True. To disable authentication, set to False.start_at – The message offset to start at in read mode. Defaults to LATEST.
persist – Whether to listen to new messages forever or stop when EOS is received in read mode. Defaults to False.
-
open
(url, mode='r')[source]¶ Opens a connection to an event stream.
- Parameters
url – Sets the broker URL to connect to.
mode – Read (‘r’) or write (‘w’) from the stream.
- Returns
An open connection to the client, either a
Producer
instance in write mode or aConsumer
instance in read mode.- Raises
ValueError – If the mode is not set to read/write or if more than one topic is specified in write mode.