abstract class KafkaProducer[F[_], K, V] extends AnyRef
KafkaProducer represents a producer of Kafka records, with the ability to produce
ProducerRecords using produce.
- Self Type
- KafkaProducer[F, K, V]
- Source
- KafkaProducer.scala
- Alphabetic
- By Inheritance
- KafkaProducer
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new KafkaProducer()
Abstract Value Members
- abstract def initTransactions: F[Unit]
Enables transactions for the producer.
Enables transactions for the producer.
If using KafkaProducer.transactional or KafkaProducer.transactionalStream, then this will automatically be done when the producer is created. If not, then this function has to be called manually before using any of the transaction methods:
- KafkaProducer#transaction,
- KafkaProducer#produceTransactionally,
- KafkaProducer#produceAndCommitTransactionally
or an
IllegalStateExceptionexception will be raised. - abstract def metrics: F[Map[MetricName, Metric]]
Returns producer metrics.
Returns producer metrics.
- See also
org.apache.kafka.clients.producer.KafkaProducer#metrics
- abstract def partitionsFor(topic: String): F[List[PartitionInfo]]
Returns partition metadata for the given topic.
Returns partition metadata for the given topic.
- See also
org.apache.kafka.clients.producer.KafkaProducer#partitionsFor
- abstract def produce(records: ProducerRecords[K, V]): F[F[ProducerResult[K, V]]]
Produces the specified ProducerRecords in two steps: the first effect puts the records in the buffer of the producer, and the second effect waits for the records to send.
Produces the specified ProducerRecords in two steps: the first effect puts the records in the buffer of the producer, and the second effect waits for the records to send.
It's possible to
flattenthe result from this function to have an effect which both sends the records and waits for them to finish sending.Waiting for individual records to send can substantially limit performance. In some cases, this is necessary, and so we might want to consider the following alternatives.
- Wait for the produced records in batches, improving the rate at which records are
produced, but loosing the guarantee where
produce >> otherActionmeansotherActionexecutes after the record has been sent. - Run several
produce.flatten >> otherActionconcurrently, improving the rate at which records are produced, and still haveotherActionexecute after records have been sent, but losing the order of produced records.
- Wait for the produced records in batches, improving the rate at which records are
produced, but loosing the guarantee where
- abstract def produceAndCommitTransactionally(records: TransactionalProducerRecords[F, K, V]): F[ProducerResult[K, V]]
Produces the
ProducerRecords in the specified TransactionalProducerRecords in four steps: first a transaction is initialized, then the records are placed in the buffer of the producer, then the offsets of the records are sent to the transaction, and lastly the transaction is committed.Produces the
ProducerRecords in the specified TransactionalProducerRecords in four steps: first a transaction is initialized, then the records are placed in the buffer of the producer, then the offsets of the records are sent to the transaction, and lastly the transaction is committed. If errors or cancellation occurs, the transaction is aborted. The returned effect succeeds if the whole transaction completes successfully. - abstract def produceTransactionally(records: ProducerRecords[K, V]): F[ProducerResult[K, V]]
Produces the
ProducerRecords in the specified ProducerRecords in three steps: first a transaction is initialized, then the records are placed in the buffer of the producer, and lastly the transaction is committed.Produces the
ProducerRecords in the specified ProducerRecords in three steps: first a transaction is initialized, then the records are placed in the buffer of the producer, and lastly the transaction is committed. If errors or cancellation occurs, the transaction is aborted. The returned effect succeeds if the whole transaction completes successfully. - abstract def sendOffsetsToTransaction(offsets: Map[TopicPartition, OffsetAndMetadata], groupMetadata: ConsumerGroupMetadata): F[Unit]
Sends the specified offsets and KafkaConsumer.groupMetadata to be committed as part of a transaction.
- abstract def transaction: Resource[F, Unit]
Return a resource which handles the transaction lifecycle.
Return a resource which handles the transaction lifecycle.
The returned resource begins a transaction on
useand:- commits the transaction if the
usefinishes successfully, or - aborts the transaction if an error occurs or the process is canceled.
Note initTransactions must have been called before using this method, either manually or automatically through the use of KafkaProducer.transactional or KafkaProducer.transactionalStream.
Also note only one transaction can be open at any time. If a second transaction is started within the lifecycle of a first transaction, the second transaction will deadlock.
producer.transaction.surround { producer.transaction.surround { // deadlocks waiting for the outer transaction IO.unit } } - commits the transaction if the
- abstract def withSerializers[K2, V2](keySerializer: KeySerializer[F, K2], valueSerializer: ValueSerializer[F, V2]): KafkaProducer[F, K2, V2]
Returns a new KafkaProducer using the same underlying producer but with different key and value serializers.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def imapK[G[_]](fk: ~>[F, G], gk: ~>[G, F])(implicit arg0: MonadCancelThrow[G], F: MonadCancelThrow[F]): KafkaProducer[G, K, V]
Creates a new KafkaProducer in which the effect type has been changed using the specified
FunctionKs. - final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()