Kafka is super fast. One design tradeoff to achieve very high through put is to put more responsibilities on client side. The underlying concept of messages fetching in Kafka client API is simple enough. But there are many behavior of the API that could be well understood only by spending some time studying how it… Continue reading
Posts Tagged → kafka
Kafka offset commit with lots of partitions
My team just had a problem with our shared Kafka cluster. Logstash processes in centralized-logging system were getting unstable and crashing. These Logstash sends logs to Kafka as a buffer before consuming the messages out and insert to ES. My colleague were suspecting that one of our new deployed application could be the cause of… Continue reading