Hard problems at scale, the future of application development, and building an open source business. If any of that is of interest, or if you want to know about Kafka, real-time data, and streaming APIs in the cloud and beyond, Jay Kreps has some thoughts to share.

If you are into real-time data and streaming applications, chances are Apache Kafka is a key component of your architecture. Recently, Confluent, the company built by the creators of Kafka, announced support for Kafka as a managed service in the cloud.

That was a great opportunity to have a long talk with Jay Kreps, Confluent’s CEO and co-founder, about everything from the future of application development to the subtle differences in streaming APIs and paradigms.

Whether you’re a streaming enthusiast or wondering what all the fuss is about, you will most likely find something of interest here.

Let’s take it from the start. Kreps, together with co-founders Neha Narkhede and Jun Rao, started working on Kafka in 2008, while they were all LinkedIn employees. The problem they were trying to solve was dealing with continuous streams of data.

Read the full article on ZDNet