Kafka makes you faster!

Apache Kafka has had a meteoric rise since its first open-source release in 2011. Today over 80% of Fortune 100 companies use it, but why is this?

 

What is Kafka?

Kafka is stream-processing software that scales from kilobytes to terabytes and, with SaaS deployments, is accessible to any organisation. Kafka solves data distribution, lineage, translation and streaming challenges in a single application.  In software development, Kafka is used for fast communication between different infrastructures and applications through a common platform.  

An Example

The image shows a system for selling products online. Customers can check if products are in stock, order products, make payments, and check their order status using the Customer App. The web services can be implemented in any technology: this might be a mix of microservices and third-party applications or a single monolithic web application.

On the business side, existing corporate systems can manage the different stages. For example, orders might be processed by a legacy inventory system, while an online portal provided by a courier handles shipping. The Enterprise Reporting System (ERP) is updated in real-time, while a third-party email and text service notifies customers. Finally, the Customer Relationship Management platform is updated with customer details.

Why Does Kafka Make The Example Faster?

For application development, the systems are decoupled. This allows departments to use the tooling that fits them; not everyone has to change to implement the online store. Secondly, fault tracing between the systems is straightforward – did the message appear in Kafka or not? Thirdly, all the messaging is happening in real-time; no need to download reports to upload to your ERP each month. Lastly, should a system be upgraded in the future, the interfaces are defined and only involve Kafka – something that will likely save a lot of time!