Microservices architectures are inherently distributed. Building Microservices always bring in the most challenging problems, such as resilient service invocation, distributed transactions, on-demand scaling, and exactly-once processing of messages.
Putting Microservices on Kubernetes doesn’t always solve these problems as Kubernetes doesn’t have an application logic. Frameworks like Spring Boot, Akka, and Quarkus offered great help when building distributed Microservices architectures. But they are tightly coupled to specific programming languages and often opinionated.
So developers remain searching for a portable runtime to build distributed, scalable, and event-driven Microservices.
Dapr is a portable, event-driven runtime that makes it easy for any developer to…
This post serves as an introduction to the Change Data Capture (CDC) practice, rather than a deep-dive on a particular tool. First, I will explore the motivation behind CDC and illustrate the components of a real-time event-driven CDC system. The latter parts discuss some potential use cases where CDC is applicable and conclude with some open-source tools available in the market.
P.S I made a YouTube videos for this post to explain this visually as well. If you are interested, you can check it below.
Applications start with a small data footprint. Initially, a single database fulfils every data…
In the previous post, I talked about an Event Streaming Platform (ESP) concept, why you should have an ESP in your organization, and the high-level architecture of a typical ESP.
In this post, I will explain the storage layer of an ESP in detail. We’ll discuss the architecture and fundamental concepts of it while taking examples from popular ESPs out there.
The storage layer is responsible for storing the ingested events in a highly scalable and durable manner. Simply put, it ensures that not a single event ingested is lost due to the failures of the entire ESP.
Debezium 1.5.0 is out with transactional metadata for the MySQL connector, LogMiner-based CDC implementation for Oracle promoted from incubating to stable, support for RedisStreams in Debezium Server, and much more.
Apache Pulsar is a cloud-native, distributed messaging and streaming platform created at Yahoo! and now a top-level Apache Software Foundation project.
That provides opinionated guidance on key topics such as Event-driven architectures, application design, debugging, and optimization.
The Design of an Event Store by Emil Koutanov
Why Threat Modelling Should Be Part of Your Serverless Development by Sheen Brisals.
Businesses today are dealing with a massive influx of real-time data. Financial transactions, application logs, website clickstreams, and IoT telemetry data are few examples.
As a business, you can’t simply ignore the importance of real-time data. Even if you do, your competitors won’t. Modern businesses use them to get meaningful insights in a timely fashion to stay ahead of the competition.
Crude oil is the best analogy we can apply to real-time data as well. They have to be extracted, transported, and refined to get a meaningful value. …
Thanks for the question. I too had that confusion when I began to go through the specification.
However, this is how I understood that.
We create an API definition to tell how it will behave to the consumers of that API. That includes specifying message formats and operation names. In my opinion, this is like ordering our consumers what to do next.
If you are creating an API that publishes messages to the consumer, then the consumer should subscribe to it. That's the order I'm giving to the consumer. Conversely, if your API consumers messages from the outside, then…
Welcome to the very first issue of our weekly review. Starting from this week, we will be bringing you a curated set of content and news related to designing, building, and managing applications in the event-driven application space.
Let’s start by exploring Event Meshes.
This article gives an overview of Event Mesh, the business and technical value, and its core features.
In this article, Kasun Indrasiri explores the potential of using Service Meshes for Event-Driven Messaging and the key capabilities such a Service Mesh should have.
Event-driven applications are getting increasingly popular these days compared to their request-reply counterpart. When you want to build asynchronous, loosely coupled, highly scalable, and resilient applications, the Event-driven Architecture (EDA) is the best option to consider.
But designing, building, and managing event-driven applications are somewhat difficult as there are many principles, frameworks, and technologies to choose from. Today, we can see tons of learning materials around the Internet related to various event-driven technologies and how to get things done with them.
Whether you are a novice or an expert in the EDA space, it is always better to have a…
P.S I did some edits to reflect the feedback I received from the audience. Thanks for your valuable contribution. I expect more! :)
Everything in Kafka revolves around partitions. They play a crucial role in structuring Kafka’s storage and the production and consumption of messages.
Understanding partitions helps you learn Kafka faster. This article walks through the concepts, structure, and behavior of Kafka’s partitions.
Before diving into partitions, we need to set the stage here. So let’s look at some high-level concepts and how they relate to partitions.
An event represents a fact that happened in the past. Events are…
Event-driven applications are loosely-coupled and distributed in nature, built with multiple self-contained components.
These applications are often blamed for the lack of standardization in the components. For example, two event-driven systems will name their operations and event schemas differently, forcing developers to redo the same things repeatedly.
Three promising specifications are emerging in the event-driven application landscape to provide a common way to do things. They are aiming at making event-driven applications portable and interoperable across multiple cloud environments.
This article explores each of them on a high level.