Open in app

Sign In

Write

Sign In

Dunith Danushka
Dunith Danushka

2.1K Followers

Home

About

Published in

Tributary Data

·Pinned

The State of Data Infrastructure Landscape in 2022 and Beyond

Key trends to expect in the data infrastructure domain in 2022 and beyond. — I happened to come across two tweets in data Twitter recently that spurred my curiosity to learn what will happen in the data infrastructure landscape in 2022 and beyond. Chris Riccomini and Gunnar Morling are two veterans in the data infrastructure space that I regularly follow to understand what’s going…

Data Engineering

8 min read

The State of Data Infrastructure Landscape in 2022 and Beyond
The State of Data Infrastructure Landscape in 2022 and Beyond
Data Engineering

8 min read


Published in

Tributary Data

·Pinned

Event-driven APIs — Understanding the Principles

What are event-driven APIs? How do they differ from REST APIs? How to use the Webhooks, WebSockets, and Server-Sent Events to build them? — In this article, you’ll learn the foundations of event-driven APIs, how they interact with consumers, the technology choices to build them, and how to document them with AsyncAPI specification. Polling is dead. We must move on We, as information consumers, have a craving desire to know things as they happen.

Event Driven Architecture

10 min read

Event-driven APIs — Understanding the Principles
Event-driven APIs — Understanding the Principles
Event Driven Architecture

10 min read


Published in

Tributary Data

·Pinned

5 Reasons Why You Should Use Microsoft Dapr to Build Event-driven Microservices

Why Dapr excels at building distributed, loosely-coupled, event-driven Microservices — Microservices architectures are inherently distributed. Building Microservices always bring in the most challenging problems, such as resilient service invocation, distributed transactions, on-demand scaling, and exactly-once processing of messages. Putting Microservices on Kubernetes doesn’t always solve these problems as Kubernetes doesn’t have an application logic. Frameworks like Spring Boot, Akka, and…

Dapr

6 min read

5 Reasons Why You Should Use Microsoft Dapr to Build Event-driven Microservices
5 Reasons Why You Should Use Microsoft Dapr to Build Event-driven Microservices
Dapr

6 min read


Published in

Tributary Data

·Pinned

A Gentle Introduction to Event-driven Change Data Capture

How to detect, capture, and propagate changes in source databases to target systems in real-time, event-driven manner — This post serves as an introduction to the Change Data Capture (CDC) practice, rather than a deep-dive on a particular tool. First, I will explore the motivation behind CDC and illustrate the components of a real-time event-driven CDC system. …

Change Data Capture

8 min read

A Gentle Introduction to Event-driven Change Data Capture
A Gentle Introduction to Event-driven Change Data Capture
Change Data Capture

8 min read


Published in

Tributary Data

·Aug 28

The Significance of In-Broker Data Transformations in Streaming Data

How WebAssembly powered data transformations are changing the data scrubbing story for streaming data platforms? — Data scrubbing or massaging is a critical aspect of data engineering, especially in streaming data pipelines where the data must be cleaned, filtered, and scrubbed in real time with the help of stream processing engines or some Python jobs. However, that is about to change. Streaming data platforms have started…

Streaming

6 min read

The Significance of In-Broker Data Transformations in Streaming Data
The Significance of In-Broker Data Transformations in Streaming Data
Streaming

6 min read


Published in

Tributary Data

·Jan 3

Operational Use case Patterns for Apache Kafka and Flink — Part 1

This is the first post of the series that shows building operational use cases with Apache Kafka and Apache Flink. — Apache Kafka is a distributed streaming data platform. It is designed for high throughput, low-latency streaming workloads, where scalable real-time data ingestion and fault-tolerant storage are critical. Apache Flink is an open-source, unified stream-processing and batch-processing framework capable of executing arbitrary dataflow programs on data streams.

Kafka

6 min read

Operational Use case Patterns for Apache Kafka and Flink — Part 1
Operational Use case Patterns for Apache Kafka and Flink — Part 1
Kafka

6 min read


Published in

Tributary Data

·Sep 29, 2022

Comparing Stateful Stream Processing and Streaming Databases

How do these two technologies work? how do they differ, and when is the right time to use them? — Choosing between a stateful stream processor and a streaming database has been a debatable question for a long time. I’ve been combing through the Internet to find a few rationales, failed, and decided to write this post to share my experience and knowledge with you. After reading this, you should…

Stream Processing

7 min read

Comparing Stateful Stream Processing and Streaming Databases
Comparing Stateful Stream Processing and Streaming Databases
Stream Processing

7 min read


Published in

Tributary Data

·Aug 11, 2022

Building CQRS Views with Debezium, Kafka, Materialize, and Apache Pinot — Part 1

How to build an incrementally updated materialized view that serves queries in a faster and scalable manner? — Microservices architecture promotes a decentralized data management practice, where each service keeps its data private and exposes it only via well-defined API interfaces. Although that is for the greater good, developers find it challenging to implement queries that span across multiple service boundaries.

Stream Processing

6 min read

Building CQRS Views with Debezium, Kafka, Materialize, and Apache Pinot — Part 1
Building CQRS Views with Debezium, Kafka, Materialize, and Apache Pinot — Part 1
Stream Processing

6 min read


Published in

Tributary Data

·Aug 11, 2022

Building CQRS Views with Debezium, Kafka, Materialize, and Apache Pinot — Part 2

How to build an incrementally updated materialized view that serves queries in a faster and scalable manner? — This post is the second and final installment of the article series that explores building a read-optimized view with Debezium, Kafka, Materialize, and Apache Pinot. Part 1 discussed the problem space of building an online pizza order tracker and a possible solution architecture. Part 2 will walk you through the…

Materialize

9 min read

Building CQRS Views with Debezium, Kafka, Materialize, and Apache Pinot — Part 2
Building CQRS Views with Debezium, Kafka, Materialize, and Apache Pinot — Part 2
Materialize

9 min read


Published in

Tributary Data

·Jul 26, 2022

CDC-based Upserts with Debezium, Apache Kafka, and Apache Pinot

How to build a streaming data pipeline to capture MySQL database changes and stream them to Apache Pinot via Debezium and Kafka — Upserting means inserting a record into a database if it does not already exist or updating it if it does exist. Analytics database at the end of a streaming data pipeline can benefit from upserts to maintain the data consistency with the source database. This article explores a minimal viable…

Kafka

9 min read

CDC-based Upserts with Debezium, Apache Kafka, and Apache Pinot
CDC-based Upserts with Debezium, Apache Kafka, and Apache Pinot
Kafka

9 min read

Dunith Danushka

Dunith Danushka

2.1K Followers

Editor of Tributary Data. Technologist, Writer, Senior Developer Advocate at Redpanda. Opinions are my own.

Following
  • ODSC - Open Data Science

    ODSC - Open Data Science

  • Felipe Hoffa

    Felipe Hoffa

  • Tim Denning

    Tim Denning

  • Eva Keiffenheim

    Eva Keiffenheim

  • Madusanka Premaratne

    Madusanka Premaratne

See all (277)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech

Teams