Change data capture - Bits by J.E

Bits by J.E

Change data capture

A collection of 4 posts

Real-Time Data Streaming and Search Indexing with Azure - Part 4 - CDC event processing

Change data capture

Real-Time Data Streaming and Search Indexing with Azure - Part 4 - CDC event processing

In the last part we looked at the CDC setup and the event structure that Debezium utilizes to stream changes from our Azure SQL Server to the Azure Event Hub. This post will go through the CDC event-processing necessary to consume the events and push them into our targets queues

Real-Time Data Streaming and Search Indexing with Azure - Part 3 - Setting up CDC

Real-Time Data Streaming and Search Indexing with Azure - Part 3 - Setting up CDC

In the previous part of this series, we did the initial work for the system infrastructure using Azure Bicep. Today, we'll get deeper into setting up a Change Data Capture (CDC) pipeline to Azure Event Hub. CDC allows us to capture and stream real-time changes from our database,

Real-Time Data Streaming and Search Indexing with Azure - Part 2 - Setup Resources

Real-Time Data Streaming and Search Indexing with Azure - Part 2 - Setup Resources

In the first part of this series, I designed and explained the high-level architecture to stream database-level changes to other components. Now, we are going to focus on the Azure Infrastructure needed to take steps towards the solution. Let's get going! Azure Resources Needed To start, we need

Real-Time Data Streaming and Search Indexing with Azure - Part 1 - Introduction and Overview

Change data capture

Real-Time Data Streaming and Search Indexing with Azure - Part 1 - Introduction and Overview

In today's digital age, we rely on seamless data integration across multiple systems to handle everyday tasks. Imagine ordering products from your favorite online store. You expect the process to be smooth and efficient, from payment and stock management to shipping and order confirmation. Behind the scenes, various