Event Delivery Guarantees – DP-203 Azure Data Engineer

3 Minutes

Azure Synapse Analytics Event Delivery Guarantees Kappa Architecture Microsoft DP-203 Microsoft Exams Process Time Series Data

Configure Checkpoints/Watermarking During Processing – Design and Implement a Data Stream Processing Solution-3

Katie Cox 2024-08-07 0 Comment

An important metric to call out is Watermark Delay. The values represented in the metric indicate the realized processing delay of the event message. Under normal operation the Watermark...

3 Minutes

Event Delivery Guarantees Microsoft DP-203

Configure Checkpoints/Watermarking During Processing – Design and Implement a Data Stream Processing Solution-1

Katie Cox 2024-06-23 0 Comment

A checkpoint is a concept that identifies a point in time when your data is in a verifiable, production‐ready state. For example, a checkpoint could be considered a backup...

4 Minutes

Azure Synapse Analytics Event Delivery Guarantees Microsoft DP-203

Azure Synapse Analytics – Design and Implement a Data Stream Processing Solution

Katie Cox 2024-04-30 0 Comment

There are numerous methods for performing upserts from an Azure Stream Analytics data stream in Azure Synapse Analytics. One approach is to replicate the pattern shown in Figure 7.25....

4 Minutes

Event Delivery Guarantees Kappa Architecture Microsoft DP-203

Develop and Create Windowed Aggregates – Design and Implement a Data Stream Processing Solution

Katie Cox 2023-11-05 0 Comment

FROM [brainwaves].[BrainwaveWindowMedians]WHERE WindowId = ’10s Tumble’ORDER BY IngestionTime The first action in Exercise 7.7 was to create a new table to store the stream of brain wave data readings....

3 Minutes

Event Delivery Guarantees Microsoft DP-203 Process Time Series Data

Use Reference Data with Azure Stream Analytics – Design and Implement a Data Stream Processing Solution

Katie Cox 2023-08-27 0 Comment

FIGURE 7.16 Configure reference data for Azure Stream Analytics use. FIGURE 7.17 Use reference data with Azure Stream Analytics. In Exercise 7.4 you uploaded a reference file in JSON...

6 Minutes

Event Delivery Guarantees Microsoft DP-203 Process Time Series Data

Stream Data with Azure Databricks – Design and Implement a Data Stream Processing Solution

Katie Cox 2023-03-30 0 Comment

FIGURE 7.26 Installing the Event Hubs library on an Azure Databricks cluster FIGURE 7.27 The installed Event Hubs library on an Azure Databricks cluster FIGURE 7.28 Streamed Event Hubs...

3 Minutes

Event Delivery Guarantees Microsoft DP-203 Microsoft Exams

Process Time Series Data – Design and Implement a Data Stream Processing Solution

Katie Cox 2023-02-26 0 Comment

The streaming of data from an IoT device, like a BCI, is an ideal fit for time series analysis. The data in this context is ideal because the data...

3 Minutes

Event Delivery Guarantees Microsoft DP-203 Microsoft Exams

Azure Event Hubs – Design and Implement a Data Stream Processing Solution

Katie Cox 2022-03-27 0 Comment

In Exercise 3.16 you provisioned a Basic tier Event Hubs namespace, and in Exercise 3.17 you configured it as an Azure Stream Analytics input. An important aspect of Azure...

2 Minutes

Event Delivery Guarantees Kappa Architecture Microsoft DP-203

Kappa Architecture – Design and Implement a Data Stream Processing Solution

Katie Cox 2021-12-17 0 Comment

Much of the interest in the kappa architecture has to do with code duplication. The code that transforms data along the cold path is often duplicated on the hot...

3 Minutes

Event Delivery Guarantees Microsoft DP-203 Process Time Series Data

Scaling – Design and Implement a Data Stream Processing Solution

Katie Cox 2021-10-28 0 Comment

Chapter 10, “Troubleshoot Data Storage Processing,” discusses performance and scaling techniques, but this is an appropriate place to introduce these concepts in the context of streaming products. It is...