The decision which programming language to use for streaming involves a few items. First, you need to know the skill sets of the people who will develop the code. It would be prudent to avoid a programming language that requires your current development staff to be retrained. If your staff is skilled with C#, you should lean heavily toward that language, but if Python is more in use, then you should use it. Another item that influences your choice of programming languages is based on what you saw in Table 7.1. The question is whether the product you want to use to implement your streaming solution supports that language. And, finally, it would make sense to keep the programming language the same across the entire data analytics solution. For example, the code written to send messages to Azure Event Hubs, the streaming product that analyzes the messages in near real time or real time, and any additional processing or presentation that happens after that, should all be created using the same language. There are many products that require programming, so you need to have a high‐level overview of them all and try to keep the variety of programming languages to a minimum.
Interoperability
One of the more challenging aspects of creating an end‐to‐end IT solution is determining which systems and technologies can work together. One of the decision points discussed in the previous section had to do with choosing a programming language. As mentioned, you should attempt to select a language that works across the numerous products that make up your data analytics architecture. However, you first need to determine which products can be used with each other. Figure 7.4 illustrates the input interoperability between Azure products. For example, Event Hubs, IoT Hub, and Blob Storage can be used to send data to Azure Stream Analytics. If you need to send data from an Azure Service Bus or a Cosmos DB, you could not use Azure Stream Analytics and would instead likely choose an Azure WebJob.
FIGURE 7.4 Input interoperability in Azure products
Input interoperability is not your only concern. Both the input and the sink (output) that your solution requires must be supported. Figure 7.5 illustrates the sink interoperability between Azure products.
FIGURE 7.5 Sink interoperability in Azure products
If your input is an event hub and your sink is an Azure storage queue, then, again, you cannot use Azure Stream Analytics. An Azure WebJob could fulfill this requirement as well.
