Word is you like to live on the Edge, stranger.

Sometimes I like it too. I wish I didn't, but I do. I do it because that's where most of the interesting things happen, and for a good reason: zero latency. I wasn't always like that. I used to struggle with my equipment, didn't know where to start from, I was intimidated, I felt completely overwhelmed and powerless by this "technology". It took me some time to build up my skills and find my comfort zone, but to get there, the ride was rough.

For the past two years I've been working on various IoT Edge technologies. For the most part of it, my work was around the Azure IoT Edge technology stack. Through this work, I came up with some ground rules that I personally think are the minimum prerequisites for any IoT development environment, especially for developers who write hardware-specific code, and some design principles that over time I've found great value in.

In this mini-series posts, I'll try to demonstrate these tools and patterns in an incremental way, starting from a simple console app that will gradually evolve to a production-ready Azure IoT Edge application.

Azure IoT Edge: the main gist

All edge technologies' purpose is to help you achieve one thing: run your code on a remote (network) device. This boils down to two distinct capabilities:

  1. Deploy and manage your code on a device.
  2. Establish communication with the device.

We just experienced the proliferation of containers and, as expected, containers are the main deployment mechanism of Azure IoT Edge too. In Azure IoT Edge, one can deploy an application by creating the application manifest: a file that holds the information of which containers compose your application, and how these containers should be instantiated.

The development tools landscape

There are development tools that help you build your containers and compose this manifest. Unfortunately, these tools rely heavily on the solution file system structure to operate, and come with unintuitive commands that mostly confuse rather than help. These tools rely on a thick stack of dependencies, that makes them flaky and slow, and print cryptic error messages. Even if you manage to properly install them on your development machine and use them to successfully compose your first IoT Edge application, the next challenge is to figure out how to debug this application. In a constant output log searching, lengthy redeployments, and remote debugging tricks, the whole development experience is frustrating.

The problem lies in the containerized nature of the deployment mechanism, and in the fact that this deployment mechanism was never abstracted out from the development experience. In addition to that, the deployment manifest is not generated by user code, and as a consequence, no code validation checks can apply to it. This means for example that tools like compilers, linters etc. are useless. If you decide to rename a class, any decent IDE/editor is clever enough to propagate this rename effect to the entire solution. But because this technology-unique manifest is exposed to the user, the application often breaks. Furthermore, it's difficult to have environment solution variants (debug, release etc.), because the compiler directives do not apply inside the manifest, and the tooling approach is to have dedicated replicas per configuration, making a large size application maintenance very difficult.  

An incremental approach

RPC happiness

In it's most fundamental form, an IoT Edge application performs two functions:

  1. Processes device sensor data and emits some telemetry.
  2. Perform some form of RPC (remote command invocation).

The typical documentation example of Azure IoT Edge is a temperature sensor, that emits the device temperature (telemetry), and has a reset method (command). As of writing this article, the source code of this example is here.

A simple console app that demonstrates this simulated temperature example looks like this:

using System;
using System.Threading;

namespace Example1
{
    class Program
    {
        // This is the program entry point
        static void Main(string[] args)
        {
            // Register the Reset command callback 
            RegisterCommandCallback("Reset", OnReset);

            // A non-blocking telemetry emission invocation
            new Thread(EmitTelemetryMessages).Start();

            // Wait until the app unloads or gets cancelled
            Console.ReadLine();
        }

        private static void RegisterCommandCallback(string command,
            Action callback)
        {
            // Perform the command registration
            // Code omitted
            return;
        }

        // A method exposed for RPC
        static void OnReset()
        {
            // Perform a temperature sensor reset
            // Code omitted
            return;
        }

        // Emit 500 telemetry messages
        static void EmitTelemetryMessages()
        {
            for (int i = 1; i <= 500; i++)
            {
                Console.WriteLine($"Sending telemetry message {i} ..");
                Thread.Sleep(1000);
            }
        }
    }
}
A simple IoT Edge Temperature Sensor

Let's break down what happens here. This example code performs three things:

  1. Registers a command callback (the actual registration code is omitted for clarity)
  2. Sends some messages in a second thread
  3. Waits for the user to press enter to exit.

The latter step is there to ensure that the program does not exit immediately, before the other thread has the chance to send any message. In practice though, in a typical IoT Edge scenario there is no user to press enter, there's not even a monitor screen to print these messages. We'll see later on what's the best pattern to deal with this issue.

As with most containerized applications, this is a command-line initiated app, a console app. Console apps come from a time when all applications were single threaded and there was no GUI to interact with. The operating system would load the application binary code into a memory block, and start a thread at the program entry point, the main function. When this function returned, the program would exit.

When multithreaded applications were eventually allowed by modern operating systems, programmers could manually create a new thread and start it at any function. This felt like running multiple applications, but sharing the same memory block. This capability allowed the creation of a new programming style, called multithreaded event-driven that was very useful in scenarios like computer networking or graphical user interfaces, or in general, when functions had to be executed in a non-predefined order and timing, but rather based on external events.

Fast forward a few decades, and the modern languages have made it easier to deal with multithreaded event driven programming, generally called asynchronous programming . Tons of literature has been written about this topic, but the fundamental idea is simple: writing code that can react to external events of undetermined timing and order. C# in particular has the async/await pattern, which is an elegant way to execute some code in parallel (code that runs in a different thread).

Using this pattern, the above example becomes:

using System;
using System.Threading.Tasks;

namespace Example2
{
    class Program
    {
        // This is the program entry point
        static async Task Main(string[] args)
        {
            // Register the Reset command callback 
            await RegisterCommandCallbackAsync("Reset", OnReset);

            // A non-blocking telemetry emission invocation
            await EmitTelemetryMessagesAsync();

            // Wait until the app unloads or gets cancelled
            Console.ReadLine();
        }

        private static async Task RegisterCommandCallbackAsync(string command,
            Action callback)
        {
            // Perform the command registration
            // Code omitted 
            return;
        }

        // A method exposed for RPC
        static void OnReset()
        {
            // Perform a temperature sensor reset
            // Code omitted
            return;
        }

        // Emit 500 telemetry messages
        static async Task EmitTelemetryMessagesAsync()
        {
            for (int i = 1; i <= 500; i++)
            {
                Console.WriteLine($"Sending telemetry message {i} ..");
                await Task.Delay(TimeSpan.FromSeconds(1));
            }
        }
    }
}
Note that we had to refactor our main function signature to be async

These two versions are very similar in functionality, but with a major difference: in the latter example, there is no code segment that runs for an extensive period, in other words, no thread is blocked by waiting for something to complete. Indeed, in the first example, the thread we created to run the EmitTelemetryMessages function would periodically wait, blocked for a second at the Thread.Sleep(1000). Blocked in this context means that the thread, although running, is waiting for something else to continue. This is not too important for now, but in real production systems becomes very important and helps to avoid the thread pool starvation issue, aka, running out of threads.

The rule of thumb here is: anything that might take a considerable amount of time to complete, has to be implemented in the async/await pattern. The important word here is might: generally speaking, things that require IO, like reading a file from the disk, sending a message over the network etc. fall into this category. Practically anything that does not run only in the context of the CPU should be awaited, because of the uncertainty of the completion time of the required additional hardware resource. But even for code that runs solely in the boundaries of the CPU, if it could take more than a few milliseconds to run, it should be awaited.

In the next article we will examine the best approach to keep the main function from returning.