In the last post we saw how to setup an F5 development experience for an IoT Edge application, without having to install the IoT Edge Runtime or any other tool on our localhost, but rather by mocking the runtime's behavior in one of our mock dependencies. In this post, we will see how to setup the same F5 experience, but with using the actual IoT Edge Runtime.
This is an alternative approach to what the official Azure IoT Edge development tools offer. In this approach we will setup a development IoT Edge Server that will allow us to debug our modules with a simple F5 experience, no redeployment, no remote debugging, no runtime installation and troubleshooting.
Emulating IoT Edge Runtime
Unfortunately, currently there is no development IoT Edge server, but we can build one!
I have found that the effort of installing and troubleshooting the runtime on my development machine exceeds the effort of building this custom emulator.
To setup this development server, we will need to build a custom tool in C#.
Let's assume we would like to debug the SimulatedTemperatureSensor we built in the last post, but using the actual IoT Edge runtime this time instead of mocking it out. Let's create an new .NET Core console app next to the SimulatedTemperatureSensor that will hold all of our development tools:
cd ..
mkdir DevelopmentTools
cd DevelopmentTools
dotnet new console
dotnet add package Microsoft.Azure.Devices
dotnet add package Docker.DotNet
code .This tool needs to perform three tasks:
- Provision a development IoT device.
- Create and push the appropriate development deployment manifest.
- Run this device locally.
Here's the self-descriptive code of our tool's Program.cs:
namespace DevelopmentTools
{
using System;
using System.IO;
using System.Threading.Tasks;
class Program
{
static async Task Main(string[] args)
{
if (args.Length != 3)
{
Console.WriteLine("Usage: dotnet run {MANIFEST_FILE} {DEVICE_ID} " +
"{IOT_HUB_OWNER_COONECTION_STRING}");
return;
}
var developmentManifest = Utilities.CreateDevelopmentManifest(
File.ReadAllText(args[0]));
var deviceConnectionString =
await Utilities.ProvisionDeviceAsync(args[1],
developmentManifest,
args[2]);
await Utilities.StartEmulatorAsync(deviceConnectionString);
}
}
}
We need to create a Utilities.cs file where we'll put the referenced three functions from above:
namespace DevelopmentTools
{
using Docker.DotNet;
using Docker.DotNet.Models;
using Microsoft.Azure.Devices;
using Microsoft.Azure.Devices.Shared;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Threading.Tasks;
public static class Utilities
{
public static async Task StartEmulatorAsync(string deviceConnectionString)
{
string deviceContainerImage = "toolboc/azure-iot-edge-device-container";
int[] exposedPorts = new[] { 15580, 15581, 443, 8883, 5671 };
string imageName = "dev_iot_edge";
var localDockerSocket = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ?
@"npipe://./pipe/docker_engine" :
@"unix:/var/run/docker.sock";
var dockerClient = new DockerClientConfiguration(new Uri(localDockerSocket))
.CreateClient();
Console.WriteLine($"Downloading the latest image:{deviceContainerImage}..");
await dockerClient.Images.CreateImageAsync(
new ImagesCreateParameters
{
FromImage = deviceContainerImage,
Tag = "latest"
},
new AuthConfig(),
new Progress<JSONMessage>((e) =>
{
if (!string.IsNullOrEmpty(e.Status))
Console.Write($"{e.Status}");
if (!string.IsNullOrEmpty(e.Status))
Console.Write($"{e.ProgressMessage}");
if (!string.IsNullOrEmpty(e.Status))
Console.Write($"{e.ErrorMessage}");
Console.WriteLine("");
}));
var containers = await dockerClient.Containers
.ListContainersAsync(new ContainersListParameters() { All = true });
foreach (var _container in containers)
{
if (_container.Names.Contains(imageName) ||
_container.Names.Contains($@"/{imageName}"))
{
if (_container.State == "running")
{
Console.WriteLine($"Stopping container {_container.ID}..");
await dockerClient.Containers.StopContainerAsync(_container.ID,
new ContainerStopParameters());
}
Console.WriteLine($"Removing container {_container.ID}..");
await dockerClient.Containers.RemoveContainerAsync(_container.ID,
new ContainerRemoveParameters());
break;
}
}
Console.WriteLine($"Creating {imageName} container..");
var container = await dockerClient.Containers
.CreateContainerAsync(new CreateContainerParameters
{
AttachStderr = true,
AttachStdin = true,
AttachStdout = true,
Tty = true,
Env = new List<string>() { $"connectionString={deviceConnectionString}" },
Name = imageName,
Image = deviceContainerImage,
ExposedPorts = exposedPorts
.ToDictionary(x => x.ToString(), x => default(EmptyStruct)),
HostConfig = new HostConfig
{
Privileged = true,
PortBindings = exposedPorts.ToDictionary(
x => x.ToString(),
x => (IList<PortBinding>)new List<PortBinding> {
new PortBinding {
HostPort = x.ToString()
}
}),
PublishAllPorts = true
}
});
Console.WriteLine($"Starting {imageName} container..");
var startResult = await dockerClient.Containers.StartContainerAsync(
container.ID, null);
if (!startResult)
throw new Exception($"Cound not start the {imageName} container!");
Console.WriteLine("Done.");
}
public static string CreateDevelopmentManifest(string template)
{
var templateContent = JsonConvert
.DeserializeObject<ConfigurationContent>(template);
var agentDesired = JObject.FromObject(
templateContent.ModulesContent["$edgeAgent"]["properties.desired"]);
if (!agentDesired.TryGetValue("modules", out var modulesSection))
throw new Exception("Cannot read modules config from $edgeAgent");
foreach (var module in modulesSection as JObject)
{
var moduleSettings = JObject.FromObject(modulesSection[module.Key]["settings"]);
moduleSettings["image"] = "wardsco/sleep:latest";
modulesSection[module.Key]["settings"] = moduleSettings;
}
agentDesired["modules"] = modulesSection;
templateContent.ModulesContent["$edgeAgent"]["properties.desired"]
= agentDesired;
return JsonConvert.SerializeObject(templateContent, Formatting.Indented,
new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore
});
}
public static async Task<string> ProvisionDeviceAsync(
string deviceId,
string manifest,
string ioTHubConnectionString)
{
var registryManager = RegistryManager
.CreateFromConnectionString(ioTHubConnectionString);
var hostName = ioTHubConnectionString.Split(";")
.SingleOrDefault(e => e.Contains("HostName="));
if (string.IsNullOrEmpty(hostName))
throw new ArgumentException(
$"Invalid ioTHubConnectionString: {ioTHubConnectionString}");
hostName = hostName.Replace("HostName=", "");
var device = await registryManager.GetDeviceAsync(deviceId) ??
await registryManager.AddDeviceAsync(
new Device(deviceId)
{
Capabilities = new DeviceCapabilities() { IotEdge = true }
});
var sasKey = device.Authentication.SymmetricKey.PrimaryKey;
var manifestContent = JsonConvert
.DeserializeObject<ConfigurationContent>(manifest);
// remove all old modules
foreach (var oldModule in await registryManager.GetModulesOnDeviceAsync(deviceId))
if (!oldModule.Id.StartsWith("$"))
await registryManager.RemoveModuleAsync(oldModule);
// create new modules
foreach (var module in manifestContent.ModulesContent.Keys)
if (!module.StartsWith("$"))
await registryManager.AddModuleAsync(new Module(deviceId, module));
await registryManager
.ApplyConfigurationContentOnDeviceAsync(deviceId, manifestContent);
return $"HostName={hostName};DeviceId={deviceId};SharedAccessKey={sasKey}";
}
}
}
We need access to a running edge daemon and an edgeHub that our code will communicate with. This communication is secure, and because of that, we need first to create the module identity to use. To create this module identity, we simply need to describe a module in the deployment manifest.
Here are the contents of the application manifest.json:
I prefer putting this file at the root directory level of the solution.
{
"modulesContent": {
"$edgeAgent": {
"properties.desired": {
"schemaVersion": "1.0",
"runtime": {
"type": "docker",
"settings": {
"minDockerVersion": "v1.25",
"loggingOptions": "",
"registryCredentials": {
}
}
},
"systemModules": {
"edgeAgent": {
"type": "docker",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.0.9",
"createOptions": ""
}
},
"edgeHub": {
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0.9",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}],\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}]}}}"
}
}
},
"modules": {
"SimulatedTemperatureSensor": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "your.azurecr.io/simulatedtemperaturesensor:latest",
"createOptions": "{}"
}
}
}
}
},
"$edgeHub": {
"properties.desired": {
"schemaVersion": "1.0",
"routes": {
"allToIoTHub": "FROM /* INTO $upstream"
},
"storeAndForwardConfiguration": {
"timeToLiveSecs": 10
}
}
},
"SimulatedTemperatureSensor": {
"properties.desired": {
"SendInterval": 1
}
}
}
}Note: we can modify this manifest to match our application requirements, for example, add more modules, define module twins, change the routes etc. The tool will simply replace the modules' container image with a mock container.
Done! Let's run this console application passing the required arguments:
dotnet run ../manifest.json dev_device "IOT HUB OWNER CONNECTION STRING"The effect of this execution is that we have a development IoT Edge device running in a local container, called dev_iot_edge. This container exposes the edge daemon and the edgeHub's ports to our host. All we have to do now us connect to it from our module.
To do that, we need to set some environment variables so that the IoT SDK can connect securely to the emulator. These variable have the prefix IOTEDGE_ and we can get them from inside the running mock module of our emulator:
docker exec dev_iot_edge bash -c "docker exec SimulatedTemperatureSensor env | grep IOTEDGE_"
We need to override the IOTEDGE_GATEWAYHOSTNAME with our host's name, and change the IP of the IOTEDGE_WORKLOADURI with 127.0.0.1.
Setting these environment variables in our application will allow us to connect to the running emulator!
In practice, it's very easy to automate this process, with this helper function:
namespace SimulatedTemperatureSensor
{
using System;
using System.Diagnostics;
using System.Linq;
using System.Net;
internal static class Utilities
{
internal static void InjectIoTEdgeVariables(string containerName)
{
var dockerCommand =
$"docker exec dev_iot_edge bash -c \"docker exec {containerName} env | grep IOTEDGE_\"";
Process p = new Process()
{
StartInfo = new ProcessStartInfo(dockerCommand)
{
FileName = "cmd.exe",
Arguments = $"/C {dockerCommand}",
RedirectStandardError = true,
RedirectStandardOutput = true,
UseShellExecute = false,
CreateNoWindow = true
},
};
p.Start();
p.WaitForExit();
var output = p.StandardOutput.ReadToEnd();
var lines = output.Split(new[] { "\n" }, StringSplitOptions.None)
.Where(e => e != null && e.Contains("="));
var variables = lines.ToDictionary(e => e.Split("=")[0], e => e.Split("=")[1]);
// Overwrite these settigns
variables["IOTEDGE_WORKLOADURI"] = "http://127.0.0.1:15581/";
variables["IOTEDGE_GATEWAYHOSTNAME"] = Dns.GetHostName();
foreach (var variable in variables)
{
Environment.SetEnvironmentVariable(variable.Key, variable.Value);
Console.WriteLine($"Injected {variable.Key}={variable.Value}");
}
}
}
} Then, when we're in the Emulated environment, we can automatically call this method from our application start up.
namespace SimulatedTemperatureSensor
{
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Serilog;
class Program
{
static async Task Main(string[] args)
{
using (var host = Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
if (hostContext.HostingEnvironment.EnvironmentName == "Emulated")
Utilities.InjectIoTEdgeVariables("SimulatedTemperatureSensor");
services.AddSingleton<IModuleClient, ModuleClientWrapper>();
services.AddHostedService<TemperatureSensorModule>();
})
.UseSerilog((hostingContext, log) =>
{
log.ReadFrom.Configuration(hostingContext.Configuration);
})
.UseConsoleLifetime()
.Build())
{
await host.RunAsync();
}
}
}
}
Note: you don't need to design your IoT Edge module project in the specific structure we used, even the monolithic C# template works with this approach. In fact, all supported programming languages will work!
Module Container
After completing our implementation and testing, we need to build the module container image. This is easily done, both in VS Code and Visual Studio.
In Visual Studio:
In VS Code:
Both tools will automatically populate the required Dockerfile, and will setup the build and run tasks. Simply hit CTRL+SHIFT+B to trigger the image build.
You can even debug the same application running inside a Linux container! This is very useful when we have system dependencies that we cannot install on our host.
Last step is to push this container to your container registry and update the manifest accordingly (currently set to your.azurecr.io )
You need to include the container registry credentials if any.
The entire application is published in this GitHub repo.
Recap
We saw how to setup an F5 development experience without using any tools, but just Docker and our C# code. This approach allows us to keep the same favorite development environment of our preference, without making any compromises.
