Which Linux Distro is Best for Embedded Development?

Joao Correia

December 1, 2022 - Technical Evangelist

If your organization deploys IoT solutions, you know that development of embedded systems is a bit different from standard desktop development. Linux’s low cost is attractive to IoT developers, so it’s often the choice for embedded development over expensive proprietary kernels. It’s not uncommon for developers to work with an environment similar to the target device using a virtual machine, but development on a VM can be awkward. Instead, developers can work with distributions specifically designed for embedded systems.

Compared to proprietary embedded operating systems, Linux is low cost; it allows for multiple suppliers of software, development and support; it has a stable kernel; and it facilitates the ability to read, modify and redistribute the source code. For these reasons and more, Linux has become the go-to option for embedded systems.

But when it comes to deploying Linux into a device, there are numerous options. When building for desktop and enterprise-grade devices, developers usually opt for a distribution such as Ubuntu to best mimic the deployment environment, with added help from tools like VirtualBox and Docker. This setup is an awkward fit for embedded systems programming, though. The development environment is very different, and the build output needs to comprise an entire software image for the target device: the kernel, device drivers, libraries, application software, maybe even the bootloader.

Approaching Linux Distro for embedded development, many people start with a desktop distribution, such as Debian or Red Hat, and strip out unnecessary components until the installed image is a fit for the target device. This is a natural way to go as many Linux devs are desktop Linux users, and can access their usual array of runtime packages. One option is actually to attach a display and keyboard to a target device, and develop directly like that. This approach can be tricky, though. Desktop distributions are not meant for low-resource systems, and manually adding and deleting packages can be a difficult and error-prone job.

What is Embedded Linux?

Automotive computing, medical devices, Internet of Things (IoT), and factory automation are just a few of the embedded systems running on embedded Linux. We know that computers run on using specific hardware (e.g. CPU, memory, motherboard), and embedded system devices also run on microprocessors and controllers but at a smaller scale. These devices need an operating system that use fewer resources, and developers must be able to deploy to the operating system from their desktops.

Embedded Linux is often the choice for IoT developers due to its low cost, open-source code, lightweight storage requirements, and its ability to run on fewer resources. It’s a stable option, which is critical for organizations who deploy systems used by several consumers. The worry of OS crashes are non-existent. For example, the Android operating system is based on Linux and powers 87% of the global market share, and the OS is very reliable.

Is Embedded Linux an RTOS?

A proprietary real-time operating system (RTOS) can be an attractive alternative to embedded Linux, but it comes with its own difficulties. When developers seek out IoT development platforms and methods, they will come across the options of embedded Linux or an RTOS. There are distinct differences between the two, so you must choose carefully before being married to one or the other.

RTOS can be more developer friendly, but for enterprise applications where you want to support customizations or integrate with other developer projects, embedded Linux provides a more intuitive integration environment. An RTOS is designed for time-sensitive applications. Unless you have a system that has critical time-sensitive events, an embedded Linux system provides more flexibility for developers.

What is the Difference between Linux and Embedded Linux?

Most developers know that you need to compile source code for the target system. When developing for embedded systems, developers write code on a desktop but deploy to another environment using cross-platform toolchains. Embedded system deployments are much different than desktops. Desktop computers target 8086 microprocessor architectures, but embedded systems can target several microprocessors, usually ARM, 8051, PIC and AVR.

A few other differences between Desktop Linux and Embedded Linux include:

Embedded systems use less power and usually run on batteries.
Embedded systems are much more compact.
Many embedded systems power critical devices such as health alert systems, medical equipment, nuclear machinery, and home security.
An embedded Linux system contains just the components needed to function on the target processor rather than all possible drivers and libraries to run on a myriad of components installed on a desktop.

What is an Example of an Embedded Linux OS?

The Linux kernel is open source and allows anyone to modify and distribute it in its altered form. Several desktop versions of Linux are on the market, just like several versions of embedded Linux are available. Developers are free to work with their own customized Linux version, but it’s much more convenient to work with an already existing and stable version.

We already mentioned the Android operating system. Android is an embedded Linux system developed by Google and released under the open source license, which allows other developers to modify and distribute it. Debian is an example of a desktop Linux distribution that also has a version embedded on Raspberry Pi devices. Raspberry is the Debian-based operating system used to power the compact Raspberry Pi computing device.

Why Linux is Used in an Embedded System?

Developers choose Linux as their embedded system target operating system for a variety of reasons. The first one is that Linux is completely customizable. Developers can freely take the Linux kernel and create their own distribution of the operating system or take an existing distribution and add small changes that create their own version.

Linux is free, so it has a low startup cost for new developers who just want to explore and get started. The more developers who work with Linux, the more likely it is to be embedded into systems.

Finally, it’s widely used making it better supported by third-party developers. If the goal is to build a system that other developers could integrate into their own projects, Linux will be more intuitive.

How to Choose an Embedded Linux Operating System?

Choosing an embedded Linux operating system depends highly on your goals, the target device operations, time to market and responsiveness required. Many developers simply go with Android, because it’s widely used and understood, but here are a few questions to consider to help you choose the right OS:

Is timing critical? If you have an application that needs precise timing, a real-time operating system should be used. Linux is not an RTOS, but extensions such as RTLinux can be considered.
Is memory size and CPU power limited? A typical Linux kernel is 1.5MB, which could be too large for some systems. A customized Linux kernel with only the right components for application functionality can be used.
Is network performance important? Mobile apps that use Android have networking capabilities, but Android is not necessary if networking is the primary focus. An alternative built specifically for network performance is OpenWRT/LEDE.
Will there be a graphical user interface? Android’s pre-installed GUI might be a good option for systems that depend heavily on user interface.
Is the time to market short? A more common Linux distribution will help reduce development time. Ubuntu is commonly used for faster launches.

Which Linux Distro is Best for Embedded Development?

Developers need an environment that makes it convenient to write code and compile to the target operating system. They could use a virtual machine and set up the target distribution, but it’s much more convenient and comfortable to work with tools that install on the local machine and deploy cross-platform. Here are a few great Linux distributions for embedded development:

Yocto

One very popular non-desktop option for Linux distro for embedded systems is Yocto, also known as Openembedded. Yocto is supported by an army of open source enthusiasts, some big-name tech advocates, and lots of semiconductor and board manufacturers. A Yocto project build wraps together target run-time binaries, package feed, and Target SDK. It is easily expanded through layers, which add functionality and features. Yocto’s modular architecture makes it very flexible, and provides a well-defined layer priority and override capability. New releases appear every six months and the documentation is always thorough.* Perhaps the only downside with Yocto is that it can be tricky to get to grips with for the newbie.

Buildroot

Buildroot aims to be a more lightweight version of Yocto. Buildroot does not support on-target package management, and disables all optional compile-time settings for all packages. You cannot install new packages into a running system. This leaves a small system; so small that many people call Buildroot simply a firmware generator (as the images are mostly fixed at build time). A Buildroot project build wraps together the root filesystem image and any other auxiliary files needed to deploy Linux; the kernel, boot-loader, and kernel modules; and the toolchain used to build all the target binaries. The main advantage of Buildroot is that it is fairly easy to learn, especially as it uses normal Makefiles and the Kconfig language. The main downside is that Buildroot’s minimal nature makes customization difficult.

OpenWRT/LEDE

OpenWRT/LEDE is aimed at producing binary images for network-accessible package repositories. It is chiefly used for custom firmware for consumer routers. If you need replacement firmware for a commercial device, especially if your embedded design is networking-focused, OpenWRT/LEDE should be your go-to. However, it imposes significant and inflexible policy decisions on your design. If you need significant customization, we recommend Buildroot or Yocto instead.

* More on benefits: https://www.embitel.com/blog/embedded-blog/why-yocto-is-a-framework-of-choice-for-creation-of-embedded-linux-distributions

How to Apply Security Updates to Embedded Systems (and Why)?

Updating your applications on embedded systems is tricky. The wrong patch scenario could result in your customers experiencing a “bricked” device. Insecure updates could result in a compromised device. You could rely on users to manually patch firmware, but users rarely take the time or even know that updates are important. This also results in a compromised system. For instance, unpatched IoT devices were responsible for one of the biggest distributed denial-of-service (DDoS) attacks. As a matter of fact, IoT attacks have increased 300% in 2019. It’s clear that patching and updates are critical to the security of the device and its users.

Communication should always be encrypted across the network, and a digital signature should be used to ensure that an attacker does not modify patch files. Encrypted communications and digital signatures stop man-in-the-middle attacks and prevent tampering. The general workflow for deploying software updates is the following:

Using a secure channel, poll a central system to determine if an update is available.
Ensure that the current update is compatible with the installed version.
Download the patch using a secure channel to avoid a man-in-the-middle attack.
Validate that the file is not corrupted using an integrity check (e.g. checksum).
Verify that the digital signature is valid.
Decrypt the file.
Extract all compressed files.
Perform any pre-install actions such as creating backups for a rollback.
Install the patch.
Perform any post-install actions such as clean-up of installation files.
Sanity-check the installation to ensure it was successful, and if not perform a rollback.

How to Apply Security Updates to Embedded Systems Without a Reboot?

The above workflow is standard for updates, but most patches require a reboot of the system. Rebooting a device interrupts the user and could cut off critical systems. For instance, rebooting a main router could result in serious downtime and data corruption on an enterprise network.

Instead of forcing a reboot, livepatching can be performed. Livepatching solves several issues with embedded systems updates. It eliminates the need for reboots and possibly bricking a device due to failed updates. Developers no longer need to create a custom patching service for embedded Linux, and they no longer need to rely on users to manually update applications.

There are three main ways to update Linux without rebooting: using the command line, using the kexec tool, or using a livepatch solution. All three including third-party solutions and the steps to patch are covered in this blog post.

Conclusion

For any developer looking for an environment in embedded systems, Linux is a popular option with several available solutions, platforms, distributions, and tools. The operating system you choose will drive many of your environment options, but one thing is clear: you always need a way to patch and update your devices. To avoid bricking customer devices but keep them updated, live patching is the best solution. Live patching keeps your consumer systems secure while giving them the flexibility to reboot when they find it appropriate.