OpenEmbedded for Deep Embedded Systems

Detlef Vollmann

vollmann engineering gmbh

P.O. Box 5423
6000 Luzern 5


Table of Contents
1. Introduction
2. Overview
2.1. Build System
2.2. Metadata
2.3. Package System
3. Working with OpenEmbedded
3.1. local.conf
3.2. Machine Configuration
3.3. Distribution Configuration
3.4. An Image Package
3.5. A Kernel Package
3.6. A Package for an Open Source Project
3.7. Own Software
4. Conclusion


OpenEmbedded has won the TuxMobil GNU/Linux Award 2005 that honors Free Software projects, which improve Linux for mobile computers. OpenEmbedded is a Linux distribution similar to Debian that has its roots in the PDA domain. It is today pretty popular among Linux fans who own Zaurus', iPAQs or similar PDAs. But OpenEmbedded is targeted at all kinds of embedded Linux systems. It features a unique cross build environment that generally allows pretty easy adaption of OpenSource software for cross compilation even if the original software authors didn't think about cross builds. That build environment also allows for easy definition and builds of complete distributions for embedded systems.

OpenEmbedded also provides a lightweight and fine-grained package system that enables easy installation of new software packages into a running system as well as updates of existing software. These features makes OpenEmbedded a first choice for the creation of embedded Linux systems.

1. Introduction

When Sharp launched its Zaurus PDA, it came with a Linux based PDA system. But not all users were happy with the original Sharp configuration and so the OpenZaurus project was created to share the modifications. Later, OpenZaurus moved from modifications to the original Sharp image to a complete distrubtion based on Debian.

But the build system for Debian was not really suited for small embedded systems and so OpenEmbedded was founded with a build system inspired from Gentoo's portage. As package system iPKG was used, which is closely related to Debians dpkg, but more tuned for small embedded devices. Later, distributions for other PDAs like Compaq's iPAQs or the Siemens SimPad moved to the OpenEmbedded build and package system. A very interesing distribution based on OpenEmbedded is OpenSLUG for LinkSys' NSLU device. The NSLU is not a PDA but originally an NAS storage system.

Today, OpenEmbedded describes itself as a "set of recipes and metadata to build Linux distributions for embedded devices with the BitBake build system".

OpenEmbedded provides three major benefits for building a distribution for an embedded system:

The remainder of this article focuses on the use of OpenEmbedded for deep embedded systems like the NSLU opposed to PDA like systems like the SimPad.

2. Overview

2.1. Build System

Like any build toold (make, ant, jam), the OpenEmbedded build tool BitBake controls how to build things and the build dependencies. But unlike single project tools like make it is not based on one makefile or a closed set of inter-dependent makefiles, but collects and manages an open set of largely independent build descriptions (package recipes) and builds them in proper order.

The OpenEmbedded set of package recipes include not only recipes for target packages, but also recipes for tools on the host required to build those target packages. So, OpenEmbedded builds a complete toolchain for cross-building before building the target packages and image.

2.2. Metadata

The metadata from which an OpenEmbedded distribution is built comes in three different forms:

  • configuration files

  • class descriptions

  • package recipes

The configuration files provide general variable definitions to control the behaviour of BitBake and how things are generally built in OpenEmbedded. This includes the build system's directory structure, version preferences, source code mirror sites as well as specific build options (e.g. the default optimizing level).

The class descriptions define common procedures to build things, like applying the auto-tools for configuration, collecting runtime library dependencies or building native build tools for the host. These class descriptions are sometimes quite specific, e.g. there exists a class to remove NLS parts of a package if NLS support is not wanted.

The package recipes provide the information how to build a specific piece of software ‐ a build tool for the host, a library or a target application. Such recipes provide the information how to get and how to build a package and dependencies on other packages.

Meta package recipes don't build a specific package, but mainly consist of dependency descriptions to build a complete set of packages, often a base image for a specific distribution.

2.3. Package System

The iPKG package system is (deliberately) very similar to Debian's dpkg, but is tuned for small systems. It contains the package data that is simply copied to the target system, metadata and optionally installation scripts. The metadata includes the (run-time) dependencies of the package.

Package systems are mainly for the benefit of users of computer-like devices who want to install their own specific set of software. Such package systems provide two major benefits:

  • easy definition of an initial image, often called 'base system'

  • controlled installation, upgrade and de-installation of packages on the running system

These benefits also apply to (deep) embedded systems. Different configurations are just different sets of packages. They can even share the already built packages from existing configurations.

In traditional embedded systems for an update first a new complete image is built that then requires on the target a shutdown, a complete re-flash of the image and finally a restart of the system. Contrasting to that image-based process, a package system allows easy updates on a live, running system that even allows to have some processes running the old version (though it is already de-installed) while other processes already run the new version.

3. Working with OpenEmbedded

To build a system based on OpenEmbedded, normally a small set of configuration files is needed:

Apart from that, typically a meta package for the base image is required. And then of course the recipes for specific packages, e.g. a kernel package, packages for additional Open Source applications and packages for project specific software.

3.1. local.conf

The local configuration file local.conf defines the local directory structure, the local build environment, some project specific preferences and other properties specific to the build system.

A very simple and short local.conf could look like this:

# DL_DIR specifies the download target directory
DL_DIR = "${PROJECT}/oesrc"

# BBFILES specifies the full set of package recipes to be parsed by BitBake
BBFILES = "${PROJECT}/*/*.bb"

# BBMASK specifies which package recipes to ignore from the full set above

# ASSUME_PROVIDED defines what local host build tools should
# not be built by BitBake but should be used from the local
# build host's installation
ASSUME_PROVIDED = "flex-native"

# For some tools exist different alternative implementations,
# e.g. for the C runtime library there exist glibc and uClibc.
# PREFERRED_PROVIDERS defines which specific package to build
PREFERRED_PROVIDERS = "virtual/kernel:mymach24"
PREFERRED_PROVIDERS += " virtual/libc:glibc"

# For many packages exist several different recipes.
# PREFERRED_VERSION defines which specific recipe to use
PREFERRED_VERSION_gcc-cross = "3.3.2"

# MACHINE defines for which hardware to build
MACHINE = "mymach"

# DISTRO defines which distribution to build
DISTRO = "mymini"

# IMAGE_FSTYPES defines which kind of images to create
IMAGE_FSTYPES = "jffs2 tar"

# For a number of package recipe versions the source code is fetched directly
# from the original CVS repository head.  To make sure that for separate
# builds this fetches the same source, use CVSDATE.
CVSDATE = "20051122"

# For some packages specific CVS versions are provided as tarballs.
# CVS_TARBALL_STASH defines where to find them.

# For a number of software sets it is possible to specify local
# mirror sites where to get the software.
export GNU_MIRROR = ""

# URL for own stuff
MY_URL = "http://myserver/projects/oe"

3.2. Machine Configuration

The machine configuration file conf/machine/mymach.conf specifies the hardware for which a distribution is built. This includes mainly the CPU architecture, specific hardware kernel modules and some size specifications.

A simple example could look like this:

#@TYPE: Machine
#@NAME: My own hardware
#@DESCRIPTION: Machine configuration for my system XYZ

# the target CPU architecture

# all compatible binary architectures
IPKG_ARCHS = "all arm armv4 armv4t armv5e armv5te ipaqpxa mymach"

# some packages for which we know they work best for our hardware
PREFERRED_PROVIDER_xserver ?= "xserver-kdrive"
PREFERRED_PROVIDER_virtual/kernel ?= "mykernel24"

# some packages we always need for this hardware
BOOTSTRAP_EXTRA_DEPENDS = "virtual/kernel sdmmc-support altboot"
BOOTSTRAP_EXTRA_RDEPENDS = "kernel sdmmc-support altboot"
BOOTSTRAP_EXTRA_RDEPENDS += " kernel-module-usbdcore kernel-module-usbdmonitor"

# autoload on boot
module_autoload_mydriver = "mydriver"

# compile with XScale optimization
include conf/machine/tune-xscale.conf

# some specific settings
SERIAL_CONSOLE = "115200 ttyS0"
GUI_MACHINE_CLASS = "smallscreen"

3.3. Distribution Configuration

The distribution configuration file conf/distro/mymini.conf specifies global configuration parameters for the whole software system on the target. The main definition here is the OS setting, but included here are also internationalization settings or a specific target filesystem layout.

A simple example could look like this:

#@TYPE: Distribution
#@NAME: MyMini
#@DESCRIPTION: A minimal base system for my system

# some general descriptions
DISTRO = "MyMini"
DISTRO_NAME = "My Minimal Embedded Linux"
DISTRO_TYPE = "release"

# feed definitions for ipkg
FEED_URIS += " \
     base##${MY_URL}/${DISTRO_VERSION}/feed/base \

# base system
TARGET_FPU = "soft"
TARGET_OS = "linux-uclibc"

# specific software versions
PREFERRED_PROVIDER_xserver ?= "xserver-kdrive"
PREFERRED_VERSION_xserver-kdrive ?= "20050207"

# i18n
USE_NLS = "yes"

# distro is based on udev

# distro is ipkg based
INHERIT += " package_ipk"

3.4. An Image Package

The image package recipe packages/meta/ builds the base system for the root filesystem image. It mainly defines the packages that are included in the base image.

A simple example could look like this:

# general description data
DESCRIPTION = "Core packages for a minimal installation for My"
PR = "r0"

MY_PACKAGES = "base-files-my \
        busybox-my initscripts-colibri netbase \
        sysvinit usbutils modutils-initscripts \
        my-modules24 e2fsprogs-mke2fs diffutils ipkg"

# binary architecture for ipkg

# name
export IMAGE_BASENAME = "my"

# which languages to include
export IMAGE_LINGUAS = ""

# which packages to include

# give the packages again so the build systems knows they must be built

# inherit the class that finally builds the image
inherit image_ipk

3.5. A Kernel Package

The kernel is typically specific to a hardware, so usually an own kernel package is required.

A simple example packages/linux/mymach24_2.4.29-mymach could look like this:

DESCRIPTION = "Linux kernel 2.4 for My hardware"
SECTION = "kernel"
PR = "r0"

# compute the kernel version strings
KV = "${'PV',d,True).split('-')[0]}"
MYV = "${'PV',d,True).split('-')[1]}"

# object suffix dependent on kernel version

# where to get the base kernel
SRC_URI = "${KERNEL_MIRROR}/v2./linux-${KV}.tar.bz2"

# where to get my specific patches
SRC_URI_append = " ${MY_URL}/patches/linux-${KV}-${MYV}.patch.gz;patch=1"

# specify the source directory
# (only necessary where it differs from the package name)
S = "${WORKDIR}/linux-${KV}"

# inherit the class that actually does the work building kernels
inherit kernel

# this not only builds the kernel itself but also the modules
PROVIDES += " my-modules24"
PACKAGES += " my-modules24"

# tell the packager where the files for the modules package are found
FILES_my-modules24 = "/lib/modules"

# which machines are supported by this kernel
COMPATIBLE_HOST = "arm.*-linux"

# nothing special is required to build the kernel, as it comes with
# full support for cross compilation

# the actual configure command
# oe_runmake just runs make
do_configure() {
        oe_runmake mymach_defconfig

# clean up after module installation
do_install_append() {
    rm -f ${D}/lib/modules/*/build
    rm -f ${D}/lib/modules/*/source

Some details for this package recipe are explained in the next section.

3.6. A Package for an Open Source Project

Though OpenEmbedded comes with recipes for many Open Source projects, sometimes a package is required for which no recipe exists yet. But providing a recipe for that project is generally quite easy. Most Open Source projects are based on the configure mechanism to build. configure is a script to collect information about the build environment and creates makefiles based on that information.

But the configure script itself is normally generated through the auto-tools. The normal OpenEmbedded build process for such a project is to rebuild the configure script based on the ultimate source and

So, a simple package file for the at tool looks like this:

DESCRIPTION = "Delayed job execution and batch processing."
SECTION = "base"

PR = "r1"

DEPENDS = "flex-native"

SRC_URI = "${DEBIAN_MIRROR}/main/a/at/at_${PV}-11.tar.gz \
	   file://configure.patch;patch=1 \

inherit autotools

That's all. Here a walkthrough for this recipe: The first three lines in this package file are just general informations (that are included into the resulting binary package).

PR defines the revision and should be incremented on each change to the package recipe.

The DEPENDS definition states that the building of this package depends on an existing flex installation on the host (therefore the -native).

The SRC_URI defines the place of the source files to be downloaded: the main distribution tarball with the URL where to find it, and two specific patches to build this package with OpenEmbedded. These patches are located together with the package file. The patch=1 specifies that this file is to be applied as patch with -p1. The ${PV} in the tarball URL is expanded from the recipe version number. And the recipe version number is taken from the file name of the recipe. So, if this recipe is provided as packages/at/, ${PV} is expanded to 3.1.8.

The next line essentially does all the work: it inherits the autotools class that adds the necessary step (task) to rebuild the configure script.

And that's all. The base class that is inherited by all packages defines all the other tasks to build the binary package:

  • do_fetch(), which does the download

  • do_unpack(), which builds the working directory and unpacks all files

  • do_patch(), which applies the patches

  • do_configure(), which runs the configure script

  • do_compile(), which basically calls make

  • do_stage(), which installs library and header files in the cross build environment for subsequent builds

  • do_install(), which installs the built tools into a special packaging area

  • do_package(), which collects the files from the packaging area and creates (possibly several) packages

All these tasks can be overwritten: in the above kernel package example the do_configure() is redefined, and in the inherited autotools class for this example the do_configure() is redefined to add a autoreconfig run to rebuild the configure script before the actual configure.

3.7. Own Software

For own software projects it is possible also to use the auto-tools and configure to create the makefiles. But this requires some familiarity with those tools and is not really necessary. A standard makefile will suffice, if some simple rules are observed:

  • don't use fix pathnames for include, library and install directories, use variables for those directories.

  • use variables for all building commands (including ar).

  • provide an install target.

So, a makefile for the standard "Hello, World" example would look like this:

CC = arm-linux-gcc
LD = arm-linux-ld
CXX = arm-linux-g++
INSTALL = install

prefix = ""
bindir = $(prefix)/usr/bin

TARGETS = hello

all: $(TARGETS)

hello: hello.c
        $(CC) $(CFLAGS) -o $@ $<

        rm -f *.o $(TARGETS) *~

        $(INSTALL) hello $(bindir)

The next decision to make is how to provide the source code: it might either be available through some download mechanism, possibly from a local CVS server, or it might be added as a local tarball to the package file.

Based on that, the actual package recipe file is pretty simple:

DESCRIPTION = "Hello world example"
SECTION = "base"

PR = "r0"

SRC_URI = "file://hello-${PV}.tar.gz"

# just don't do any configuring
do_configure() {

The recipes shown here were all pretty simple. But actually 90% of the recipes in OpenEmbedded are not much more complex. And for more complex packages normally some recipes already exist, if not for exactly the wanted package then for a similar one.

And for the really complicated cases the OpenEmbedded developers on the mailing list are always helpful.

4. Conclusion

Most embedded Linux systems currently follow the full image approach: if something changes, the complete image is rebuilt and deployed.

An embedded Linux distribution that provides a package system follows a different approach: the original image provides only a base system that is augmented incrementally by separate packages.

OpenEmbedded provides not only such a package system, but also the tools to build these packages, i.e. the BitBake build tools and all the metadata in form of predefined classes for most common tasks for building an embedded Linux distribution.

And OpenEmbedded comes with lots of ready-to-use package recipes for Open Source tools, libraries and applications.

But OpenEmbedded has also drawbacks: It is quite complex and though this complexity is often hidden in the provided classes, it is sometimes necessary to understand that complexity. And though most package recipes are quite simple, even these simple must be learned, and documentation is a bit scarce. But the OpenEmbedded developers on the mailing list are generally friendly and willing to provide some pointers to solve simple and complex tasks.

Another drawback is the amounts of resources required to build OpenEmbedded: to build a basic distribution including a GUI takes several hours; to build everything takes nearly two days on a Pentium M @ 2GHz. And it takes about 30GHz disk space.

A last drawback is the SCM monotone used by OpenEmbedded: pulling and updating is quite slow.

Some of these drawbacks are just due to the fact that OpenEmbedded now provides a huge repository of recipes: to build one packages and its dependencies, OpenEmbedded must parse all recipes to know which recipe provides what, and with more than 3000 recipes this takes some time. But the OpenEmbedded developers are aware especially of the performance problems (they are bitten themselves most by them) and try to solve at least some of them.


OpenEmbedded Homepage.

Developer Documentation.

OpenEmbedded recipe hints.

BitBake manual.


OpenEmbedded monotone hints.