LXC containers

LXC, or Linux Containers, is a virtualization technology that enables the execution of multiple isolated Linux systems (containers) on a single host. Unlike traditional virtual machines that emulate entire hardware stacks, LXC specializes in operating system-level virtualization. It provides a lightweight environment that shares the host’s kernel while ensuring process and network isolation.

LXC uses Linux’s cgroups functionality to allow the host CPU to better partition memory allocation into isolation levels called namespaces. This isolation enabled by cgroups is the basis of LXC containers.

For information on how LXC compares to Docker, see Comparison between Docker and LXC

Configure your project for LXC support

Follow these steps to build an image with LXC support:

Edit your project’s conf/bblayers.conf configuration file and add meta-virtualization and meta-filesystems layers:

conf/bblayers.conf

   /usr/local/dey-4.0/sources/meta-digi/meta-digi-arm \
   /usr/local/dey-4.0/sources/meta-digi/meta-digi-dey \
+  /usr/local/dey-4.0/sources/meta-virtualization \
+  /usr/local/dey-4.0/sources/meta-openembedded/meta-filesystems \
"

Include virtualization in your project. Edit conf/local.conf, and add the following:
conf/local.conf
```
DISTRO_FEATURES:append = " virtualization"
IMAGE_INSTALL:append = " lxc"
```
Note the required white space when appending a value to an array variable using the :append override syntax.

Build image with LXC support

Once you have configured your project for LXC support, build the image. For example:

$ bitbake dey-image-qt

Bitbaking an image recipe implies downloading and building the source code of all the recipes that form part of the root file system, which takes several hours the first time. Some source code repositories, such as the Linux kernel, represent a large download that might time out and make your build process fail. If this happens, run the following command to just fetch the source code of the offending recipe separately (to dedicate all CPU resources to it):

$ bitbake -k --runall=fetch <image-recipe>

When this task finishes successfully (you may need several retries), you can proceed to build your image recipe. Do the same with any recipe that fails with a timeout during the fetch operation.

Program the resulting images as explained in Update firmware.

Configure LXC

There are two main configuration files for LXC containers:

/etc/lxc/lxc.conf: this is the system configuration file. It is used to set values such as default lookup paths and storage backend settings for LXC. For reference, see https://linuxcontainers.org/lxc/manpages/man5/lxc.system.conf.5.html
/etc/lxc/default.conf: contains configuration options that apply to every container created in the target device. These configurations are appended to each container specific configuration file. For reference, see https://linuxcontainers.org/lxc/manpages/man5/lxc.container.conf.5.html

Change the default path for containers

Each container is stored on a folder of its own in /var/lib/lxc/ by default. You can change the default path for containers by adding the following line to /etc/lxc/lxc.conf (create the file if it doesn’t exist):

/etc/lxc/lxc.conf

lxc.lxcpath = <your-path>

where <your-path> is a path of your choice.

On devices with little storage space, storing containers on the root file system may not be viable due to storage limitations. In this case, Digi recommends changing the default path to external storage media such as a USB stick or a microSD card.

Create a minimal BusyBox LXC container

A BusyBox container can be ideal for environments where minimal overhead is crucial.

Use the lxc-create command to create the container:

# lxc-create --name my-bb-container --template busybox

Container file structure

Each LXC container is stored by default in a folder of its own in /var/lib/lxc/ by default, or in the path specified on /etc/lxc/lxc.conf.

# ls /var/lib/lxc/
/var/lib/lxc/my-bb-container/

Each container has:

a config file that contains the container configuration
a rootfs folder that contains the container root file system

# ls /var/lib/lxc/my-bb-container/
/var/lib/lxc/my-bb-container/config
/var/lib/lxc/my-bb-container/rootfs

Start the container

To start the BusyBox container, run:

# lxc-start -n my-bb-container -F

The -F flag attaches the container in foreground mode so it spawns the sbin/init process and you can use it immediately. This is required in the BusyBox container or else the container stops right after starting.

You can type exit to quit the container and go back to the native device.

Work with containers

The following list contains common LXC commands.

If you created/stored the container on a different path than the default /var/lib/lxc or the one specified at /etc/lxc/lxc.conf, you must add -P <my-container-path> to specify the path on every command.

List available containers

# lxc-ls -f

Start a container

# lxc-start -n <container-name>

Start a container and let it run a specific command

# lxc-start -n <container-name> -- <command>

Attach a container

# lxc-attach -n <container-name>

Stop a container

# lxc-stop <container-name>

Destroy a container

# lxc-destroy -n <container-name>

Create a container from a pre-built image

https://images.linuxcontainers.org/ is a library of container images that contains a variety of distributions, releases, and architectures.

Creating a container from a prebuilt image does the following:

Downloads the prebuilt image to a temporary folder
Moves the downloaded image to a cache folder for future use
Uncompresses the image root file system to the container specific root file system folder

If your device does not have enough free storage space to hold the image and the container, you may need to create the container in the host computer and copy it to the target (possibly to another partition or external storage media).

Create the container on the target device

If you have enough storage space, you can create the container directly on the target. To create a container based on, for example, the Debian "buster" release, run:

$ lxc-create --name my-debian-container --template download -- --dist debian --release buster --arch arm64

Create the container on the host computer

If you don’t have enough space on the device, or if you prefer to create the container on your host computer where you have more tools and resources, follow these instructions:

On the host computer, you need to run most commands as root, prepending sudo.
When you create a container on the host computer, the settings that apply are those on your host computer’s /etc/lxc/default.conf file, not the ones on your device. This sometimes requires changing the container configuration.

On the host computer, install lxc if not already available, for instance:
```
$ sudo apt-get install lxc
```

To create a container based on, for example, the Debian "buster" release, run:

$ sudo lxc-create --name my-debian-container --template download -- --dist debian --release buster --arch arm64

Check the container directory:

$ sudo ls /var/lib/lxc/my-debian-container
config  rootfs

[Optional step] If you want to install additional packages (or remove others), start the container while in the host computer, and install anything you need. Should you require Ethernet access, refer to Share Ethernet.

When you’re done, stop the container and return to the host computer system.

Compress the container to more easily transfer it to the device.

$ cd /var/lib/lxc/
$ sudo tar cfvz mydebcontainer.tgz my-debian-container

Copy the compressed container to the target. For instance, to copy over the network to a path of your choice:
```
$ scp mydebcontainer.tgz root@192.168.42.30:/<my-container-path>
```

On the target, decompress the container.

# cd <my-container-path>
# tar xfvz mydebcontainer.tgz

Edit the container configuration file and make the following changes:

change the container’s rootfs path to your selected path

remove the default network configuration settings (they create a bridge that doesn’t work)

<my-container-path>/my-debian-container/config

 lxc.arch = linux32

 # Container specific configuration
-lxc.rootfs.path = dir:/var/lib/lxc/my-debian-container/rootfs
+lxc.rootfs.path = dir:<my-container-path>/my-debian-container/rootfs
 lxc.uts.name = my-debian-container

 # Network configuration
-lxc.net.0.type = veth
-lxc.net.0.link = lxcbr0
-lxc.net.0.flags = up
-lxc.net.0.hwaddr = 00:16:3e:3c:17:15

See Work with containers for instructions on starting and attaching the container.

You can define specific resources of the native device for your containers to use, such as network interfaces or GPIO pins. This allows for precise control over what resources a container can use.

The following example shares GPIO3 port (/dev/gpiochip2) of the native device with a container.

List the device on the native system to see its type (char, block…) and its major/minor numbers:
```
# ls -la /dev/gpiochip2
crw-rw----    1 root     digiapix  254,   2 Apr 28 17:42 /dev/gpiochip2
```
Add the following lines to the container’s configuration file:
<container-path>/<container-name>/config
```
lxc.cgroup.devices.allow = c 254:2 rwm
lxc.mount.entry = /dev/gpiochip2 dev/gpiochip2 none bind,create=file
```
- The first line grants read, write, and mknod permissions (rwm) to character devices (c) with major number 254 and minor number 2 (254:2).
- The second line binds the native device node (/dev/gpiochip2) to the same path on the container (note the omitted leading slash: dev/gpiochip2), making the device file accessible to processes running in the container.

With these changes, the GPIO port is accessible when you restart the container.

You can use the same logic for /dev/fb0, /dev/video0, or any other node.

Copy files to the container

Basic containers, such as the BusyBox one, may be missing tools to access the GPIO pins. On more complex containers, such as the Debian one, you can use the package manager to install additional packages. However, it is important to note that files from the native device can be copied to the container rootfs and therefore be immediately available for use in the container.

For instance, copy the libgpiod applications and libraries from the native device to the container’s rootfs:

# cp /usr/bin/gpio* <container-path>/<container-name>/rootfs/usr/bin
# cp /usr/lib/libgpiod* <container-path>/<container-name>/rootfs/usr/lib

On the minimal BusyBox container, copying the libraries is not needed cause the configuration file for the container binds the /usr/lib folder of the native device with the container, so all the libraries are available in the container.

You can now use libgpiod applications to access GPIO pins on the shared port. See Using the GPIOs, for reference.

To share Ethernet between the native device and an LXC container, you can either share the native Ethernet or use a bridge.

This setup configures the container to directly share the native device’s primary network interface.

Edit the config file and set the network type to none (the default value empty does not share the network):
<container-path>/<container-name>/config
```
lxc.net.0.type = none
```
Restart the container (stop > start > attach) to apply the network changes.

The container now has network access.

This approach is simpler but doesn’t isolate the network between the native device and the container. As a consequence, if the container brings down the network (for example, when you stop the container), the native device’s network will also be brought down.

The recommended method is to use a bridge.

Use a bridge

This method creates a bridge on the native device and attaches the container’s virtual network interface to this bridge.

On the target, create a bridge named br0, attach the native device’s physical network interface eth0 to it, and obtain a dynamic IP:

# ip link add name br0 type bridge
# ip link set br0 up
# ip addr flush dev eth0
# ip link set eth0 master br0
# ifconfig br0 up
# udhcpc -i br0

Edit the container’s config file and set the network as follows:
/var/lib/lxc/debian/config
```
lxc.net.0.type = veth
lxc.net.0.link = br0
lxc.net.0.flags = up
lxc.net.0.name = eth0
lxc.net.0.hwaddr = XX:XX:XX:XX:XX:XX
```
where XX:XX:XX:XX:XX:XX is the MAC address of the container’s network interface, and must be different than the MAC of the native Ethernet device.
Restart the container (stop > start > attach) to apply the network changes.

The container now has network access, isolated from the native device.