Devices
Every container needs access to basic devices, such as /dev/null, /dev/zero,
TTYs, console, etc. Although access to these devices is managed through the
devices cgroup, manipulating the cgroup directly is tricky. You'd have to
add devices to the root group and then manually propagate them to child groups
and containers. To make device management and manipulation simpler, osctld
manages the devices itself. Use osctl ct/group devices instead of
osctl ct/group cgparams for device management.
Device access trees
Since every container needs access to a minimal set of devices, we're utilizing groups to provide a common set of devices to containers. Like with the devices cgroup, child groups can access only those devices that the parent group has access to. Child groups can only restrict access granted by the parent, not expand it. Therefore, every device that some container wants to use must be enabled in the root group and then in all groups, that are a direct or an indirect parent to that container.
Let's review the default device access list for the root group:
osctl group devices ls /
TYPE MAJOR MINOR MODE NAME INHERIT INHERITED
char 1 3 rwm /dev/null true -
char 1 5 rwm /dev/zero true -
char 1 7 rwm /dev/full true -
char 1 8 rwm /dev/random true -
char 1 9 rwm /dev/urandom true -
char 1 11 rwm /dev/kmsg true -
char 5 0 rwm /dev/tty true -
char 5 1 rwm - true -
char 5 2 rwm - true -
char 136 all rwm - true -
This is the minimal set of devices that all containers have to have access to.
Child groups will inherit all devices where the INHERIT column is true,
making them available to their own child groups and containers.
If we look at the default group, which is a direct descendant of the root group, we can see that all those devices were in fact inherited:
osctl group devices ls /default
TYPE MAJOR MINOR MODE NAME INHERIT INHERITED
char 1 3 rwm /dev/null true true
char 1 5 rwm /dev/zero true true
char 1 7 rwm /dev/full true true
char 1 8 rwm /dev/random true true
char 1 9 rwm /dev/urandom true true
char 1 11 rwm /dev/kmsg true true
char 5 0 rwm /dev/tty true true
char 5 1 rwm - true true
char 5 2 rwm - true true
char 136 all rwm - true true
osctld makes sure that all these listed devices are actually enabled in the devices cgroups and that the containers have corresponding device nodes created.
Adding a new device to all containers
If you wish to add a device to all containers, all you have to do is add it to the root group and make it inheritable. All child groups and containers will then be able to use it:
osctl group devices add --inherit / char 10 200 rw /dev/net/tun
Access to the device is permitted immediately, but the device node is created
the next time the container starts, when all the device nodes will be provided
by osctld. You can call mknod at any time though.
Adding a new device to one group/container
When adding a new device to a non-root group or a container, the device has to be provided by the parent groups, otherwise an error is reported:
osctl ct devices add myct01 char 10 229 rw /dev/fuse
error: device not available in group 'default'
You can either add the device to the parent groups manually, or you can use
the -p, --parents switch:
osctl ct devices add --parents myct01 char 10 229 r /dev/fuse
The device will then be enabled in all parent groups that do not already have it, but only the one selected container will have access to it. The device will not be automatically inherited to other groups or containers.
Since we have provided the device name, osctld has created the device node within the container:
osctl ct exec myct01 ls -l /dev/fuse
crw-r--r-- 1 root root 10, 229 Mar 4 14:48 /dev/fuse
To add the device to a group, simply replace osctl ct devices with
osctl group devices.
Inherited and promoted devices
When a device is marked as inheritable (i.e. the INHERIT column in
osctl ct/group devices ls is true), the device will automatically be
propagated to child groups and containers. Inherited devices can be provided
and taken away by the adminsitrator at any time. To declare an explicit
requirement of a device, or change its access mode, it has to be promoted.
The parent group then cannot simply remove the device, because its child group
or container has declared that it requires it.
Devices can be promoted by either by changing its access mode (see
below) or osctl ct/group device promote. To remove
the requirement declaration, i.e. revert promotion, the device can be inherited
again using osctl ct/group devices inherit. Note that if the parent group
does not have the device marked as inheritable, the device will be removed from
the group or container you're manipulating and all child groups and containers.
To set or unset the inheritability flag, i.e. decide whether child groups and
containers should automatically inherit a device, you can use
osctl group devices set/unset inherit.
Changing access mode
If you need to change access mode of an existing device, you can do so with
osctl ct/group devices chmod. By default, you can change the mode only when
it is safe, i.e.:
- no child group or container is using the device with a broader access mode
- all parent groups can provide the requested access mode
If you wish to override the child groups and containers, you can use switch
-r, --recursive, which will change all their modes as well. Similarly,
to update parent groups to provide necessary access mode, you can use switch
-p, --parents.
For example, we've created the /dev/fuse device above, but we've allowed only
read access to it. Let's add permission to write as well:
# This won't work, since both the default and root group only have `r`
osctl ct devices chmod myct01 char 10 229 rw
error: group 'default' provides only mode 'r'
# Add the missing permission to all parents:
osctl ct devices chmod --parents myct01 char 10 229 rw
Removing devices
Devices can be removed only if no group or container uses them:
osctl group devices del / char 10 229
error: device is used by child groups/containers, use recursive mode
To remove the device from all child groups and containers, you can use switch
-r, --recursive:
osctl group devices del --recursive / char 10 229
By removing the device from the root group, we have removed it from all child groups and all containers. If you remove a device from a non-root group or a specific container, the parent groups can still use it.
It is not possible to remove an inherited device, because, as it is designed,
the device would automatically get inherited again when the pool is imported
again. You can, however, restrict access to inherited devices using
osctl ct/group devices chmod. To deny access completely, you can use mode -.
Send/receive, export/import
Since we're using device inheritance, we cannot ensure that a container will have access to all expected devices when sent or imported to another node with different device hierarchy. osctld can check only for explicitly defined devices for the container, i.e. non-inherited devices.
When importing a container, osctld will by default check that all required
devices are present. If not, the import is aborted. You can use option
--missing-devices check|provide|remove to change this behaviour.
Migrations enforce the check mode. All required devices have to be present,
or the send will fail in the first step, i.e. osctl ct send config.
It is up to the administrator to configure his vpsAdminOS nodes to provide
the same environment for sends to work properly.