1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
|
+
-
+
-
+
-
+
+
-
+
-
+
-
+
-
+
-
+
-
-
|
# Motivation
The [RouterOS `container.npk` feature](https://help.mikrotik.com/docs/display/ROS/Container) is highly useful, but it is a custom development written in-house by MikroTik, not a copy of Docker Engine or any of the other server-grade container engines.(^Podman, LXC/LXD, etc.) Because of the stringent resource constraints on the bulk of MikroTik's devices, it is exceptionally small, thus unavoidably very thinly featured compared to its big-boy competition. If we can use installed size as a proxy for expected feature set size, we find:
* **Docker Engine**: 422 MiB(^Version 27.1.1, according to `dnf remove docker-ce…` after installing these packages [per the instructions](https://docs.docker.com/engine/install/rhel/#install-docker-engine).)
* **`containerd`+`nerdctl`**: 174 MiB(^Version 2.0.0-rc1 of `nerdctl` plus the `containerd` from the Docker Engine CE install according to `sudo dnf remove containerd` and `du -sh nerdctl`.)
* **Podman**: 107 MiB(^Version 4.9.4 on EL9, according to `sudo dnf remove podman conmon crun`.)
* **systemd-nspawn**: 1.3 MiB(^This is the bare-bones OCI image runner built into systemd, with a feature set fairly close to that of `container.npk`. The size above is for version 252 of this program's parent `systemd-container` package as shipped on EL9.)
* **`container.npk`**: _0.0626 MiB_(^Version 7.15.2, according to `/system/package/print`.)
And this is fine! RouterOS serves a particular market, and its developers are working within those constraints. The intent here is to provide a mapping between what people expect of a fully-featured container engine and what you actually get in RouterOS. Where it makes sense, I try to provide workarounds for missing features and guidance to alternative methods where RouterOS's way merely *works* differently.
<font color=red>This document is a **Work in Progress**.</font>
# General Observations
# <a id="general"></a>General Observations
Allow me to present a distilled version of the details below, both to satisfy the **tl;dr** crowd and to set broad expectations for the rest of my readers.
Allow me to begin with a distilled version of the details below, both to satisfy the **tl;dr** crowd and to set broad expectations for the rest of my readers.
RouterOS's `container.npk` lacks:
* a local image cache
* a local image cache(^One knock-on effect of this not covered above is that removing and reinstalling a container requires RouterOS to re-download the image, even when done back-to-back, even if you never start the container between and thereby cause it to make changes to the expanded image's files.)
* a CoW/overlay file system(^This is not a verified fact, but an inference based on the observation that if RouterOS _did_ have this facility underlying its containers, I would expect to find equivalents to Docker's `commit` and `diff` commands. This pairs with the lack of an image cache: no CoW means no need for a baseline source to compute deltas against.)
* image building
* orchestration
* JSON and REST APIs
A good many of the `container.npk` limitations stem from those of RouterOS itself. For instance, while RouterOS proper is built atop Linux, and it provides a feature-rich CLI, it is nothing like a Linux command shell. This means equivalent commands to the likes of "`docker run --attach std…`" would not make a lot of sense on RouterOS, there being nothing like the termios/pty subsystem visible at the RouterOS CLI level.
While I could also point out the lack of a background management daemon(^`containerd` in modern setups, `dockerd` in old ones) a good bit of Docker's competition also lacks this, on purpose, so I cannot ding RouterOS for this same lack.
With this grounding, let us get to the per-command details…
# Top-Level Commands
# <a id="top"></a>Top-Level Commands
For lack of any better organization principle, I've chosen to structure this document along the lines of the `docker` CLI, duplicating their command hierarchy, sorted alphabetically at each level. I skip over short aliases like `docker rmi` for `docker image rm` in order to cover things only once. Because Podman cloned the Docker CLI, this matches fairly well with it, except that I do not currently go into any of its pure extensions, like its eponymous `pod` command.
## <a id="attach"></a>`attach`
There is no interactive terminal (stdin/stdout/stderr) in RouterOS to speak of. Containers normally run in the background, with logging suppressed by default. If you say `/container/set logging=yes`, the standard output streams go to the configured logging destination, but there is no way to interactively type commands at the container short of `/container/shell`, which requires that `/bin/sh` exist inside the container. Even then, you're typing commands at the shell, not at the container's `ENTRYPOINT` process.
There is no interactive terminal (stdin/stdout/stderr) in RouterOS to speak of, and you normally run these boxes headless, connecting to their Terminal via WinBox or SSH only long enough to reconfigure something before logging back out. This means it is meant to run containers purely in the background, with logging suppressed by default. If you say `/container/set logging=yes`, the standard output streams go to the configured logging destination, but there is no way to interactively type commands at the container short of `/container/shell`, which requires that `/bin/sh` exist inside the container. Even then, you're typing commands at the shell, not at the container's `ENTRYPOINT` process.
In short, there is no equivalent in RouterOS to the common `docker run -it` invocation option.
RouterOS provides a bare-bones container runtime only, not any of the image build tooling. It is closer in nature to the `runc` command underlying `containerd` than to Docker Engine proper. An even closer match is the lightweight `crun` command at the heart of Podman, and even more so the elementary runner that ships with systemd, variously called either [`systemd-nspawn`][sdnsp] or [`systemd-container`][sdcnt], depending on the tastes of whoever is packaging it.
RouterOS provides a bare-bones container runtime only, not any of the image building toolchain. It is closer in nature to the `runc` command underlying `containerd` than to Docker Engine proper. An even closer match is the lightweight `crun` command at the heart of Podman, and even more so the elementary runner that ships with systemd, variously called either [`systemd-nspawn`][sdnsp] or [`systemd-container`][sdcnt], depending on the tastes of whoever is packaging it.
[sdcnt]: https://packages.fedoraproject.org/pkgs/systemd/systemd-container/
[sdnsp]: https://wiki.archlinux.org/title/Systemd-nspawn
## <a id="commit"></a>`commit`
RouterOS doesn't maintain an image cache, thus has no way to commit changes made to the current image layer to a new layer.
Given the [generalities above](#general), it should be no surprise that RouterOS has no way to commit changes made to the current image layer to a new layer.
It is for this same reason that removing and reinstalling the container re-downloads the image, even when done back-to-back, without starting the container and thereby making changes to the downloaded image.
|