| ︙ | | |
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
|
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
|
-
-
+
+
-
+
|
The single biggest area of difference between the likes of Docker and the RouterOS `container.npk` feature is how you create containers from [OCI] images. It combines Docker’s `create` and `load` commands under `/container/add`, the distinction expressed by whether you give it the `remote-image` or `file` option, respectively.
Given the size of the output from `docker create --help`, it should not be surprising that the bulk of that is either not available in RouterOS or exists in a very different form. Most of these limitations stem from [the list above](#global). For instance, the lack of any CPU usage limit features means there is no equivalent under `/container` for the several `docker create --cpu*` options. Rather than go into these options one by one, I’ll cover the ones where the answers cannot be gleaned through a careful reading of the rest of this article:
* **`--env`**: The equivalent is this RouterOS command pair:
/container/envs/add name=NAME…
/container/add envlist=NAME…
/container/envs/add name=NAME …
/container/add envlist=NAME …
This is in fact closer to the way the **`--env-file`** option works, except that under RouterOS, this particular “file” isn’t stored under `/file`!
* **`--expose`/`--publish`**: <a id="publish"></a>The VETH you attach the container to makes every listening socket visible by default; the `EXPOSE` directive given in your `Dockerfile` is completely ignored. Everything the big-boy container engines do related to this is left up to you, the RouterOS administrator, to do manually:
* block **unwanted** services exposed within the container with `/ip/firewall/filter` rules
* port-forward **wanted** services in via `dstnat` rules
* **`--health-cmd`**: Because health-checks are often implemented by periodic API calls to verify that the container continues to run properly, the logical equivalent under RouterOS is to [script] calls to [`/fetch`](https://help.mikrotik.com/docs/display/ROS/Fetch), which then issues `/container/{stop,start}` calls to remediate any problems it finds.
* **`--init`**: Although there is no direct equivalent to this in RouterOS, nothing stops you from doing it the old-school way, creating a container that calls “`ENTRYPOINT /sbin/init`” or similar, which then starts the subordinate services inside that container. It would be somewhat silly to use systemd for this in a container meant to run on RouterOS in particular; a more suitable alternative would be [Alpine’s OpenRC](https://wiki.alpinelinux.org/wiki/OpenRC) init system, a popular option for managing in-container services.
* **`--label`**: The closest equivalent is RouterOS’s `comment` facility, which you can apply to a running container with “`/container/set 0 comment=MYLABEL`”.
* **`--mac-address`**: If RouterOS had this, I would expect it to be offered as “`/interface/veth/set mac-address=…`”, but that does not currently exist. As it stands, a VETH interface’s MAC address is random, same as the default behavior of Docker.
* **`--mount`**: The closest equivalent to this in RouterOS is quite different, being the `/container/mounts/add` mechanism. The fact that you create this ahead of instantiating the container might make you guess this to be a nearer match to a “`docker volume create…`” command, but alas, there is no container volume storage manager. In Docker-speak, RouterOS offers bind-mounts only, not separately-managed named volumes that only containers can see.
* **`--mount`**: The closest equivalent to this in RouterOS is quite different, being the `/container/mounts/add` mechanism. The fact that you create this ahead of instantiating the container might make you guess this to be a nearer match to a “`docker volume create …`” command, but alas, there is no container volume storage manager. In Docker-speak, RouterOS offers bind-mounts only, not separately-managed named volumes that only containers can see.
Atop this, `container.npk` can bind-mount whole directories only, not single files as Docker and Podman allow. This can be a particular problem when trying to inject a single file under `/etc` since it tends to require that you copy in all of the “peer” files in that same subdirectory hierarchy merely to override one of them.
* **`--network`**: This one is tricky. While there is certainly nothing like “`/container/add network=…`”, it’s fair to say the equivalent is, “RouterOS.” You are, after all, running this container atop a highly featureful network operating system. Bare-bones the `container.npk` runtime may be, but any limitations you run into with the network it attaches to are more a reflection of your imagination and skill than to lack of command options under `/container`.
* **`--pid/uts`**: The RouterOS container runner must use Linux namespaces under the hood, but it does not offer you control over which PID, file, network, user, etc. namespaces each container uses. See also [this](#root).
|
| ︙ | | |
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
|
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
|
-
+
-
+
-
+
-
+
|
sh-5.1# exit
…may end up expressed under RouterOS as…
> /container
> add remote-image=alpine:latest veth=veth1 entrypoint=sleep cmd=3600
> print
… nope, still downloading, wait…
… nope, still downloading, wait…
> print
… nope, still extracting, wait longer…
… nope, still extracting, wait longer…
> print
… oh, good, got the container ID…
… oh, good, got the container ID…
> start 0
… wait for it to launch…
… wait for it to launch…
> shell 0
sh-5.1# <do something inside the container>
sh-5.1# exit
> stop 0
> remove 0
Whew! 😅
|
| ︙ | | |
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
|
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
|
-
+
-
+
-
+
-
+
|
But how then do you say something akin to the following under RouterOS?
docker run -it alpine:latest ls -lR /etc
You might want to do that in debugging to find out what a given config file is called and exactly where it is in the hierarchy so that you can target it with a `mount=…` override. If you try to pass it all as…
/container/add… entrypoint="ls -lR /etc"
/container/add … entrypoint="ls -lR /etc"
…the kernel will complain that there is no command in the container’s `PATH` called “`ls -lR /etc`”.
You may then try to split it as…
/container/add… entrypoint="ls" cmd="-lR /etc"
/container/add … entrypoint="ls" cmd="-lR /etc"
…but that will earn you a refusal by `/bin/ls` to accept “ ” (space) as an option following the `R`!
If you get cute and try to “cuddle” the options with the arguments as…
/container/add… entrypoint="ls" cmd="-lR/etc"
/container/add … entrypoint="ls" cmd="-lR/etc"
…the `/bin/ls` implementation will certainly attempt to treat `/` as an option and die with an error message.(^Yes, for certain. I tested the GNU, BSD, _and_ BusyBox implementations of `ls`, and they all do this.)
Things aren’t always this grim. For instance, you can run [my `iperf3` container](/dir/iperf3) as a client instead of its default server mode by saying something like:
/container/add… cmd="-c192.168.88.99"
/container/add … cmd="-c192.168.88.99"
This relies on the fact that the `iperf3` command parser knows how to break the host name part out from the `-c` option itself, something not all command parsers are smart enough to do. There’s 50 years of Unix and Linux history encouraging programs to rely on the shell to do a lot of work before the program’s `main()` function is even called. The command line processing that `container.npk` applies to its `cmd` argument lacks all that power. If you want Bourne shell parsing of your command line, you have to set it via `ENTRYPOINT` or `CMD` in the `Dockerfile`, then rebuild the image.
There is one big exception to all this: a common pattern is to have the `ENTRYPOINT` to a container be a shell script and for that to do something like this at the end:
/path/to/actual/app $@
|
| ︙ | | |
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
|
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
|
-
+
|
# <a id="logs"></a>Log Handling
Although Docker logging is tied into this same Linux terminal I/O design, we cannot blame the lack of an equivalent to “`docker logs`” on the RouterOS design principles in the same manner as [above](#terminal). The cause here is different, stemming first from the fact that RouterOS boxes try to keep logging to a minimum by default, whereas Docker logs everything the container says, without restriction. RouterOS takes the surprising default of logging to volatile RAM in order to avoid burning out the flash. Additionally, it ignores all messages issued under “topics” other than the four preconfigured by default, which does not include the “container” topic you get access to by installing `container.npk`.
To prevent your containers’ log messages from being sent straight to the bit bucket, you must say:
/container/{add,set}… logging=yes
/container/{add,set} … logging=yes
/system/logging add topics=container action=…
Having done so, we have a new limitation to contend with: RouterOS logging isn’t as powerful as the Docker “`logs`” command, which by default works as if you asked it, “Tell me what this particular container logged since the last time I asked.” RouterOS logging, on the other hand, mixes everything together in real time, requiring you to dig through the history manually.
(The same is true of `podman logs`, except that it ties into systemd’s unified “journal” subsystem, a controversial design choice that ended up paying off handsomely when Podman came along and wanted to pull up per-container logs to match the way Docker behaved.)
|
| ︙ | | |
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
|
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
|
-
+
|
This shows up in a number of guises, but the overall effect is that all containers run as a nerfed `root` user under `container.npk`, same as Docker did from the start. This remains the Docker default, but starting with the 20.10 release, it finally got a [rootless mode][drl] to compete with [Podman’s rootless-by-default][prl] nature. I bring up this history to show that RouterOS is not unconditionally “wrong” to operate as it does, merely limited.
This design choice may be made reasonably safe through the grace of [user namespaces](https://www.man7.org/linux/man-pages/man7/user_namespaces.7.html), which cause the in-container `root` user to be meaningfully different from the Linux `root` user that RouterOS itself runs as. RouterOS does have a `/user` model, but they are not proper Linux users as understood by the kernel, with permissions enforced by Linux user IDs; RouterOS users have _no meaningful existence at all_ inside the container. One practical effect of this is that when you start a container as RouterOS user `fred`, you will not find a `fred` entry in its `/etc/passwd` file, and if you create one at container build time (e.g. with a `RUN useradd` command) it will not be the same `fred` as the RouterOS user on the outside.
Files created by that nerfed `root` user will show up as owned by `root` when using bind-mounted directories on file systems like `ext4` which preserve file ownership. One possible solution for this is:
/disk/format-drive file-system=exfat…
/disk/format-drive file-system=exfat …
It is because of this same limitation that there is no RouterOS equivalent to the `create --user*` or `--group-add` flags.
If your container was designed to have non-root users inside with meaningful distinctions from root, it may require massaging to work on RouterOS. There are no UID maps to convert in-container user IDs to RouterOS user IDs, etc. This is one of the key reasons why it matters that [containers are not VMs][cvm]; persisting in this misunderstanding is liable to lead you to grief under `container.npk`. Let go of your preconceptions and use the RouterOS container runner the way it was meant to be applied: running well-focused single services.(^This philosophy is not specific to RouterOS, nor is it special pleading on its behalf, meant to justify its limitations. [Microservices][msc] are good idea atop _all_ container runtimes.)
[cvm]: /wiki?name=Containers%20Are%20Not%20VMs
[drl]: https://docs.docker.com/engine/security/rootless/
|
| ︙ | | |
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
|
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
|
-
+
|
## <a id="info" name="inspect"></a>`info`/`inspect`
With the understanding that RouterOS has far fewer configurables than a big-boy container engine, the closest commands in RouterOS are:
* `/container/config/print`
* `/container/print detail where…`
* `/container/print detail where …`
* `:put [:serialize value=[/container/get 0] to=json options=json.pretty]`
That last one was crafted by @Nick on the [MikroTik Discord][MTDisc]. It gives a pretty-printed JSON version of what you get from the second command, which is useful when automating `/container` commands via SSH, as with Ansible. Even so, it’s far short of the pages and pages of detail you get from the Docker and Podman CLI equivalents.
A related limitation is that configurable parameters are often global in RouterOS, set for all containers running on the box, not available to be set on a per-container basis. A good example of this is the memory limit, set via `/container/config/set ram-high=…`.
[MTDisc]: https://discord.gg/exGj6whYw7
|
| ︙ | | |
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
|
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
|
-
+
-
+
-
+
|
## <a id="update"></a>`update`
There is no equivalent short of this:
/container/stop 0
…wait for it to stop…
…wait for it to stop…
/container/remove 0
/container/add…
/container/add …
The last step is the tricky one since `/container/print` shows most but not all of the options you gave to create it. If you didn’t write down how you did that, you’re going to have to work that out to complete the command sequence.
## <a id="version"></a>`version`
While RouterOS’s `container.npk` technically does have an independent version number of its own, it is meant to always match that of the `routeros.npk` package you have installed. RouterOS automatically upgrades both in lock-step, making this the closest equivalent command:
/system/package/print
## <a id="wait"></a>`wait`
The closest equivalent to this would be to call `/container/stop` in a RouterOS script and then poll on `/container/print where…` until it stopped.
The closest equivalent to this would be to call `/container/stop` in a RouterOS script and then poll on `/container/print where …` until it stopped.
# <a id="license"></a>License
This work is © 2024-2025 by Warren Young and is licensed under <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank" rel="license noopener noreferrer">CC BY-NC-SA 4.0<img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/nc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1"></a>
<div style="height: 50em" id="this-space-intentionally-left-blank"></div>
|