MikroTik Solutions

Changes To Container Limitations
Login

Changes To Container Limitations

Next Previous

Changes to "Container Limitations" between 2024-07-25 03:02:55 and 2024-07-25 04:02:22

︙
20
21
22
23
24
25
26













27



28

29
30
31
32
33
34
35

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51








+
+
+
+
+
+
+
+
+
+
+
+
+

+
+
+
-
+








RouterOS's `container.npk` lacks:

*   a local image cache(^One knock-on effect of this not covered above is that removing and reinstalling a container requires RouterOS to re-download the image, even when done back-to-back, even if you never start the container between and thereby cause it to make changes to the expanded image's files.)
*   a CoW/overlay file system(^This is not a verified fact, but an inference based on the observation that if RouterOS _did_ have this facility underlying its containers, I would expect to find equivalents to Docker's `commit` and `diff` commands. This pairs with the lack of an image cache: no CoW means no need for a baseline source to compute deltas against.)
*   image building
*   orchestration
*   JSON and REST APIs
*   per-container limit controls:(^The only configurable resource limit is on maximum RAM usage, and it's global, not settable on a per-container basis.)
    *   FD count
    *   PID limit
    *   CPU usage
    *   storage IOPS
    *   `/dev/shm` size limit
    *   terminal/logging bps
    *   syscall restrictions
    *   [capability][caps] restrictions
    *   [rlimit]
*   hardware pass-thru:
    *   USB device entries under `/dev` are on the wish list, but not currently available.
    *   There is no GPU support, not even for bare-metal x86 installs.

[caps]:   https://www.man7.org/linux/man-pages/man7/capabilities.7.html
[rlimit]: https://www.man7.org/linux/man-pages/man2/getrlimit.2.html

A good many of the `container.npk` limitations stem from those of RouterOS itself. For instance, while RouterOS proper is built atop Linux, and it provides a feature-rich CLI, it is nothing like a Linux command shell. This means equivalent commands to the likes of "`docker run --attach std…`" would not make a lot of sense on RouterOS, there being nothing like the termios/pty subsystem visible at the RouterOS CLI level.
A good many of these limitations stem from those of RouterOS itself. For instance, while RouterOS proper is built atop Linux, and it provides a feature-rich CLI, it is nothing like a Linux command shell. This means equivalent commands to the likes of "`docker run --attach std…`" would not make a lot of sense on RouterOS, there being nothing like the termios/pty subsystem visible at the RouterOS CLI level.

While I could also point out the lack of a background management daemon(^`containerd` in modern setups, `dockerd` in old ones) a good bit of Docker's competition also lacks this, on purpose, so I cannot ding RouterOS for this same lack.

With this grounding, let us get to the per-command details…


# <a id="top"></a>Top-Level Commands
︙
60
61
62
63
64
65
66
67



68
69
70
71

72


73


74




















75
76
77
78


79
80
81
82
83
84
85

76
77
78
79
80
81
82

83
84
85
86
87
88
89
90

91
92
93
94
95

96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128








-
+
+
+




+
-
+
+

+
+
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+




+
+








## <a id="compose"></a>`compose`

RouterOS completely lacks multi-container orchestration features, including lightweight single-box ones like [Compose](https://docs.docker.com/compose/) or [Kind](https://kind.sigs.k8s.io) virtual clusters.


## <a id="cp"></a>`cp`

There is no direct equivalent of this command. The closest RouterOS comes is when you mount a volume, then use the regular `/file` facility to copy files in under that volume's mount point. There is no direct way to copy a file into the container proper, as you might when overwriting a stock config file.
RouterOS does let you mount a volume inside a container, then use the regular `/file` facility to copy files in under that volume's mount point, but this is not at all the same thing as the "`docker cp`" command. There is no way to overwrite in-container files with external data short of rebuilding the container or using in-container mechanisms like `/bin/sh` to do the copying for you.

If you come from a Docker or Podman background, their local overlay image stores might lead you into thinking you could drill down into the GUID-named "container store" directories visible under `/file` and perform _ad hoc_ administration operations like overwriting existing config files inside the container, but alas, it does not.


## <a id="create"></a>`create`/`load`/`run`

RouterOS combines these commands as `/container/add`, though with many limitations, most of which stem from the list of [generalities above](#general). For instance, the lack of any CPU usage limit features means there is no equivalent under `/container` for the several `docker create --cpu*` options. Rather than go into these one by one, I'll add a few notes on matters you cannot glean by a careful reading of the rest of this article:
The RouterOS command `/container/add` provides a basic version of this, though with many limitations relative to a fully-featured container engine:

*   **`--env`**: The equivalent is this RouterOS command pair:

        /container/envs/add name=NAME …
        /container/add envlist=NAME …
<font color=red>**TODO**</font>

*   **`--expose`/`--publish`**: Every TCP/UDP port published by the container is exposed to the VETH you attach the container to. It is left up to you to manually block off anything the container exposes against your wishes by use of `/ip/firewall/filter` commands.

*   **`--health-cmd`**: Because health-checks are often implemented by periodic API calls to verify that the container continues to run properly, the logical equivalent under RouterOS is to [script] calls to [`/fetch`](https://help.mikrotik.com/docs/display/ROS/Fetch), which then issues `/container/{stop,start}` calls to remediate any problems it finds.

*   **`--init`**: Although there is no direct equivalent, nothing stops you from doing it the old-school way, creating a container that calls "`ENTRYPOINT /sbin/init`" or similar, which then starts the subordinate services inside that container.

*   **`--label`**: The closest equivalent is RouterOS's `comment` facility, which you can apply to a running container with "`/container/set 0 comment=MYLABEL`".

*   **`--mac-address`**: If RouterOS had this, I would expect it to be offered as "`/interface/veth/set mac-address=…`", but that does not currently exist. As it stands, a VETH interface's MAC address is random, same as the default behavior of Docker.

*   **`--network`**: This one is tricky. While there is certainly nothing like "`/container/add network=…`", it's fair to say the equivalent is, "RouterOS." You are, after all, running this container atop a highly featureful network operating system. Bare-bones the `container.npk` runtime may be, but any limitations you run into with the network it attaches to are more a reflection of your imagination and skill than to lack of command options under `/container`.

*   **`--pid/userns/uts`**: The RouterOS container runner must use Linux namespaces under the hood, but it does not offer you control over which PID, file, network, etc. namespaces each container uses.

*   **`--read-only`**: RouterOS offers precious little in terms of file system permission adjustment. As a rule, it is best to either shell into the container and adjust permissions there or rebuild the container with the permissions you want from go. Any expectations based on being able to adjust any of this between image download time and container creation time are likely to founder.

*   **`--restart`**: The closest RouterOS gets to this is its `start-on-boot` setting, meaning you'd have to reboot the router to get the container to restart. If you want automatic restarts, you will have to [script] it.

*   **`--rm`**: No direct equivalent. There is a manual `/container/remove` command, but nothing like this option, which causes the container runtime to automatically remove the instantiated container after it exits. It's just as well since this option is most often used when running _ad hoc_ containers made from a previously downloaded image; RouterOS's lack of an image cache means you have to go out of your way to export a tarball of the image and upload it to the router, then use "`/container/add file=…`" if you want to avoid re-downloading the image from the repository on each relaunch.

RouterOS doesn't have separate top-level commands for creating a container from an OCI image registry versus loading it from a tarball. They're both `/container/add`, differing in whether you give the `remote-image` or `file` options, respectively.

RouterOS has no shorthand command like `docker run` for creating and starting a container in a single step. You must `add` it, then `start` it.

[script]: https://help.mikrotik.com/docs/display/ROS/Scripting


## <a id="diff"></a>`diff`

With no local image cache, there can be no equivalent command.
︙
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160

176
177
178
179
180
181
182














183
184
185
186
187
188
189








-
-
-
-
-
-
-
-
-
-
-
-
-
-









    /container/print detail where …

You get only a few lines of information back from this, mainly what you gave it to create the container from the image. You will not get the pages of JSON data the Docker CLI gives.

A related limitation is that the configurable items are often global in RouterOS, set for all containers running on the box, not available to be set on a per-container basis. A good example of this is the memory limit, set via `/container/config/set ram-high=…`.

Atop this, there aren't even _global_ settings in RouterOS for:

*   FD limits
*   PID limits
*   maximum CPU usage
*   syscall restrictions
*   [capabilities][caps]
*   [rlimit]

…much less per-container settings as you get in Docker, Podman, LXC, etc.

[caps]:   https://www.man7.org/linux/man-pages/man7/capabilities.7.html
[rlimit]: https://www.man7.org/linux/man-pages/man2/getrlimit.2.html


## <a id="kill" name="stop"></a>`kill`/`stop`

RouterOS doesn't make a distinction between "kill" and "stop". The `/container/stop` command behaves more like `docker kill` or `docker stop -t0` in that it doesn't try to bring the container down gracefully before giving up and killing it.


## <a id="login"></a>`login`/`logout`
︙