*   storage IOPS
    *   `/dev/shm` size limit
    *   terminal/logging bps
    *   [capability][caps] restrictions
    *   [seccomp profiles](https://docs.docker.com/engine/security/seccomp/)
    *   [rlimit]
*   hardware pass-thru:
    *   USB device entries under `/dev` are on the wish list, but not currently available.(^Not unless RouterOS itself sees the USB device, as with storage media, which you can bind-mount into the container with “`/container/add mounts=…`”.)
    *   USB and serial device entries under `/dev` are [on the wish list](https://forum.mikrotik.com/viewtopic.php?p=1109498&hilit=serial#p1109498), but are not yet implemented.(^RouterOS itself may see the USB device and let your container use it indirectly, as with storage media, which you can bind-mount into the container with “`/container/add mounts=…`”.)
    *   There is no GPU support, not even for bare-metal x86 installs.

Lack of a management daemon(^`containerd` in modern setups, `dockerd` in old ones) is not in that list because a good bit of Docker’s competition also lacks this, on purpose. Between that and the other items on the list, the fairest comparison is not to fully-featured container *engines* like Docker and Podman but to the container *runner* at their heart:

* **runc**: 14 MiB(^This is the runner underpinning `containerd`, thus also Docker, although it precedes it. Long before they created `containerd`, it underpinned `dockerd` instead. Because it is so primordial, a good many other container engines are also based on it.)
* **systemd-nspawn**: 1.3 MiB(^[This][sdnsp] is the bare-bones [OCI] image runner built into systemd, with a feature set fairly close to that of `container.npk`. The size above is for version 252 of this program’s parent [`systemd-container`][sdcnt] package as shipped on EL9.)
* **crun**: 0.5 MiB(^This is Podman’s alternative to `runc`, written in C to make it smaller. Early versions of Podman once relied on `runc`, and it can still be configured to use it, but the new default is to use the slimmer but feature-equivalent `crun`.)
* **runc**: 14 MiB(^This is the runner underpinning `containerd`, thus also Docker, although it precedes it. Long before they created `containerd`, it underpinned `dockerd` instead. Because it is so primordial, a good many other container engines are also based on it.)
* **systemd-nspawn**: 1.3 MiB(^[This][sdnsp] is the bare-bones [OCI] image runner built into systemd, with a feature set fairly close to that of `container.npk`. The size above is for version 252 of this program’s parent [`systemd-container`][sdcnt] package as shipped on EL9.)
* **crun**: 0.5 MiB(^This is Podman’s alternative to `runc`, written in C to make it smaller. Early versions of Podman once relied on `runc`, and it can still be configured to use it, but the new default is to use the slimmer but feature-equivalent `crun`.)

One reason `container.npk` is far smaller than even the smallest of these runners is that the engines delegate much of what RouterOS lacks to the runner, so that even then it’s an unbalanced comparison. The [`kill`](#kill), [`ps`](#ps), and [`pause`](#pause) commands missing from `container.npk` are provided in Docker Engine way down at the `runc` level, not up at the top-level CLI.

With this grounding, let us dive into the details.

[caps]:   https://www.man7.org/linux/man-pages/man7/capabilities.7.html
[CoW]:    https://en.wikipedia.org/wiki/Copy-on-write

*   **`--read-only`**: RouterOS offers precious little in terms of file system permission adjustment. As a rule, it is best to either shell into the container and adjust permissions there or rebuild the container with the permissions you want from go. Any expectations based on being able to adjust any of this between image download time and container creation time are likely to founder.

*   **`--restart`**: <a id="restart"></a>The closest RouterOS gets to this is its `start-on-boot` setting, meaning you’d have to reboot the router to get the container to restart. If you want automatic restarts, you will have to [script] it.

*   **`--rm`**: No direct equivalent. There is a manual `/container/remove` command, but nothing like this option, which causes the container runtime to automatically remove the instantiated container after it exits. It’s just as well since this option is most often used when running _ad hoc_ containers made from a previously downloaded image; RouterOS’s lack of an image cache means you have to go out of your way to export a tarball of the image and upload it to the router, then use “`/container/add file=…`” if you want to avoid re-downloading the image from the repository on each relaunch.

*   **`--volume`**: This is largely covered under `--mount` above, but it’s worth repeating that `container.npk` has no concept of what Docker calls ”volumes,” it _only_ has bind-mounts. In that sense, RouterOS does not blur lines as Docker and Podman attempt to do in their handling of the `--volume` option.
*   **`--volume`**: This is largely covered under `--mount` above, but it’s worth repeating that `container.npk` has no concept of what Docker calls ”volumes;” it _only_ has bind-mounts. In that sense, RouterOS does not blur lines as Docker and Podman attempt to do in their handling of the `--volume` option.

That brings us to the related matter of…

[script]: https://help.mikrotik.com/docs/display/ROS/Scripting


# <a id="run"></a>There Is No “Run”

There’s nothing stopping anyone reading this that has the skill and motivation to do this from doing so, but you’ll have to prove out your containers under emulation. Not until then do I see MikroTik being forced to take notice and provide a build of `container.npk` for that platform. It’s not quite a classic chicken-and-egg situation, but I can’t ignore the hiss of radio silence I got in response to [this challenge](https://forum.mikrotik.com/viewtopic.php?t=204868#p1058351) on the forum.

Until someone breaks this logjam, it’s fair enough to say that RouterOS’s container runner only supports ARM and Intel CPUs.


# <a id="auto"></a>Automation

Included in the list of lacks [above](#global) is the [Docker Engine API][DEAPI]. The closest equivalent feature is the [RouterOS REST API][ROAPI], which can issue commands equivalent to those available at the CLI via `/container`. With this, you can programmatically add, remove, start, and stop containers, plus more.
Included in the list of lacks [above](#global) is the [Docker Engine API][DEAPI]. The closest extant feature is the [RouterOS REST API][ROAPI], which can issue commands equivalent to those available at the CLI via `/container`. With this, you can programmatically add, remove, start, and stop containers, plus more.

What RouterOS does _not_ offer is a way for common control plane software like Docker Desktop or [Portainer](https://www.portainer.io/) to manage the containers running on your routers. This is because these programs were written with the assumption that everyone’s running Docker or Podman underneath, and as long as you stick to a compatible subset of the Docker Engine API, implementation details cease to matter up at the control plane’s level of abstraction.
What RouterOS does _not_ offer is a way for common control plane software like Docker Desktop or [Portainer](https://www.portainer.io/) to manage the containers running on your routers. This is because these programs were written with the assumption that everyone’s running Docker or Podman underneath, and as long as they stick to a compatible subset of the Docker Engine API, implementation details cease to matter up at these programs’ level of abstraction.

If you find yourself needing a control plane for your routers’ containers, you will likely need to write it yourself.
If you find yourself needing a control plane for your routers’ containers, you will likely need to write it yourself. Third-party ones are unlikely to be compatible out of the box.

[ROAPI]: https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API


# <a id="oci" name="compliance"></a>OCI Compliance

RouterOS benefits greatly from the existence of the [Open Container Initiative][OCI] specification process. It gives them a vendor-independent standard to target, rather than continually chasing Docker’s current implementation in an _ad hoc_ fashion.

    /system/package/print


## <a id="wait"></a>`wait`

The closest equivalent to this would be to call `/container/stop` in a RouterOS script and then poll on `/container/print where …` until it stopped.


# <a id="license"></a>License

This work is © 2024 by Warren Young and is licensed under <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank" rel="license noopener noreferrer">CC BY-NC-SA 4.0<img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/nc.svg?ref=chooser-v1"><img style="height:22px!important;margin-left:3px;vertical-align:text-bottom;" src="https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1"></a>

<div style="height: 50em" id="this-space-intentionally-left-blank"></div>