A broad workaround for _some_ of the above is having the foresight to pull the image using Docker or Podman, then save the image out as a tarball and using `/container/add file=` instead of `remote-image`. There are landmines along this path owing to the [OCI compatibility issue](#compliance) covered separately below.


# <a id="root"></a>Everything Is Rootful

This shows up in a number of guises, but the overall effect is that all containers run as a nerfed `root` user under `container.npk`, same as Docker did from the start. This remains the Docker default, but starting with the 20.10 release, it finally got a [rootless mode][drl] to compete with [Podman’s rootless-by-default][prl] nature. I bring up this history to show that RouterOS is not unconditionally “wrong” to operate as it does, merely limited.

This design choice may be made reasonably safe through the grace of [user namespaces](https://www.man7.org/linux/man-pages/man7/user_namespaces.7.html), which cause the in-container `root` user to be meaningfully different from the Linux `root` user that RouterOS itself runs as. RouterOS does have a `/user` model, but they are not proper Linux users as understood by the kernel, with permissions enforced by Linux user IDs; RouterOS users have _no meaningful existence at all_ inside the container. One practical effect of this is that when you start a container as RouterOS user `fred`, you will not find a `fred` entry in its `/etc/passwd` file, and if you create one at container build time (e.g. with a `RUN useradd` command) it will not be the same `fred` as the RouterOS user on the outside.
This design choice may be made reasonably safe through the grace of [user namespaces](https://www.man7.org/linux/man-pages/man7/user_namespaces.7.html), which cause the in-container `root` user to be meaningfully different from the Linux `root` user that RouterOS itself runs as.

Files created by that nerfed `root` user will show up as owned by `root` when using bind-mounted directories on file systems like `ext4` which preserve file ownership. One possible solution for this is:

    /disk/format-drive file-system=exfat …

It is because of this same limitation that there is no RouterOS equivalent to the `create --user*` or `--group-add` flags.

If your container was designed to have non-root users inside with meaningful distinctions from root, it may require massaging to work on RouterOS. There are no UID maps to convert in-container user IDs to RouterOS user IDs, etc. This is one of the key reasons why it matters that [containers are not VMs][cvm]; persisting in this misunderstanding is liable to lead you to grief under `container.npk`. Let go of your preconceptions and use the RouterOS container runner the way it was meant to be applied: running well-focused single services.(^This philosophy is not specific to RouterOS, nor is it special pleading on its behalf, meant to justify its limitations. [Microservices][msc] are good idea atop _all_ container runtimes.)
If your container was designed to have non-root users inside with meaningful distinctions from root, it may require massaging to work on RouterOS. Its `/user` model has no connection to proper Linux users as understood by the kernel; while a RouterOS user does have an ID, it is not a UID recognized by the underlying Linux kernel, which it uses in enforcing permissions. One practical effect of this is that when you start a container as RouterOS user `fred`, you will not find a `fred` entry in the generated`/etc/passwd` file inside the container, and if you create one at container build time (e.g. with a `RUN useradd` command) it will not be the same `fred` as the RouterOS user on the outside.

This brings us to the oft-repeated observation that [containers are not VMs][cvm]. Persisting in misunderstanding the distinction is liable to lead you to grief under `container.npk`. Let go of your preconceptions and use the RouterOS container runner the way it was meant to be applied: running well-focused single services.(^This philosophy is not specific to RouterOS, nor is it special pleading on its behalf, meant to justify its limitations. [Microservices][msc] are good idea atop _all_ container runtimes.)

[cvm]: /wiki?name=Containers%20Are%20Not%20VMs
[drl]: https://docs.docker.com/engine/security/rootless/
[msc]: https://www.bmc.com/blogs/containers-vs-microservices/
[prl]: https://developers.redhat.com/blog/2020/09/25/rootless-containers-with-podman-the-basics

When you next come across one of the huge number of containers based on Alpine, you’ll be back in the soup once again. While [its CPU support list](https://wiki.alpinelinux.org/wiki/Requirements) is broader than [the one for Ubuntu](https://ubuntu.com/cpu-compatibility), there is no TILE or MIPS at all, and its PPC support is 64-bit only. Are you going to port the Alpine base image and enough of its package repository to get your container building?

Then there’s Debian, another popular OCI image base, one that’s been ported to a lot of strange platforms, but chances are that it was someone’s wild project, now abandoned. It’s likely the APT package repo isn’t working any more, for one, because who wants to host a huge set of packages for a dead project?

In brief, the reason MikroTik doesn’t ship `container.npk` for 32-bit PPC, 32-bit MIPS, and TILE is that there are few Linux distro images in OCI format to use as base images, and it isn’t greatly in their interest to pull that together along with the QEMU and `binfmt_misc` pieces for you, nor is it in the financial interest of Docker, Podman, etc.

There’s nothing stopping anyone reading this that has the skill and motivation to do this from doing so, but you’ll have to prove out your containers under emulation. Not until then do I see MikroTik being forced to take notice and provide a build of `container.npk` for that platform. It’s not quite a classic chicken-and-egg situation, but I can’t ignore the hiss of radio silence I got in response to [this challenge](https://forum.mikrotik.com/viewtopic.php?t=204868#p1058351) on the forum.
There’s nothing stopping anyone reading this that has the skill and motivation to do this from doing so, but you’ll have to prove out your containers under emulation. Not until then do I see MikroTik being forced to take notice and provide a build of `container.npk` for that platform. It’s not quite a classic chicken-and-egg situation, but I can’t ignore the hiss of radio silence I got in response to [this challenge](https://forum.mikrotik.com/t/ccr1xxx-with-containers/173851/31) on the forum.

Until someone breaks this logjam, it’s fair enough to say that RouterOS’s container runner only supports ARM and Intel CPUs.

Incidentally, exploration of the binfmts available to you on your container build host of choice might result in output like `linux/mips64le`, leaving you exulting, “See, there _is_ MIPS support!” But no. First off, this is 64-bit MIPS, while all MIPS CPUs shipped by MikroTik to this date have been 32-bit. Second, it’s [little-endian](https://en.wikipedia.org/wiki/Endianness) (LE) which means it wouldn’t work with the big-endian MIPS CPUs that were more popular historically. Third, even if you find/build a platform that includes support for the MIPSBE, MMIPS, and SMIPS CPU types MikroTik shipped, you’re likely back to lack of a base OS to build from.


### <a id="armv5"></a>ARMv5