1 - VM types
Lima supports two ways of running guest machines:
The vmType can be specified only on creating the instance.
The vmType of existing instances cannot be changed.
See the following flowchart to choose the best vmType for you:
flowchart
host{"Host OS"} -- "Windows" --> wsl2["WSL2"]
host -- "Linux" --> qemu["QEMU"]
host -- "macOS" --> intel_on_arm{"Need to run <br> Intel binaries <br> on ARM?"}
intel_on_arm -- "Yes" --> just_elf{"Just need to <br> run Intel userspace (fast), <br> or entire Intel VM (slow)?"}
just_elf -- "Userspace (fast)" --> vz
just_elf -- "VM (slow)" --> qemu
intel_on_arm -- "No" --> vz["VZ"]
The default vmType is QEMU in Lima prior to v1.0.
Starting with Lima v1.0, Lima will use VZ by default on macOS (>= 13.5) for new instances,
unless the config is incompatible with VZ. (e.g., legacyBIOS or 9p is enabled)
QEMU
“qemu” option makes use of QEMU to run guest operating system.
VZ
⚡ Requirement | Lima >= 0.14, macOS >= 13.0 |
---|
“vz” option makes use of native virtualization support provided by macOS Virtualization.Framework.
An example configuration:
limactl start --vm-type=vz --mount-type=virtiofs
# Example to run ubuntu using vmType: vz instead of qemu (Default)
vmType: "vz"
images:
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img"
arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img"
arch: "aarch64"
mounts:
- location: "~"
mountType: "virtiofs"
Caveats
- “vz” option is only supported on macOS 13 or above
- Virtualization.framework doesn’t support running “intel guest on arm” and vice versa
Known Issues
- “vz” doesn’t support
legacyBIOS: true
option, so guest machine like centos-stream and oraclelinux will not work - When running lima using “vz”,
${LIMA_HOME}/<INSTANCE>/serial.log
will not contain kernel boot logs - On Intel Mac with macOS prior to 13.5, Linux kernel v6.2 (used by Ubuntu 23.04, Fedora 38, etc.) is known to be unbootable on vz.
kernel v6.3 and later should boot, as long as it is booted via GRUB.
https://github.com/lima-vm/lima/issues/1577#issuecomment-1565625668
The issue is fixed in macOS 13.5.
WSL2
Warning
“wsl2” mode is experimental
⚡ Requirement | Lima >= 0.18 + (Windows >= 10 Build 19041 OR Windows 11) |
---|
“wsl2” option makes use of native virtualization support provided by Windows’ wsl.exe
(more info).
An example configuration:
limactl start --vm-type=wsl2 --mount-type=wsl2 --containerd=system
# Example to run Fedora using vmType: wsl2
vmType: wsl2
images:
# Source: https://github.com/runfinch/finch-core/blob/main/Dockerfile
- location: "https://deps.runfinch.com/common/x86-64/finch-rootfs-production-amd64-1690920103.tar.zst"
arch: "x86_64"
digest: "sha256:53f2e329b8da0f6a25e025d1f6cc262ae228402ba615ad095739b2f0ec6babc9"
mountType: wsl2
containerd:
system: true
user: false
Caveats
- “wsl2” option is only supported on newer versions of Windows (roughly anything since 2019)
Known Issues
- “wsl2” currently doesn’t support many of Lima’s options. See this file for the latest supported options.
- When running lima using “wsl2”,
${LIMA_HOME}/<INSTANCE>/serial.log
will not contain kernel boot logs - WSL2 requires a
tar
formatted rootfs archive instead of a VM image - Windows doesn’t ship with ssh.exe, gzip.exe, etc. which are used by Lima at various points. The easiest way around this is to run
winget install -e --id Git.MinGit
(winget is now built in to Windows as well), and add the resulting C:\Program Files\Git\usr\bin\
directory to your path.
2 - Intel-on-ARM and ARM-on-Intel
Lima supports two modes for running Intel-on-ARM and ARM-on-Intel:
Lima can run a VM with a foreign architecture, just by specifying arch
in the YAML.
arch: "x86_64"
# arch: "aarch64"
images:
- location: "https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img"
arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-arm64.img"
arch: "aarch64"
# Disable mounts and containerd, otherwise booting up may time out if the host is slow
mounts: []
containerd:
system: false
user: false
Running a VM with a foreign architecture is extremely slow.
Consider using Fast mode or Fast mode 2 whenever possible.
This mode uses QEMU User Mode Emulation.
QEMU User Mode Emulation is significantly faster than QEMU System Mode Emulation, but it often sacrifices compatibility.
Set up:
lima sudo systemctl start containerd
lima sudo nerdctl run --privileged --rm tonistiigi/binfmt:qemu-v7.0.0-28@sha256:66e11bea77a5ea9d6f0fe79b57cd2b189b5d15b93a2bdb925be22949232e4e55 --install all
Run containers:
$ lima nerdctl run --platform=amd64 --rm alpine uname -m
x86_64
$ lima nerdctl run --platform=arm64 --rm alpine uname -m
aarch64
Build and push container images:
$ lima nerdctl build --platform=amd64,arm64 -t example.com/foo:latest .
$ lima nerdctl push --all-platforms example.com/foo:latest
See also https://github.com/containerd/nerdctl/blob/master/docs/multi-platform.md
⚡ Requirement | Lima >= 0.14, macOS >= 13.0, ARM |
---|
Rosetta is known to be much faster than QEMU User Mode Emulation.
Rosetta is available for VZ instances on ARM hosts.
limactl start --vm-type=vz --rosetta
vmType: "vz"
rosetta:
# Enable Rosetta for Linux.
# Hint: try `softwareupdate --install-rosetta` if Lima gets stuck at `Installing rosetta...`
enabled: true
# Register rosetta to /proc/sys/fs/binfmt_misc
binfmt: true
3 - Network
See the following flowchart to choose the best network for you:
flowchart
connect_to_vm_via{"Connect to the VM via"} -- "localhost" --> default["Default"]
connect_to_vm_via -- "IP" --> connect_from{"Connect to the VM IP from"}
connect_from -- "Host" --> vm{"VM type"}
vm -- "vz" --> vzNAT["vzNAT"]
vm -- "qemu" --> shared["socket_vmnet (shared)"]
connect_from -- "Other VMs" --> userV2["user-v2"]
connect_from -- "Other hosts" --> bridged["socket_vmnet (bridged)"]
Default user-mode network (192.168.5.0/24)
By default Lima only enables the user-mode networking aka “slirp”.
Guest IP (192.168.5.15)
The guest IP address is set to 192.168.5.15
.
This IP address is not accessible from the host by design.
Use VMNet (see below) to allow accessing the guest IP from the host and other guests.
Host IP (192.168.5.2)
The loopback addresses of the host is 192.168.5.2
and is accessible from the guest as host.lima.internal
.
DNS (192.168.5.3)
If hostResolver.enabled
in lima.yaml
is true, then the hostagent is going to run a DNS server over tcp and udp - each on a separate randomly selected free port. This server does a local lookup using the native host resolver, so it will deal correctly with VPN configurations and split-DNS setups, as well as mDNS, local /etc/hosts
etc. For this the hostagent has to be compiled with CGO_ENABLED=1
as default Go resolver is broken.
These tcp and udp ports are then forwarded via iptables rules to 192.168.5.3:53
, overriding the DNS provided by QEMU via slirp.
Currently following request types are supported:
For all other queries hostagent will redirect the query to the nameservers specified in /etc/resolv.conf
(or, if that fails - to 8.8.8.8
and 1.1.1.1
).
DNS over tcp is rarely used. It is usually only used either when user explicitly requires it, or when request+response can’t fit into a single UDP packet (most likely in case of DNSSEC), or in the case of certain management operations such as domain transfers. Neither DNSSEC nor management operations are currently supported by a hostagent, but on the off chance that the response may contain an unusually long list of records - hostagent will also listen for the tcp traffic.
During initial cloud-init bootstrap, iptables
may not yet be installed. In that case the repo server is determined using the slirp DNS. After iptables
has been installed, the forwarding rule is applied, switching over to the hostagent DNS.
If hostResolver.enabled
is false, then DNS servers can be configured manually in lima.yaml
via the dns
setting. If that list is empty, then Lima will either use the slirp DNS (on Linux), or the nameservers from the first host interface in service order that has an assigned IPv4 address (on macOS).
Lima user-v2 network
⚡ Requirement | Lima >= 0.16.0 |
---|
user-v2 network provides a user-mode networking similar to the default user-mode network and also provides support for vm -> vm
communication.
To enable this network mode, define a network with mode: user-v2
in networks.yaml
By default, the below network configuration is already applied (Since v0.18).
...
networks:
user-v2:
mode: user-v2
gateway: 192.168.104.1
netmask: 255.255.255.0
...
Instances can then reference these networks from their lima.yaml
file:
limactl start --network=lima:user-v2
networks:
- lima: user-v2
An instance’s IP address is resolvable from another instance as lima-<NAME>.internal.
(e.g., lima-default.internal.
).
Note
VMNet networks
VMNet assigns a “real” IP address that is reachable from the host.
The configuration steps are different for each network type:
vzNAT
⚡ Requirement | Lima >= 0.14, macOS >= 13.0 |
---|
For VZ instances, the “vzNAT” network can be configured as follows:
limactl start --vm-type=vz --network=vzNAT
The range of the IP address is not specifiable.
The “vzNAT” network does not need the socket_vmnet
binary and the sudoers
file.
socket_vmnet
Managed (192.168.105.0/24)
socket_vmnet
can be used for adding another guest IP that is accessible from the host and other guests,
without depending on vz.
It must be installed according to the instruction provided on https://github.com/lima-vm/socket_vmnet.
Note that installation using Homebrew is not secure and not recommended by the Lima project.
Homebrew installation will only work with Lima if password-less sudo
is enabled for the current user.
The limactl sudoers
command requires that socket_vmnet
is installed into a secure path only
writable by root
and will reject socket_vmnet
installed by Homebrew into a user-writable location.
# Install socket_vmnet as root from source to /opt/socket_vmnet
# using instructions on https://github.com/lima-vm/socket_vmnet
# This assumes that Xcode Command Line Tools are already installed
git clone https://github.com/lima-vm/socket_vmnet
cd socket_vmnet
# Change "v1.1.5" to the actual latest release in https://github.com/lima-vm/socket_vmnet/releases
git checkout v1.1.5
make
sudo make PREFIX=/opt/socket_vmnet install.bin
# Set up the sudoers file for launching socket_vmnet from Lima
limactl sudoers >etc_sudoers.d_lima
less etc_sudoers.d_lima # verify that the file looks correct
sudo install -o root etc_sudoers.d_lima /etc/sudoers.d/lima
rm etc_sudoers.d_lima
Note
Lima before v0.12 used vde_vmnet
for managing the networks.
vde_vmnet
is no longer supported.
Lima v0.14.0 and later used to also accept socket_vmnet
installations if they were
owned by the admin
user. Starting with v1.0.0 only root
ownership is acceptable.
The networks are defined in $LIMA_HOME/_config/networks.yaml
. If this file doesn’t already exist, it will be created with these default
settings:
Default
# Path to socket_vmnet executable. Because socket_vmnet is invoked via sudo it should be
# installed where only root can modify/replace it. This means also none of the
# parent directories should be writable by the user.
#
# The varRun directory also must not be writable by the user because it will
# include the socket_vmnet pid file. Those will be terminated via sudo, so replacing
# the pid file would allow killing of arbitrary privileged processes. varRun
# however MUST be writable by the daemon user.
#
# None of the paths segments may be symlinks, why it has to be /private/var
# instead of /var etc.
paths:
# socketVMNet requires Lima >= 0.12 .
socketVMNet: /opt/socket_vmnet/bin/socket_vmnet
varRun: /private/var/run/lima
sudoers: /private/etc/sudoers.d/lima
group: everyone
networks:
shared:
mode: shared
gateway: 192.168.105.1
dhcpEnd: 192.168.105.254
netmask: 255.255.255.0
bridged:
mode: bridged
interface: en0
# bridged mode doesn't have a gateway; dhcp is managed by outside network
host:
mode: host
gateway: 192.168.106.1
dhcpEnd: 192.168.106.254
netmask: 255.255.255.0
Instances can then reference these networks:
limactl start --network=lima:shared
networks:
# Lima can manage the socket_vmnet daemon for networks defined in $LIMA_HOME/_config/networks.yaml automatically.
# The socket_vmnet binary must be installed into a secure location only alterable by the "root" user.
# - lima: shared
# # MAC address of the instance; lima will pick one based on the instance name,
# # so DHCP assigned ip addresses should remain constant over instance restarts.
# macAddress: ""
# # Interface name, defaults to "lima0", "lima1", etc.
# interface: ""
The network daemon is started automatically when the first instance referencing them is started,
and will stop automatically once the last instance has stopped. Daemon logs will be stored in the
$LIMA_HOME/_networks
directory.
Since the commands to start and stop the socket_vmnet
daemon requires root, the user either must
have password-less sudo
enabled, or add the required commands to a sudoers
file. This can
be done via:
limactl sudoers >etc_sudoers.d_lima
less etc_sudoers.d_lima # verify that the file looks correct
sudo install -o root etc_sudoers.d_lima /etc/sudoers.d/lima
rm etc_sudoers.d_lima
The IP address is automatically assigned by macOS’s bootpd.
If the IP address is not assigned, try the following commands:
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd
Unmanaged
Lima can also connect to “unmanaged” networks addressed by “socket”. This
means that the daemons will not be controlled by Lima, but must be started
before the instance. The interface type (host, shared, or bridged) is
configured in socket_vmnet
and not in lima.
networks:
- socket: "/var/run/socket_vmnet"
4 - Port Forwarding
Lima supports automatic port-forwarding of localhost ports from guest to host.
Port forwarding types
Lima supports two port forwarders: SSH and GRPC.
The default port forwarder is SSH.
The default was once changed to GRPC in Lima v1.0, but it was reverted to SSH in v1.0.1 due to stability reasons.
In future, it is expected that GRPC will take over the default position again.
Using SSH
SSH based port forwarding is the default and current model that is supported in Lima.
To explicitly use SSH forwarding use the below command
LIMA_SSH_PORT_FORWARDER=true limactl start
Caveats
- Doesn’t support UDP based port forwarding
- Spans child process on host for running SSH master.
Using GRPC
In this model, lima uses existing GRPC communication (Host <-> Guest) to tunnel port forwarding requests.
For each port forwarding request, a GRPC tunnel is created and this will be used for transmitting data
To enable this feature, set LIMA_SSH_PORT_FORWARDER
to false
:
LIMA_SSH_PORT_FORWARDER=false limactl start
Advantages
- Supports both TCP and UDP based port forwarding
- Performs faster compared to SSH based forwarding
- No additional child process for port forwarding
Benchmarks
Use case | GRPC | SSH |
---|
TCP | 3.80 Gbits/sec | 3.38 Gbits/sec |
TCP Reverse | 4.77 Gbits/sec | 3.08 Gbits/sec |
The benchmarks detail above are obtained using the following commands
Host -> limactl start vz
VZ Guest -> iperf3 -s
Host -> iperf3 -c 127.0.0.1 //Benchmark for TCP
Host -> iperf3 -c 127.0.0.1 -R //Benchmark for TCP Reverse
5 - Filesystem mounts
Lima supports several methods for mounting the host filesystem into the guest.
The default mount type is shown in the following table:
Lima Version | Default |
---|
< 0.10 | reverse-sshfs + Builtin SFTP server |
>= 0.10 | reverse-sshfs + OpenSSH SFTP server |
>= 0.17 | reverse-sshfs + OpenSSH SFTP server for QEMU, virtiofs for VZ |
>= 1.0 | 9p for QEMU, virtiofs for VZ |
Mount types
reverse-sshfs
The “reverse-sshfs” mount type exposes the host filesystem by running an SFTP server on the host.
While the host works as an SFTP server, the host does not open any TCP port,
as the host initiates an SSH connection into the guest and let the guest connect to the SFTP server via the stdin.
An example configuration:
limactl start --mount-type=reverse-sshfs
mountType: "reverse-sshfs"
mounts:
- location: "~"
sshfs:
# Enabling the SSHFS cache will increase performance of the mounted filesystem, at
# the cost of potentially not reflecting changes made on the host in a timely manner.
# Warning: It looks like PHP filesystem access does not work correctly when
# the cache is disabled.
# 🟢 Builtin default: true
cache: null
# SSHFS has an optional flag called 'follow_symlinks'. This allows mounts
# to be properly resolved in the guest os and allow for access to the
# contents of the symlink. As a result, symlinked files & folders on the Host
# system will look and feel like regular files directories in the Guest OS.
# 🟢 Builtin default: false
followSymlinks: null
# SFTP driver, "builtin" or "openssh-sftp-server". "openssh-sftp-server" is recommended.
# 🟢 Builtin default: "openssh-sftp-server" if OpenSSH SFTP Server binary is found, otherwise "builtin"
sftpDriver: null
The default value of sftpDriver
has been set to “openssh-sftp-server” since Lima v0.10, when an OpenSSH SFTP Server binary
such as /usr/libexec/sftp-server
is detected on the host.
Lima prior to v0.10 had used “builtin” as the SFTP driver.
Caveats
- A mount is disabled when the SSH connection was shut down.
- A compromised
sshfs
process in the guest may have access to unexposed host directories.
9p
The “9p” mount type is implemented by using QEMU’s virtio-9p-pci devices.
virtio-9p-pci is also known as “virtfs”, but note that this is unrelated to virtio-fs.
An example configuration:
limactl start --vm-type=qemu --mount-type=9p
vmType: "qemu"
mountType: "9p"
mounts:
- location: "~"
9p:
# Supported security models are "passthrough", "mapped-xattr", "mapped-file" and "none".
# "mapped-xattr" and "mapped-file" are useful for persistent chown but incompatible with symlinks.
# 🟢 Builtin default: "none" (since Lima v0.13)
securityModel: null
# Select 9P protocol version. Valid options are: "9p2000" (legacy), "9p2000.u", "9p2000.L".
# 🟢 Builtin default: "9p2000.L"
protocolVersion: null
# The number of bytes to use for 9p packet payload, where 4KiB is the absolute minimum.
# 🟢 Builtin default: "128KiB"
msize: null
# Specifies a caching policy. Valid options are: "none", "loose", "fscache" and "mmap".
# Try choosing "mmap" or "none" if you see a stability issue with the default "fscache".
# See https://www.kernel.org/doc/Documentation/filesystems/9p.txt
# 🟢 Builtin default: "fscache" for non-writable mounts, "mmap" for writable mounts
cache: null
The “9p” mount type requires Lima v0.10.0 or later.
Caveats
- The “9p” mount type is known to be incompatible with CentOS, Rocky Linux, and AlmaLinux as their kernel do not support
CONFIG_NET_9P_VIRTIO
.
virtiofs
Warning
“virtiofs” mode is experimental on Linux hosts
⚡ Requirement | Lima >= 0.14, macOS >= 13.0 | Lima >= 0.17.0, Linux, QEMU 4.2.0+, virtiofsd (Rust version) |
---|
The “virtiofs” mount type is implemented via the virtio-fs device by using apple Virtualization.Framework shared directory on macOS and virtiofsd on Linux.
Linux guest kernel must enable the CONFIG_VIRTIO_FS support for this support.
An example configuration:
limactl start --vm-type=vz --mount-type=virtiofs
vmType: "vz" # only for macOS; Linux uses 'qemu'
mountType: "virtiofs"
mounts:
- location: "~"
Caveats
- For macOS, the “virtiofs” mount type is supported only on macOS 13 or above with
vmType: vz
config. See also vmtype
. - For Linux, the “virtiofs” mount type requires the Rust version of virtiofsd.
Using the version from QEMU (usually packaged as
qemu-virtiofsd
) will not work, as it requires root access to run.
wsl2
Warning
“wsl2” mode is experimental
⚡ Requirement | Lima >= 0.18 + (Windows >= 10 Build 19041 OR Windows 11) |
---|
The “wsl2” mount type relies on using WSL2’s native disk sharing, where the root disk is available by default at /mnt/$DISK_LETTER
(e.g. /mnt/c/
).
An example configuration:
limactl start --vm-type=wsl2 --mount-type=wsl2
vmType: "wsl2"
mountType: "wsl2"
Caveats
- WSL2 file permissions may not work exactly as expected when accessing files that are natively on the Windows disk (more info)
- WSL2’s disk sharing system uses a 9P protocol server, making the performance similar to Lima’s 9p mode (more info)
Mount Inotify
Warning
“mountInotify” is experimental
⚡ Requirement | Lima >= 0.21.0 |
---|
The mountInotify
support enables inotify support for all different mountTypes like 9p, virtiofs etc.
When mountInotify is enabled,
- hostagent will listen and send inotify events from host machine to guest.
- Guest will modify the file to trigger inotify on guest side
This support will be enabled only for writable mounts because only for writable mount guest will be able to trigger inotify
An example configuration:
limactl start --mount-inotify
mountInotify: true
mounts:
- location: "~"
writable: true
Caveats
- For
mountType: 9p
, Inotify events are not triggered for nested files from the listening directory. - Inotify events are not triggered when files are removed from host