Developers' guide

• 1: Internal data structure
• 2: Virtual Machine Drivers
• 3: Testing
• 3.1: BATS Style Guide
• 4: Git tips

1 - Internal data structure

Lima home directory (${LIMA_HOME})

Defaults to ~/.lima.

Note that we intentionally avoid using ~/Library/Application Support/Lima on macOS.

We use ~/.lima so that we can have enough space for the length of the socket path, which must be less than 104 characters on macOS.

Unix: The directory can not be located on an NFS file system, it needs to be local.

Config directory (${LIMA_HOME}/_config)

The config directory contains global lima settings that apply to all instances.

User identity:

Lima creates a default identity and uses its public key as the authorized key to access all lima instances. In addition, lima will also configure all public keys from ~/.ssh/*.pub as well, so the user can use the ssh endpoint without having to specify an identity explicitly.

• user: private key
• user.pub: public key

Instance directory (${LIMA_HOME}/<INSTANCE>)

An instance directory contains the following files:

Metadata:

• lima-version: the Lima version used to create this instance
• lima.yaml: the YAML
• protected: empty file, used by limactl protect

cloud-init:

• cloud-config.yaml: cloud-init configuration, for reference only.
• cidata.iso: cloud-init ISO9660 image. See cidata.iso.

Ansible:

• ansible-inventory.yaml: the Ansible node inventory. See ansible.

disk:

• image: the downloaded VM image; renamed to disk or iso during setup
• image.ipsw: hardlink to image, created for running VZMacOSInstaller that requires the image file to have the .ipsw suffix
• disk: the VM disk (can be a symlink to legacy diffdisk)
• iso: optional CDROM image for ISO-based installations (can be a symlink to legacy basedisk)
• basedisk: legacy name for the downloaded image (pre-v2.1 instances; may remain as a qcow2 backing file)
• diffdisk: legacy name for disk (pre-v2.1 instances)

disk mount:

• mnt: the mount point directory for the disk, used for macOS guests

kernel:

• kernel: the kernel
• kernel.cmdline: the kernel cmdline
• initrd: the initrd

QEMU:

• qemu.pid: QEMU PID
• qmp.sock: QMP socket
• qemu-efi-code.fd: QEMU UEFI code (not always present)

VZ:

• vz.pid: VZ PID
• vz-identifier: Unique machine identifier file for a VM
• vz-hwmodel: Hardware model information for a Mac VM
• vz-aux: Auxiliary storage for a Mac VM
• vz-efi: EFIVariable store file for a VM

Serial:

• serial.log: default serial log (QEMU only), for debugging
• serial.sock: default serial socket (QEMU only), for debugging (Usage: socat -,echo=0,icanon=0 unix-connect:serial.sock)
• serialp.log: PCI serial log (QEMU (ARM) only), for debugging
• serialp.sock: PCI serial socket (QEMU (ARM) only), for debugging (Usage: socat -,echo=0,icanon=0 unix-connect:serialp.sock)
• serialv.log: virtio serial log, for debugging
• serialv.sock: virtio serial socket (QEMU only), for debugging (Usage: socat -,echo=0,icanon=0 unix-connect:serialv.sock)

SSH:

• ssh.sock: SSH control master socket
• ssh.config: SSH config file for ssh -F. Not consumed by Lima itself.

VNC:

• vncdisplay: VNC display host/port
• vncpassword: VNC display password

TPM:

• swtpm.sock: SWTPM (Software TPM Emulator) socket.
• swtpm: A directory where TPM’s state will be written.

Installation Marker for Windows:

• win-done-installation: An indicator showing whether Windows OS installation has completed.

Guest agent:

Each drivers use their own mode of communication

• qemu: uses virtio-port io.lima-vm.guest_agent.0
• vz: uses vsock port 2222
• wsl2: uses free random vsock port The fallback is to use port forward over ssh port
• ga.sock: Forwarded to /run/lima-guestagent.sock in the guest, via SSH

Host agent:

• ha.pid: hostagent PID
• ha.sock: hostagent REST API
• ha.stdout.log: hostagent stdout (JSON lines, see pkg/hostagent/events.Event)
• ha.stderr.log: hostagent stderr (human-readable messages)

External drivers:

• <driver-name>.drv.pid: external driver gRPC server process PID. A single server instance runs per Lima instance and is shared by all callers (CLI, hostagent, inspect). The server is started on first use and stopped only when the instance shuts down.
• <driver-name>.drv.sock: Unix socket for gRPC communication with the external driver server.

Disk directory (${LIMA_HOME}/_disk/<DISK>)

A disk directory contains the following files:

data disk:

• datadisk: the qcow2 or raw disk that is attached to an instance

lock:

• in_use_by: symlink to the instance directory that is using the disk

When using vmType: vz (Virtualization.framework), on boot, any qcow2 (default) formatted disks that are specified in additionalDisks will be converted to RAW since Virtualization.framework only supports mounting RAW disks. This conversion enables additional disks to work with both Virtualization.framework and QEMU, but it has some consequences when it comes to interacting with the disks. Most importantly, a regular macOS default cp command will copy the entire virtual disk size, instead of just the used/allocated portion. The easiest way to copy only the used data is by adding the -c option to cp: cp -c old_path new_path. cp -c uses clonefile(2) to create a copy-on-write clone of the disk, and should return instantly.

ls will also only show the full/virtual size of the disks. To see the allocated space, du -h disk_path or qemu-img info disk_path can be used instead. See #1405 for more details.

Templates directory (${LIMA_HOME}/_templates)

The templates directory can store additional template files that can be referenced with the template: schema.

If the template directory exists (and $LIMA_TEMPLATES_PATH is not set), then this directory will be searched before the /usr/local/share/lima/templates default directory that contains all the templates bundled with Lima itself.

Lima cache directory (~/Library/Caches/lima)

Currently hard-coded to ~/Library/Caches/lima on macOS.

Uses $XDG_CACHE_HOME/lima, normally $HOME/.cache/lima, on Linux.

Uses %LocalAppData%\lima, C:\Users\<USERNAME>\AppData\Local\lima, on Windows.

Download cache (~/Library/Caches/lima/download/by-url-sha256/<SHA256_OF_URL>)

The directory contains the following files:

• url: raw url text, without “\n”
• data: data
• <ALGO>.digest: digest of the data, in OCI format. e.g., file name sha256.digest, with content sha256:5ba3d476707d510fe3ca3928e9cda5d0b4ce527d42b343404c92d563f82ba967
• imgconv/: (Optional) a converted copy of the image, in a format the current driver can use. The downloader creates this on demand when the driver reports its SupportedImageFormats and the cached data is not one of them (for example, a qcow2 image requested by the vz driver, which only boots raw). The original data and its digest file are left as-is, so another driver can still use them.
  • raw: the converted raw image. If data is newer than raw, the raw copy is treated as stale and rebuilt.
  • raw.digest: digest of the raw image. Uses the same algorithm as the digest the caller passed in, and is only read back when the caller passes an expected digest.

Ansible

The instance directory contains an inventory file, that might be used with Ansible playbooks and commands. See Building Ansible inventories about dynamic inventories.

cidata.iso

cidata.iso contains the following files:

• user-data: Cloud-init user-data
• meta-data: Cloud-init meta-data
• network-config: Cloud-init Networking Config Version 2
• lima.env: The LIMA_CIDATA_* environment variables (see below) available during boot.sh processing
• param.env: The PARAM_* environment variables corresponding to the param settings from lima.yaml
• lima-guestagent: Lima guest agent binary. On Linux, omitted in plain mode (the guest agent daemon does not run). On macOS, always injected, as it is required for fake-cloud-init.
• nerdctl-full.tgz: nerdctl-full-<VERSION>-<OS>-<ARCH>.tar.gz
• boot.sh: Boot script
• boot.<OS>/*: Boot script modules
• boot.essential.<OS>/*: Essential boot script modules, executed in plain mode too (unlike boot.<OS>/*, which is skipped in plain mode).
• util/*: Utility command scripts, executed in the boot script modules
• provision.data/*: Custom provision files (data)
• provision.dependency/*: Custom provision scripts (dependency)
• provision.system/*: Custom provision scripts (system)
• provision.user/*: Custom provision scripts (user)
• provision.yq/*: Custom provision scripts (yq)
• etc_environment: Environment variables to be added to /etc/environment (also loaded during boot.sh)

Max file name length = 30

Volume label

The volume label is “cidata”, as defined by cloud-init NoCloud.

Environment variables

• LIMA_CIDATA_DEBUG: the value of the --debug flag of the limactl start command.
• LIMA_CIDATA_IID: the instance ID, regenerated on every boot.
• LIMA_CIDATA_NAME: the lima instance name.
• LIMA_CIDATA_MNT: the mount point of the disk. /mnt/lima-cidata.
• LIMA_CIDATA_USER: the username string.
• LIMA_CIDATA_UID: the numeric UID.
• LIMA_CIDATA_COMMENT: the full name or comment string.
• LIMA_CIDATA_HOME: the guest home directory.
• LIMA_CIDATA_SHELL: the guest login shell.
• LIMA_CIDATA_HOSTHOME_MOUNTPOINT: the mount point of the host home directory, or empty if not mounted.
• LIMA_CIDATA_MOUNTS: the number of the Lima mounts.
• LIMA_CIDATA_MOUNTS_%d_MOUNTPOINT: the N-th mount point of Lima mounts (N=0, 1, …).
• LIMA_CIDATA_MOUNTTYPE: the type of the Lima mounts (“reverse-sshfs”, “9p”, …).
• LIMA_CIDATA_DISKS: the number of additional disks attached to the instance.
• LIMA_CIDATA_DISK_%d_NAME: the name of the N-th additional disk (N=0, 1, …).
• LIMA_CIDATA_DISK_%d_DEVICE: the guest block device path of the N-th additional disk.
• LIMA_CIDATA_DISK_%d_FORMAT: set to “true” when the N-th additional disk should be formatted on first boot.
• LIMA_CIDATA_DISK_%d_FSTYPE: the filesystem type to format the N-th additional disk with (e.g. ext4).
• LIMA_CIDATA_DISK_%d_FSARGS: extra arguments passed to mkfs for the N-th additional disk (space-separated).
• LIMA_CIDATA_DATAFILE_%08d_OVERWRITE: set to “true” if the datafile should be overwritten if it already exists.
• LIMA_CIDATA_DATAFILE_%08d_OWNER: set to the owner of the datafile.
• LIMA_CIDATA_DATAFILE_%08d_PATH: set to the path the datafile should be copied to.
• LIMA_CIDATA_DATAFILE_%08d_PERMISSIONS: set to the file permissions (in octal) for the datafile.
• LIMA_CIDATA_YQ_PROVISION_%08d_FORMAT: the input format (“yaml” or “json”) for the N-th yq provision step.
• LIMA_CIDATA_YQ_PROVISION_%08d_OWNER: the owner (user[:group]) for the file produced by the N-th yq provision step.
• LIMA_CIDATA_YQ_PROVISION_%08d_PATH: the destination path for the N-th yq provision step.
• LIMA_CIDATA_YQ_PROVISION_%08d_PERMISSIONS: the file permissions (in octal) for the N-th yq provision step.
• LIMA_CIDATA_GUEST_INSTALL_PREFIX: the install prefix used for guest tools (e.g. /usr/local).
• LIMA_CIDATA_UPGRADE_PACKAGES: set to “1” if upgradePackages is enabled in lima.yaml, empty otherwise.
• LIMA_CIDATA_CONTAINERD_USER: set to “1” if rootless containerd to be set up.
• LIMA_CIDATA_CONTAINERD_SYSTEM: set to “1” if system-wide containerd to be set up.
• LIMA_CIDATA_CONTAINERD_ARCHIVE: the name of the containerd archive. nerdctl-full.tgz.
• LIMA_CIDATA_SLIRP_GATEWAY: set to the IP address of the host on the SLIRP network. 192.168.5.2.
• LIMA_CIDATA_SLIRP_DNS: set to the IP address of the DNS on the SLIRP network. 192.168.5.3.
• LIMA_CIDATA_SLIRP_IP_ADDRESS: set to the IP address of the guest on the SLIRP network. 192.168.5.15.
• LIMA_CIDATA_UDP_DNS_LOCAL_PORT: set to the udp port number of the hostagent dns server (or 0 when not enabled).
• LIMA_CIDATA_TCP_DNS_LOCAL_PORT: set to the tcp port number of the hostagent dns server (or 0 when not enabled).
• LIMA_CIDATA_ROSETTA_ENABLED: set to “true” if Rosetta x86_64 emulation is enabled in the guest (Apple Silicon, vz driver).
• LIMA_CIDATA_ROSETTA_BINFMT: set to “true” if the guest should register the Rosetta binfmt_misc handler at boot.
• LIMA_CIDATA_SKIP_DEFAULT_DEPENDENCY_RESOLUTION: set to “1” to skip Lima’s built-in package dependency resolution during provisioning, empty otherwise.
• LIMA_CIDATA_VMTYPE: the VM driver type (“qemu”, “vz”, “wsl2”, …), used to conditionally enable VM-specific behavior (e.g. virtiofs mounts on vz).
• LIMA_CIDATA_VSOCK_PORT: the vsock port used by the guest agent (0 when vsock is not used).
• LIMA_CIDATA_VIRTIO_PORT: the virtio-serial port name used by the guest agent (empty when virtio-serial is not used).
• LIMA_CIDATA_PLAIN: set to “1” when the instance is in plain mode (no mounts, port forwarding, or containerd), empty otherwise.
• LIMA_CIDATA_NO_CLOUD_INIT: set to “1” if cloud-init should be skipped on this boot, empty otherwise.

Guest runtime files

Boot scripts create the following marker files inside the guest at /run/:

• /run/lima-boot-done: Written by boot.sh when provisioning is complete. Contains LIMA_CIDATA_IID.
• /run/lima-ssh-ready: Written by boot.Linux/07-etc-environment.sh when SSH is ready. Contains LIMA_CIDATA_IID.
• /run/lima-fuse-ready: Written by boot.Linux/35-setup-packages.sh after fuse.conf is configured for allow_other. Contains LIMA_CIDATA_IID. Only created when LIMA_CIDATA_MOUNTTYPE=reverse-sshfs.

VM lifecycle

(based on Lima 0.8.3)

2 - Virtual Machine Drivers

Lima supports two types of drivers: internal and external. This architecture allows for extensibility and platform-specific implementations. Drivers are unware whether they are internal or external.

💡 See also: VM Types for user configuration of different virtualization backends.

Internal vs External Drivers

Internal Drivers are compiled directly into the limactl binary and are registered automatically at startup by passing the driver object into registry.Register() function and importing the package in the main limactl code using Go’s blank import _. For example:

• qemu: registration file & import file

Build tags control which drivers are compiled as internal vs external (e.g., external_qemu, external_vz, external_wsl2).

External Drivers are separate executables that communicate with Lima via gRPC. They are discovered at runtime from configured directories.

⚠️ Note: External drivers are experimental and the API may change in future releases.

Building Drivers as External

You can build existing internal drivers as external drivers using the ADDITIONAL_DRIVERS Makefile variable:

# Build QEMU as external driver
make ADDITIONAL_DRIVERS=qemu limactl additional-drivers

# Build multiple drivers as external
make ADDITIONAL_DRIVERS="qemu vz wsl2" limactl additional-drivers

This creates external driver binaries in _output/libexec/lima/ with the naming pattern lima-driver-<name> (or lima-driver-<name>.exe on Windows).

Driver Discovery

Lima discovers external drivers from these locations:

1. Custom directories: Set path to the external driver’s directory via LIMA_DRIVERS_PATH environment variable
2. Standard directory: <LIMA-PREFIX>/libexec/lima/, where <LIMA_PREFIX> is the location path where the Lima binary is present

The discovery process is handled by pkg/registry/registry.go.

Creating Custom External Drivers

To create a new external driver:

1. Implement the interface: Your driver must implement the driver.Driver interface:

type Driver interface {
	Lifecycle
	GUI
	SnapshotManager
	GuestAgent

	Info(ctx context.Context) Info
	Configure(ctx context.Context, inst *limatype.Instance) (*ConfiguredDriver, error)
	SSHAddress(ctx context.Context) (string, error)
	AdditionalSetupForSSH(ctx context.Context) error
}

The gRPC transport that carries these methods to an external driver is defined in driver.proto, whose per-RPC doc comments are the authoritative contract (call ordering, idempotency and per-RPC semantics) for driver authors. The Go side is documented as godoc on the driver.Driver interface.

2. Create main.go: Use server.Serve() to expose your driver:

package main

import (
    "context"
    "github.com/lima-vm/lima/v2/pkg/driver/external/server"
)

func main() {
    driver := &MyDriver{}
    server.Serve(context.Background(), driver)
}

3. Build and deploy:

   • Build your driver: go build -o lima-driver-mydriver main.go
   • Place the binary in a directory accessible via LIMA_DRIVERS_PATH
   • Ensure the binary is executable

4. Use the driver: Explicitly specify the driver when creating instances:

limactl create myinstance --vm-type=mydriver template:default

Driver lifecycle & call ordering

Drivers are unaware whether they are internal or external, but external driver authors need to know when each method is invoked. The sequence below is the instance start path: instance.Prepare (in pkg/instance) runs steps 1-4, then the hostagent (pkg/hostagent) boots the VM and runs steps 5-7:

1. Configure - sets the instance configuration on the driver. It must run before any method that depends on the config.
2. Validate - rejects a config the driver cannot support. Returning an error aborts the start.
3. Create - first-time provisioning (e.g. writing a driver identifier file). It must be a no-op (succeed) when the instance already exists, and it must not create the disks.
4. CreateDisk - creates the instance disk(s).
5. Start - begins booting the VM. It is streaming and returns as soon as boot has been initiated (the initial “started” message), not once the guest has finished booting; it then streams any runtime errors, then a final “done” (see the StartResponse message in driver.proto).
6. AdditionalSetupForSSH - runs right after Start returns (boot initiated, the guest is not necessarily up yet), before the first SSH connection.
7. SSHAddress - re-queried after Start only when Info reports Features.dynamicSSHAddress (e.g. WSL2, whose address is not known until the VM is up).

The guest agent is reached in one of two ways, chosen by what ForwardGuestAgent returns. When it returns true, the host agent forwards the guest-agent socket over SSH and GuestAgentConn returns nil. When it returns false, the driver provides a direct guest-agent connection (e.g. vsock/virtio) through GuestAgentConn.

These start-path methods are invoked sequentially in the order above. Creating an instance without starting it (limactl create) only runs Configure and Create. Once the VM is running, the host agent polls read-only methods (notably Info and ForwardGuestAgent) from background goroutines, so a driver’s gRPC server must be safe to handle concurrent calls.

For the full per-method contract, see the doc comments in driver.proto and the driver.Driver godoc.

Examples

See existing external driver implementations:

• cmd/lima-driver-qemu/main.go
• cmd/lima-driver-vz/main_darwin.go
• cmd/lima-driver-wsl2/main_windows.go

3 - Testing

Unit tests

The unit tests are written in Go and can be executed with the following commands:

go test -v ./...

The unit tests do not execute actual virtual machines.

Integration tests

The integration tests incurs actual execution of virtual machines.

The integration tests are written in BATS (Bash Automated Testing System).

Run the following commands to run the BATS tests:

git submodule update --init --recursive
make bats

The BATS tests are located under hack/bats/tests.

Extra tests

There are also extra tests (hack/bats/extras) that are not automatically invoked from make bats.

Run the following command to run the extra BATS tests:

./hack/bats/lib/bats-core/bin/bats ./hack/bats/extras

Template-specific tests

Tests that are specific to template files are written in bash and partially in Perl.

Use hack/test-templates.sh to execute tests, with a virtual machine template file, e.g.,:

./hack/test-templates.sh ./templates/default.yaml
./hack/test-templates.sh ./templates/fedora.yaml
./hack/test-templates.sh ./hack/test-templates/test-misc.yaml

CI

.github/workflows/test.yml executes the tests on the GitHub Actions with the “Tier 1” templates.

Most tests are executed on Linux runners, as macOS runners are slow and flaky.

The tests about macOS-specific features (e.g., vz and vmnet) are still executed on macOS runners.

Currently, the Intel version of macOS is used, as the ARM version of macOS on GitHub Actions still do not support nested virtualization.

3.1 - BATS Style Guide

Lima uses BATS with the bats-support, bats-assert, and bats-file helper libraries.

All tests run with errexit enabled (via BATS_RUN_ERREXIT=1 in helpers/load.bash), so any failing command aborts the test immediately.

When to use run

Use run only when you need to capture output or assert a non-zero exit code. Do not use it just to check that a command succeeds.

Command should succeed, output does not matter

Call the command directly. errexit handles the failure case.

# Good
limactl shell "$INSTANCE" -- mkdir -p /tmp/foo

# Bad — unnecessary run and status check
run limactl shell "$INSTANCE" -- mkdir -p /tmp/foo
[[ $status == 0 ]]

# Bad — unnecessary run and assert_success
run limactl shell "$INSTANCE" -- mkdir -p /tmp/foo
assert_success

Command should succeed, and you need its output

Use run -0 to assert success and capture $output/$lines, then use assert_output or assert_line to verify the output.

# Good
run -0 limactl shell "$INSTANCE" -- cat /tmp/hello.txt
assert_output "hello"

# Bad — manual status and output checks
run limactl shell "$INSTANCE" -- cat /tmp/hello.txt
[[ $status == 0 ]]
[[ $output == "hello" ]]

Command should fail with a specific exit code

Use run -N where N is the expected exit code.

run -1 limactl yq -n foo
assert_output --partial "invalid input"

Checking stderr (log messages)

Use run_e (a wrapper for run --separate-stderr) when you need to check both stdout and stderr. The helpers assert_fatal, assert_warning, assert_info, assert_error, and assert_debug match Lima’s structured log output in stderr.

run_e -1 limactl ls foo foobar bar
assert_warning 'No instance matching foobar found.'
assert_fatal 'unmatched instances'

Checking files and directories

Use bats-file assertions instead of test expressions.

# Good
assert_file_exists "$LIMA_HOME/$INSTANCE/protected"
assert_dir_exists "$BATS_TEST_TMPDIR/foo/bar"

# Bad — raw test expressions give poor failure messages
[[ -f "$LIMA_HOME/$INSTANCE/protected" ]]
[[ -d "$BATS_TEST_TMPDIR/foo/bar" ]]

Test lifecycle

Define local_setup_file, local_teardown_file, local_setup, and local_teardown instead of overriding setup_file, setup, etc. directly. The base implementations in helpers/load.bash call these local_ variants automatically.

Set INSTANCE at file scope to have setup_file create (or reuse) a Lima instance and teardown_file delete it.

Summary

4 - Git tips

Squashing Commits

To combine multiple commits into one (recommended unless your PR covers multiple topics):

# Adjust the number based on how many commits you want to squash
git rebase -i HEAD~3

In the interactive editor that appears:

1. Keep the first commit as pick
2. Change subsequent commits from pick to fixup (short formf). You may also choose squash (s), however, fixup is recommended to keep the commit message clean.
3. Save and close the editor to proceed

Example:

pick aaaaaaa First commit message
pick bbbbbbb Second commit message
pick ccccccc Fix typo

To:

pick aaaaaaa First commit message
f bbbbbbb Second commit message
f ccccccc Fix typo

Rebasing onto Upstream Master

To update your branch with the latest changes from upstream:

git remote add upstream https://github.com/lima-vm/lima.git  # Only needed once
git fetch upstream
git rebase upstream/master

Troubleshooting

If you encounter issues during rebase:

git rebase --abort  # Cancel the rebase and return to original state
git status          # Check current state

For merge conflicts during rebase:

1. Resolve the conflicts in the files
2. git add the resolved files
3. git rebase --continue