Part II: Setup Windows Subsystem for Linux and Docker in Windows 10

This is Part II of a 3-part blog post on how I configured my working environment. Here I’ll explain how to configure Windows Subsystem for Linux (WSL) and Docker.

• Part I: Setup Dev Environment in Windows 10
• Part II: Windows Subsystem for Linux (WSL) and Docker setup
• Part III: Installation of additional packages and tools in WSL

Since Microsoft announced WSL, Windows 10 became a possible alternative to Linux and MacOS for software development. WSL combines the advantages of having a Linux Kernel backend (running Python, bash etc.) and a Windows graphical frontend, e.g. for working with Visual Studio Code.

Preparations

Enable Windows Features:

• WSL
• Virtual Machine Platform Windows (“VM Plattform)
• Hyper-V

Install a Linux distro with LxRunOffline

Setup of WSL is pretty straight forward if you use the Linux images available from the Microsoft Store. However, this is not always possible - e.g. if you’re running Windows 10 Education, the Microsoft Store may be disabled for reasons of security. With LxRunOffline, it is possible to set up WSL with any Linux image, and also chose where WSL default folder is located.

To install LxRunOffline, I prefer Chocolatey package manager. But you can also install lxrunoffline manually.

Open CMD with elevated privileges.

If you have choco, run the following to install LxRunOffline.

choco install lxrunoffline

Decide whether to use WSL1 or 2

See the section at the end of this page for instructions to switch between WSL1 and WSL2. By default, as of 2023, WSL2 will be used.

Install Linux Image

It is up to you what Linux distro to use. These days, it seems developers prefer Debian over Ubuntu, but both are valid choices.

• Download the latest Ubuntu Linux Image *.tar.gz from canonical.com/core/
• Download the latest Debian Linux Image *.tar.xz from docker.debian.net. These are the official Debian Images used in Docker. For example, this is the link to the latest bullseye image, go to the folder and download rootfs.tar.xz.

Below, I use the Debian Bullseye image.

Run the following command to install this image in WSL:

LxRunOffline i -n UF ^
    -d c:\WSL\UFDebian ^
    -f "c:\temp\rootfs.tar.xz" -s

.. where

• -d c:\WSL\UFDebian is the path to the WSL Install. You can also set this to your second hard drive, to safe space on C, e.g. d:\WSL\UFDebian.
• -f c:\temp\rootfs.tar.xz is the path to the image you just downloaded.
• -s means, create a Desktop Shortcut

By filling up the absolute path to the rootfs.tar.xz file, the command will create a distribution named as UF, in directory c:\WSL\UFDebian

You can open the WSL Bash by creating a Shortcut, e.g. on your Desktop, with the following Target:

C:\tools\lxrunoffline\LxRunOffline.exe run -w -n "UF"

(the path may be different, depending on how you installed lxrunoffline)

Once you opened WSL bash, install base packages.

apt-get update \
    && apt-get install -y \
        lsb-release sudo nano git wget rsync \
    && apt-get clean all

.. and check your installation with: lsb_release -a.

It should output something along these lines:

> No LSB modules are available.
> Distributor ID: Debian
> Description:    Debian GNU/Linux 11 (bullseye)
> Release:        11
> Codename:       bullseye

Error: 0x800703fa Illegal operation attempted on a registry key that has been marked for deletion

sc stop lxssmanager
sc start lxssmanager

I also recommend adding a password to user root. The password doesn’t have to be bullet proof, since you’re working in WSL inside Windows, which provides the main security layer. I’d recommend using a simple, easy to remember password that can be typed fast here. Set password with:

passwd root

choco update lxrunoffline

lxrunoffline uninstall -n UF

lxrunoffline unlink -n UF

lxrunoffline move -n UF -d d:\wsl\UF

lxrunoffline sm -n UF

Add WSL to your right-click menu

We want to be able to open WSL console from any folder location. To achieve this, we add WSL to the right click context menu with the following steps.

First, open the registry editor by pressing the Windows key, typing “regedit” into the Start menu, and pressing “Enter”.

Navigate to the following key:

HKEY_CLASSES_ROOT\Directory\Background\shell

• Right-click the “shell” key and select New > Key.
• Name the key “bash” or something similar. You can name it anything you want. This name doesn’t appear in Windows anywhere, and is just used to keep track of the entry in the registry.
• Select “bash” (or whatever you named the key) in the left pane.
• Double-click “(Default)” in the right pane and enter whatever name you want to appear in File Explorer’s context menu. For example, you could enter “Open WSL Bash here” or just “Bash”.
• Next, right-click the “bash” key and select New > Key.
• Name it “command”.
• With the “command” key selected in the left pane, double-click “(Default)” in the right pane and enter the following value: C:\Windows\System32\bash.exe

You are done. You can now right-click a folder in File Explorer and select “Open WSL Bash here” (or whatever you named the option) to quickly open a Bash shell to that specific folder. This option will appear immediately, so you don’t have to sign out or reboot first.

Create WSL default user

You don’t want to work with the root user by default.

Follow the steps below to create a new user account and give it sudo access. If you want to configure sudo for an existing user, skip to step 3.

1. Open your WSL-Bash

2. Create a new user account.

   Create a new user account using the adduser command. Don’t forget to replace {your-username-here} with the user name that you want to create:

   export username={your-username-here}
   adduser $username
   

   You will be prompted to set and confirm the new user password.

   Similar to the root user, you can use a relatively weak password that is easy to remember.

   Adding user `username' ...
   Adding new group `username' (1001) ...
   Adding new user `username' (1001) with group `username' ...
   Creating home directory `/home/username' ...
   Copying files from `/etc/skel' ...
   New password:
   Retype new password:
   passwd: password updated successfully
   

   Once you set the password the command will create a home directory for the user, copy several configuration files in the home directory and prompts you to set the new user’s information. If you want to leave all of this information blank just press ENTER to accept the defaults.

   Changing the user information for username
   Enter the new value, or press ENTER for the default
       Full Name []:
       Room Number []:
       Work Phone []:
       Home Phone []:
       Other []:
   Is the information correct? [Y/n]
   

3. Add the new user to the sudo group

   By default on Ubuntu systems, members of the group sudo are granted with sudo access. To add the user you created to the sudo group use the usermod command:

   usermod -aG sudo $username
   

Change default user for WSL

According to this issue, WSL sometimes starts with the wrong user account (e.g. you don’t want to use root by default). To correct this, first find out which user-id you need to provide (in WSL terminal):

id $username

will show something like this:

uid=1000(my-user-name) gid=1000(my-user-name) groups=1000(my-user-name),27(sudo)

Then use the given uid to update the default user (in Windows Command Line):

lxrunoffline su -n UF -v 1000

Fix WSL mount

By default, folders outside of WSL will have a /mnt/ in front of the actual path (because external paths are mounted into Linux WSL). This makes working with WSL a bit cumbersome, we can remove this /mnt/ with a WSL configuration file.

Create and modify the new WSL configuration file:

sudo nano /etc/wsl.conf

• Now make it look like this and save the file when you’re done:

[automount]
root = /
options = "metadata"

Afterwards, in cmd, shutdown WSL with wsl --shutdown and reopen the WSL shell. You should see your paths starting with the drive letters, without /mnt:

/c/User/...

Install & configure packages

This is rather open ended, but there are some packages that you may want to install and configure such as locales or configuration of time zone.

Locales:

sudo bash -c 'apt-get clean && apt-get update && apt-get install -y locales'
sudo dpkg-reconfigure --frontend readline locales
# when asked, select the number for en_US.UTF-8 (e.g. 158) and confirm
# when asked again, select the number for `en_US.UTF-8` as default locale

Time zone:

sudo dpkg-reconfigure tzdata

Install Docker

You want to be able to run docker-compose and docker client from WSL1, but the backend needs to be run through native Windows 10 Docker Server. Therefore, install Docker Desktop for Windows 10 first.

Option 1: Install with Chocolatey Package Manager with choco install docker-desktop

Option 2: Download from here and install manually

Setup Docker

In Docker Desktop (Windows), under General, make sure you uncheck Expose daemon on tcp://localhost/2375 without TLS.

Also, disable Use the WSL 2 based engine.

Under Shared Drives, select the drive where you keep your project folders (e.g. docker-compose.yml etc.).

This will allow docker from WSL1 access files on this drive, which is necessary if you want to execute docker-compose in a folder on d:.

I’ve also changed some of the Resource/Advanced settings to fit my needs:

As you can see, I’ve also changed the location where Docker stores my virtual containers to D:

From now on, we’re working in the WSL bash (open with right-click and Open WSL bash here).

• Update the apt package list.

sudo apt-get update

• Install Docker’s package dependencies.

sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

• Download and add Docker’s official public PGP key.

sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

• Use the following command to set up the repository

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

• Install Docker Engine

sudo chmod a+r /etc/apt/keyrings/docker.gpg
sudo apt-get update

• Install Docker Engine, containerd, and Docker Compose.

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

• Allow your user to access the Docker CLI without needing root access.

sudo usermod -aG docker $USER

Docker Compose

There is no need to install docker-compose anymore.

docker compose is now available by default (note the missing - in the command)

Setup secure WSL1-Docker connection

A secure connection between WSL1 and Docker Desktop is available through socat¹ and npiperelay.

The steps to set up npiperelay:

1. Install Socat and tmux

• tmux for starting the relay in a background session
• socat for the docker relay stream

sudo apt update && sudo apt install socat tmux

2. Get the the npiperelay.exe and setup connection

You need to use your Windows username (/c/Users/<username>).

Export your Windows user name as a variable:

export wuser=<username>

Get npiperelay.exe from Github releases, build, and store in Windows user directory:

cd /c/Users/$wuser/
mkdir -p go/bin/
wget -O \
    go/bin/npiperelay_windows_amd64.zip \
    https://github.com/jstarks/npiperelay/releases/download/v0.1.0/npiperelay_windows_amd64.zip
unzip -d go/bin/ go/bin/npiperelay_windows_amd64.zip
rm go/bin/npiperelay_windows_amd64.zip

Symlink npiperelay.exe from Windows to WSL1:

sudo ln -s \
    /c/Users/$wuser/go/bin/npiperelay.exe \
    /usr/local/bin/npiperelay.exe

Get docker-relay script and run as background task:

cd ~
git clone https://github.com/jstarks/npiperelay.git
sudo ln -s ~/npiperelay/scripts/docker-relay /usr/local/bin/
chmod +x /usr/local/bin/docker-relay

We modify docker-relay so it starts the Socket inside the user directory.

mkdir ~/sockets
nano /usr/local/bin/docker-relay

Add the following. Make sure to replace {replace-with-your-username} with your WSL1 username.

#!/bin/sh
SOCKET=/home/{replace-with-your-username}/sockets/docker.sock
if [ -e $SOCKET ]; then rm $SOCKET; fi
exec socat UNIX-LISTEN:$SOCKET,fork,group=docker,umask=007 EXEC:"npiperelay.exe -ep -s //./pipe/docker_engine",nofork

Store with CTRL+O.

I also had to manually remove the docker.sock and symlink with the new one:

cd /var/run
sudo rm docker.sock
sudo ln -s /home/alex/sockets/docker.sock /var/run/docker.sock

Test it with:

sudo docker-relay &
docker info

> Client:
>  Context:    default
>  Debug Mode: false
>  Plugins:
>   app: Docker App (Docker Inc., v0.9.1-beta3)
>   buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
>   scan: Docker Scan (Docker Inc., v0.8.0)

You may need to add permissions to the docker-sock:

sudo chown $USER:$USER /home/$USER/sockets/docker.sock

To autostart the relay, we need to add these lines to ~/.bashrc:

# set WSL env; configure secure Docker connection through
# socat and npiperelay
if cat /proc/version | grep Microsoft > /dev/null; then
  export WSL=true
fi
if [ "$WSL" ]; then
  export DOCKER_HOST="unix://$HOME/sockets/docker.sock"
  if ! pgrep socat > /dev/null; then
    tmux new -s docker-relay-session -d docker-relay
  fi
fi

(nano ~/.bashrc)

Finally, some commands may use sudo to start docker. In this case, DOCKER_HOST will not be available. Modify sudoers to keep this entry:

sudo nano /etc/sudoers

and add the following line:

Defaults env_keep += DOCKER_HOST

Test docker in WSL

Restart Docker for windows!

docker run hello-world

Some additional information for working with WSL & Docker follow below.

sudo apt-get update && sudo apt-get upgrade

Cannot restart container my_container: driver failed programming external connectivity on endpoint my_container 
...

nano ~/.bashrc

alias update_wsl="sudo bash -c 'apt-get update && apt-get upgrade && apt-get dist-upgrade && apt-get autoremove'"

source ~/.bashrc

Use a different WSL terminal

The default Ubuntu Terminal is fine for most situations. If you like to optimize workflows, it can be tempting to check some of the other WSL Terminals available for WSL, such as MobaXTerm, ConEmu or Hyper.

I’ve changed to wsltty because this terminal is small, has a lot of nice features that help reduce daily work, and there are many nice themes available.

You can install wsltty with choco:

choco install wsltty

Some things you may want to do after installation:

• go to WSLtty entry folder in start menu and run
  • “add default to context menu” (this will add the “open”-shortcut to right-click menu)
  • “configure WSL shortcuts” (this should be automatically run with choco)
• on the upper left, click on the Linux icon:
  • set “right click to paste”
  • set “copy on select” (all selected text is copied)
  • set “CTRL+SHIFT+ letter shortcuts” (allows you to use CTRL+SHIFT+V to paste)
• install theme, for example gruvbox.minttyrc
  • open WSL, then go to this folder (replace AD with your username):

    /c/Users/AD/AppData/Roaming/wsltty/themes/
    

    (Explorer: %AppData%/wsltty/themes/)
  • then:

    wget https://raw.githubusercontent.com/morhetz/gruvbox-contrib/master/mintty/gruvbox.minttyrc
    

  • optional: nano gruvbox.minttyrc and use 38,38,38 as BackgroundColor (slightly darker - nice tip from this guy)
  • now you can chose the theme under options
  • additional settings for WSLtty:
    • activate CTRL+Shift+letter shortcuts (lets you use e.g. CTRL+Shift+V for copy - note that any mouse selection is also always copied)
    • Window size 180x24 (wider display)
    • “no beep”
    • “no scrollbar” (I can use CTRL+scroll for zoom/shrink)

Here’s how wsltty with the gruvbox.minttyrc theme looks:

Backup WSL distro

Use LxRunOffline, to back up (or restore) your WSL distro in/from a tar.gz file.

Optionally, clean up the tmp folder, before backing up:

sudo find /tmp -type f -atime +10 -delete

Optionally, purge unused old Linux Kernels:

sudo apt --purge autoremove

Then, shutdown WSL and store or load from .tar.gz.

wsl --list --all -v
wsl --shutdown
LxRunOffline.exe export -n UF -f c:/temp/UF.tar.gz
LxRunOffline.exe install -n UF2 -d D:/wsl/UF2 -f c:/temp/UF.tar.gz

Update from WSL 1 to WSL 2

You can upgrade distros from WSL1 to WSL2 (and vice versa). It is a good idea to make a backup before upgrading.

Use cmd for the following commands.

dism.exe /online ^
    /enable-feature ^
    /featurename:VirtualMachinePlatform ^
    /all /norestart

WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel For information on key differences with WSL 2 please visit https://aka.ms/wsl2

Convert existing WSL distros:

wsl --list --verbose

NAME STATE VERSION

• UF Running 1

wsl --set-version UF 2

Note: If you ever have trouble starting a specific distro through a shortcut, try opening it directly from within cmd:

wsl -d UF

Importing the distribution failed.

This may happen for various reasons. In my case, hard links used in ruby gems caused this failure.

After removing all gems, conversion did work.

gem cleanup --dryrun
gem cleanup
apt remove ruby
rm -rf /home/$USER/gems

See WSL/issues/4457.

Fix Docker Connectivity after WSL2 upgrade

If you used WSL1 before, there may be an entry in ~/.bashrc file that must be removed.

1. Removed export DOCKER_HOST=tcp://localhost:2375 from ~/.bashrc;

2. Disabled Expose daemon on tcp://localhost:2375 without TLS in Docker Desktop;

3. Enabled Use the WSL 2 based engine in Docker Desktop

See WSL/issues/4321

Update default distro

If you have multiple WSL distros, set the default one to open with the command line or power shell:

wsl --list --all -v
> UF (Default)
> UF2
wsl --setdefault UF2

If using WSLtty, don’t forget to update your shortcuts and default context menu.

To remove a distro completely:

wsl --unregister <distro name>

Update WSL Kernel

This should be as simple as running:

wsl --update

Just that it isn’t, sometimes.

I would get (Windows Education with disabled App Store):

To install this application you need either a Windows developer license or a sideloading-enabled system.

These steps worked for me:

1. Download latest AppX WSL manually from WSL Releases

2. Use Admin PowerShell and run Add-AppxPackage ./Microsoft.WSL_2.1.5.0_x64_ARM64.msixbundle

3. This failed, for the following reason as stated above.

4. Enable Developer mode. Use Show-WindowsDeveloperLicenseRegistration (powerShell)

5. The Button is grayed out

6. Open Policies gpedit.msc

7. Go to Computer Configuration > Administrative Templates > Windows Components > App Package Deployment.

   • Click “Allows development of Windows Store apps.”
   • Change to “Enabled”

8. Developer Mode should now be active

9. Run Add-AppxPackage ./Microsoft.WSL_2.1.5.0_x64_ARM64.msixbundle

10. If it works, disable developer mode in policies afterwards.

Check the version afterwards (cmd):

wsl --version # (or wsl --status) 
> WSL-Version: 2.1.5.0
> Kernelversion: 5.15.146.1-2
> WSLg-Version: 1.0.60
> MSRDC-Version: 1.2.5105
> Direct3D-Version: 1.611.1-81528511
> DXCore-Version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
> Windows-Version: 10.0.22631.3155

Upgrade Linux distro

This is not related to WSL. You can follow any of the official instructions to update (e.g.) from Bionic Beaver 18.04 to Focal Fossa 20.04.

Make a backup before you upgrade!

sudo apt update && sudo apt upgrade
sudo apt install update-manager-core

You may need to uninstall screen to get do-release-upgrade to work in WSL:

sudo apt-get remove screen

Edit release-upgrades

sudo nano /etc/update-manager/release-upgrades

And change:

prompt=lts

to:

prompt=normal

Restart, then:

sudo do-release-upgrade
sudo reboot

Validate afterwards:

lsb_release -a

After the upgrade, you may need to enable 3rd-Party repos.

cd /etc/apt/sources.list.d/
ls -l

Edit the files with nano to uncomment repos.

Afterwards, run:

sudo apt update
sudo apt upgrade

List of WSL issues

Over the years, I have seen many different issues with WSL under different systems. This is a collection of issues and solutions.

On some rainy day, I suddenly saw this for my WSL2:

wsl -d UF
> Logon failure: the user has not been granted the requested logon type at this computer.
> Error code: Wsl/Service/CreateInstance/CreateVm/HCS/0x80070569

Solved with:

Get-Service vmcompute | Restart-Service