Part II: Setup Windows Subsystem for Linux and Docker in Windows 10
This is Part II of a 3-part blog post on how I configured my working environment. Here I’ll explain how to configure Windows Subsystem for Linux (WSL) and Docker.
- Part I: Setup Dev Environment in Windows 10
- Part II: Windows Subsystem for Linux (WSL) and Docker setup
- Part III: Installation of additional packages and tools in WSL
Since Microsoft announced WSL, Windows 10 became a possible alternative to Linux and MacOS for software development. WSL combines the advantages of having a Linux Kernel backend (running Python, bash etc.) and a Windows graphical frontend, e.g. for working with Visual Studio Code.
WSL1 or WSL2?
Preparations
Enable Windows Features:
- WSL
- Virtual Machine Platform Windows (“VM Plattform)
- Hyper-V
Non-admin user?
Install a Linux distro with LxRunOffline
Setup of WSL is pretty straight forward if you use the Linux images available from the Microsoft Store. However, this is not always possible - e.g. if you’re running Windows 10 Education, the Microsoft Store may be disabled for reasons of security. With LxRunOffline, it is possible to set up WSL with any Linux image, and also chose where WSL default folder is located.
To install LxRunOffline, I prefer Chocolatey package manager. But you can also install lxrunoffline manually.
Open CMD with elevated privileges.
If you have choco, run the following to install LxRunOffline
.
choco install lxrunoffline
2024-02-19 Note for outdated lxrunoffline
on choco
Decide whether to use WSL1 or 2
See the section at the end of this page for instructions to switch between WSL1 and WSL2. By default, as of 2023, WSL2 will be used.
Install Linux Image
It is up to you what Linux distro to use. These days, it seems developers prefer Debian over Ubuntu, but both are valid choices.
- Download the latest Ubuntu Linux Image
*.tar.gz
from canonical.com/core/ - Download the latest Debian Linux Image
*.tar.xz
from docker.debian.net. These are the official Debian Images used in Docker. For example, this is the link to the latest bullseye image, go to the folder and downloadrootfs.tar.xz
.
Below, I use the Debian Bullseye image.
Run the following command to install this image in WSL:
LxRunOffline i -n UF ^
-d c:\WSL\UFDebian ^
-f "c:\temp\rootfs.tar.xz" -s
.. where
-d c:\WSL\UFDebian
is the path to the WSL Install. You can also set this to your second hard drive, to safe space onC
, e.g.d:\WSL\UFDebian
.-f c:\temp\rootfs.tar.xz
is the path to the image you just downloaded.-s
means, create a Desktop Shortcut
By filling up the absolute path to the rootfs.tar.xz
file,
the command will create a distribution named as UF
, in directory c:\WSL\UFDebian
You can open the WSL Bash by creating a Shortcut, e.g. on your Desktop, with the following Target:
C:\tools\lxrunoffline\LxRunOffline.exe run -w -n "UF"
(the path may be different, depending on how you installed lxrunoffline)
Once you opened WSL bash, install base packages.
apt-get update \
&& apt-get install -y \
lsb-release sudo nano git wget rsync \
&& apt-get clean all
.. and check your installation with: lsb_release -a
.
It should output something along these lines:
> No LSB modules are available.
> Distributor ID: Debian
> Description: Debian GNU/Linux 11 (bullseye)
> Release: 11
> Codename: bullseye
Error: 0x800703fa Illegal operation attempted on a registry key that has been marked for deletion
If the following error occurs:
Error: 0x800703fa Illegal operation attempted on a registry key that has been marked for deletion
Restart lxssmanager (in Windows command prompt)
sc stop lxssmanager
sc start lxssmanager
I also recommend adding a password to user root
.
The password doesn’t have to be bullet proof, since you’re working in WSL inside Windows,
which provides the main security layer. I’d recommend using a simple, easy to remember
password that can be typed fast here. Set password with:
passwd root
Additional commands: lxrunoffline
To keep lxrunoffline up to date (in choco):
choco update lxrunoffline
To remove an existing WSL distro (e.g. named UF
):
lxrunoffline uninstall -n UF
If, for any reason, a WSL distro got corrupt, it is possible to unlink (=remove)
it with the following command (e.g. named UF
):
lxrunoffline unlink -n UF
If you later decide you want to move your WSL installation
(e.g. from C:/wsl/
to D:/wsl/
) ^1
instructions:
lxrunoffline move -n UF -d d:\wsl\UF
To view additional information for a distro (such as the path it is installed to):
lxrunoffline sm -n UF
Add WSL to your right-click menu
We want to be able to open WSL console from any folder location. To achieve this, we add WSL to the right click context menu with the following steps.
First, open the registry editor by pressing the Windows key, typing “regedit” into the Start menu, and pressing “Enter”.
Navigate to the following key:
HKEY_CLASSES_ROOT\Directory\Background\shell
- Right-click the “shell” key and select New > Key.
- Name the key “bash” or something similar. You can name it anything you want. This name doesn’t appear in Windows anywhere, and is just used to keep track of the entry in the registry.
- Select “bash” (or whatever you named the key) in the left pane.
- Double-click “(Default)” in the right pane and enter whatever name you want to appear in File Explorer’s context menu. For example, you could enter “Open WSL Bash here” or just “Bash”.
- Next, right-click the “bash” key and select New > Key.
- Name it “command”.
- With the “command” key selected in the left pane, double-click “(Default)” in
the right pane and enter the following value:
C:\Windows\System32\bash.exe
You are done. You can now right-click a folder in File Explorer and select “Open WSL Bash here” (or whatever you named the option) to quickly open a Bash shell to that specific folder. This option will appear immediately, so you don’t have to sign out or reboot first.
Create WSL default user
You don’t want to work with the root
user by default.
Follow the steps below to create a new user account and give it sudo access. If you want to configure sudo for an existing user, skip to step 3.
Open your WSL-Bash
Create a new user account.
Create a new user account using the
adduser
command. Don’t forget to replace{your-username-here}
with the user name that you want to create:export username={your-username-here} adduser $username
You will be prompted to set and confirm the new user password.
Similar to the root user, you can use a relatively weak password that is easy to remember.
Adding user `username' ... Adding new group `username' (1001) ... Adding new user `username' (1001) with group `username' ... Creating home directory `/home/username' ... Copying files from `/etc/skel' ... New password: Retype new password: passwd: password updated successfully
Once you set the password the command will create a home directory for the user, copy several configuration files in the home directory and prompts you to set the new user’s information. If you want to leave all of this information blank just press
ENTER
to accept the defaults.Changing the user information for username Enter the new value, or press ENTER for the default Full Name []: Room Number []: Work Phone []: Home Phone []: Other []: Is the information correct? [Y/n]
Add the new user to the
sudo
groupBy default on Ubuntu systems, members of the group
sudo
are granted with sudo access. To add the user you created to the sudo group use the usermod command:usermod -aG sudo $username
Change default user for WSL
According to this issue, WSL sometimes starts with the wrong user account (e.g. you don’t want to use root by default). To correct this, first find out which user-id you need to provide (in WSL terminal):
id $username
will show something like this:
uid=1000(my-user-name) gid=1000(my-user-name) groups=1000(my-user-name),27(sudo)
Then use the given uid
to update the default user (in Windows Command Line):
lxrunoffline su -n UF -v 1000
Fix WSL mount
By default, folders outside of WSL will have a /mnt/
in front of the actual path
(because external paths are mounted into Linux WSL).
This makes working with WSL a bit cumbersome, we can remove this /mnt/ with a WSL
configuration file.
Create and modify the new WSL configuration file:
sudo nano /etc/wsl.conf
- Now make it look like this and save the file when you’re done:
[automount]
root = /
options = "metadata"
Afterwards, in cmd
, shutdown WSL with wsl --shutdown
and reopen the WSL shell. You should see
your paths starting with the drive letters, without /mnt
:
/c/User/...
Install & configure packages
This is rather open ended, but there are some packages that you may want to install and configure such as locales or configuration of time zone.
Locales:
sudo bash -c 'apt-get clean && apt-get update && apt-get install -y locales'
sudo dpkg-reconfigure --frontend readline locales
# when asked, select the number for en_US.UTF-8 (e.g. 158) and confirm
# when asked again, select the number for `en_US.UTF-8` as default locale
Time zone:
sudo dpkg-reconfigure tzdata
Install Docker
You want to be able to run docker-compose and docker client from WSL1, but the backend needs to be run through native Windows 10 Docker Server. Therefore, install Docker Desktop for Windows 10 first.
Option 1: Install with Chocolatey Package Manager with choco install docker-desktop
Option 2: Download from here and install manually
Setup Docker
In Docker Desktop (Windows), under General, make sure you uncheck Expose daemon on tcp://localhost/2375 without TLS
.
Also, disable Use the WSL 2 based engine
.
Under Shared Drives,
select the drive where you keep your project folders (e.g. docker-compose.yml
etc.).
This will allow docker from WSL1 access files on this drive,
which is necessary if you want to execute docker-compose in a folder on d:
.
I’ve also changed some of the Resource/Advanced settings to fit my needs:
As you can see, I’ve also changed the location where Docker stores my virtual containers to D:
From now on, we’re working in the WSL bash (open with right-click and Open WSL bash here
).
- Update the apt package list.
sudo apt-get update
- Install Docker’s package dependencies.
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
- Download and add Docker’s official public PGP key.
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
- Use the following command to set up the repository
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
- Install Docker Engine
sudo chmod a+r /etc/apt/keyrings/docker.gpg
sudo apt-get update
- Install Docker Engine, containerd, and Docker Compose.
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
- Allow your user to access the Docker CLI without needing root access.
sudo usermod -aG docker $USER
Docker Compose
There is no need to install docker-compose anymore.
docker compose
is now available by default (note the missing -
in the command)
Setup secure WSL1-Docker connection
A secure connection between WSL1 and Docker Desktop is available
through socat
1 and
npiperelay.
Connect WSL1<->Docker the insecure way?
The steps to set up npiperelay
:
- Install Socat and tmux
- tmux for starting the relay in a background session
- socat for the docker relay stream
sudo apt update && sudo apt install socat tmux
- Get the the
npiperelay.exe
and setup connection
You need to use your Windows username (/c/Users/<username>
).
Export your Windows user name as a variable:
export wuser=<username>
Get npiperelay.exe
from Github releases,
build, and store in Windows user directory:
cd /c/Users/$wuser/
mkdir -p go/bin/
wget -O \
go/bin/npiperelay_windows_amd64.zip \
https://github.com/jstarks/npiperelay/releases/download/v0.1.0/npiperelay_windows_amd64.zip
unzip -d go/bin/ go/bin/npiperelay_windows_amd64.zip
rm go/bin/npiperelay_windows_amd64.zip
Symlink npiperelay.exe
from Windows to WSL1:
sudo ln -s \
/c/Users/$wuser/go/bin/npiperelay.exe \
/usr/local/bin/npiperelay.exe
Get docker-relay
script and run as background task:
cd ~
git clone https://github.com/jstarks/npiperelay.git
sudo ln -s ~/npiperelay/scripts/docker-relay /usr/local/bin/
chmod +x /usr/local/bin/docker-relay
We modify docker-relay so it starts the Socket inside the user directory.
mkdir ~/sockets
nano /usr/local/bin/docker-relay
Add the following. Make sure to replace {replace-with-your-username}
with your WSL1 username.
#!/bin/sh
SOCKET=/home/{replace-with-your-username}/sockets/docker.sock
if [ -e $SOCKET ]; then rm $SOCKET; fi
exec socat UNIX-LISTEN:$SOCKET,fork,group=docker,umask=007 EXEC:"npiperelay.exe -ep -s //./pipe/docker_engine",nofork
Store with CTRL+O.
I also had to manually remove the docker.sock
and symlink with the new one:
cd /var/run
sudo rm docker.sock
sudo ln -s /home/alex/sockets/docker.sock /var/run/docker.sock
Test it with:
sudo docker-relay &
docker info
> Client:
> Context: default
> Debug Mode: false
> Plugins:
> app: Docker App (Docker Inc., v0.9.1-beta3)
> buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
> scan: Docker Scan (Docker Inc., v0.8.0)
You may need to add permissions to the docker-sock
:
sudo chown $USER:$USER /home/$USER/sockets/docker.sock
To autostart the relay, we need to add these lines to ~/.bashrc
:
# set WSL env; configure secure Docker connection through
# socat and npiperelay
if cat /proc/version | grep Microsoft > /dev/null; then
export WSL=true
fi
if [ "$WSL" ]; then
export DOCKER_HOST="unix://$HOME/sockets/docker.sock"
if ! pgrep socat > /dev/null; then
tmux new -s docker-relay-session -d docker-relay
fi
fi
(nano ~/.bashrc
)
Finally, some commands may use sudo
to start docker. In this case,
DOCKER_HOST
will not be available. Modify sudoers to keep this entry:
sudo nano /etc/sudoers
and add the following line:
Defaults env_keep += DOCKER_HOST
Test docker in WSL
Restart Docker for windows!
docker run hello-world
Some additional information for working with WSL & Docker follow below.
Keep WSL distro up to date
To upgrade your WSL installation and packages, use:
sudo apt-get update && sudo apt-get upgrade
Fix Docker Connectivity Bug
There seems to be a bug, where Docker on Windows 10 may see errors similar to this after restarting the computer:
Cannot restart container my_container: driver failed programming external connectivity on endpoint my_container
...
The steps described below may solve this issue.
How to disable fast startup on Windows 10
Disable it in just a few steps:
- Right-click the Start button.
- Click Search.
- Type
Control Panel
and hit Enter on your keyboard. - Click
Power Options
. - Click
Choose what the power buttons do
. - Click
Change settings that are currently unavailable
. - Click
Turn on fast startup (recommended)
so that the check-mark disappears. - Click
Save changes
.
Annoying WSL bug: Missing Access rights
Now this may occur on very few occasions, only for WSL1 - for me it happened mainly when building
python packages (e.g. setuptools
, distutils
, pypi
). Since the latest Windows Update,
some access rights changed and those affect how
packages in WSL can access folders.
Wherever you keep your code, right click main folder, select Security and allow “Full Access” to authenticated users.
Good things to know when working with WSL-Bash
- Never edit any files of the WSL distro with Windows
E.g. anything below C:\WSL\UFDebian
is holy Linux territory that should never be touched
from Windows. Windows will mess with Linux file permissions, which is quite difficult to fix afterwards.
- Exit Shell Shortcut
Use Ctrl + D to exit the current shell.
- Add Command Shortcuts
The .bashrc
file can be used to add aliases for commands.
The following instructions will add an alias update_wsl
that will update WSL.
Open .bashrc in nano text editor:
nano ~/.bashrc
Add the following line at any position:
alias update_wsl="sudo bash -c 'apt-get update && apt-get upgrade && apt-get dist-upgrade && apt-get autoremove'"
Reload .bashrc
file:
source ~/.bashrc
Use a different WSL terminal
The default Ubuntu Terminal is fine for most situations. If you like to optimize workflows, it can be tempting to check some of the other WSL Terminals available for WSL, such as MobaXTerm, ConEmu or Hyper.
I’ve changed to wsltty because this terminal is small, has a lot of nice features that help reduce daily work, and there are many nice themes available.
You can install wsltty with choco:
choco install wsltty
Some things you may want to do after installation:
- go to WSLtty entry folder in start menu and run
- “add default to context menu” (this will add the “open”-shortcut to right-click menu)
- “configure WSL shortcuts” (this should be automatically run with choco)
- on the upper left, click on the Linux icon:
- set “right click to paste”
- set “copy on select” (all selected text is copied)
- set “CTRL+SHIFT+ letter shortcuts” (allows you to use CTRL+SHIFT+V to paste)
- install theme, for example gruvbox.minttyrc
- open WSL, then go to this folder (replace AD with your username):
(Explorer:/c/Users/AD/AppData/Roaming/wsltty/themes/
%AppData%/wsltty/themes/
) - then:
wget https://raw.githubusercontent.com/morhetz/gruvbox-contrib/master/mintty/gruvbox.minttyrc
- optional:
nano gruvbox.minttyrc
and use38,38,38
as BackgroundColor (slightly darker - nice tip from this guy) - now you can chose the theme under options
- additional settings for WSLtty:
- activate CTRL+Shift+letter shortcuts (lets you use e.g. CTRL+Shift+V for copy - note that any mouse selection is also always copied)
- Window size 180x24 (wider display)
- “no beep”
- “no scrollbar” (I can use CTRL+scroll for zoom/shrink)
- open WSL, then go to this folder (replace AD with your username):
Here’s how wsltty with the gruvbox.minttyrc theme looks:
Backup WSL distro
Use LxRunOffline, to back up (or restore) your WSL distro in/from a tar.gz
file.
Optionally, clean up the tmp
folder, before backing up:
sudo find /tmp -type f -atime +10 -delete
Optionally, purge unused old Linux Kernels:
sudo apt --purge autoremove
Then, shutdown WSL and store or load from .tar.gz
.
wsl --list --all -v
wsl --shutdown
LxRunOffline.exe export -n UF -f c:/temp/UF.tar.gz
LxRunOffline.exe install -n UF2 -d D:/wsl/UF2 -f c:/temp/UF.tar.gz
Update from WSL 1 to WSL 2
You can upgrade distros from WSL1 to WSL2 (and vice versa). It is a good idea to make a backup before upgrading.
Use cmd
for the following commands.
dism.exe /online ^
/enable-feature ^
/featurename:VirtualMachinePlatform ^
/all /norestart
WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel For information on key differences with WSL 2 please visit https://aka.ms/wsl2
Convert existing WSL distros:
wsl --list --verbose
NAME STATE VERSION
- UF Running 1
wsl --set-version UF 2
Note: If you ever have trouble starting a specific distro through a shortcut, try opening it directly from within cmd:
wsl -d UF
Importing the distribution failed.
This may happen for various reasons. In my case, hard links used in ruby gems caused this failure.
After removing all gems, conversion did work.
gem cleanup --dryrun
gem cleanup
apt remove ruby
rm -rf /home/$USER/gems
See WSL/issues/4457.
Fix Docker Connectivity after WSL2 upgrade
If you used WSL1 before, there may be an entry in ~/.bashrc
file that must be removed.
Removed
export DOCKER_HOST=tcp://localhost:2375
from~/.bashrc
;Disabled
Expose daemon on tcp://localhost:2375 without TLS
in Docker Desktop;Enabled
Use the WSL 2 based engine
in Docker Desktop
See WSL/issues/4321
Update default distro
If you have multiple WSL distros, set the default one to open with the command line or power shell:
wsl --list --all -v
> UF (Default)
> UF2
wsl --setdefault UF2
If using WSLtty, don’t forget to update your shortcuts and default context menu.
To remove a distro completely:
wsl --unregister <distro name>
Update WSL Kernel
This should be as simple as running:
wsl --update
Just that it isn’t, sometimes.
I would get (Windows Education with disabled App Store):
To install this application you need either a Windows developer license or a sideloading-enabled system.
These steps worked for me:
Download latest AppX WSL manually from WSL Releases
Use Admin PowerShell and run
Add-AppxPackage ./Microsoft.WSL_2.1.5.0_x64_ARM64.msixbundle
This failed, for the following reason as stated above.
Enable Developer mode. Use
Show-WindowsDeveloperLicenseRegistration
(powerShell)The Button is grayed out
Open Policies
gpedit.msc
Go to Computer Configuration > Administrative Templates > Windows Components > App Package Deployment.
- Click “Allows development of Windows Store apps.”
- Change to “Enabled”
Developer Mode should now be active
Run
Add-AppxPackage ./Microsoft.WSL_2.1.5.0_x64_ARM64.msixbundle
If it works, disable developer mode in policies afterwards.
Check the version afterwards (cmd
):
wsl --version # (or wsl --status)
> WSL-Version: 2.1.5.0
> Kernelversion: 5.15.146.1-2
> WSLg-Version: 1.0.60
> MSRDC-Version: 1.2.5105
> Direct3D-Version: 1.611.1-81528511
> DXCore-Version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
> Windows-Version: 10.0.22631.3155
Upgrade Linux distro
This is not related to WSL. You can follow any of the official instructions to update (e.g.) from Bionic Beaver 18.04 to Focal Fossa 20.04.
Make a backup before you upgrade!
sudo apt update && sudo apt upgrade
sudo apt install update-manager-core
You may need to uninstall screen to get do-release-upgrade
to work in WSL:
sudo apt-get remove screen
Edit release-upgrades
sudo nano /etc/update-manager/release-upgrades
And change:
prompt=lts
to:
prompt=normal
Restart, then:
sudo do-release-upgrade
sudo reboot
Validate afterwards:
lsb_release -a
After the upgrade, you may need to enable 3rd-Party repos.
cd /etc/apt/sources.list.d/
ls -l
Edit the files with nano
to uncomment repos.
Afterwards, run:
sudo apt update
sudo apt upgrade
List of WSL issues
Over the years, I have seen many different issues with WSL under different systems. This is a collection of issues and solutions.
On some rainy day, I suddenly saw this for my WSL2:
wsl -d UF
> Logon failure: the user has not been granted the requested logon type at this computer.
> Error code: Wsl/Service/CreateInstance/CreateVm/HCS/0x80070569
Solved with:
Get-Service vmcompute | Restart-Service