Part II: Setup Windows Subsystem for Linux and Docker in Windows 10

This is Part II of a 3-part blog post on how I configured my working environment. Here I’ll explain how to configure Windows Subsystem for Linux (WSL) and Docker.

  • Part I: Setup Dev Environment in Windows 10
  • Part II: Windows Subsystem for Linux (WSL) and Docker setup
  • Part III: Installation of additional packages and tools in WSL

Since Microsoft announced WSL, Windows 10 became a possible alternative to Linux and MacOS for software development. WSL combines the advantages of having a Linux Kernel backend (running Python, bash etc.) and a Windows graphical frontend, e.g. for working with Visual Studio Code.

Install Ubuntu with LxRunOffline

Setup of WSL is pretty straight forward if you use the Linux images available from the Microsoft Store. However, this is not always possible - e.g. if you’re running Windows 10 Education, the Microsoft Store may be disabled for reasons of security. With LxRunOffline, it is possible to set up WSL with any Linux image, and also chose where WSL default folder is located.

To install LxRunOffline, I prefer Chocolatey package manager. But you can also install lxrunoffline manually.

Open CMD with elevated privileges.

If you have choco, run the following to install LxRunOffline.

choco install lxrunoffline

Decide whether to use WSL1 or 2

See the section at the end of this page for instructions to switch between WSL1 and WSL2. If you do nothing, I believe that WSL1 will be used as default.

Install Ubuntu Linux Image

First, download the latest Ubuntu Linux Image *.tar.gz from canonical.com/core/

Here, I use the Ubuntu Focal Fossa image.

The file name will be similar to ubuntu-focal-core-cloudimg-amd64-root.tar.gz.

Run the following command to install this image in WSL:

LxRunOffline i -n UF ^
    -d c:\WSL\UFull ^
    -f "c:\temp\ubuntu-bionic-core-cloudimg-amd64-root.tar.gz" -s

.. where

  • c:\WSL\UFull is the path to the WSL Install. You can also set this to your second harddrive to safe space on C, e.g. d:\WSL\UFull.
  • c:\temp\ubuntu-bionic-core-cloudimg-amd64-root.tar.gz is the path to the image you just downloaded.

By filling up the precise absolute path to the ubuntu-bionic-core-cloudimg-amd64-root.tar.gz file, the command will create a distribution named as UF, in directory c:\WSL\UFull

You can open the WSL Bash by creating a Shortcut, e.g. on your Desktop, with the following Target:

C:\tools\lxrunoffline\LxRunOffline.exe run -w -n "UF"

(the path may be different, depending on how you installed lxrunoffline)

Once you opened WSL bash, check your installation with: lsb_release -a. It should output something along these lines:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

If the following error occurs:

Error: 0x800703fa Illegal operation attempted on a registry key that has been marked for deletion

Restart lxssmanager (in Windows command prompt)

sc stop lxssmanager
sc start lxssmanager

After several setups of WSL with lxrunoffline, I also noticed that the steps above may slightly vary, depending on the versions used. Here are some additional steps that may be required to follow the instructions below:

  • lsb_release -a command not available, install with:
    • apt-get update && apt-get install lsb-release
  • sudo command not available, install with:
    • apt-get install sudo
  • nano command not available, install with:
    • apt-get install nano

I also recommend adding a password to user root. The password doesn’t have to be bullet proof, since you’re working in WSL inside Windows, which provides the main security layer. I’d recommend using a simple, easy to remember password that can be typed fast here. Set password with:

passwd root
Additional commands: lxrunoffline

To keep lxrunoffline up to date (in choco):

choco update lxrunoffline

To remove an existing WSL distro (e.g. named UF):

lxrunoffline uninstall -n UF

If, for any reason, a WSL distro got corrupt, it is possible to unlink (=remove) it with the following command (e.g. named UF):

lxrunoffline unlink -n UF

If you later decide you want to move your WSL installation (e.g. from C:/wsl/ to D:/wsl/), check these instructions.

Add WSL to your right-click menu

We want to be able to open WSL console from any folder location. To achive this, we add WSL to the right click context menu with the following steps.

First, open the registry editor by pressing the Windows key, typing “regedit” into the Start menu, and pressing “Enter”.

Navigate to the following key:

HKEY_CLASSES_ROOT\Directory\Background\shell
  • Right-click the “shell” key and select New > Key.
  • Name the key “bash” or something similar. You can name it anything you want. This name doesn’t appear in Windows anywhere, and is just used to keep track of the entry in the registry.
  • Select “bash” (or whatever you named the key) in the left pane.
  • Double-click “(Default)” in the right pane and enter whatever name you want to appear in File Explorer’s context menu. For example, you could enter “Open WSL Bash here” or just “Bash”.
  • Next, right-click the “bash” key and select New > Key.
  • Name it “command”.
  • With the “command” key selected in the left pane, double-click “(Default)” in the right pane and enter the following value: C:\Windows\System32\bash.exe

You are done. You can now right-click a folder in File Explorer and select “Open WSL Bash here” (or whatever you named the option) to quickly open a Bash shell to that specific folder. This option will appear immediately, so you don’t have to sign out or reboot first.

Create WSL default user

You don’t want to work with the root user by default.

Follow the steps below to create a new user account and give it sudo access. If you want to configure sudo for an existing user, skip to step 3.

  1. Open your WSL-Bash

  2. Create a new user account.

    Create a new user account using the adduser command. Don’t forget to replace username with the user name that you want to create:

    adduser username
    

    You will be prompted to set and confirm the new user password.

    Similar to the root user, you can use a relatively weak password that is easy to remember.

    Adding user `username' ...
    Adding new group `username' (1001) ...
    Adding new user `username' (1001) with group `username' ...
    Creating home directory `/home/username' ...
    Copying files from `/etc/skel' ...
    New password:
    Retype new password:
    passwd: password updated successfully
    

    Once you set the password the command will create a home directory for the user, copy several configuration files in the home directory and prompts you to set the new user’s information. If you want to leave all of this information blank just press ENTER to accept the defaults.

    Changing the user information for username
    Enter the new value, or press ENTER for the default
        Full Name []:
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
    Is the information correct? [Y/n]
    
  3. Add the new user to the sudo group

    By default on Ubuntu systems, members of the group sudo are granted with sudo access. To add the user you created to the sudo group use the usermod command:

    usermod -aG sudo username
    

Change default user for WSL

According to this issue, WSL sometimes starts with the wrong user account (e.g. you don’t want to use root by default). To correct this, first find out which user-id you need to provide (in WSL terminal):

id

will show something like this:

uid=1000(my-user-name) gid=1000(my-user-name) groups=1000(my-user-name),27(sudo)

Then use the given uid to update the default user (in Windows Command Line):

lxrunoffline su -n UF -v 1000

Fix WSL mount

By default, folders outside of WSL will have a /mnt/ in front of the actual path (because external paths are mounted into Linux WSL). This makes working with WSL a bit cumbersome, we can remove this /mnt/ with a WSL configuration file.

Create and modify the new WSL configuration file:

sudo nano /etc/wsl.conf

  • Now make it look like this and save the file when you’re done:
[automount]
root = /
options = "metadata"

Install Docker

You want to be able to run docker-compose and docker client from WSL, but the backend needs to be run through native Windows 10 Docker Server. Therefore, install Docker Desktop for Windows 10 first.

Option 1: install with Chocolatey Package Manager with choco install docker-desktop

Option 2: Download from here and install manually

If you use WSL1, open Docker Desktop setup (task manager/right-click/setup) and select

Expose daemon on tcp://localhost:2375 without TLS

Additionally, under Shared Drives, select the drive where you keep your project folders (e.g. docker-compose.yml etc.).

This will allow docker from WSL access files on this drive, which is necessary if you want to execute docker-compose in a folder on d:.

I’ve also changed some of the Resource/Advanced settings to fit my needs:

As you can see, I’ve also changed the location where Docker stores my virtual containers to D:

From now on, we’re working in the WSL bash (open with right-click and Open WSL bash here).

  • Update the apt package list.

sudo apt-get update -y

  • Install Docker’s package dependencies.
sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common
  • Download and add Docker’s official public PGP key.
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  • Verify the fingerprint.
sudo apt-key fingerprint 0EBFCD88
  • Add the stable channel’s Docker upstream repository.
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
  • Update the apt package list (for the new apt repo).
sudo apt-get update -y
  • Install the latest version of Docker CE.
sudo apt-get install -y docker-ce
  • Allow your user to access the Docker CLI without needing root access.
sudo usermod -aG docker $USER

Install Docker Compose

You should follow the official docker-compose installation instructions in linux.

sudo curl -L \
    "https://github.com/docker/compose/releases/download/1.25.1/docker-compose-$(uname -s)-$(uname -m)" \
    -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Optionally, follow these instructions to configure docker-compose to run as non-root user.

If any problems occur, check that which docker-compose points to /usr/local/bin/docker-compose.

Configure WSL to Connect to Docker for Windows

Only necessary for WSL1: Connect to a remote Docker daemon with this 1 liner:

echo "export DOCKER_HOST=tcp://localhost:2375" >> ~/.bashrc && source ~/.bashrc

Test docker in WSL

Restart Docker for windows!

docker run hello-world

Keep WSL distro up to date

To upgrade your WSL installation and packages, use:

sudo apt-get update && sudo apt-get upgrade

Fix Docker Connectivity Bug

There seems to be a bug, where Docker on Windows 10 may see errors similar to this after restarting the computer:

Cannot restart container my_container: driver failed programming external connectivity on endpoint my_container 
...

The steps described below may solve this issue.

How to disable fast startup on Windows 10

Disable it in just a few steps:

  1. Right-click the Start button.
  2. Click Search.
  3. Type Control Panel and hit Enter on your keyboard.
  4. Click Power Options.
  5. Click Choose what the power buttons do.
  6. Click Change settings that are currently unavailable.
  7. Click Turn on fast startup (recommended) so that the check-mark disappears.
  8. Click Save changes.

Annoying WSL bug: Missing Access rights

Now this may occur on very few occasions, only for WSL1 - for me it happened mainly when building python packages (e.g. setuptools, distutils, pypi). Since the latest Windows Update, some access rights changed and those affect how packages in WSL can access folders.

Wherever you keep your code, right click main folder, select Security and allow “Full Access” to authenticated users.

full access

Good things to know when working with WSL-Bash

  • Never edit any files of the WSL distro with Windows

E.g. anything below C:\WSL\UFull is holy Linux territory that should never be touched from Windows. Windows will mess with Linux file permissions, which is quite difficult to fix afterwards.

  • Exit Shell Shortcut

Use Ctrl + D to exit the current shell.

  • Add Command Shortcuts

The .bashrc file can be used to add aliases for commands. The following instructions will add an alias update_wsl that will update WSL.

Open .bashrc in nano text editor:

nano ~/.bashrc

Add the following line at any position:

alias update_wsl="sudo bash -c 'apt-get update && apt-get upgrade && apt-get dist-upgrade && apt-get autoremove'"

Reload .bashrc file:

source ~/.bashrc

Use different WSL terminal

The default Ubuntu Terminal is fine for most situations. If you like to optimize workflows, it can be tempting to check some of the other WSL Terminals available for WSL, such as MobaXTerm, ConEmu or Hyper.

I’ve changed to wsltty because this terminal is small, has a lot of nice features that help reduce daily work, and there are many nice themes available.

You can install wsltty with choco:

choco install wsltty

Some things you may want to do after installation:

  • go to WSLtty entry folder in start menu and run
    • “add default to context menu” (this will add the “open”-shortcut to right-click menu)
    • “configure WSL shortcuts” (this should be automatically run with choco)
  • install theme, for example gruvbox.minttyrc
    • open WSL, then go to this folder (replace AD with your username):
      /c/Users/AD/AppData/Roaming/wsltty/themes/
      
    • then:
      wget https://raw.githubusercontent.com/morhetz/gruvbox-contrib/master/mintty/gruvbox.minttyrc
      
    • optional: nano gruvbox.minttyrc and use 38,38,38 as BackgroundColor (slightly darker - nice tip from this guy)
    • now you can chose the theme under options
    • additional settings for WSLtty:
      • activate CTRL+Shift+letter shortcuts (lets you use e.g. CTRL+Shift+V for copy - note that any mouse selection is also always copied)
      • Window size 180x24 (wider display)
      • “no beep”
      • “no scrollbar” (I can use CTRL+scroll for zoom/shrink)

Here’s how wsltty with the gruvbox.minttyrc theme looks:

A nice addition to wsltty is byobu - it is a “window manager and terminal multiplexer”: the important part is that

  1. it allows opening multiple windows next to each other and
  2. running commands in background.

To install byobu:

sudo apt-get install byobu

Start it with byobu. Here’s a list with the most frequent shortcuts I use:

  • F2 - Create a new window
  • F3/F4 - Move focus among windows
  • F6 - Detach session and then logout (byobu window keeps running, even if WSL is closed)
  • Ctrl-F6 - Kill current Window in focus (window closed)
  • F7 - Enter scrollback history
  • F8 - Rename the current window

When you close WSL and reopen it later, type byobu and you’ll have all your Sessions still running.

Backup WSL distro

Use LxRunOffline, to back up (or restore) your WSL distro in/from a tar.gz file:

LxRunOffline.exe export -n UF -f c:/temp/UF.tar.gz
LxRunOffline.exe install -n UF2 -d D:/wsl/UF2 -f c:/temp/UF.tar.gz

Optionally, clean up the tmp folder, before backing up:

sudo find /tmp -type f -atime +10 -delete

Update from WSL 1 to WSL 2

You can upgrade distros from WSL1 to WSL2 (and vice versa). It is a good idea to make a backup before upgrading.

Use cmd for the following commands.

dism.exe /online ^
    /enable-feature ^
    /featurename:VirtualMachinePlatform ^
    /all /norestart

WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel For information on key differences with WSL 2 please visit https://aka.ms/wsl2

Convert existing WSL distros:

wsl --list --verbose

NAME STATE VERSION

  • UF Running 1
wsl --set-version UF 2

Importing the distribution failed.

This may happen for various reasons. In my case, hard links used in ruby gems caused this failure.

After removing all gems, conversion did work.

gem cleanup --dryrun
gem cleanup
apt remove ruby
rm -rf /home/$USER/gems

See WSL/issues/4457.

Fix Docker Connectivity after WSL2 upgrade

If you used WSL1 before, there may be an entry in ~/.bashrc file that must be removed.

  1. Removed export DOCKER_HOST=tcp://localhost:2375 from ~/.bashrc;

  2. Disabled Expose daemon on tcp://localhost:2375 without TLS in Docker Desktop;

  3. Enabled Use the WSL 2 based engine in Docker Desktop

See WSL/issues/4321

Update default distro

If you have multiple WSL distros, set the default one to open with the command line or power shell:

wsl --list --all -v
> UF (Default)
> UF2
wsl --setdefault UF2

If using WSLtty, don’t forget to update your shortcuts and default context menu.