Addie's place on the web...

Just some stuff I want to share with you

Dev Containers Fail Fast

Friday 27 October 2023
Dev ContainersLinuxPodmanWSL 2

I started to experiment with Development Containers using Podman. The Linux system setup by the podman machine init command was so basic that things like git were not installed. So, let’s install git to find out that another thing was missing and a setting had to be changed an yet another one and and another one.

And finally, things still fail… Time for another approach: replace the Podman installed and configured Linux system with another one that comes with Windows/WSL integration.

Fail fast

Instead of trying to make this Linux system work, why not install another one and add Podman to it? Let’s drop the Podman installed system by doing a podman machine stop and podman machine rm and do a wsl --install to get an Ubuntu system up and running. Once that is in place, we’ll follow the instructions on the Podman website to get Podman installed.

Within 5 minutes, the system was up and running with Podman version 3.4.4. Sorry, say again. Which version? 3.4.4. But hang on, we’re at 4.7.1 now. Hmmm… A bit of searching showed that I should not expect to see a newer version of Podman on this Ubuntu system any time soon.

Okay, no worries. There are other Linux distributions we can install on WSL. So, let’s do a wsl --unregister Ubuntu to say goodbye to Ubuntu for now and let’s see what other options we have.

C:\Users\WindowsUser>wsl --list --online
The following is a list of valid distributions that can be installed.
Install using 'wsl.exe --install <Distro>'.

NAME                                   FRIENDLY NAME
Ubuntu                                 Ubuntu
Debian                                 Debian GNU/Linux
kali-linux                             Kali Linux Rolling
Ubuntu-18.04                           Ubuntu 18.04 LTS
Ubuntu-20.04                           Ubuntu 20.04 LTS
Ubuntu-22.04                           Ubuntu 22.04 LTS
OracleLinux_7_9                        Oracle Linux 7.9
OracleLinux_8_7                        Oracle Linux 8.7
OracleLinux_9_1                        Oracle Linux 9.1
openSUSE-Leap-15.5                     openSUSE Leap 15.5
SUSE-Linux-Enterprise-Server-15-SP4    SUSE Linux Enterprise Server 15 SP4
SUSE-Linux-Enterprise-15-SP5           SUSE Linux Enterprise 15 SP5
openSUSE-Tumbleweed                    openSUSE Tumbleweed

I decided to go for openSUSE-Tumbleweed. This Linux distribution is using a rolling release model which means that updates are delivered frequently and it’s providing access to the latest and most secure versions of tools.

Here we go again, but now with a wsl --install openSUSE-Tumbleweed. The installation is straightforward and the distribution has a similar integration with Windows as Ubuntu. This means that we can (without any further configuration) get to files and folders on the Windows filesystem from the Linux system and vice versa. VS Code Remote works out of the box.

Before I installed Podman, I first made sure that everying was up to date using sudo zypper dup and once all the updates were done, I logged out and in again using the wsl command on Windows. Time to install Podman: sudo zypper install podman which resulted in:

LinuxUser@w10laptop:~> podman -v
podman version 4.7.1

Yes, that’s more like it. So, does it work?

LinuxUser@w10laptop:~> podman run hello-world
WARN[0000] "/" is not a shared mount, this could cause issues or missing mounts with rootless containers
Resolved "hello-world" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull quay.io/podman/hello:latest...
Getting image source signatures
Copying blob d08b40be6878 done   |
Copying config e2b3db5d4f done   |
Writing manifest to image destination
!... Hello Podman World ...!

         .--"--.
       / -     - \
      / (O)   (O) \
   ~~~| -=(,Y,)=- |
    .---. /`  \   |~~
 ~/  o  o \~~~~.----. ~~
  | =(X)= |~  / (O (O) \
   ~~~~~~~  ~| =(Y_)=-  |
  ~~~~    ~~~|   U      |~~

Project:   https://github.com/containers/podman
Website:   https://podman.io
Documents: https://docs.podman.io
Twitter:   @Podman_io

Yes, it does. Meaning we can use containers on this computer from this point onward.

Let’s build some documentation

This Linux distribution also doesn’t come with git installed, but it’s pretty easy to set it up. I used the Microsoft WSL documentation to do the following steps:

LinuxUser@w10laptop:~> sudo zypper install git
...
LinuxUser@w10laptop:~> git config --global user.name "Addie Janssen"
...
LinuxUser@w10laptop:~> git config --global user.email "my email address"
...
LinuxUser@w10laptop:~> git config --global credential.helper "/mnt/c/Program\ Files/Git/mingw64/bin/git-credential-manager.exe"
...

The last command in that list is important in my context since the git repositories are not publicly accessible. Once these steps were done, I tried to clone my documentation.

LinuxUser@w10laptop:~> git clone https://azure_repo_url/_git/documentation
Cloning into 'documentation'...
remote: Azure Repos
remote: Found 2261 objects to send. (401 ms)
Receiving objects: 100% (2261/2261), 11.72 MiB | 40.69 MiB/s, done.
Resolving deltas: 100% (992/992), done.

Flawlessly. So, can we do a hugo server on it?

LinuxUser@w10laptop:~> cd documentation
LinuxUser@w10laptop:~/documentation> podman run --rm -it -v ./src:/src -p 1313:1313 klakegg/hugo:ext-ubuntu server --disableFastRender --noHTTPCache --buildDrafts --verbose
✔ docker.io/klakegg/hugo:ext-ubuntu
Trying to pull docker.io/klakegg/hugo:ext-ubuntu...
Getting image source signatures
Copying blob e2103eff9f1b done   |
Copying blob 361237ddf358 done   |
Copying blob 17e036648e36 done   |
Copying blob 09681a3abe6b done   |
Copying blob 21158ea1b70a done   |
Copying blob 4f4fb700ef54 done   |
Copying blob 3a19f3baab81 done   |
Copying config 05e36fdff2 done   |
Writing manifest to image destination
hugo: downloading modules …
hugo: collected modules in 20331 ms
Start building sites …
hugo v0.111.3-5d4eb5154e1fed125ca8e9b5a0315c4180dab192+extended linux/amd64 BuildDate=2023-03-12T11:40:50Z VendorInfo=hugoguru
INFO 2023/10/27 14:19:32 syncing static files to /

                   | EN
-------------------+------
  Pages            | 122
  Paginator pages  |   0
  Non-page files   |  84
  Static files     |  45
  Processed images |   7
  Aliases          |   3
  Sitemaps         |   0
  Cleaned          |   0

Built in 603 ms
Watching for changes in /src/{archetypes,assets,content,i18n,layouts,package.json,static}
Watching for config changes in /src/config.toml, /src/go.mod
Environment: "DEV"
Serving pages from memory
Web Server is available at //localhost:1313/ (bind address 0.0.0.0)
Press Ctrl+C to stop
INFO 2023/10/27 14:19:45 Received System Events: [WRITE         "/src/content/en/docs/_index.md" WRITE         "/src/content/en/docs/_index.md"]

Change detected, rebuilding site.
2023-10-27 14:19:45.298 +0000
Source changed WRITE         "/src/content/en/docs/_index.md"
Total in 210 ms

No problem at all. Performance is equivalent to the Podman installed Linux distribution. And the Hugo container is nicely monitoring the src folder. And I can browse the documentation using my web browser on Windows and visiting http://localhost:1313.

Podman remote config on Linux

All the Podman work we did so far was done on the Linux system. Now we need to make the Podman command on Windows work.

Do you remember that little paragraph from my previous post: Podman on Windows is using an ssh-connection to a port on the Linux system to interact with a Podman socket. That Podman socket triggers the Podman API service that will act as a proxy to the local (on the Linux system) Podman software.

That’s what we now need to get going without the help of the podman machine init command. First, we need to make sure that the Linux system is using systemd by creating (as root) a /etc/wsl.conf file with the following lines in it:

[boot]
systemd=true

Then we need to configure the ssh server on the Linux system by creating a /etc/ssh/sshd_config file with the following lines in it:

Port 58366

Just pick a random port number. I would recommend using something in the 58xxx range for that purpose.

With these files in place, we need to restart the Linux system. Easiest way of doing so: logout of the Linux system and do a wsl --shutdown to shutdown all WSL systems. Then login again using the wsl command.

We need to make sure that the ssh server starts on every reboot. This must be done using the root user:

LinuxUser@w10laptop:~> sudo systemctl enable sshd
Created symlink /etc/systemd/system/multi-user.target.wants/sshd.service → /usr/lib/systemd/system/sshd.service.
LinuxUser@w10laptop:~> sudo systemctl start sshd

You can verify if it is enabled and running:

LinuxUser@w10laptop:~> sudo systemctl status sshd
● sshd.service - OpenSSH Daemon
     Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; preset: disabled)
     Active: active (running) since Fri 2023-10-27 16:45:16 CEST; 1s ago
    Process: 649 ExecStartPre=/usr/sbin/sshd-gen-keys-start (code=exited, status=0/SUCCESS)
    Process: 651 ExecStartPre=/usr/sbin/sshd -t $SSHD_OPTS (code=exited, status=0/SUCCESS)
   Main PID: 653 (sshd)
      Tasks: 1
     CGroup: /system.slice/sshd.service
             └─653 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups

Last step on the Linux system is to enable and start the Podman socket. This step MUST NOT be done as root, but as a regular user. We want to make sure that the whole setup is rootless. Here we go:

LinuxUser@w10laptop:~> systemctl --user enable podman.socket
Created symlink /home/LinuxUser/.config/systemd/user/sockets.target.wants/podman.socket → /usr/lib/systemd/user/podman.socket.
LinuxUser@w10laptop:~> systemctl --user start podman.socket

And verify:

LinuxUser@w10laptop:~> systemctl --user status podman.socket
● podman.socket - Podman API Socket
     Loaded: loaded (/usr/lib/systemd/user/podman.socket; enabled; preset: disabled)
     Active: active (listening) since Fri 2023-10-27 16:52:00 CEST; 25s ago
   Triggers: ● podman.service
       Docs: man:podman-system-service(1)
     Listen: /run/user/1000/podman/podman.sock (Stream)
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/podman.socket

Oct 27 16:52:00 w10laptop systemd[260]: Listening on Podman API Socket.

At this point we need to restart the Linux system one more time by doing a wsl --shutdown and doing a login using the wsl command. Once the system is up again, we can start to configure Podman on Windows.

Podman on Windows

When we removed the Podman configured Linux system earlier, it also removed the connection to that machine. Therefore, we have no Podman connections setup on Windows anymore:

C:\Users\WindowsUser>podman system connection list
Name        URI         Identity    Default

The Windows Podman command will use ssh to connect to the Linux system and for that it needs to have a set of ssh key files. If you already have ssh key files available in your C:\Users\WindowsUser\.ssh folder, you can reuse them. But it’s also an option to create a new set of ssh key files specifically for Podman, like this:

C:\Users\WindowsUser>ssh-keygen -t ed25519
Generating public/private ed25519 key pair.
Enter file in which to save the key (C:\Users\WindowsUser/.ssh/id_ed25519): .ssh/podman
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in .ssh/podman.
Your public key has been saved in .ssh/podman.pub.
The key fingerprint is:
SHA256:95Hdv8qw/06EjHSMafHxiDZhytb4V+EzTi23vH9r50g WindowsUser@w10laptop
The key's randomart image is:
+--[ED25519 256]--+
|           + . . |
|        . = O = o|
|         = X =.Bo|
|        . = =o*++|
|        S .oo+.=.|
|         . .... o|
|           .. Eo.|
|            +..o=|
|           ..+***|
+----[SHA256]-----+

This will result in 2 files in the C:\Users\WindowsUser\.ssh folder:

podman
podman.pub

We now need copy the contents of the podman.pub file to the .ssh/authorized_keys file on the Linux system:

C:\Users\WindowsUser>wsl
LinuxUser@w10laptop:/mnt/c/Users/WindowsUser> cd .ssh
LinuxUser@w10laptop:/mnt/c/Users/WindowsUser/.ssh> mkdir ~/.ssh
LinuxUser@w10laptop:/mnt/c/Users/WindowsUser/.ssh> cp podman.pub ~/.ssh/authorized_keys
LinuxUser@w10laptop:/mnt/c/Users/WindowsUser/.ssh> chmod 600 ~/.ssh/authorized_keys

Once that is done, we have the ssh stuff in place; time to create a Podman connection on Windows using the port number that we configured earlier. So back on the Window Command Prompt:

C:\Users\WindowsUser>podman system connection add --identity C:\Users\WindowsUser\.ssh\podman podman ssh://LinuxUser@127.0.0.1:58366
C:\Users\WindowsUser>podman system connection list
Name        URI                                                               Identity                          Default
podman      ssh://LinuxUser@127.0.0.1:58366/run/user/1000/podman/podman.sock  C:\Users\WindowsUser\.ssh\podman  true

Let’s give it a go:

C:\Users\WindowsUser>podman run hello-world
!... Hello Podman World ...!

         .--"--.
       / -     - \
      / (O)   (O) \
   ~~~| -=(,Y,)=- |
    .---. /`  \   |~~
 ~/  o  o \~~~~.----. ~~
  | =(X)= |~  / (O (O) \
   ~~~~~~~  ~| =(Y_)=-  |
  ~~~~    ~~~|   U      |~~

Project:   https://github.com/containers/podman
Website:   https://podman.io
Documents: https://docs.podman.io
Twitter:   @Podman_io

YES.

Where are we now?

That were a lot of steps, so where are we now?

We now have an openSUSE-Tumbleweed Linux system on WSL with Podman installed. And since this Linux system has all the Windows integration stuff configured out of the box, we finally have everything in place to start looking at Development Containers. More on that in the next post.

Want to respond to this post?
Look me up on twitter Twitter, facebook Facebook or linkedin LinkedIn.

Categories

Tags