WSL 2 – Docker https://www.docker.com Fri, 09 Jan 2026 15:36:04 +0000 en-US hourly 1 https://wordpress.org/?v=6.9 https://www.docker.com/app/uploads/2024/02/cropped-docker-logo-favicon-32x32.png WSL 2 – Docker https://www.docker.com 32 32 Docker Desktop 4.50: Indispensable for Daily Development  https://www.docker.com/blog/docker-desktop-4-50/ Wed, 12 Nov 2025 14:00:00 +0000 https://www.docker.com/?p=81883 Docker Desktop 4.50 represents a major leap forward in how development teams build, secure, and ship software. Across the last several releases, we’ve delivered meaningful improvements that directly address the challenges you face every day: faster debugging workflows, enterprise-grade security controls that don’t get in your way, and seamless AI integration that makes modern development accessible to every team member.

Whether you’re debugging a build failure at 2 AM, managing security policies across distributed teams, or leveraging AI capabilities to build your applications, Docker Desktop delivers clear, real-world value that keeps your workflows moving and your infrastructure secure.

4.50

Accelerating Daily Development: Productivity and Control for Every Developer

Modern development teams face mounting pressures: complex multi-service applications, frequent context switching between tools, inconsistent local environments, and the constant need to balance productivity with security and governance requirements. For principal engineers managing these challenges, the friction of daily development workflows can significantly impact team velocity and code quality.

Docker Desktop addresses these challenges head-on by delivering seamless experiences that eliminate friction and giving organizations the control necessary to maintain security and compliance without slowing teams down.

Seamless Developer Experiences

Docker Debug is now free for all users, removing barriers to troubleshooting and making it easier for every developer on your team to diagnose issues quickly. The enhanced IDE integration goes deeper than ever before: the Dockerfile debugger in the VSCode Extension enables developers to step through build processes directly within their familiar editing environment, reducing the cognitive overhead of switching between tools. Whether you’re using VSCode, Cursor, or other popular editors, Docker Desktop integrates naturally into your existing workflow. For Windows-based enterprises, Docker Desktop’s ongoing engineering investments are delivering significant stability improvements with WSL2 integration, ensuring consistent performance for development teams at scale.

Getting applications from local development to production environments requires reducing the gap between how developers work locally and how applications run at scale. Compose to Kubernetes capabilities enable teams to translate local multi-service applications into production-ready Kubernetes deployments, while cagent provides a toolkit for running and developing agents that simplifies the development process. Whether you’re orchestrating containerized microservices or developing agentic AI workflows, Docker Desktop accelerates the path from experimentation to production deployment.

Enterprise-Level Control and Governance

For organizations requiring centralized management, Docker Desktop delivers enterprise-grade capabilities that maintain security without sacrificing developer autonomy. Administrators can set proxy settings via macOS configuration profiles, and can specify PAC files and Embedded PAC scripts with installer flags for macOS and Windows Docker, ensuring corporate network policies are automatically enforced during deployment without requiring manual developer configuration, further extending enterprise policy enforcement.

A faster release cadence with continuous updates ensures every developer runs the latest stable version with critical security patches, eliminating the traditional tension between IT requirements and developer productivity. The Kubernetes Dashboard is now part of the left navigation, making it easier to find and use.

Kind (k8s) Enterprise Support brings production-grade Kubernetes tooling to local development, enabling teams to test complex orchestration scenarios before deployment. 

k8s settings

Figure 1: K8 Settings

Together, these capabilities build on Docker Desktop’s position as the foundation for modern development, adding enterprise-grade management that scales with your organization’s needs. You get the visibility and control that enterprise architecture teams require while preserving the speed and flexibility that keeps developers productive.

Securing Container Workloads: Enterprise-Grade Protection Without Sacrificing Speed

As containerized applications move from development to production and AI workloads proliferate across enterprises, security teams face a critical challenge: how do you enforce rigorous security controls without creating bottlenecks that slow development velocity? Traditional approaches often force organizations to choose between security and speed, but that’s a false choice that puts both innovation and infrastructure at risk.

Docker Desktop’s recent releases address this tension directly, delivering enterprise-grade security controls that operate transparently within developer workflows. These aren’t afterthought features; they’re foundational protections designed to give security and platform teams confidence at scale while keeping developers productive.

Granular Control Over Container Behavior

Enforce Local Port Bindings prevents services running in Docker Desktop from being exposed across the local network, ensuring developers maintain network isolation during local development while retaining full functionality. For teams in regulated industries where network segmentation requirements extend to development environments, this capability helps maintain compliance standards without disrupting developer workflows.

Building on Secure Foundations

These runtime protections work in tandem with secure container foundations. Docker’s new Hardened Images, secure, minimal, production-ready container images maintained by Docker with near-zero CVEs and enterprise SLA backing. Recent updates introduced unlimited catalog pricing and the addition of Helm charts to the catalog. We also outlined Docker’s five pillars for Software Supply Chain Security, delivering transparency and eliminating the endless CVE remediation cycle. While Hardened Images are available as a separate add-on, they’re purpose-built to extend the secure-by-default foundation that Docker Desktop provides, giving teams a comprehensive approach to container security from development through production.

Seamless Enterprise Policy Integrations

The Docker CLI now gracefully handles certificates issued by non-conforming certificate authorities (CAs) that use negative serial numbers. While the X.509 standard specifies that certificate serial numbers must be positive, some enterprise PKI systems still produce certificates that violate this rule. Previously, organizations had to choose between adhering to their CA configuration and maintaining Docker compatibility, a frustrating trade-off that often led to insecure workarounds. Now, Docker Desktop works seamlessly with enterprise certificate infrastructure, ensuring developers can authenticate to private registries without security teams compromising their PKI standards.

These improvements reflect Docker’s commitment to being secure by default. Rather than treating security as a feature developers must remember to enable, Docker Desktop builds protection into the platform itself, giving enterprises the confidence to scale container adoption while maintaining the developer experience that drives innovation.

Unlocking AI Development: Making Model Context Protocol (MCP)Accessible for Every Developer

As AI-native development becomes central to modern software engineering, developers face a critical challenge: integrating AI capabilities into their workflows shouldn’t require extensive configuration knowledge or create friction that slows teams down. The Model Context Protocol (MCP) offers powerful capabilities for connecting AI agents to development tools and data sources, but accessing and managing these integrations has historically been complex, creating barriers to adoption, especially for teams with varying technical expertise.

Docker is addressing these challenges directly by making MCP integration seamless and secure within Docker Desktop.

Guided Onboarding Through Learning Center and MCP Toolkit Walkthroughs and Improved MCP Server Discovery

Understanding that accessibility drives adoption, Docker has introduced a redesigned onboarding experience through the Learning Center. The new MCP Toolkit Walkthroughs guide teams through complex setup processes step-by-step, ensuring that engineers of all skill levels can confidently adopt AI-powered workflows. Further, Docker’s MCP Server Discovery feature simplifies discovery by enabling developers to search, filter, and sort available MCP servers efficiently.  By eliminating the knowledge barriers and frictions around discovery, these improvements accelerate time to productivity and help organizations scale AI development practices across their teams.

Expanded Catalog: 270+ MCP Servers and Growing

The Docker MCP Catalog now includes over 270 MCP servers, with support for more than 60 remote servers. We’ve also added one-click connections for popular clients like Claude Code and Codex, making it easier than ever to supercharge your AI coding agents with powerful MCP tools. Getting started takes just a few clicks.

Remote MCP Server Support with Built-In OAuth

Connecting to MCP servers has traditionally meant dealing with manual tokens, fragile config files, and scattered credential management. It’s frustrating, especially for developers new to these workflows, who often don’t know where to find the right credentials in third-party tools. With the latest update to the Docker MCP Toolkit, developers can now securely connect to 60+ remote MCP servers, including Notion and Linear, using built-in OAuth support. This update goes beyond convenience; it lays the foundation for a more connected, intelligent, and automated developer experience, all within Docker Desktop. Read more about connecting to remote MCP servers.

MCP Servers with OAuth

Figure 2: Docker MCP Toolkit now supports remote MCP Servers with OAuth built-in

Smarter, More Efficient, and More Capable Agents with Dynamic MCPs

In this release, we’re introducing dynamic MCPs, a major step forward in enabling AI agents to discover, configure, and compose tools autonomously. Previously, integrating MCP servers required manual setup and static configurations. Now, with new features like Smart Search and Tool Composition, agents can search the MCP Catalog, pull only the tools they need, and even generate code to compose multi-tool workflows, all within a secure, sandboxed environment. These enhancements not only increase agent autonomy but also improve performance by reducing token usage and minimizing context bloat. Ultimately, this leads to less context switching and more focused time for developers. Read more about dynamic MCPs.

Together, these advancements represent Docker’s commitment to making AI-native development accessible and practical for development teams of any size.

Conclusion: Committed to Your Development Success

The innovations across Docker Desktop 4.45 through 4.50 reinforce our commitment to being the development solution teams rely on every day, for every workflow, at any scale.

We’ve made daily development faster and more integrated, with free debugging tools, native IDE support, and enterprise governance that actually works. We’ve strengthened security with controls that protect your infrastructure without creating bottlenecks. And we’ve made AI development accessible, turning complex integrations into guided experiences that accelerate your team’s capabilities. The impact is measurable. Independent research from theCUBE found that Docker Desktop users achieve 50% faster build times and reclaim 10-40+ hours per developer each month, time that goes directly back into innovation

This is Docker Desktop operating as your indispensable foundation: giving developers the tools they need to stay productive, giving security teams the controls they need to stay protected, and giving organizations the confidence they need to innovate at scale.

As we continue our accelerated release cadence, expect Docker to keep delivering the features that matter most to how you build, ship, and run modern applications. We’re committed to being the solution you can count on today and as your needs evolve.

Upgrade to the latest Docker Desktop now

Learn more

]]>
Docker Desktop 4.36: New Enterprise Administration Features, WSL 2, and ECI Enhancements https://www.docker.com/blog/docker-desktop-4-36/ Fri, 22 Nov 2024 16:38:18 +0000 https://www.docker.com/?p=65523 Key features of the Docker Desktop 4.36 release include: 

Docker Desktop 4.36 introduces powerful updates to simplify enterprise administration and enhance security. This release features streamlined macOS sign-in enforcement via configuration profiles, enabling IT administrators to deploy tamper-proof policies at scale, alongside a new PKG installer for efficient, consistent deployments. Enhancements like the unified WSL 2 mono distribution improve startup speeds and workflows, while updates to Enhanced Container Isolation (ECI) and Desktop Settings Management allow for greater flexibility and centralized policy enforcement. These innovations empower organizations to maintain compliance, boost productivity, and streamline Docker Desktop management across diverse enterprise environments.

2400x1260 4.36 rectangle docker desktop release

Sign-in enforcement: Streamlined alternative for organizations for macOS 

Recognizing the need for streamlined and secure ways to enforce sign-in protocols, Docker is introducing a new sign-in enforcement mechanism for macOS configuration profiles. This Early Access update delivers significant business benefits by enabling IT administrators to enforce sign-in policies quickly, ensuring compliance and maximizing the value of Docker subscriptions.

Key benefits

  • Fast deployment and rollout: Configuration profiles can be rapidly deployed across a fleet of devices using Mobile Device Management (MDM) solutions, making it easy for IT admins to enforce sign-in requirements and other policies without manual intervention.
  • Tamper-proof enforcement: Configuration profiles ensure that enforced policies, such as sign-in requirements, cannot be bypassed or disabled by users, providing a secure and reliable way to manage access to Docker Desktop (Figure 1).
  • Support for multiple organizations: More than one organization can now be defined in the allowedOrgs field, offering flexibility for users who need access to Docker Desktop under multiple organizational accounts (Figure 2).

How it works

macOS configuration profiles are XML files that contain specific settings to control and manage macOS device behavior. These profiles allow IT administrators to:

  • Restrict access to Docker Desktop unless the user is authenticated.
  • Prevent users from disabling or bypassing sign-in enforcement.

By distributing these profiles through MDM solutions, IT admins can manage large device fleets efficiently and consistently enforce organizational policies.

Screenshot of Enforced Sign-in Configuration Profile showing Description, Signed, Installed, Settings, Details, and Custom Settings.
Figure 1: macOS configuration profile in use.
Screenshot of macOS configuration profile showing "allowedOrgs"
Figure 2: macOS configuration profile in use with multiple allowedOrgs visible.

Configuration profiles, along with the Windows Registry key, are the latest examples of how Docker helps streamline administration and management. 

Enforce sign-in for multiple organizations

Docker now supports enforcing sign-in for more than one organization at a time, providing greater flexibility for users working across multiple teams or enterprises. The allowedOrgs field now accepts multiple strings, enabling IT admins to define more than one organization via any supported configuration method, including:

  • registry.json
  • Windows Registry key
  • macOS plist
  • macOS configuration profile

This enhancement makes it easier to enforce login policies across diverse organizational setups, streamlining access management while maintaining security (Figure 3).

Learn more about the various sign-in enforcement methods.

Screenshot of Sign-in required box, saying "Sign-in to continue using Docker Desktop. You must be a member of one of the following organizations" with Docker-internal and Docker listed.
Figure 3: Docker Desktop when sign-in is enforced across multiple organizations. The blue highlights indicate the allowed company domains.

Deploy Docker Desktop for macOS in bulk with the PKG installer

Managing large-scale Docker Desktop deployments on macOS just got easier with the new PKG installer. Designed for enterprises and IT admins, the PKG installer offers significant advantages over the traditional DMG installer, streamlining the deployment process and enhancing security.

  • Ease of use: Automate installations and reduce manual steps, minimizing user error and IT support requests.
  • Consistency: Deliver a professional and predictable installation experience that meets enterprise standards.
  • Streamlined deployment: Simplify software rollouts for macOS devices, saving time and resources during bulk installations.
  • Enhanced security: Benefit from improved security measures that reduce the risk of tampering and ensure compliance with enterprise policies.

You can download the PKG installer via Admin Console > Security and Access > Deploy Docker Desktop > macOS. Options for both Intel and Arm architectures are also available for macOS and Windows, ensuring compatibility across devices.

Start deploying Docker Desktop more efficiently and securely today via the Admin Console (Figure 4). 

Screenshot of Admin console showing option to download PKG installer.
Figure 4: Admin Console with PKG installer download options.

Desktop Settings Management (Early Access) 

Managing Docker Desktop settings at scale is now easier than ever with the new Desktop Settings Management, available in Early Access for Docker Business customers. Admins can centrally deploy and enforce settings policies for Docker Desktop directly from the cloud via the Admin Console, ensuring consistency and efficiency across their organization.

Here’s what’s available now:

  • Admin Console policies: Configure and enforce default Docker Desktop settings from the Admin Console.
  • Quick import: Import existing configurations from an admin-settings.json file for seamless migration.
  • Export and share: Export policies as JSON files to easily share with security and compliance teams.
  • Targeted testing: Roll out policies to a smaller group of users for testing before deploying globally.

What’s next?

Although the Desktop Settings Management feature is in Early Access, we’re actively building additional functionality to enhance it, such as compliance reporting and automated policy enforcement capabilities. Stay tuned for more!

This is just the beginning of a powerful new way to simplify Docker Desktop management and ensure organizational compliance. Try it out now and help shape the future of settings management: Admin Console > Security and Access > Desktop Settings Management (Figure 5).

Screenshot of Admin console showing Desktop Setting Management page, which includes Global policy, Settings policy, User policies, and more.
Figure 5: Admin console with Desktop Settings Management.

Streamlining data workflow with WSL 2 mono distribution 

Simplify the Windows Subsystem for Linux (WSL 2) setup by eliminating the need to maintain two separate Docker Desktop WSL distributions. This update streamlines the WSL 2 configuration by consolidating the previously required dual Docker Desktop WSL distributions into a single distribution, now available on both macOS and Windows operating systems.

The simplification of Docker Desktop’s WSL 2 setup is designed to make the codebase easier to understand and maintain. This enhances the ability to handle failures more effectively and increases the startup speed of Docker Desktop on WSL 2, allowing users to begin their work more quickly.

The value of streamlining data workflows and relocating data to a different drive on macOS and Windows with the WSL 2 backend in Docker Desktop encompasses these key areas:

  • Improved performance: By separating data and system files, I/O contention between system operations and data operations is reduced, leading to faster access and processing.
  • Enhanced storage management: Separating data from the main system drives allows for more efficient use of space.
  • Increased flexibility with cross-platform compatibility: Ensuring consistent data workflows across different operating systems (macOS and Windows), especially when using Docker Desktop with WSL 2.
  • Enhanced Docker performance: Docker performs better when processing data on a drive optimized for such tasks, reducing latency and improving container performance.

By implementing these practices, organizations can achieve more efficient, flexible, and high-performing data workflows, leveraging Docker Desktop’s capabilities on both macOS and Windows platforms.

Enhanced Container Isolation (ECI) improvements 

  • Allow any container to mount the Docker socket: Admins can now configure permissions to allow all containers to mount the Docker socket by adding * or *:* to the ECI Docker socket mount permission image list. This simplifies scenarios where broad access is required while maintaining security configuration through centralized control. Learn more in the advanced configuration documentation.
  • Improved support for derived image permissions: The Docker socket mount permissions for derived images feature now supports wildcard tags (e.g., alpine:*), enabling admins to grant permissions for all versions of an image. Previously, specific tags like alpine:latest had to be listed, which was restrictive and required ongoing maintenance. Learn more about managing derived image permissions.

These enhancements reduce administrative overhead while maintaining a high level of security and control, making it easier to manage complex environments.

Upgrade now

The Docker Desktop 4.36 release introduces a suite of features designed to simplify enterprise administration, improve security, and enhance operational efficiency. From enabling centralized policy enforcement with Desktop Settings Management to streamlining deployments with the macOS PKG installer, Docker continues to empower IT administrators with the tools they need to manage Docker Desktop at scale.

The improvements in Enhanced Container Isolation (ECI) and WSL 2 workflows further demonstrate Docker’s commitment to innovation, providing solutions that optimize performance, reduce complexity, and ensure compliance across diverse enterprise environments.  

As businesses adopt increasingly complex development ecosystems, these updates highlight Docker’s focus on meeting the unique needs of enterprise teams, helping them stay agile, secure, and productive. Whether you’re managing access for multiple organizations, deploying tools across platforms, or leveraging enhanced image permissions, Docker Desktop 4.36 sets a new standard for enterprise administration.  

Start exploring these powerful new features today and unlock the full potential of Docker Desktop for your organization.

Learn more

]]>
Docker Desktop 4.34: MSI Installer GA, Upgraded Host Networking, and Powerful Enhancements for Boosted Productivity & Administration https://www.docker.com/blog/docker-desktop-4-34/ Tue, 03 Sep 2024 14:00:48 +0000 https://www.docker.com/?p=57659 Key GA features of the Docker Desktop 4.34 release include: 

Docker Desktop 4.34 introduces key features to enhance security, scalability, and productivity for all development team sizes, making deploying and managing environments more straightforward. With the general availability (GA) of the MSI installer for bulk deployment, managing installations across Windows environments becomes even simpler. Enhanced authentication features offer an improved administration experience while reinforcing security. Automatically reclaim valuable disk space with Docker Desktop’s new smart compaction feature, streamlining storage management for WSL2 users. Additionally, the integration with NVIDIA AI Workbench provides developers with a seamless connection between model training and local development. Explore how these innovations simplify your workflows and foster a culture of innovation and reliability in your development practices.

2400x1260 4.34 rectangle docker desktop release

Deploy Docker Desktop in bulk with the MSI installer

We’re excited to announce that the MSI installer for Docker Desktop is now generally available to all our Docker Business customers. This powerful tool allows you to customize and deploy Docker Desktop across multiple users or machines in an enterprise environment, making it easier to manage Docker at scale. 

Features include:

  • Interactive and silent installations: Choose between an interactive setup process or deploy silently across your organization without interrupting your users.
  • Customizable installation paths: Tailor the installation location to fit your organization’s needs.
  • Desktop shortcuts and automatic startup: Simplify access for users with automatic creation of desktop shortcuts and Docker Desktop starting automatically after installation.
  • Set usage to specific Docker Hub organizations: Control which Docker Hub organizations your users are tied to during installation.

Docker administrators can download the MSI installer directly from the Docker Admin Console.

One of the standout features of this installer is the --allowed-org flag. This option enables the creation of a Windows registry key during installation, enforcing sign-in to a specified organization. By requiring sign-in, you ensure that your developers are using Docker Desktop with their corporate credentials, fully leveraging your Docker Business subscription. This also adds an extra layer of security, protecting your software supply chain.

Additionally, this feature paves the way for Docker to provide you with valuable usage insights across your organization and enable cloud-based control over application settings for every user in your organization in the future.

dd 434 f1
Figure 1: Docker admins can download the MSI installer directly from the Docker Admin Console.

What’s next

We’re also working on releasing a PKG enterprise installer for macOS, config profiles for macOS, and supporting multiple organizations in all supported sign-in enforcement mechanisms. 

Refer to our docs to learn about MSI configuration and discover more about sign-in enforcement via Windows registry key.

Host networking support to Docker Desktop 

Previously, Docker Desktop lacked seamless host networking capability, complicating the integration between host and container network services. Developers had to take time to set up and enable communication between the host and containers. Docker Desktop now supports host networking capability directly into Docker Desktop. 

Host networking allows containers that are started with --net=host to use localhost to connect to TCP and UDP services on the host. It will automatically allow software on the host to use localhost to connect to TCP and UDP services in the container. This simplifies the setup for scenarios in which close integration between host and container network services is required. Additionally, we’re driving cross-platform consistency and simplifying configuration by reducing the need for additional steps, such as setting up port forwarding or bridge networks. 

While this has previously been available in the Docker Engine, we’re now extending this capability to Docker Desktop for Windows, macOS, and Linux. We’re dedicated to improving developer productivity, and this is another way we help developers spend less time configuring network settings and more time building and testing applications, accelerating development cycles. 

This new capability is available for all users logged into Docker Desktop. To enable this feature, navigate to Settings > Resources > Network. Learn more about this feature on Docker Docs. 

dd 434 f2
Figure 2: Enable the host networking support feature in the Settings menu.

Automatic reclamation of disk space in Docker Desktop for WSL2 

Previously, when customers using Docker Desktop for WSL2 deleted Docker objects such as containers, images, or builds (for example via a docker system prune), the freed storage space was not automatically reclaimed on their host. Instead, they had to use external tools to “compact” the virtual disk/distribution backing Docker Desktop.

Starting with Docker 4.34, we are rolling out automatic reclamation of disk space. When you quit the app, Docker Desktop will automatically check whether there is storage space that can be returned to the host. It will then scan the virtual disk used for Docker storage, and compact it by returning all zeroed blocks to the operating system. Currently Docker Desktop will only start the scan when it estimates that at least 16GB of space can be returned. In the future, we plan to make this threshold adaptive and configurable by the user.

The feature is now enabled for all customers running the Mono distribution architecture for Docker Desktop on WSL2. This new architecture, which was rolled out starting with Docker Desktop 4.30 for all fresh installations of Docker Desktop, removed the need for a dedicated docker-desktop-data WSL2 distribution to store docker data. We will be rolling out the new architecture to all customers in the upcoming Docker Desktop releases.

Customers with installations still using the docker-desktop-data WSL2 distribution can compact storage manually via VHDX compaction tools, or change the WSL2 configuration to enable the experimental WSL2 feature for disk cleanup.

(Pro tip: Did you know you can use the Disk Usage extension to see how Docker Desktop is using your storage and use it to prune dangling objects with a single click?)

Authentication enhancements 

Previously, authenticating via the CLI required developers to either type their password into the command-line interface — which should generally be avoided by the security-minded — or manually create a personal access token (PAT) by navigating to their Docker account settings, generating the token, and then copying it into the CLI for authentication. This process was time-consuming and forced developers to switch contexts between the CLI and the web portal.

In this latest Docker Desktop release, we’re streamlining the CLI authentication flow. Now, users can authenticate through a seamless browser-based process, similar to the experience in CLIs like GitHub’s gh or Amazon’s AWS CLI. With this improved flow, typing docker login in the CLI will print a confirmation code and open your browser for authentication, automating PAT creation behind the scenes and eliminating the need for manual PAT provisioning. This enhancement saves time, reduces complexity, and delivers a smoother and more secure user experience. Additionally, when you authenticate using this workflow, you’ll be logged in across both Docker CLI and Docker Desktop. 

This new flow also supports developers in organizations that require single sign-on (SSO), ensuring a consistent and secure authentication process.

dd 434 f3 resized
Figure 3: When you log in via the new workflow, you’ll be logged in across both Docker CLI and Docker Desktop.

Enterprise-grade AI application development with Docker Desktop and NVIDIA AI Workbench  

AI development is a complex journey, often hindered by the challenge of connecting the dots between model training, local development, and deployment. Developers frequently encounter a fragmented and inconsistent development environment and toolchain, making it difficult to move seamlessly from training models in the cloud to running them locally. This fragmentation slows down innovation, introduces errors, and complicates the end-to-end development process.

To solve this, we’re proud to announce the integration of Docker Desktop with NVIDIA AI Workbench, a collaboration designed to streamline every stage of AI development. This solution brings together the power of Docker’s containerization with NVIDIA’s leading AI tools, providing a unified environment that bridges the gap between model training and local development.

With this integration, you can now train models in the cloud using NVIDIA’s robust toolkit and effortlessly transition to local development on Docker Desktop. This eliminates the friction of managing different environments and configurations, enabling a smoother, more efficient workflow from start to finish. 

To learn more about this collaboration and how Docker Business supports enterprise-grade AI application development, read our blog post. 

Multi-platform UX improvements and the containerd image store  

In February 2024, we announced the general availability of the containerd image store in Docker Desktop. Since then, we’ve been working on improving the output of our commands to make multi-platform images easier to view and manage. 

Now, we are happy to announce that the docker image list CLI command now supports an experimental --tree flag. This offers a completely new tree view of the image list, which is more suitable for describing multi-platform images.

dd 434 f4
Figure 4: New CLI tree view of the image list.

If you’re looking for multi-platform support, you need to ensure that you have the containerd image store enabled in Docker Desktop (see General settings in Docker Desktop, select Use containerd for pulling and storing images). As of the Docker Desktop 4.34 release, fresh installs or factory resets of Docker Desktop will now default to using the containerd image store, meaning that you get multi-platform building capability out of the box. 

dd 434 f5
Figure 5: You can enable the containerd image store in the Docker Desktop general settings.

To learn more about the containerd image store, check out our containerd documentation. 

Wrapping up 

Docker Desktop 4.34 marks a significant milestone in our commitment to providing an industry-leading container development suite. With key features such as the MSI installer for bulk deployment, enhanced authentication mechanisms, and the integration with NVIDIA AI Workbench, Docker Desktop is transforming how teams manage deployments, protect their environments, and accelerate their development workflows. 

These advancements simplify your development processes and help drive a culture of innovation and reliability. Stay tuned for more exciting updates and enhancements as we continue to deliver solutions designed to empower your development teams and secure your operations at scale. 

Upgrade to Docker Desktop 4.34 today and experience the future of container development. 

Learn more

]]>
Optimizing Deep Learning Workflows: Leveraging Stable Diffusion and Docker on WSL 2 https://www.docker.com/blog/stable-diffusion-and-docker-on-wsl2/ Tue, 11 Jul 2023 14:15:00 +0000 https://www.docker.com/?p=43799 Deep learning has revolutionized the field of artificial intelligence (AI) by enabling machines to learn and generate content that mimics human-like creativity. One advancement in this domain is Stable Diffusion, a text-to-image model released in 2022. 

Stable Diffusion has gained significant attention for its ability to generate highly detailed images conditioned on text descriptions, thereby opening up new possibilities in areas such as creative design, visual storytelling, and content generation. With its open source nature and accessibility, Stable Diffusion has become a go-to tool for many researchers and developers seeking to harness the power of deep learning. 

In this article, we will explore how to optimize deep learning workflows by leveraging Stable Diffusion alongside Docker on WSL 2, enabling seamless and efficient experimentation with this cutting-edge technology.

Dark purple background with the Docker logo in the bottom left corner and a paint palette in the center

In this comprehensive guide, we will walk through the process of setting up the Stable Diffusion WebUI Docker, which includes enabling WSL 2 and installing Docker Desktop. You will learn how to download the required code from GitHub and initialize it using Docker Compose

The guide provides instructions on adding additional models and managing the system, covering essential tasks such as reloading the UI and determining the ideal location for saving image output. Troubleshooting steps and tips for monitoring hardware and GPU usage are also included, ensuring a smooth and efficient experience with Stable Diffusion WebUI (Figure 1).

Screenshot of Stable Diffusion WebUI showing five different cat images.
Figure 1: Stable Diffusion WebUI.

Why use Docker Desktop for Stable Diffusion?

In the realm of image-based generative AI, setting up an effective execution and development environment on a Windows PC can present particular challenges. These challenges arise due to differences in software dependencies, compatibility issues, and the need for specialized tools and frameworks. Docker Desktop emerges as a powerful solution to tackle these challenges by providing a containerization platform that ensures consistency and reproducibility across different systems.

By leveraging Docker Desktop, we can create an isolated environment that encapsulates all the necessary components and dependencies required for image-based generative AI workflows. This approach eliminates the complexities associated with manual software installations, conflicting library versions, and system-specific configurations.

Using Stable Diffusion WebUI

The Stable Diffusion WebUI is a browser interface that is built upon the Gradio library, offering a convenient way to interact with and explore the capabilities of Stable Diffusion. Gradio is a powerful Python library that simplifies the process of creating interactive interfaces for machine learning models.

Setting up the Stable Diffusion WebUI environment can be a tedious and time-consuming process, requiring multiple steps for environment construction. However, a convenient solution is available in the form of Stable Diffusion WebUI Docker project. This Docker image eliminates the need for manual setup by providing a preconfigured environment.

If you’re using Windows and have Docker Desktop installed, you can effortlessly build and run the environment using the docker-compose command. You don’t have to worry about preparing libraries or dependencies beforehand because everything is encapsulated within the container.

You might wonder whether there are any problems because it’s a container. I was anxious before I started using it, but I haven’t had any particular problems so far. The images, models, variational autoencoders (VAEs), and other data that are generated are shared (bind mounted) with my Windows machine, so I can exchange files simply by dragging them in Explorer or in the Files of the target container on Docker Desktop. 

The most trouble I had was when I disabled the extension without backing it up, and in a moment blew away about 50GB of data that I had spent half a day training. (This is a joke!)

Architecture

I’ve compiled a relatively simple procedure to start with Stable Diffusion using Docker Desktop on Windows. 

Prerequisites:

  • Windows 10 Pro, 21H2 Build 19044.2846
  • 16GB RAM
  • NVIDIA GeForce RTX 2060 SUPER
  • WSL 2 (Ubuntu)
  • Docker Desktop 4.18.0 (104112)

Setup with Docker Compose

We will use the WebUI called AUTOMATIC1111 to utilize Stable Diffusion this time. The environment for these will be constructed using Docker Compose. The main components are shown in Figure 2.

Illustration of execution environment using Automatic1111 showing Host, portmapping, Docker image, bind mount information, etc.
Figure 2: Configuration built using Docker Compose.

The configuration of Docker Compose is defined in docker-compose.yml. We are using a Compose extension called x-base_service to describe the major components common to each service.

To start, there are settings for bind mount between the host and the container, including /data, which loads modes, and /output, which outputs images. Then, we make the container recognize the GPU by loading the NVIDIA driver.

Furthermore, the service named sd-auto:58 runs AUTOMATIC1111, WebUI for Stable Diffusion, within the container. Because there is a port mapping (TCP:7860), between the host and the container in the aforementioned common service settings, it is possible to access from the browser on the host side to the inside of the container.

Getting Started

Prerequisite

WSL 2 must be activated and Docker Desktop installed.

On the first execution, it downloads 12GB of Stable Diffusion 1.5 models, etc. The Web UI cannot be used until this download is complete. Depending on your connection, it may take a long time until the first startup.

Downloading the code

First, download the Stable Diffusion WebUI Docker code from GitHub. If you download it as a ZIP, click Code > Download ZIP and the stable-diffusion-webui-docker-master.zip file will be downloaded (Figure 3). 

Unzip the file in a convenient location. When you expand it, you will find a folder named stable-diffusion-webui-docker-master. Open the command line or similar and run the docker compose command inside it.

 Screenshot showing Stable Diffusion WebUI Docker being downloaded as ZIP file from GitHub.
Figure 3: Downloading the configuration for Docker Compose from the repository.

Or, if you have an environment where you can use Git, such as Git for Windows, it’s quicker to download it as follows:

git clone https://github.com/AbdBarho/stable-diffusion-webui-docker.git

In this case, the folder name is stable-diffusion-webui-docker. Move it with cd stable-diffusion-webui-docker.

Supplementary information for those familiar with Docker

If you just want to get started, you can skip this section.

By default, the timezone is UTC. To adjust the time displayed in the log and the date of the directory generated under output/txt2img to Japan time, add TZ=Asia/Tokyo to the environment variables of the auto service. Specifically, add the following description to environment:.

auto: &automatic
    <<: *base_service
    profiles: ["auto"]
    build: ./services/AUTOMATIC1111
    image: sd-auto:51
    environment:
      - CLI_ARGS=--allow-code --medvram --xformers --enable-insecure-extension-access --api
      - TZ=Asia/Tokyo

Tasks at first startup

The rest of the process is as described in the GitHub documentation. Inside the folder where the code is expanded, run the following command:

docker compose --profile download up --build

After the command runs, the log of a container named webui-docker-download-1 will be displayed on the screen. For a while, the download will run as follows, so wait until it is complete:

webui-docker-download-1  | [DL:256KiB][#4561e1 1.4GiB/3.9GiB(36%)][#42c377 1.4GiB/3.9GiB(37%)]

If the process ends successfully, it will be displayed as exited with code 0 and returned to the original prompt:

…(snip)
webui-docker-download-1  | https://github.com/xinntao/Real-ESRGAN/blob/master/LICENSE 
webui-docker-download-1  | https://github.com/xinntao/ESRGAN/blob/master/LICENSE 
webui-docker-download-1  | https://github.com/cszn/SCUNet/blob/main/LICENSE 
webui-docker-download-1 exited with code 0

If a code other than 0 comes out like the following, the download process has failed:

webui-docker-download-1  | 42c377|OK  |   426KiB/s|/data/StableDiffusion/sd-v1-5-inpainting.ckpt 
webui-docker-download-1  | 
webui-docker-download-1  | Status Legend: 
webui-docker-download-1  | (OK):download completed.(ERR):error occurred. 
webui-docker-download-1  | 
webui-docker-download-1  | aria2 will resume download if the transfer is restarted. 
webui-docker-download-1  | If there are any errors, then see the log file. See '-l' option in help/m 
an page for details. 
webui-docker-download-1 exited with code 24

In this case, run the command again and check whether it ends successfully. Once it finishes successfully, run the command to start the WebUI. 

Note: The following is for AUTOMATIC1111’s UI and GPU specification:

docker compose --profile auto up --build

When you run the command, loading the model at the first startup may take a few minutes. It may look like it’s frozen like the following display, but that’s okay:

webui-docker-auto-1  | LatentDiffusion: Running in eps-prediction mode
webui-docker-auto-1  | DiffusionWrapper has 859.52 M params.

If you wait for a while, the log will flow, and the following URL will be displayed:

webui-docker-auto-1  | Running on local URL:  http://0.0.0.0:7860

Now the startup preparation of the Web UI is set. If you open http://127.0.0.1:7860 from the browser, you can see the Web UI. Once open, select an appropriate model from the top left of the screen, write some text in the text field, and select the Generate button to start generating images (Figure 4).

Screenshot showing text input and large orange "generate" button for creating images.
Figure 4: After selecting the model, input the prompt and generate the image.

When you click, the button will be reversed. Wait until the process is finished (Figure 5).

Screenshot of gray "interrupt" and "skip" buttons.
Figure 5: Waiting until the image is generated.

At this time, the log of image generation appears on the terminal you are operating, and you can also check the similar display by looking at the log of the container on Docker Desktop (Figure 6).

Screenshot of progress log, showing 100% completion.
Figure 6: 100% indicates that the image generation is complete.

When the status reaches 100%, the generation of the image is finished, and you can check it on the screen (Figure 7).

Screenshot of Stable Diffusion WebUI showing "space cat" as text input with image of gray and white cat on glowing purple background.
Figure 7: After inputting “Space Cat” in the prompt, a cat image was generated at the bottom right of the screen.

The created images are automatically saved in the output/txt2img/date folder directly under the directory where you ran the docker compose command.

To stop the launched WebUI, enter Ctrl+C on the terminal that is still running the docker compose command.

Gracefully stopping... (press Ctrl+C again to force)
Aborting on container exit...
[+] Running 1/1
 ? Container webui-docker-auto-1  Stopped                                                     11.4s
canceled

When the process ends successfully, you will be able to run the command again. To use the WebUI again after restarting, re-run the docker compose command:

docker compose --profile auto up --build

To see the operating hardware status, use the task manager to look at the GPU status (Figure 8).

Screenshot of Windows Task Manager showing GPU status.
Figure 8: From the Performance tab of the Windows Task Manager, you can monitor the processing of CUDA and similar tasks on the GPU.

To check whether the GPU is visible from inside the container and to see whether the information comes out, run the nvidia-smi command from docker exec or the Docker Desktop terminal.

root@e37fcc5a5810:/stable-diffusion-webui# nvidia-smi 
Mon Apr 17 07:42:27 2023 
+---------------------------------------------------------------------------------------+ 
| NVIDIA-SMI 530.41.03              Driver Version: 531.41       CUDA Version: 12.1     | 
|-----------------------------------------+----------------------+----------------------+ 
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC | 
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. | 
|                                         |                      |               MIG M. | 
|=========================================+======================+======================| 
|   0  NVIDIA GeForce RTX 2060 S...    On | 00000000:01:00.0  On |                  N/A | 
| 42%   40C    P8                6W / 175W|   2558MiB /  8192MiB |      2%      Default | 
|                                         |                      |                  N/A | 
+-----------------------------------------+----------------------+----------------------+ 
+---------------------------------------------------------------------------------------+ 
| Processes:                                                                            | 
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory | 
|        ID   ID                                                             Usage      | 
|=======================================================================================| 
|    0   N/A  N/A       149      C   /python3.10                               N/A      | 
+---------------------------------------------------------------------------------------+

Adding models and VAEs

If you download a model that is not included from the beginning, place files with extensions, such as .safetensors in stable-diffusion-webui-docker\data\StableDiffusion. In the case of VAE, place .skpt files in stable-diffusion-webui-docker\data\VAE.

If you’re using Docker Desktop, you can view and operate inside on the Files of the webui-docker-auto-1 container, so you can also drag it into Docker Desktop. 

Figure 9 shows the Docker Desktop screen. It says MOUNT in the Note column, and it shares the information in the folder with the container from the Windows host side.

Screenshot of Docker Desktop showing blue MOUNT indicator under Note column.
Figure 9: From the Note column, you can see whether the folder is mounted or has been modified.

Now, after placing the file, a link to Reload UI is shown in the footer of the WebUI, so select there (Figure 10).

Screenshot of Stable Diffusion WebUI showing Reload UI option.
Figure 10: By clicking Reload UI, the WebUI settings are reloaded.

When you select Reload UI, the system will show a loading screen, and the browser connection will be cut off. When you reload the browser, the model and VAE files are automatically loaded. To remove a model, delete the model file from data\StableDiffusion.

Conclusion

With Docker Desktop, image generation using the latest generative AI environment can be done easier than ever. Typically, a lot of time and effort is required just to set up the environment, but Docker Desktop solves this complexity. If you’re interested, why not take a challenge in the world of generative AI? Enjoy!

Learn more

]]>
WSL 2 GPU Support for Docker Desktop on NVIDIA GPUs https://www.docker.com/blog/wsl-2-gpu-support-for-docker-desktop-on-nvidia-gpus/ Wed, 15 Dec 2021 17:39:05 +0000 https://www.docker.com/blog/wsl-2-gpu-support-for-docker-desktop-on-nvidia-gpus/ It’s been a year since Ben wrote about Nvidia support on Docker Desktop. At that time, it was necessary to take part in the Windows Insider program, use Beta CUDA drivers, and use a Docker Desktop tech preview build. Today, everything has  changed:

  • On the OS side, Windows 11 users can now enable their GPU without participating in  the Windows Insider program. Windows 10 users still need to register.
  • Nvidia CUDA drivers have been released.
  • Last, the GPU support has been merged in Docker Desktop (in fact since version 3.1).

Nvidia used the term near-native to describe the performance to be expected.

Where to find the Docker images

Base Docker images are hosted at https://hub.docker.com/r/nvidia/cuda. The original project is located at https://gitlab.com/nvidia/container-images/cuda.

What they contain

The nvidia-smi utility allows users to query information on the accessible devices.

$ docker run -it --gpus=all --rm nvidia/cuda:11.4.2-base-ubuntu20.04 nvidia-smi
Tue Dec  7 13:25:19 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.00       Driver Version: 510.06       CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
| N/A    0C    P0    13W /  N/A |    132MiB /  4096MiB |     N/A      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------

The dmon function of nvidia-smi allows monitoring the GPU parameters :

$ docker exec -ti $(docker ps -ql) bash
root@7d3f4cbdeabb:/src# nvidia-smi dmon
# gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk
# Idx     W     C     C     %     %     %     %   MHz   MHz
    0    29    69     -     -     -     0     0  4996  1845
    0    30    69     -     -     -     0     0  4995  1844

The nbody utility is a CUDA sample that provides a benchmarking mode.

$ docker run -it --gpus=all --rm nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -benchmark
...
> 1 Devices used for simulation
GPU Device 0: "Turing" with compute capability 7.5

> Compute 7.5 CUDA device: [NVIDIA GeForce GTX 1650 Ti]
16384 bodies, total time for 10 iterations: 25.958 ms
= 103.410 billion interactions per second
= 2068.205 single-precision GFLOP/s at 20 flops per interaction

Quick comparison to a CPU suggest a different order of magnitude of performance. GPU is 2000 times faster:

> Simulation with CPU
4096 bodies, total time for 10 iterations: 3221.642 ms
= 0.052 billion interactions per second
= 1.042 single-precision GFLOP/s at 20 flops per interaction

What can you do with a paravirtualized GPU?

Run cryptographic tools

Using a GPU is of course useful when operations can be heavily parallelized. That’s the case for hash analysis. dizcza hosted its nvidia-docker based images of hashcat on Docker hub. This image magically works on Docker Desktop!

$ docker run -it --gpus=all --rm dizcza/docker-hashcat //bin/bash
root@a6752716788d:~# hashcat -I
hashcat (v6.2.3) starting in backend information mode

clGetPlatformIDs(): CL_PLATFORM_NOT_FOUND_KHR

CUDA Info:
==========

CUDA.Version.: 11.6

Backend Device ID #1
  Name...........: NVIDIA GeForce GTX 1650 Ti
  Processor(s)...: 16
  Clock..........: 1485
  Memory.Total...: 4095 MB
  Memory.Free....: 3325 MB
  PCI.Addr.BDFe..: 0000:01:00.0

From there it is possible to run hashcat benchmark

hashcat -b
...
Hashmode: 0 - MD5
Speed.#1.........: 11800.8 MH/s (90.34ms) @ Accel:64 Loops:1024 Thr:1024 Vec:1
Hashmode: 100 - SHA1
Speed.#1.........:  4021.7 MH/s (66.13ms) @ Accel:32 Loops:512 Thr:1024 Vec:1
Hashmode: 1400 - SHA2-256
Speed.#1.........:  1710.1 MH/s (77.89ms) @ Accel:8 Loops:1024 Thr:1024 Vec:1
...

Draw fractals

The project at https://github.com/jameswmccarty/CUDA-Fractal-Flames uses CUDA for generating fractals. There are two steps to build and run on Linux. Let’s see if we can have it running on Docker Desktop. A simple Dockerfile with nothing fancy helps for that.

# syntax = docker/dockerfile:1.3-labs
FROM nvidia/cuda:11.4.2-base-ubuntu20.04
RUN apt -y update
RUN DEBIAN_FRONTEND=noninteractive apt -yq install git nano libtiff-dev cuda-toolkit-11-4
RUN git clone --depth 1 https://github.com/jameswmccarty/CUDA-Fractal-Flames /src
WORKDIR /src
RUN sed 's/4736/1024/' -i fractal_cuda.cu # Make the generated image smaller
RUN make

And then we can build and run:

$ docker build . -t cudafractal
$ docker run --gpus=all -ti --rm -v ${PWD}:/tmp/ cudafractal ./fractal -n 15 -c test.coeff -m -15 -M 15 -l -15 -L 15

Note that the --gpus=all is only available to the run command. It’s not possible to add GPU intensive steps during the build.

Here’s an example image:

delete

Machine learning

Well really, looking at GPU usage without looking at machine learning would be a miss. The tensorflow:latest-gpu image can take advantage of the GPU in Docker Desktop. I will simply point you to Anca’s blog earlier this year. She described a tensorflow example and deployed it in the cloud: https://www.docker.com/blog/deploy-gpu-accelerated-applications-on-amazon-ecs-with-docker-compose/

Conclusion: What are the benefits for developers? 

At Docker, we want to provide a turn key solution for developers to execute their workflows seamlessly:

DockerCon2022

Join us for DockerCon2022 on Tuesday, May 10. DockerCon is a free, one day virtual event that is a unique experience for developers and development teams who are building the next generation of modern applications. If you want to learn about how to go from code to cloud fast and how to solve your development challenges, DockerCon 2022 offers engaging live content to help you build, share and run your applications. Register today at https://www.docker.com/dockercon/

]]>
WSL 2 GPU Support is Here https://www.docker.com/blog/wsl-2-gpu-support-is-here/ Mon, 21 Dec 2020 14:00:00 +0000 https://www.docker.com/blog/?p=27355 November 2024 update: Learn about the upgraded Docker plans and choose the best Docker Subscription for you. Simpler, more value, better development and productivity.

Read our Docker Desktop release collection to learn about our latest enhancements and innovations.

At Microsoft Build in the first half of the year, Microsoft demonstrated some awesome new capabilities and improvements that were coming to Windows Subsystem for Linux 2 including the ability to share the host machine’s GPU with WSL 2 processes. Then in June Craig Loewen from Microsoft announced that developers working on the Windows insider ring machines could now make use of GPU for the Linux workloads. This support for NVIDIA CUDA enabled developers and data scientists to use their local Windows machines for inner-loop development and experimentation. 

Last week, during the Docker Community All Hands, we announced the availability of a developer preview build of Docker Desktop for WSL 2 supporting GPU for our Developer Preview Program. We already have more than 1,000 who have joined us to help test preview builds of Docker Desktop for Windows (and Mac!). If you’re interested in joining the program for future releases you should do it now!

Today we are excited to announce the general preview of Docker Desktop support for GPU with Docker in WSL2. There are over one and a half million users of Docker Desktop for Windows today and we saw in our roadmap how excited you all were for us to provide this support.

Preview of Docker Desktop with GPU support in WSL2

To get started with Docker Desktop with Nvidia GPU support on WSL 2, you will need to download our technical preview build.

Once you have the preview build installed there are still a couple of steps you will need to do to get started using your GPU: 

  • You will need access to a PC with an Nvidia GPU (if you don’t have this we would still like feedback on this build, we have changed a fair bit in our VM to get this working!) 
  • The latest Windows Insider version from the Dev preview ring 
  • Beta drivers from Nvidia supporting WSL 2 GPU paravirtualization: https://developer.nvidia.com/cuda/wsl
  • WSL 2 backend enabled in Docker Desktop

Once you have all of this working you can have a go at the command below to check that GPU support is working. 

Screen Shot 2020 12 18 at 10.37.15 AM

Keep in mind that this is a technical preview release: it may break, it has not been tested as thoroughly as our normal releases and ‘here be dragons’. 

If you do find issues and want to give us feedback, please raise bugs on our public repo https://github.com/docker/for-win. We use this feedback to improve the product and your support in testing this will help us get this ready for GA sooner 🙂 

Enjoy the tech preview and happy GPU Hacking!

]]>
Creating the best Linux Development experience on Windows & WSL 2 https://www.docker.com/blog/creating-the-best-linux-development-experience-on-windows-wsl-2/ Wed, 20 May 2020 16:00:00 +0000 https://www.docker.com/blog/?p=26260 We are really excited to have had Docker Desktop be featured in a breakout session titled “The Journey to One .NET” at MSFT Build by @Scott Hanselman  with WSL 2. Earlier in the his  keynote, we learned about the great new enhancements for GPU support for WSL 2 and we want to hear from our community about your interest in adding this functionality to Docker Desktop. If you are eager to see GPU support come to Docker Desktop, please let us know by voting up our roadmap item and feel free to raise any new requests here as well.

With this announcement, the launch of the Windows 2004 release imminently and Docker Desktop v2.3.02 reaching WSL2 GA , we thought this would be a good time to reflect on how we got to where we are today with WSL 2.

April 2019

Casting our minds back to 2019 (a very different time!), we first discussed WSL 2 with Microsoft in April. We were excited to get started and wanted to find a way to get a build as soon as possible.

May 2019

It turned out the easiest way to do this was to collect a laptop at Kubecon EU (never underestimate the bandwidth of a 747). We brought this back and started work on what would be our first ‘hacky’ version of WSL 2 for Docker Desktop.

June 2019

With some internal demo’s done we decided to announce what we were planning <3

This announcement was a bit like watching a swan on a lake, our blog post was calm and collected, but beneath the water we were kicking madly to take us towards something we could share more widely.

July 2019

We finally got far enough along that we were ready to share something!

https://www.docker.com/blog/docker-wsl2-tech-preview/

And not long after we released our first preview of Docker Desktop using WSL 2

https://www.docker.com/blog/5-things-docker-desktop-wsl2-tech-preview/

August-September 2019

Once again, with a preview out and things seeming calm we went back to work. We were talking with Microsoft weekly about how we could improve what we had, on fixing bugs and generally on improving the experience. Simon and Ben did have enough time though to head over to the USA to talk to Microsoft about how we were getting on.

October 2019

We released a major rework to how Docker Desktop would integrate with WSL 2:

Along with adding K8s support and providing feature parity with our old Hyper-V based back end. We also made the preview more visible in Docker Desktop and our user numbers started to increase

November 2019 – Feb 2020

This time flew by, we spent a lot of this time chasing down bugs, looking at how we could improve the local experience and also what the best ways of working would be:

March 2020

We had built up a fair bit of confidence in what we had built and finally addressed one of the largest outstanding items we still had in our backlog – we added Windows Home support

This involved us removing the functionality associated with running our old Moby VM in Hyper V and all of the options associated with running Windows containers – as these are not supported on Windows Home. With this we were now able to focus on heading straight to GA…

April 2020

We doubled down how we were getting ready for GA, learning lessons on improving our development practice. We wanted to share how we were preparing and testing WSL 2 ready for the 2.7m people out there running Docker Desktop.

https://www.docker.com/how-we-test-docker-desktop-with-wsl-2/

May 2020

We finally reached our GA with Docker Desktop v2.3.02!

Now we are out in the wild, we shared some ideas and best practices to make sure you are getting the best experience out of Docker Desktop when working with WSL 2. 

(And of course for Windows Pro users this still comes with all the same features you know and love including the ability to switch back over to using Windows Containers.)

What’s next?

Next, is that people start to use Docker Desktop with WSL 2! To try out Docker Desktop with WSL 2 today, make sure you are on Windows 2004 or higher and download the latest Docker Desktop to get started.

If you are enjoying  Docker Desktop but have ideas of what we could do to make it better then please give us feedback. You can let us know what features you want to see next via our roadmap, including voting up GPU support for WSL 2. 

]]>
Docker Desktop: WSL 2 Best practices https://www.docker.com/blog/docker-desktop-wsl-2-best-practices/ Mon, 04 May 2020 16:30:00 +0000 https://www.docker.com/blog/?p=26100 Docker Desktop WSL 2 backend has now been available for a few months for Windows 10 insider users and Microsoft just released WSL 2 on the Release Preview channel (which means GA is very close). We and our early users have accumulated some experience working with it and are excited to share a few best practices to implement in your Linux container projects!

Docker Desktop with the WSL 2 backend can be used as before from a Windows terminal. We focused on compatibility to keep you happy with your current development workflow.

But to get the most out of Windows 10 2004 we have some recommendations for you.

Fully embrace WSL 2

The first and most important best practice we want to share, is to fully embrace WSL 2. Your project files should be stored within your WSL 2 distro of choice, you should run the docker CLI from this distro, and you should avoid accessing files stored on the Windows host as much as possible.

For backward compatibility reasons, we kept the possibility to interact with Docker from the Windows CLI, but it is not the preferred option anymore.

Running docker CLI from WSL will bring you…

Awesome mounts performance

Both your own WSL 2 distro and docker-desktop run on the same utility VM. They share the same Kernel, VFS cache etc. They just run in separate namespaces so that they have the illusion of running totally independently. Docker Desktop leverages that to handle bind mounts from a WSL 2 distro without involving any remote file sharing system. This means that when you mount your project files in a container (with docker run -v ~/my-project:/sources <...>), docker will propagate inotify events and share the same cache as your own distro to avoid reading file content from disk repeatedly.

A little warning though: if you mount files that live in the Windows file system (such as with docker run -v /mnt/c/Users/Simon/windows-project:/sources <...>), you won’t get those performance benefits, as /mnt/c is actually a mountpoint exposing Windows files through a Plan9 file share. 

Compatibility with Linux toolchains and build scripts

Most reasonably sized projects involving Linux containers come with a bunch of automation scripts. Those scripts are often developed for Linux first (because most of the time, CI/CD pipelines for those projects run on Linux), and developers running on Windows are often considered second-class citizens. They are often using less polished versions of those scripts, and have to deal with subtle behavioral differences.

By fully embracing WSL 2, Windows developers can use the exact same build and automation scripts as on Linux. This means that Windows-specific scripts don’t need to be maintained anymore. Also, that means that you won’t experience issues with different line endings between Windows and Mac/Linux users!

What about my IDE?

If you want an IDE for editing your files, you can do that even if they are hosted within your WSL 2 distro. There are 3 different ways:

  • Use Visual Studio Code Remote to WSL extension

If your IDE is Visual Studio Code, using Remote to WSL is the best way to continue working on your project. Visual Studio Code architecture is based on a client/server approach where pretty much everything except rendering and input processing is done in a server process, while the UI itself runs in a client process. Remote to WSL leverages that to run the whole server process within WSL while the UI runs as a classic win32 process.

That means that you get the same experience as before, but all your language services, terminals etc. run within WSL.

For more information, see Microsoft’s VS Code Remote to WSL documentation.

  • Point your IDE to your distro network share

WSL provides a network share for each of your running distros. For example, if I have a project in my Ubuntu distro at `~/wall-e`, I can access it from Windows Explorer (and from any Windows Process) via the special network share `\\wsl$\Ubuntu\home\simon\wall-e`. 

  • Run an X11 server on Windows, and run a Linux native IDE

The setup is a bit more complicated, but you always have the possibility to run an X11 server on Windows (VcXsrv, X410,…) and configure your DISPLAY environment variable such that GUI apps on Linux get rendered properly.

Use BuildKit and multi-stage builds

Docker Desktop WSL 2 backend has access to all your CPU cores. To leverage this as much as possible (and also to get access to the latest build features), you should enable BuildKit by default.

The easiest way to do that is to add the following line to your ~/.profile file:

export DOCKER_BUILDKIT=1.

This way, anytime you run docker build, it will run the build with the awesome BuildKit which is capable of running different build stages concurrently.

Use resource limits

Docker Desktop WSL 2 backend can use pretty much all CPU and memory resources on your machine. This is awesome for most cases, but there is a category of workloads where this can cause issues. Indeed, some containers (mainly databases, or caching services) tend to allocate as much memory as they can, and leave other processes (Linux or Win32) starving. Docker provides a way to impose limits in allocatable memory (as well as quotas on CPU usage) by a container. You can find documentation about it here: https://docs.docker.com/config/containers/resource_constraints/.

Reclaim cached memory

WSL 2 automatically reclaims memory when it is freed, to make it available to Windows processes. However, if the kernel decides to keep content in cache (and with Docker, it tends to happen quite a lot), the amount of memory reclaimed might not be sufficient.

To reclaim more memory, after stopping your containers, you can run echo 1 > /proc/sys/vm/drop_caches as root to drop the kernel page cache and make WSL 2 reclaim memory used by its VM.

What next

We are excited for people to use Docker Desktop with WSL 2 and hope that the tips and tricks in this article will help you get the best performance for all of your workloads. 

If you have another tip or idea you want to share with us for using Docker send us a tweet @docker or if you have feedback on our implementation then raise a ticket against our Github Repo.

]]>
How we test Docker Desktop with WSL 2 https://www.docker.com/blog/how-we-test-docker-desktop-with-wsl-2/ Mon, 13 Apr 2020 16:00:00 +0000 https://www.docker.com/blog/?p=25965 Recently we have released a new Edge version 2.2.3.0 of Docker Desktop for Windows. This can be considered as a release candidate for the next Stable version that will officially support WSL 2. With Windows 10 version 2004 in sight we are giving the next version of Docker Desktop the final touches to give you the best experience running Linux containers on Windows 10.

One of the great benefits is that with the next update of Windows 10 we will also support running Docker Desktop on Windows 10 Home. We worked closely with Microsoft during the last few months to make Docker Desktop and WSL 2 fit together.

In this blog post we look behind the scenes at how we set up new WSL 2 capable test machines to run automated tests in our CI pipeline.

It started with a laptop

Let’s keep in mind that all automation somehow starts with manual steps and you evolve from there to get better and more automated. At the beginning of this project we were given a laptop back at KubeCon 2019 with an early version of WSL 2.

desktop wsl 2 1

With that single laptop our development team could start getting their hands on that new feature and integrating it into Docker Desktop. But of course, this doesn’t really scale for a whole team and we also needed automated tests.

The Docker Desktop test matrix

In the Docker Desktop team we run several test suites across several Windows and Mac machines with different operating system versions installed. Each code change is tested with a matrix of tests on selected machines.

desktop wsl 2 2

One of our challenges was to add Windows machines to this matrix with WSL 2 enabled. At that time the Windows Insider program started to ship first releases and we could start automating the process to keep new test machines up to date.

On-demand test runners

The startup time of Docker Desktop is much faster with the WSL 2 backend. This gave us the option to run the end-to-end tests in virtual machines. We enhanced our CI infrastructure to spin up Windows 10 Insider machines in Azure on demand. This gave us more flexibility to keep the test machines at a working version of WSL 2 in our pool and also trying out the latest Insider builds.

Our internal CI dashboard shows all the test machines and the jobs running on them changed every few weeks. We constantly moved from one Insider release to the next. Currently we are concentrating on the final Slow Ring builds 19041.x, but we are also continuing with the next Fast Ring machines to have feedback from upcoming Windows builds.

desktop wsl 2 3

Automated pipeline to build the test machines

The Azure VM images we use to spin up WSL 2 test machines are created with a separate CI pipeline. We use Packer to create the VM image from an ISO file and run provision scripts to prepare everything we need to run it as a CI runner. The pipeline of how we build and upload the VM image also contains more than just the build step. We first check the source code of the Packer template and the PowerShell and Unix shell scripts to fail fast if a code change broke something. The Packer build itself takes the longest time, it also runs a Windows Update in the VM to get the latest OS version. After the build we added a verification step using InSpec to check if the software we need is installed correctly.

desktop wsl 2 4

The output of this Packer pipeline is an Azure VM image that can be used to spin up new on-demand runners in other CI pipelines. We normally run some tests in a canary environment to see if the VM image really boots up and attaches to our CI infrastructure. If everything is fine we update the configuration for the Docker Desktop CI for our end-to-end tests.

A new challenge: Windows 10 Home

With that automation for Windows 10 Pro machines at hand we were able to add Windows 10 Home very easily. Of course there were some challenges, for example Windows 10 Home does not provide Remote Desktop support. We added a VNC server to be able to attach to the cloud runners if we want to investigate problems.

Conclusion

In the last 12 months the Docker Desktop team worked hard to bring not only the WSL 2 support to Docker Desktop, but also enabled Windows 10 Home users to easily run Docker on their machines. We really look forward to the official release of Windows 10, version 2004 and love to hear your feedback.

]]>
Docker Desktop for Windows Home is here! https://www.docker.com/blog/docker-desktop-for-windows-home-is-here/ Thu, 05 Mar 2020 15:01:13 +0000 https://www.docker.com/blog/?p=25713 Last year we announced that Docker had released a preview of Docker Desktop with WSL 2 integration. We are now pleased to announce that we have completed the work to enable experimental support for Windows Home WSL 2 integration. This means that Windows Insider users on 19040 or higher can now install and use Docker Desktop!

Feedback on this first version of Docker Desktop for Windows Home is welcomed! To get started, you will need to be on Windows Insider Preview build 19040 or higher and install the Docker Desktop Edge 2.2.2.0.

What’s in Docker Desktop for Windows Home?

Docker Desktop for WSL 2 Windows Home is a full version of Docker Desktop for Linux container development. It comes with the same feature set as our existing Docker Desktop WSL 2 backend. This gives you: 

  • Latest version of Docker on your Windows machine 
  • Install Kubernetes in one click on Windows Home 
  • Integrated UI to view/manage your running containers 
  • Start Docker Desktop in <5 seconds
  • Use Linux Workspaces
  • Dynamic resource/memory allocation 
  • Networking stack, support for http proxy settings, and trusted CA synchronization 

How do I get started developing with Docker Desktop? 

For the best experience of developing with Docker and WSL 2, we suggest having your code inside a Linux distribution. This improves the file system performance and thanks to products like VSCode mean you can still do all of your work inside the Windows UI and in an IDE you know and love. 

Firstly make sure you are on the Windows insider program, are on 19040 and have installed Docker Desktop Edge.

Next install a WSL distribution of Linux (for this example I will assume something like Ubuntu from the Microsoft store).

You may want to check your distro is set to V2, to check in powershell run

wsl -l -v 

If you see your distro is a version one you will need to run 

wsl ‐‐set-version DistroName 2

Once you have a V2 WSL distro, Docker Desktop will automatically set this up with Docker.

The next step is to start working with your code inside this Ubuntu distro and ideally with your IDE still in Windows. In VSCode this is pretty straightforward.

You will want to open up VSCode and install the Remote WSL extension, this will allow you to work with a remote server in the Linux distro and your IDE client still on Windows. 

 Now we need to get started working in VSCode remotely, the easiest way to do this is to open up your terminal and type:

Wsl
code .

This will open a new VSCode connected remotely to your default distro which you can check in the bottom corner of the screen. 

(or you can just look for Ubuntu in your start menu, open it and then run  code . )

windows home

Once in VSCode there I use the terminal in VSCode to pull my code and start working natively in Linux with Docker from my Windows Home Machine!

Other tips and tricks:

Your feedback needed!

We are excited to get your feedback on the first version of Docker Desktop for Windows Home and for you to tell us how we can make it even better.

To get started with WSL 2 Docker Desktop on Windows home today you will need to be on Windows Insider Preview build 19040 or higher and install the Docker Desktop Edge 2.2.2.0.

]]>