Skip to content

Commit 77d8909

Browse files
compnerdcompnerd
committed
Windows: add a toolchain image for 10.0.19044.1706 (21H2)
Add rules for building a docker image for the toolchain build at 10.0.19044.1706 (21H2). As the base image for the release is not available, use a slightly older release which should be compatible to avoid any issues due to kernel mismatches (the newer image will not run). This allows building the toolchain in a docker image with the latest stable release of Windows 10.
1 parent 594ab4b commit 77d8909

File tree

5 files changed

+211
-0
lines changed

5 files changed

+211
-0
lines changed

‎swift-ci/README.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,114 @@ docker run \
3535
/bin/bash -lc "cp -r /source/* /home/build-user/; ./swift/utils/build-script --preset buildbot_linux install_destdir=/home/build-user/swift-install installable_package=/home/build-user/swift-DEVELOPMENT-SNAPSHOT-$(date +'%F')-a.tar.gz"
3636
```
3737

38+
## Quick Start for Windows Development
39+
40+
The Windows Docker image will setup an enviornment with Python, Visual Studio
41+
Build Tools, and Git. It is setup to assume that the sources will be available
42+
in `S:\SourceCache`.
43+
44+
### Building and Tagging the image
45+
46+
```powershell
47+
cd master\windows\10.0.19044.1706
48+
docker image build --compress -t swift:swiftci .
49+
```
50+
51+
### Running the image
52+
53+
```powershell
54+
docker run --rm -it -v %UserProfile%\data:S: swift:swiftci
55+
```
56+
57+
### Building the Toolchain
58+
59+
While we can build the toolchain in the containerized environment, the sources
60+
are expected to reside on the host and is passed into the docker container as a
61+
volume. The rest of automation expects that the Sources reside under a
62+
directory with the name `SourceCache`.
63+
64+
#### Clone the Sources
65+
66+
```cmd
67+
md %UserProfile%\data\SourceCache
68+
cd %UserProfile%\data\SourceCache
69+
70+
git clone -b stable/20220426 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/llvm-project
71+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift
72+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-cmark cmark
73+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-experimental-string-processing
74+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-corelibs-libdispatch
75+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-corelibs-foundation
76+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-corelibs-xctest
77+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-argument-parser
78+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-crypto
79+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-driver
80+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-llbuild llbuild
81+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-package-manager
82+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-system
83+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-tools-support-core
84+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/swift-installer-scripts
85+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/indexstore-db
86+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/apple/sourcekit-lsp
87+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/jpsim/Yams
88+
git clone -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/compnerd/swift-build
89+
git clone -t curl-7_77_0 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/curl/curl
90+
git clone -t v2.9.12 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/gnome/libxml2
91+
git clone -t v1.2.11 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/madler/zlib
92+
git clone -b maint/maint-69 -c core.autocrlf=input -c core.symlink=true -c core.useBuiltinFSMonitor=false https://github.com/unicode-org/icu
93+
```
94+
95+
#### Run Docker
96+
97+
```cmd
98+
docker run --rm -it -v %UserProfile%\data:S: swift:swiftci
99+
```
100+
101+
#### Build the Toolchain
102+
103+
This will build a full Swift toolchain distribution (llvm, clang, lld, lldb,
104+
swift, swift-package-manger, SourceKit-LSP) and the Windows SDK (x86, x64,
105+
ARM64).
106+
107+
```cmd
108+
S:
109+
S:\SourceCache\swift\utils\build.cmd
110+
```
111+
112+
#### Running Swift Tests
113+
114+
The toolchain tests require some modifications to the path to find some of the
115+
dependencies. The following will run the Swift test suite within the docker
116+
container:
117+
118+
```cmd
119+
path S:\b\1\bin;S:\b\1\tools\swift\libdispatch-windows-x86_64-prefix\bin;%Path%;%ProgramFiles%\Git\usr\bin
120+
ninja -C S:\b\1 check-swift
121+
```
122+
123+
#### Using the Toolchain
124+
125+
> **NOTE**: Running the test suite and using the toolchain near the production mode are mututally incompatible (due to the path changes).
126+
127+
The build will generate a toolchain image that is roughly similar to the
128+
installed version. The following can be run inside the docker container to use
129+
the toolchain:
130+
131+
```cmd
132+
set SDKROOT=S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
133+
path S:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\x64;S:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin;%Path%
134+
```
135+
136+
Because the toolchain is built in the volume which is backed by the host, the
137+
toolchain can be used on the host (assuming the dependencies such as Visual
138+
Studio is installed and the module modules deployed). The adjusted paths below
139+
should enable that:
140+
141+
```cmd
142+
set SDKROOT=%UserProfile%\data\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk
143+
path %UserProfile%\data\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\bin\x64;%UserProfile%\data\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin;%Path%
144+
```
145+
38146
## Contributions
39147

40148
Contributions via pull requests are welcome and encouraged :)

‎swift-ci/master/windows/10.0.19044.1706/Dockerfile‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# escape=`
2+
3+
# 10.0.19044.1706 is not yet published, use the 20H2 snapshot as that is
4+
# believed to work better than the newer image.
5+
FROM mcr.microsoft.com/windows/servercore:10.0.19042.1706 AS windows
6+
7+
LABEL maintainer="Swift Infrastructure <swift-infrastructure@forums.swift.org>"
8+
LABEL description="Docker Container for the Swift programming language"
9+
10+
ARG GIT=https://github.com/git-for-windows/git/releases/download/v2.36.1.windows.1/Git-2.36.1-64-bit.exe
11+
ARG PYTHON=https://www.python.org/ftp/python/3.10.4/python-3.10.4-amd64.exe
12+
13+
# restore the default Windows shell for correct batch processing
14+
SHELL ["cmd", "/S", "/C"]
15+
16+
# Enable Developer Mode.
17+
RUN reg add "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock" /t REG_DWORD /f /v "AllowDevelopmentWithoutDevLicense" /d "1"
18+
19+
# Enable Long Paths
20+
RUN reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem" /t REG_DWORD /f /v "LongPathsEnabled" /d "1"
21+
22+
# Install Git.
23+
# See: git-[version]-[bit].exe /SAVEINF=git.inf and /?
24+
COPY git.inf .
25+
RUN `
26+
curl -SLo git.exe %GIT% `
27+
&& (start /w git.exe /SP- /VERYSILENT /SUPPRESSMSGBOXES /NOCANCEL /NORESTART /CLOSEAPPLICATIONS /FORCECLOSEAPPLICATIONS /LOADINF=git.inf ) `
28+
&& del /q git.exe git.inf
29+
30+
# Install Python.
31+
# See: https://docs.python.org/3.10/using/windows.html
32+
# FIXME: it appears that `PYTHONHOME` and `PYTHONPATH` are unset
33+
COPY unattend.xml .
34+
RUN `
35+
curl -SLo python.exe %PYTHON% `
36+
&& (start /w python.exe /quiet ) `
37+
&& del /q python.exe unattend.xml
38+
39+
# Install Visual Studio Build Tools
40+
RUN `
41+
curl -SLo vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
42+
&& (start /w vs_buildtools.exe --quiet --wait --norestart --nocache `
43+
--add Microsoft.VisualStudio.Component.Windows11SDK.22000 `
44+
--add Microsoft.VisualStudio.Component.VC.ATL `
45+
--add Microsoft.VisualStudio.Component.VC.ATL.ARM `
46+
--add Microsoft.VisualStudio.Component.VC.ATL.ARM64 `
47+
--add Microsoft.VisualStudio.Component.VC.CMake.Project `
48+
--add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
49+
--add Microsoft.VisualStudio.Component.VC.Tools.ARM64 `
50+
--add Microsoft.VisualStudio.Component.VC.Tools.ARM `
51+
|| IF "%EXITCODE%"=="3010" EXIT 0) `
52+
&& del /q vs_buildtools.exe
53+
COPY InstallModules.cmd .
54+
RUN InstallModules.cmd && del /q InstallModules.cmd
55+
56+
# FIXME: we should use a non-Administrator user
57+
# USER ContainerUser
58+
59+
ENV PYTHONUTF8=1
60+
# Default to powershell
61+
CMD ["powershell.exe", "-nologo", "-ExecutionPolicy", "Bypass"]

‎swift-ci/master/windows/10.0.19044.1706/InstallModules.cmd‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
@echo off
2+
setlocal
3+
set vswhere=%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe
4+
FOR /F "tokens=* usebackq" %%r IN (`"%vswhere%" -nologo -latest -all -prerelease -products * -property installationPath`) DO SET VsDevCmd=%%r\Common7\Tools\VsDevCmd.bat
5+
CALL "%VsDevCmd%" -no_logo -host_arch=amd64 -arch=amd64
6+
mklink "%UniversalCRTSdkDir%\Include\%UCRTVersion%\ucrt\module.modulemap" S:\SourceCache\swift\stdlib\public\Platform\ucrt.modulemap
7+
mklink "%UniversalCRTSdkDir%\Include\%UCRTVersion%\um\module.modulemap" S:\SourceCache\swift\stdlib\public\Platform\winsdk.modulemap
8+
mklink "%VCToolsInstallDir%\include\module.modulemap" S:\SourceCache\swift\stdlib\public\Platform\visualc.modulemap
9+
mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\SourceCache\swift\stdlib\public\Platform\visualc.apinotes
10+
endlocal
11+
@echo on

‎swift-ci/master/windows/10.0.19044.1706/git.inf‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[Setup]
2+
NoIcons=1
3+
Components=gitlfs
4+
EditorOption=VIM
5+
PathOption=Cmd
6+
SSHOption=OpenSSH
7+
TurtoiseOption=false
8+
CURLOption=WinSSL
9+
BashTerminalOption=ConHost
10+
PerformanceTweaksFSCache=Enabled
11+
EnableSymlinks=Enabled
12+
EnablePseudoConsoltSupport=Disabled
13+
EnableFSMonitor=Enabled

‎swift-ci/master/windows/10.0.19044.1706/unattend.xml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
<?xml version="1.0" encoding="utf-8"?>
2+
<Options>
3+
<Option Name="InstallAllUsers" Value="1" />
4+
<Option Name="AssociateFiles" Value="0" />
5+
<Option Name="PrependPath" Value="1" />
6+
<Option Name="Shortcuts" Value="0" />
7+
<Option Name="Include_doc" Value="0" />
8+
<Option Name="Include_debug" Value="0" />
9+
<Option Name="Include_dev" Value="1" />
10+
<Option Name="Include_exe" Value="1" />
11+
<Option Name="Include_launcher" Value="0" />
12+
<Option Name="InstallLauncherAllUsers" Value="0" />
13+
<Option Name="Include_lib" Value="1" />
14+
<Option Name="Include_symbols" Value="0" />
15+
<Option Name="Include_tcltk" Value="0" />
16+
<Option Name="Include_test" Value="0" />
17+
<Option Name="Include_tools" Value="0" />
18+
</Options>

0 commit comments

Comments
 (0)