Tests with old cmake are slow. (no Ninja, no unity, and running slower
than recent versions.)
It also revealed that 3.7.2 2017-01-13 is too old to consume curl via
`find_package()` due to:
```
CMake Error at bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:69 (add_library):
add_library cannot create ALIAS target "CURL::libcurl" because target
"CURL::libcurl_shared" is IMPORTED.
Call Stack (most recent call first):
CMakeLists.txt:48 (find_package)
CMake Error at bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:69 (add_library):
add_library cannot create ALIAS target "CURL::libcurl" because target
"CURL::libcurl_shared" is IMPORTED.
Call Stack (most recent call first):
CMakeLists.txt:49 (find_package)
```
The mitigation for this issue requires 3.11.
Also:
- rename a few existing envs to use the `TEST_` prefix.
- make the `find_package` test provider stage verbose.
- fix issue when consuming with cmake 3.7.2 (all platforms):
```
CMake Error at /home/runner/cmake-3.7.2-Linux-x86_64/share/cmake-3.7/Modules/CMakeFindDependencyMacro.cmake:25 (message):
Invalid arguments to find_dependency. VERSION is empty
Call Stack (most recent call first):
bld-curl/_pkg/lib/cmake/CURL/CURLConfig.cmake:52 (find_dependency)
CMakeLists.txt:48 (find_package)
```
Ref: https://github.com/curl/curl/actions/runs/14906066962/job/41868621979?pr=17293#step:9:1199Closes#17293
The pedantic level is experimental. If it causes issues, we may just
disable it alongside the ignore comments.
Also:
- silence error:
```
INFO audit: zizmor: completed label.yml
error[dangerous-triggers]: use of fundamentally insecure workflow trigger
--> label.yml:13:1
|
13 | 'on': [pull_request_target]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ pull_request_target is almost always used insecurely
|
= note: audit confidence -> Medium
```
- fix pedantic warning:
```
INFO audit: zizmor: completed label.yml
warning[excessive-permissions]: overly broad permissions
--> label.yml:1:1
... |
24 | | with:
25 | | repo-token: '${{ secrets.GITHUB_TOKEN }}'
| |____________________________________________________- default permissions used due to no permissions: block
|
= note: audit confidence -> Medium
```
- silence `template-injection` false positives like:
```
- note: ${{ matrix.build.torture && 'test-torture' || 'test-ci' }} may expand into attacker-controllable code
- note: ${{ contains(matrix.build.install_steps, 'pytest') && 'caddy httpd vsftpd' || '' }} may expand into attacker-controllable code
```
It doesn't seem like these could be controlled by an attacker.
Let me know if I'm missing something.
Closes#17278
The unplanned dropping of the granular vcpkg binary cache indeed fell
into the cracks between Microsoft's various departments. The old method
is now official dropped, without replacement either on the vcpkg side or
the GitHub cache provider side.
Without a granular cache, vcpkg is impractical for builds larger than
a small dependency tree in CI, for performance reasons.
A granular cache is critical for CI use. Building dependencies is not
a goal of this CI, so a more desirable option would be pre-built binary
downloads. This would also allow keeping job timeouts low, which is
important for quick iteration in GHA when a flaky job requiring a manual
retry needs all other jobs to finish first. (GHA often disregards
step timeouts, which is another contributing factor here.)
Windows remains tested extensively with MSYS2, curl-for-win, and via
AppVeyor CI with MSVC + OpenSSL, and also in GHA via scaled back vcpkg
jobs that perform well without caching. What's lost is the recently
added Android OpenSSL build tests.
We may consider building/cachine important dependencies manually as in
GHA/linux-http3, and/or try integrating MSVC jobs with MSYS2 UCRT DLLs.
Ref: https://github.com/microsoft/vcpkg-tool/pull/1662
Ref: https://github.com/microsoft/vcpkg/issues/45073
Follow-up to cd0ec4784c#17089
Follow-up to e3912f0f9f#17086
Follow-up to 15fb1dc7f8#17069Closes#17200
- GHA: add cmake integration tests for Windows.
- make them run faster with prefill, unity, Ninja, omitting curl tool.
- also test static libcurl.
- add old-cmake support with auto-detection.
- auto-detect Ninja.
- run consumer test apps to see if they work.
- add support for Windows.
- make it more verbose.
- re-add `ExternalProject` cmake consumer test. It's broken.
- tidy up terminology.
Cherry-picked from #16973Closes#17203
This step could take from 5 seconds to 5 minutes, sometimes making it
run out of its time slot. It affected 60 CI jobs.
Saving an estimated minimum of 5 minutes per CI run.
Also fixing:
```
Fri, 25 Apr 2025 06:19:14 GMT
Processing triggers for man-db (2.12.0-4build2) ...
Fri, 25 Apr 2025 06:23:40 GMT
Running kernel seems to be up-to-date.
[...]
Error: The action 'install packages' has timed out after 5 minutes.
```
Ref: https://github.com/curl/curl/actions/runs/14658212268/job/41136971525?pr=17180#step:2:169Closes#17181
They typically finish (well) within 10 minutes.
A notable exception was vcpkg jobs when a rebuild was triggered.
With caching lost and reducing them to short builds, this is not
an issue at the moment.
The advantage of shorter timeouts is hung/crashed jobs giving back
control earlier for a manual retry.
Closes#17173
Build/install the man page. Install the script.
This an "import" rather than just adding the script into the release tarball
with maketgz to make sure the tarball build is completely reproducible.
wcurl development, issues etc is still to be done at
https://github.com/curl/wcurl
Assisted-by: Viktor Szakats
Assisted-by: Samuel Henrique
Closes#17035
- fix filter expression for cache step.
It did not cause an issue because `wolfssh` is always paired with
a `wolfssl*`.
- build `wolfssh` against `wolfssl-opensslextra` (was: `wolfssl-all`).
It makes `wolfssh` builds `OPENSSL_COEXIST`-compatible, and clarifies
its use in the clang-tidy job. The earlier mixup didn't cause issues
because the clang-tidy job is compile-only (using their headers only.)
Useful side-effect is making the wolfssh build valgrind-tested.
Reported-by: bo0tzz on github
Ref: https://github.com/curl/curl/discussions/16970#discussioncomment-12752019Closes#17109
To keep testing these with MSVC:
- UWP !ssl
- arm64 build on the `windows-11-arm` runner
- examples
- OpenSSH-Windows (fix install on `windows-11-arm`)
- `windows-2025` runner
- cmake pre-fill checker
Surprise: UWP doesn't support SSPI, which is required by curl's Schannel
backend. Thus, no TLS support for this UWP build. It also suggests
the Schannel UWP mingw-w64 binaries may be broken and just a happy build
accident thanks to mingw-w64 headers being inaccurate.
Building zlib + libssh2 might actually take up to 2+ minutes with vcpkg,
instead of the previously estimated <1.5 minutes.
Follow-up to e3912f0f9f#17086
Follow-up to 15fb1dc7f8#17069Closes#17089
With the new addition of OpenSSL QUIC API support and the support in
ngtcp2 main branch, make the necessary adjustments in curl to support
this combination.
- add support in configure.ac to detect the feature OPENSSL_QUIC_API2 in
openssl
- initialise ngtcp2 properly in this combination
- add a Curl_vquic_init() for global initialisation that ngtcp2 likes
for performance reasons
- add documentation on how to build in docs/HTTP3.md
- add CI testing in http3-linux.yml
Assisted-by: Viktor Szakats
Closes#17027
In the last 1-2 days, curl's vcpkg binary cache entries stored by GHA
apparently disappeared, and each vcpkg job is now rebuilding everything
on every run. This takes up to 40 minutes per job. New cache entries
fail on upload with messages like:
```
Completed submission of nghttp2:arm64-android@1.65.0 to 0 binary cache(s) in 172 ms
```
(note the zero.)
This coincided with my adding support for windows-11-arm and touching
some jobs affected. Yet, unrelated. It also affects Android jobs. The root
cause is a runner image update bumping the vcpkg tool.
The latest vcpkg tool also has this issue.
As a temporary workaround, stop using vcpkg in CI.
Last good run: https://github.com/curl/curl/actions/runs/14473789417/job/40594304523
First bad run: https://github.com/curl/curl/actions/runs/14474616982/job/40597095025
Assisted-by: Tal Regev
Ref: #17070Closes#17069
- bump an MSYS2/mingw job to windows-2025 runner.
(MSVC is possible, but vcpkg needs to build for windows-2025, and
can't share these with windows-2022 builds, so not optimal for
a single canary job.)
- skip installing OpenSSH-Windows-builtin on windows-2025.
It's preinstalled:
```
ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 9.5.0
```
Still older than the manual preview install (9.8.1), so keep using that.
Closes#17066
To allow configuring paths styles for SCP and SFTP servers separately.
- make `scp://` URLs use `%SCP_PWD` (was: `%SSH_PWD`).
- make `%SCP_PWD` equal to `%POSIX_PWD`.
To fix test 3022 with OpenSSH-Windows 9.8.0 server.
The fix works on a local machine. Remains broken in CI.
Before this patch, it was equal to `%FILE_PWD` when using
OpenSSH-Windows, otherwise it was `%POSIX_PWD`.
Notice that no matter what path-style we pass, test 3022
was and still is broken with earlier OpenSSH-Windows versions.
(as tested with 9.5.0, 9.5.0-beta20240403, 8.0.0.1)
- rename rest of `%SSH_PWD` uses to `%SFTP_PWD`.
- drop unused `%POSIX_PWD`.
- GHA/windows: test with OpenSSH-Windows server again.
In the LibreSSL MSVC job. This job is short enough to fit the slow
install of the built-in OpenSSH-Windows tools, if needed.
Follow-up to 1abb087a9c#5298
Ref: #16803Closes#17041
On the windows-2022 runner it installs these client/server versions:
```
ssh client found /c/Windows/System32/OpenSSH/ssh.exe is OpenSSH-Windows 9.5.0
ssh server found /c/Windows/System32/OpenSSH/sshd.exe is OpenSSH-Windows 8.1.0
```
Not currently enabled. Slight downside (when enabled) that Windows needs
over 1 minute to install these two tiny programs.
Closes#17046
Found by improving verify-examples.pl:
- Operate directly on markdown files to remove the need to render nroff files
first.
- Add -Wall as a compiler option to find more issues
Closes#17028
Add workaround for an issue related to the gcc "hacklayer" after the
GitHub macos-15-arm64 runner bumped to 20250408.1231.
Fixes:
```
configure:5175: gcc-13 -o conftest --sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk -w conftest.c >&5
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:71,
from /opt/homebrew/Cellar/gcc@13/13.3.0/lib/gcc/13/gcc/aarch64-apple-darwin24/13/include-fixed/stdio.h:75,
from conftest.c:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h: In function 'fmemopen':
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:457:107: error: expected declaration specifiers before '__API_AVAILABLE_GET_MACRO_93585900'
457 | FILE *fmemopen(void * __restrict __buf _LIBC_SIZE(__size), size_t __size, const char * __restrict __mode) __API_AVAILABLE(macos(10.13), ios(11.0), tvos(11.0), watchos(4.0));
| ^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/14378524390/job/40316589059?pr=17012#step:7:169
Assisted-by: Bo Anderson
Bug: https://github.com/curl/curl/pull/17012#issuecomment-2792572344
Bug: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2792601570Closes#17017
Sometimes it's useful to have a look at the generated `libcurl.pc` and
`curl-config` files.
`cmp-config.pl` normalizes them before diffing, thus doesn't show their
original content.
Closes#16981
To allow making per-job variations for SSH backends.
Also:
- fix Cygwin builds to not ignore per-job `install:` items.
It worked by accident before this patch.
Follow-up to 66313cc036#16629Closes#16911
curl now has a working GnuTLS CI job, with tests, with MSYS2.
The MultiSSL build scenario is now tested on macOS.
The vcpkg GnuTLS package seems to have a deep dependency tree with large
packages that need to be rebuilt relatively frequently. Since they can't
fit into to the time limit, these cause CI failures.
To stabilize CI, drop the `shiftmedia-libgnutls` dependency.
Partial revert of e86f99824c#16623
Ref: https://github.com/curl/curl/actions/runs/14192680124/job/39760753274?pr=16902Closes#16904
In the memory and address sanitizer builds.
Verify that nothing unexpected happens.
Starting out with 60 second runs.
The script does not set any seed so it runs with a new random every
time, meaning that if it fails in a single CI run it might not fail in a
subsequent one: but it should still show the full command that failed to
enable us to reproduce it locally. We can work on improving the seed
situation later if this script turns useful.
Closes#16884
Add the last (*) missing bit for feature parity with autotools.
Also test in CI. Add a new `cmake install` step to GHA/macos.
(*) AFAIK. Let us know if there's something else missing.
Closes#16833
- move pytest from the valgrind variant of the mbedTLS and Rustls jobs
to their non-valgrind counterpart (they different in C compiler and
build tool respectively). To parallelize more and finish the workflow
faster.
- drop pytest from the valgrind variant of the two identical (other than
the build tool) 'libressl heimdal' jobs. Saves 1.5 minutes CI time.
- drop pytest from the longest valgrind job to make the workflow finish
almost 2 minutes faster. `sync-resolver` is its unique build propery.
It wasn't pytested on Azure.
- explicitly install `libnghttp2-dev` and `libldap-dev` to keep them in
jobs where pytest deps were installing them implicitly before this
patch.
Before: https://github.com/curl/curl/actions/runs/14118080563
After: https://github.com/curl/curl/actions/runs/14118903372?pr=16851Closes#16851
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.
Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.
As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.
Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.
Details:
- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
```
+ openssl genrsa -out test-ca.key -passout fd:0 2048
Invalid password argument, starting with "fd:"
```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
(`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
the Visual Studio generator miss to create the `clean-certs`,
`build-certs` targets. No target depend on them, so they don't execute
implicitly anyway. Fixes:
```
MSBUILD : error MSB1009: Project file does not exist.
Switch: clean-certs.vcxproj
```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.
Follow-up to 556f722fe3#16593
Follow-up to fa461b4eff#14486Closes#16824
