Fix SC2046: "Quote this to prevent word splitting."
Ref: https://www.shellcheck.net/wiki/SC2046
Also:
- shellcheck.sh: add `set -eu`.
- shellcheck.sh, yamlcheck.sh: always run from repo root.
- pass `--` before passing the list of files, where missing.
- badwords.pl, cleancmd.pl: rework to accept `git ls-files` arguments.
Requires Perl 5.22+ (2015-Jun-01) on Windows.
Ref: https://perldoc.perl.org/functions/open
- INTERNALS.md: require Perl 5.22 on Windows.
- spacecheck.pl: formatting.
- GHA/http3-linux: rework command to avoid SC2046.
- stop suppressing SC2046 warnings.
The yamlcheck.sh issue reported-by: Stanislav Fort (Aisle Research)
Ref: 20251109163515_6eb31da3-deb2-4f4d-8327-935904f27da5
Closes#19432
It also does shellcheck on `run:` elements, overlapping with
the homegrown `shellcheck-ci.sh` with the same purpose. But it also does
more and perhaps could replace the script too, especially in curl
sub-repos.
Also:
- GHA/macos: delete potentially useful, but commented, and ultimately
unused, non-default Xcode-testing logic. It's causing unused matrix
exceptions, upsetting actionlint.
Ref: https://github.com/rhysd/actionlintCloses#19373
To silence zizmor 1.16.0 warnings.
Also:
- http3-linux: replace hard-coded workflow name with variable.
Follow-up to a8174176b5#13841
- codeql: set `cancel-in-progress: true`.
zizmor apparently does not allow `false` in pedantic mode anymore:
https://github.com/zizmorcore/zizmor/pull/1227
- codeql: sync concurrency setting with the rest of the jobs.
(I'm not sure this is correct, or why it was previously special-cased.)
Expressions used (before and after this patch):
- `group: ${{ github.workflow }}-${{ github.event.sha }}-${{ github.event.target_url }}`
for GHA/appveyor-status.
- `group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}`
for all the rest.
Ref: https://github.com/curl/curl/actions/runs/18776245057/job/53571438139?pr=19209Closes#19215
Make autotools and cmake detect DES support in OpenSSL and mbedTLS.
Forward feature macros to C and omit NTLM from the feature preview list.
Use the feature macros in source. This ensure that `-V` output matches
the preview.
OpenSSL doesn't support DES when built with `no-des` or `no-deprecated`.
mbedTLS 4.x no longer supports it, and it's possible to disable it in
<4 with `scripts/config.py unset MBEDTLS_DES_C`.
Before this patch this worked for
mbedTLS 4 only, and with a regression for pending PR #16973.
Also:
- drop NTLM feature check from `curl_setup.h` in favour of autotools/
cmake feature macros. This makes `curl_setup.h` no longer need
to include an mbedTLS header, which in turn makes tests/server build
without depending on mbedTLS.
Fixing, in #16973:
```
In file included from tests/server/first.h:40,
from bld/tests/server/servers.c:3:
lib/curl_setup.h:741:10: fatal error: mbedtls/version.h: No such file or directory
741 | #include <mbedtls/version.h>
| ^~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/18689537893/job/53291322012?pr=16973
Ref: #19181 (initial fix idea)
Follow-up to 3a305831d1#19077
- move back mbedTLS header include and version check from
`curl_setup.h` to each source which consumes mbedTLS.
- GHA/http3-linux: drop workaround that disabled NTLM for
`no-deprecated` OpenSSL builds.
Follow-up to 006977859d#12384
- curl_ntlm_core: drop pointless macro `CURL_NTLM_NOT_SUPPORTED`.
Follow-up to 006977859d#12384Closes#19206
- http3-linux: move local nghttpx (nghttp2) build to openssl (from
quictls). Also tried LibreSSL, but it made some HTTP/2 tests fails.
- http3-linux: drop quictls ngtcp2 build.
- http3-linux: build local openssl with `no-deprecated`.
(previously tested in the quictls local build.)
- http3-linux: explicitly disable LDAP in cmake openssl jobs.
cmake builds auto-detect OpenLDAP (autotools don't), and when enabled,
linking curl fails because system `libsasl.so` requires MD5 openssl
functions, which are missing from openssl no-deprecated builds.
- macos: move options tested in quictls jobs to other ones.
- linux: drop unused quictls local build. (it was used for msh3.)
Follow-up to 91138b014d#17729
- renovate: drop quictls bump detection.
Closes#18833
Also:
- point the source tarball to a working URL.
The GitHub release page misses the official source tarball for 4.1.1.
- GHA/linux: switch LibreSSL build to cmake (syncing with http3-linux.)
- GHA/macos: drop no longer needed LibreSSL build workaround.
Closes#18792
To avoid typos and non-existing options passed to `./configure` in CI
builds.
Also delete obsolete option `--enable-test-bundles` from Circle CI jobs.
Closes#18759
- requirements.txt: shorten copyright headers.
- requirements.txt: pin packages to versions.
- GHA/windows: use `tests/requirements.txt`.
Pick a `cryptography` package version that satifies both `impacket`
and pytests dependencies.
- GHA/checksrc: move pip deps into a new `requirements.txt`.
To make Dependabot detect and bump them.
- GHA/checksrc: replace apt packages for python test deps with pip
install `tests/**/requirements.txt` to a venv.
- GHA/checksrc: use venv and drop `--break-system-packages`.
- GHA/linux: fix to actually activate venvs.
Follow-up to 2638570241#15578
- GHA/linux: fixup (did not cause an issue)
Follow-up to d75785c7de#18660
- GHA: create venvs later, simplify commands.
- GHA: sync pip command-line options, e.g. drop progress-bar,
everywhere.
Assisted-by: Dan Fandrich
Closes#18708
- fix `nghttp2` build to also build the `nghttpx` application.
Restore required `libc-ares-dev`. Also confirm that `libev-dev` is
required too. Document these requirements.
Follow-up to 0455d8772a#18509
- explicitly enable `nghttpx` for the `nghttp2` build to make it fail if
requirements aren't met:
```
configure: error: applications were requested (--enable-app) but dependencies are not met.
```
- explicitly install brotli, zstd, zlib for the dependency builds.
Of these, zstd and zlib are preinstalled. zlib is required for
`nghttpx`. zstd and brotli doesn't seem to be used, but keep them
there just in case and to match the test env.
Follow-up to 0455d8772a#18509
- enable brotli for `nghttpx`. It doesn't change the tests, and also
cost almost nothing, so I figure why not.
Closes#18522
- explicitly install `libldap-dev` to not rely on test-specific packages
installing it implicitly, to have the same `curl -V` output for each
TLS backend build pair.
Follow-up to 0455d8772a#18509
- install `libev-dev` for tests. It's a runtime dependency for
the local build of `nghttpx`. Missing it made pytest skip 178 tests.
Also skewing the 'Gain' time. I estimate it to account for 3 minutes,
making the total gain ~20 minutes.
Follow-up to 0455d8772a#18509
(It may be a better solution to disable libev for the local nghttp2
build, to avoid this hidden dependency.)
- fix quiche jobs to use the local build of `libnghttp2`.
- stop installing the `clang` package for Windows-cross. `clang` and
`clang-tidy` tools are preinstalled on the Ubuntu 24.04 runner.
Closes#18519
In the last couple of months some jobs started taking a lot of time and
often timing out due to slow `apt install` from the Azure Ubuntu mirror.
The jobs affected were those that installed large packages:
GHA/http3-linux and the 3 cross-build jobs in GHA/windows.
This patch reduces the installed packaged to the minimum required
to complete the jobs. Saving a minute+ for each http3-linux job (a total
of 20+ minutes for the workflow.) Also saving bandwidth and reducing
the chance for long downloads or timeouts with slow Azure repos.
Details:
- http3: delete redundant packages from the `build-cache` job.
- http3: install gnutls dependencies for gnutls jobs only.
- http3: do not install test dependencies in jobs not running tests.
- http3: drop redundant packages from the curl jobs.
- Windows-cross: replace `mingw-w64` with `gcc-mingw-w64-x86-64-win32`
for the 3 Windows cross-build job. Dropping C++, 32-bit, and 64-bit
POSIX-threaded parts. Saving time and significant bandwidth for each
of the 3 jobs:
Download size: 277 MB -> 65 MB (installed: 1300 MB -> 400 MB)
- Windows-cross: restore previous job time limit of 15m (from 45m)
Follow-up to ff5140a25f#18163
Before:
https://github.com/curl/curl/actions/runs/17611514207 (http3)
https://github.com/curl/curl/actions/runs/17611514185/job/50034354923 (Windows cross)
After:
https://github.com/curl/curl/actions/runs/17628406362?pr=18509 (http3)
https://github.com/curl/curl/actions/runs/17627562551/job/50088055529?pr=18509 (Windows cross)
http3 job | Bef. | Aft. |
:------------------ | ------: | ------: |
Build caches (hot) | 10s | 12s |
AM awslc | 3m 0s | 1m 54s |
CM awslc | 4m 32s | 3m 4s |
AM boringssl | 3m 9s | 1m 48s |
CM boringssl | 3m 43s | 3m 2s |
AM gnutls | 3m 9s | 2m 18s |
CM gnutls | 4m 19s | 2m 55s |
AM libressl | 2m 14s | 1m 24s |
CM libressl | 5m 30s | 2m 57s |
AM openssl | 5m 16s | 4m 17s |
CM openssl | 1m 50s | 1m 47s |
AM openssl-quic | 2m 58s | 1m 7s |
CM openssl-quic | 4m 16s | 2m 43s |
AM quiche | 2m 54s | 1m 34s |
CM quiche | 5m 0s | 3m 15s |
AM quictls | 2m 34s | 1m 13s |
CM quictls | 4m 20s | 3m 17s |
AM wolfssl | 2m 48s | 1m 30s |
CM wolfssl | 4m 49s | 3m 22s |
Total: | 66m 21s | 43m 27s |
Gain: | | 22m 54s |
Out of curiousity, build times as seen in the http3 build-cache job:
- TLS backends:
- openssl: 2m25s
- libressl: 27s
- aws-lc: 41s
- boringssl: 1m8s
- quictls: 1m46s
- gnutls: 6m30s
- wolfssl: 51s
- quiche + boringssl: 1m9s
- ng* libs (not yet optimized for build speed):
- nghttp3: 13s
- ngtcp2: 52s (with 6 backends, 3 runs)
- ngtcp2: 19s (boringssl)
- nghttp2: 21s
Ref: https://github.com/curl/curl/actions/runs/17626120054/job/50083344805
A similar effort in curl-for-win, affecting 2 GHA/curl-for-win Windows
jobs (though they use the default Debian repo, with no issues):
- with llvm/clang:
Download size: 648 MB -> 430 MB (installed: 3344 MB -> 2333 MB)
- with gcc:
Download size: 550 MB -> 328 MB (installed: 2815 MB -> 1804 MB)
Ref: e19665d948
Ref: 6b14c3946a
Bug: https://github.com/curl/curl/pull/18502#issuecomment-3270259744Closes#18509
In CI we want to ensure that examples build cleanly, but we don't want
to actually run them there. Meaning it's enough to just compile, but not
link them in CI. Saving time up to 2-4x (MSVC), and disk space up
to 1.2GB (or 8-70x).
Add a new cmake target that compiles all examples without linking them
into runnable binaries. Keep a full build for a single example to test
if it links correctly.
Also:
- CI: switch over all `curl-examples` targets to `curl-examples-build`
- GHA/linux-old: build examples in one of the cmake builds.
Result highlights:
Job | Bef. | Bef. | Aft. | Aft. |
:------------------ | ---: | ----: | ---: |----: |
cygwin | 15s | 9MB | 10s | 1MB |
msys | 13s | 8MB | 7s | 1MB |
dl-mingw 15 | 39s | 113M | 34s | 2MB |
dl-mingw 9.5.0 | 49s | 115MB | 42s | 2MB |
dl-mingw 7.3.0 | 19s | 113MB | 14s | 2MB |
dl-mingw 6.4.0 | 9s | 12MB | 7s | 4MB |
Linux cross | 19s | 28MB | 19s | 2MB |
MSVC UWP | 65s | 374MB | 9s | 17MB |
MSVC x64 | 22s | 846MB | 9s | 17MB |
VS2010 | 48s | 105MB | 15s | 9MB |
VS2022 clang-cl | 195s | 1.2GB | 51s | 20MB |
iOS Xcode | 8s | | 5s | |
macOS LibreSSL | 16s | | 11s | |
Linux aws-lc | 3s | | 1s | |
Follow-up to dda251ef10#18232Closes#18209
- add startup check for 'danted' to avoid fails on low cpu
- rename 'sockd' to 'danted' everywhere to clarify what we use
- add proper defaults for 'danted' for debian
- install 'dante-server' in pytest ci runs
Closes#18075
ngtcp2 1.14.0 added crypto dependencies to the ngtcp2 crypto `.pc`
files. It broke GHA builds, because how curl's `configure` is setting up
the per-dependency custom prefixes for pkg-config.
`configure` uses `PKG_CONFIG_LIBDIR` to set per-dependency custom
prefixes, as specified via `--with-ngtcp2=<custom-dir>`. In classic
`pkg-config` this overrides any previously configured `PKG_CONFIG_DIR`.
This in turn break detecting transitive pkg-config modules unless they
are found at locations `pkg-config` is searching by default. This
doesn't affect `pkgconf` because it appends `PKG_CONFIG_LIBDIR` to
the custom `PKG_CONFIG_DIR`, according to its man page.
It may make sense to fix this in`acinclude.m4`, to make sure to honor
global custom pkg-config paths while detecting components at custom
locations, regardless of pkg-config implementation. But this PR doesn't
do this.
Instead it drops the ngtcp2 custom path and lets detection rely on
`PKG_CONFIG_DIR` that's already set up for all custom-built dependencies
anyway.
Also:
- fix `openssl-quic` job to use the custom-built nghttp2 (like other
jobs do) instead of the system default.
- configure nghttp3 via `PKG_CONFIG_DIR` in the `openssl-quic` job,
to sync with other jobs. And drop `--with-nghttp3` option.
cb9b1a4c4e/acinclude.m4 (L1376-L1381)https://manpages.debian.org/unstable/pkg-config/pkg-config.1.en.htmlhttps://man.archlinux.org/man/pkgconf.1.en
Ref: https://github.com/ngtcp2/ngtcp2/pull/1689#issuecomment-3121576712Closes#18022Closes#18028
