- GHA/windows: disable building certs in the MSVC job that's not running
tests. Saves 4-5 seconds for MSVC, makes logs shorter for the rests.
- GHA/linux: build tests in two more jobs (LTO, CM Rustls), 5s each.
- GHA/linux: skip 'install test prereqs' for `skiprun` jobs.
(there were no such jobs before this patch.)
Closes#18034
ngtcp2 1.14.0 added crypto dependencies to the ngtcp2 crypto `.pc`
files. It broke GHA builds, because how curl's `configure` is setting up
the per-dependency custom prefixes for pkg-config.
`configure` uses `PKG_CONFIG_LIBDIR` to set per-dependency custom
prefixes, as specified via `--with-ngtcp2=<custom-dir>`. In classic
`pkg-config` this overrides any previously configured `PKG_CONFIG_DIR`.
This in turn break detecting transitive pkg-config modules unless they
are found at locations `pkg-config` is searching by default. This
doesn't affect `pkgconf` because it appends `PKG_CONFIG_LIBDIR` to
the custom `PKG_CONFIG_DIR`, according to its man page.
It may make sense to fix this in`acinclude.m4`, to make sure to honor
global custom pkg-config paths while detecting components at custom
locations, regardless of pkg-config implementation. But this PR doesn't
do this.
Instead it drops the ngtcp2 custom path and lets detection rely on
`PKG_CONFIG_DIR` that's already set up for all custom-built dependencies
anyway.
Also:
- fix `openssl-quic` job to use the custom-built nghttp2 (like other
jobs do) instead of the system default.
- configure nghttp3 via `PKG_CONFIG_DIR` in the `openssl-quic` job,
to sync with other jobs. And drop `--with-nghttp3` option.
cb9b1a4c4e/acinclude.m4 (L1376-L1381)https://manpages.debian.org/unstable/pkg-config/pkg-config.1.en.htmlhttps://man.archlinux.org/man/pkgconf.1.en
Ref: https://github.com/ngtcp2/ngtcp2/pull/1689#issuecomment-3121576712Closes#18022Closes#18028
- runtests: fix `codeset-utf8` feature detection. Before this patch it
detected if the calling environment had UTF-8 enabled. If not, UTF-8
tests were all skipped. After this patch, it detects if UTF-8 is
supported by the calling environment regardless of what's currently
enabled.
Follow-up to 0b70b23ef4#15039
- GHA/linux: sync `codeset-test` to also reset `LC_CTYPE` and
`LC_NUMBER`. To give it more spin.
Follow-up to c221c0ee59#17938
- GHA/macos: fix to actually enable `codeset-test`. Also set `LC_ALL`,
which seems necessary to trigger issues.
Follow-up to c221c0ee59#17938
- tests/data: replace `LC_CTYPE` env with `LC_ALL` in all tests
requiring a locale. Also to avoid potential issues with a blank or
unset `LC_ALL`, as seen earlier. And to ensure that the override works
on all platforms (as tested in CI.)
Slight downside is that this now resets the language/culture to `C`.
Ref: b4c9982382#4743
Ref: 23208e330a#4738
- replace `en_US.UTF-8` with `C.UTF-8` to be language/culture-agnostic.
- TEST-SUITE.md: drop `UTF-8` as a requirement for tests.
Tests shall work (or least be skipped) without UTF-8 support.
Tests requiring UTF-8 locale:
165, 962, 963, 964, 965, 966, 967, 1448, 1560, 2046, 2047
Tests requiring UTF-8 locale, but passing without one anyway:
955, 956, 957, 958, 959, 960, 961, 968, 1034, 1035
Spec 1997: https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html
Spec 2008: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
Ref: c221c0ee59#17938
Ref: 7cf8414fab
Ref: 4c140a5628
Ref: 28faaacee2#2436
Ref: ecd1d020abCloses#17988
To fix running test 1560 when `LC_ALL` is set to something unexpected
(e.g. `C`). Also syncing it with the rest of tests.
Also:
- GHA/linux: enable `libidn2` in more jobs.
Also to enable test 1560 reproducing this issue in more jobs.
- GHA/linux: run tests with `LC_ALL=C` in one of the jobs.
- GHA/linux: switch to the non-deprecated package name for libidn2.
- GHA/macos: run tests with non-default locale settings in one job.
- GHA/macos: enable AppleIDN in that job.
Ref: https://github.com/curl/curl/pull/17933#issuecomment-3074582840
Follow-up to f27262b179#10196Closes#17938
- codespell: break logic out into its own runnable script. Allowing
to run it on local machines.
- codespell: install via `pip`, bump to latest version.
- codespell: show version number in CI log.
- codespell: drop no longer needed word exception: `msdos`.
- codespell: include all curl source tree, except `packages` and
`winbuild`. Drop an obsolete file exclusion.
- add new spellchecker job using the `typos` tool. It includes
the codespell dictionary and a couple more. Use linuxbrew to install
it. This takes 10 seconds, while installing via `cargo` from source
would take over a minute.
- codespell: introduce an inline ignore filter compatible with `cspell`
Make `typos` recognize it, too. Move single exceptions inline.
Fix new typos found. Also rename variables and words to keep
spellchecking exceptions at minumum. This involves touching some tests.
Also switch base64 strings to `%b64[]` to avoid false positives.
Ref: https://github.com/crate-ci/typos/blob/master/docs/reference.md
Ref: https://github.com/codespell-project/codespell?tab=readme-ov-file#inline-ignore
Ref: https://github.com/codespell-project/codespell/issues/1212#issuecomment-1721152455
Ref: https://cspell.org/docs/Configuration/document-settingsCloses#17905
- autotools: fix auto-detection on the Windows platform.
It was mis-detected when targeting Windows XP/2003 64-bit.
It was permanently undetected when building for Windows 32-bit.
```
lib/url.c: In function 'zonefrom_url':
lib/url.c:1802:18: error: implicit declaration of function 'if_nametoindex' [-Wimplicit-function-declaration]
1802 | scopeidx = if_nametoindex(zoneid);
| ^~~~~~~~~~~~~~
lib/url.c:1802:18: error: nested extern declaration of 'if_nametoindex' [-Werror=nested-externs]
```
Ref: https://github.com/curl/curl/actions/runs/16405598782/job/46351023138?pr=17982#step:10:29
Reported-by: LoRd_MuldeR
Fixes#17979
Without this patch the workaround for the 8.15.0 release is:
`export ac_cv_func_if_nametoindex=0` for Windows XP/2003 64-bit.
Background: Checking for the `if_nametoindex()` function via
`AC_CHECK_FUNCS()` (autotools) or `check_function_exists()` (cmake) do
not work on Windows, for two reasons:
- the function may be disabled at compile-time in Windows headers
when targeting old Windows versions (XP or WS2003 in curl context)
via `_WIN32_WINNT`. But it's always present in the system implib
`iphlpapi` where these checks are looking.
- for 32-bit Windows the function signature in the implib requires
a 4-byte argument, while these checks always use no arguments,
making them always fail.
- cmake: call `if_nametoindex` dynamically with mingw-w64 v1.0.
This mingw-w64 version lacks prototype and implib entry for it.
- cmake: add auto-detection for Windows and use as a fallback for
non-pre-fill cases.
- cmake: disable pre-fill with `_CURL_PREFILL=OFF`. (for testing)
- cmake: disable pre-fill for untested compilers. (i.e. non-MSVC,
non-mingw64)
- GHA/windows: make an autotools job build for Windows XP.
Follow-up to 0d71b18153#17413Closes#17982
To fix test 1308 in MultiSSL builds.
Failure was caused by the random number generator virtual function being
NULL, instead of pointing to the implementation in the runtime-selected
TLS backend. This could happen in MultiSSL builds when a functionality
was asking for a random number without triggering a VTLS function table
initialization first. Such functionality is MIME, or form data via MIME.
The reason CI did not catch it in an earlier MultiSSL GHA/windows job,
is that it was a debug-enabled one. In debug-enabled builds the test
runner was overriding the random number generator for all tests.
Fixed this by moving the override to the tests requiring it, via
1fcf22585f#17971, enabling debug builds
to catch this issue.
Enable MultiSSL in two CI jobs, to verify this patch.
Fixing:
```
test 1308...[formpost tests]
libtests returned 44, when expecting 0
1308: exit FAILED
[...]
=== Start of file stderr1308
URL: log/3/test-1308
tests/libtest/lib1308.c:70 Assertion 'res == 0' FAILED: curl_formget returned error
tests/libtest/lib1308.c:72 Assertion 'total_size == 518' FAILED: curl_formget got wrong size back
tests/libtest/lib1308.c:88 Assertion 'res == 0' FAILED: curl_formget returned error
tests/libtest/lib1308.c:89 Assertion 'total_size == 899' FAILED: curl_formget got wrong size back
```
Ref: https://github.com/curl/curl/actions/runs/16387693424/job/46309536359?pr=17963#step:16:2515
Bug: https://github.com/curl/curl/pull/17963#issuecomment-3092282057Closes#17970
Bind them to the (arbitrary choice of) Apple clang and gcc compilers,
respectively.
Also:
- bind existing mbedTLS job to the llvm compiler, to keep the number of
jobs the same as before this patch.
- move OpenLDAP from mbedTLS over to LibreSSL to keep testing it with
all 3 compilers.
- simplify exclusions for clang-tidy and torture jobs.
- tag clang-tidy and torture jobs via `install_steps`.
To avoid keeping around special bool fields.
Closes#17963
In preparation for the removal of Windows CE support.
These jobs were also using a macOS runner, which is a waste. And it'd be
too much effort to move them over to Windows or Linux.
Follow-up to 8491e6574c#17379Closes#17924
No instant benefit, but it brings all Apple jobs into the same workflow.
Then perhaps allows intergrating iOS jobs with macOS ones, and in
the meantime keep them more tightly aligned.
Closes#17912
- GHA/checksrc: make a step name more accurate.
- cmakelint.sh: make sure to run from project root.
To make it easier to run locally.
- cmakelint.sh: use `set -eu`.
Closes#17915
CI check used macOS before this patch, but with the help of Linuxbrew,
latest zizmor and shellcheck are also available on Linux.
Also:
- migrate CI checks to the misc check workflow, to make both shellcheck
use the same, latest, shellcheck version, and to save the overhead of
an extra workflow.
Closes#17911
`checksrc.bat` was outdated and required Perl for `checksrc.pl` anyway.
Rewrite `checksrc-all.sh` in Perl, making it usable in envs without
a POSIX shell.
Closes#17882
- circleci: pipe to tar.
- use long options uniformly.
- sync option order.
- set timeout where missing.
- set retry where missing.
- set `--retry-connrefused` where missing.
- set `--disable` where missing.
- lower 999s timeouts to 120s.
Closes#17851
Replace the `libstubgss.so`-based overload solution with one built into
libcurl at compile-time.
The previous, `LD_PRELOAD`-based, solution was non-portable, allowlisted
for Linux, BSD and Solaris. It also required non-debug builds, which
turned out to be an accidental condition:
7d342c723c. It also required a curl tool
built against a shared libcurl. Detecting this condition wasn't always
accurate, e.g. with certain cmake configurations.
The overload solution also didn't work on macOS, though it theoretically
should have:
- #17653
- #2394
Experiments on making the overload solution work in more envs:
- #17759
That revealed that it also did not work on NetBSD, in CI.
The replacement solution is overloading the necessary GSS-API functions
for test 2056 and 2057 at compile time. It requires a debug-enabled curl
build (due to its insecure nature).
This makes these tests run on all platforms. Including most GSS jobs in
CI, that are running tests. (the exception is old-linux, non-debug jobs,
where it felt overkill to enable debug for this.)
The refactored GSS stub code needs to overload less than before because
it's free to use the official GSS API. (This didn't work with
the overload solution on Alpine for example). It can also use libcurl
functions, allowing to replace `snprintf()` with `msnprintf()`.
OS/400 is also overloading GSS API functions. I haven't tested how this
works after this PR. In theory it should, because this PR doesn't rely
on preprocessor overrides.
Note that for future GSS tests, it may be necessary to stub these GSS
API functions: `gss_inquire_context()`, `gss_unwrap()`, `gss_wrap()`.
They are on codepaths not (yet) touched by tests.
Also:
- stub-gss: check for token buffer overrun.
- stub-gss: replace size macros with `sizeof()`.
- GHA: enable debug for some jobs with GSS.
- GHA/linux: ignore results for 2056 and 2057 in the valgrind job.
They leak the same way as seen with 2077 and 2078.
Ref: 7020ba7979#17462
Ref: 146759716c#14430
- GHA/linux: fix to ignore `gss_import_name()` leaks in valgrind builds.
only.
- lib/vauth/krb5_gssapi: reduce variable scope.
- lib/vauth/spnego_gssapi: reduce variable scope.
- tests/libtest: drop code and build logic dealing with `libstubgss`.
- runtests:
- drop `ld_preload` feature.
- drop special handling of `LD_PRELOAD` env in tests.
- drop logic dealing with shared curl tool detection.
- drop `LD_PRELOAD` envs from tests.
Follow-up to 56d949d31a#1687Closes#17752
Job `name:` now mandatory in zizmor.
Also:
- enclose `name:` values in single-quotes, for uniformity.
- drop `name: checkout` where set, for uniformity.
- dist: also install with cmake.
- dist: replace `make` with `cmake --build` for cmake.
(to make this make-tool agnostic)
- appveyor-status: double-quote shell arguments.
- tweak existing names to be shorter, to sync terms and style across
jobs and steps.
Ref: https://github.com/zizmorcore/zizmor/releases/tag/v1.10.0Closes#17773
`brotlicommon` is necessary when linking brotli statically.
E.g. on Alpine Linux with `PKG_CONFIG='pkg-config --static'`.
It also syncs brotli libs listed in `libcurl.pc` and `curl-config` with
those generated by cmake, which already listed `brotlicommon` there.
Also:
- remove workaround from GHA/configure-vs-cmake CI jobs.
Ref: 69cda1b704
Follow-up to f2adb3b6d7#15431
Reported-by: Christopher Boyd
Fixes#17678Closes#17723
Replace existing `mk-unity.pl` `--embed` workaround with running
`clang-tidy` manually on individual test source instead. This aligns
with how clang-tidy works and removes `mk-unity.pl` from the solution.
Also:
- mqttd: fix potentially uninitialized buffer by zero filling it.
```
tests/server/mqttd.c:484:41: error: The left operand of '<<' is a garbage value
[clang-analyzer-core.UndefinedBinaryOperatorResult,-warnings-as-errors]
484 | payload_len = (size_t)(buffer[10] << 8) | buffer[11];
| ^
[...]
tests/server/mqttd.c:606:45: error: The left operand of '<<' is a garbage value
[clang-analyzer-core.UndefinedBinaryOperatorResult,-warnings-as-errors]
606 | topiclen = (size_t)(buffer[1 + bytes] << 8) | buffer[2 + bytes];
| ^
```
- sockfilt: fix potential out-of-bound pointer:
```
tests/server/sockfilt.c:1128:33: error: The 2nd argument to 'send' is a buffer
with size 17010 but should be a buffer with size equal to or greater than
the value of the 3rd argument (which is 18446744073709551615)
[clang-analyzer-unix.StdCLibraryFunctions,-warnings-as-errors]
1128 | ssize_t bytes_written = swrite(sockfd, buffer, buffer_len);
| ^
```
- clang-tidy: suppress bogus `bzero()` warnings that happens
inside the notorious `FD_ZERO()` macros, on macOS.
Ref: https://github.com/curl/curl/pull/17680#issuecomment-2991730158Closes#17705
No longer necessary after bumping the default runtime to a version
fixing the previously experienced performance drop.
Thanks to MSYS2/Cygwin teams for the help and fix.
Follow-up to 9a26be1e6a#17708
Follow-up to d4896d94f2#16424Closes#17710
Used by test 2056 and 2057, in a way that's Linux- & autotools-specific.
This patch builds it for all Unix, syncing cmake with autotools.
Adapt the two tests to find the library in CMake builds as well.
Tested OK on Linux. (CI does not test this. The corresponding jobs build
in debug mode, while the `LD_PRELOAD` feature is locked to non-debug.)
On macOS it didn't load without building everything for aarch64e arch:
"../bld/tests/libtest/libstubgss.dylib' (mach-o file, but is
an incompatible architecture (have 'arm64', need 'arm64e'))"
With that fixed it still did not load correctly and/or the tests did not
pass. So, for macOS these tests remain disabled.
Also:
- GHA/macos: build for aarch64e. (recognized by Apple clang as of this
patch. llvm and gcc fall back to aarch64.)
Follow-up to 56d949d31a#1687Closes#17653
- GHA/windows: make a mingw autotools build static only.
- GHA/windows: fix a CI script issue with the build above.
- src: fix to pass `LIBCURL_PC_LIBS_PRIVATE` instead of `LINKFLAGS`.
This makes the libs propagate to tunits, making the local hack there
unnecessary. `LINKFLAGS` had this single use in the repo, and it was
empty in local tests.
- tests: drop passing redundant `LIBCURL_PC_LDFLAGS_PRIVATE`.
- tests: drop redundant target name from config variables.
- examples, tests/client: drop `LIBDIR` temp variables with single uses.
- examples, tests: formatting to sync `Makefile.am` scripts with each
other.
Closes#17661
