Commit Graph

200 Commits

Author SHA1 Message Date
Viktor Szakats
8d004781a5
build: drop the winbuild build system
In favor of CMake.

Closes #18040
2025-09-20 01:20:25 +02:00
renovate[bot]
c0e63bab6f
GHA: update actions/checkout action to v5
Closes https://github.com/curl/curl/issues/18250
2025-08-12 11:17:08 +02:00
renovate[bot]
4db91c4411
GHA: update actions/checkout digest to 08eba0b
Closes #18249
2025-08-11 14:59:35 +02:00
Viktor Szakats
fe5225b5ea
cmake: optimize building examples in CI
In CI we want to ensure that examples build cleanly, but we don't want
to actually run them there. Meaning it's enough to just compile, but not
link them in CI. Saving time up to 2-4x (MSVC), and disk space up
to 1.2GB (or 8-70x).

Add a new cmake target that compiles all examples without linking them
into runnable binaries. Keep a full build for a single example to test
if it links correctly.

Also:
- CI: switch over all `curl-examples` targets to `curl-examples-build`
- GHA/linux-old: build examples in one of the cmake builds.

Result highlights:

Job                 | Bef. |  Bef. | Aft. | Aft. |
:------------------ | ---: | ----: | ---: |----: |
cygwin              |  15s |   9MB |  10s |  1MB |
msys                |  13s |   8MB |   7s |  1MB |
dl-mingw 15         |  39s |  113M |  34s |  2MB |
dl-mingw 9.5.0      |  49s | 115MB |  42s |  2MB |
dl-mingw 7.3.0      |  19s | 113MB |  14s |  2MB |
dl-mingw 6.4.0      |   9s |  12MB |   7s |  4MB |
Linux cross         |  19s |  28MB |  19s |  2MB |
MSVC UWP            |  65s | 374MB |   9s | 17MB |
MSVC x64            |  22s | 846MB |   9s | 17MB |
VS2010              |  48s | 105MB |  15s |  9MB |
VS2022 clang-cl     | 195s | 1.2GB |  51s | 20MB |
iOS Xcode           |   8s |       |   5s |      |
macOS LibreSSL      |  16s |       |  11s |      |
Linux aws-lc        |   3s |       |   1s |      |

Follow-up to dda251ef10 #18232

Closes #18209
2025-08-09 02:27:43 +02:00
renovate[bot]
56584ed6cc
GHA: update actions/cache digest to 0400d5f
Closes #18222
2025-08-07 23:01:34 +02:00
Viktor Szakats
1644a49ab8
configure: if no perl, disable unity and shell completion, related tidy ups
Also:
- GHA/linux, macos: test `install` with autotools too.
- GHA/linux, macos: enable fish and zsh completion in an autotools job.
- scripts: drop dynamic perl checks redundant after this patch.
- scripts: drop two interim variables.
- autotools: make `build-certs` target check perl first.
- autotools: replace `$(PERL)` with `@PERL@` to match the pattern used
  in most automake scripts. For consistency. This makes `PERL` be
  defined at configure-time, as opposed to make-time, for these cases
  now.

Closes #18141
2025-08-04 14:55:10 +02:00
Viktor Szakats
1cc8a5235f
tests: fix UTF-8 detection, per-test LC_* settings, CI coverage
- runtests: fix `codeset-utf8` feature detection. Before this patch it
  detected if the calling environment had UTF-8 enabled. If not, UTF-8
  tests were all skipped. After this patch, it detects if UTF-8 is
  supported by the calling environment regardless of what's currently
  enabled.
  Follow-up to 0b70b23ef4 #15039

- GHA/linux: sync `codeset-test` to also reset `LC_CTYPE` and
  `LC_NUMBER`. To give it more spin.
  Follow-up to c221c0ee59 #17938

- GHA/macos: fix to actually enable `codeset-test`. Also set `LC_ALL`,
  which seems necessary to trigger issues.
  Follow-up to c221c0ee59 #17938

- tests/data: replace `LC_CTYPE` env with `LC_ALL` in all tests
  requiring a locale. Also to avoid potential issues with a blank or
  unset `LC_ALL`, as seen earlier. And to ensure that the override works
  on all platforms (as tested in CI.)
  Slight downside is that this now resets the language/culture to `C`.
  Ref: b4c9982382 #4743
  Ref: 23208e330a #4738

- replace `en_US.UTF-8` with `C.UTF-8` to be language/culture-agnostic.

- TEST-SUITE.md: drop `UTF-8` as a requirement for tests.
  Tests shall work (or least be skipped) without UTF-8 support.

Tests requiring UTF-8 locale:
165, 962, 963, 964, 965, 966, 967, 1448, 1560, 2046, 2047
Tests requiring UTF-8 locale, but passing without one anyway:
955, 956, 957, 958, 959, 960, 961, 968, 1034, 1035

Spec 1997: https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html
Spec 2008: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Ref: c221c0ee59 #17938
Ref: 7cf8414fab
Ref: 4c140a5628
Ref: 28faaacee2 #2436
Ref: ecd1d020ab

Closes #17988
2025-07-23 22:17:07 +02:00
Viktor Szakats
c221c0ee59
test1560: set locale/codeset with LC_ALL (was: LANG), test in CI
To fix running test 1560 when `LC_ALL` is set to something unexpected
(e.g. `C`). Also syncing it with the rest of tests.

Also:
- GHA/linux: enable `libidn2` in more jobs.
  Also to enable test 1560 reproducing this issue in more jobs.
- GHA/linux: run tests with `LC_ALL=C` in one of the jobs.
- GHA/linux: switch to the non-deprecated package name for libidn2.
- GHA/macos: run tests with non-default locale settings in one job.
- GHA/macos: enable AppleIDN in that job.

Ref: https://github.com/curl/curl/pull/17933#issuecomment-3074582840
Follow-up to f27262b179 #10196

Closes #17938
2025-07-21 17:27:23 +02:00
Viktor Szakats
bf9d9fe0bd
multissl: initialize when requesting a random number
To fix test 1308 in MultiSSL builds.

Failure was caused by the random number generator virtual function being
NULL, instead of pointing to the implementation in the runtime-selected
TLS backend. This could happen in MultiSSL builds when a functionality
was asking for a random number without triggering a VTLS function table
initialization first. Such functionality is MIME, or form data via MIME.

The reason CI did not catch it in an earlier MultiSSL GHA/windows job,
is that it was a debug-enabled one. In debug-enabled builds the test
runner was overriding the random number generator for all tests.

Fixed this by moving the override to the tests requiring it, via
1fcf22585f #17971, enabling debug builds
to catch this issue.

Enable MultiSSL in two CI jobs, to verify this patch.

Fixing:
```
test 1308...[formpost tests]

libtests returned 44, when expecting 0
 1308: exit FAILED
[...]
=== Start of file stderr1308
 URL: log/3/test-1308
 tests/libtest/lib1308.c:70 Assertion 'res == 0' FAILED: curl_formget returned error
 tests/libtest/lib1308.c:72 Assertion 'total_size == 518' FAILED: curl_formget got wrong size back
 tests/libtest/lib1308.c:88 Assertion 'res == 0' FAILED: curl_formget returned error
 tests/libtest/lib1308.c:89 Assertion 'total_size == 899' FAILED: curl_formget got wrong size back
```
Ref: https://github.com/curl/curl/actions/runs/16387693424/job/46309536359?pr=17963#step:16:2515

Bug: https://github.com/curl/curl/pull/17963#issuecomment-3092282057

Closes #17970
2025-07-20 12:03:56 +02:00
Viktor Szakats
f2d1d4747d
GHA/macos: drop redundant build option
Follow-up to 7c23e88d17 #17973
2025-07-20 12:03:53 +02:00
Viktor Szakats
7c23e88d17
GHA/macos: enable QUIC API with OpenSSL
In one autotools and one cmake job.

Also:
- enable OpenSSL QUIC in two more jobs.
- pytest: add checks to skip h3 tests when there is no h3 server to
  tests 26, 27a, 27b, 27c, 29 in test_02_download. Fixing:
  ```
  FAILED tests/http/test_02_download.py::TestDownload::test_02_26_session_shared_reuse[h3] - AssertionError: expected exit code 0, got 1
  [...]
  ```
  Ref: https://github.com/curl/curl/actions/runs/16392680316/job/46320739635?pr=17973

Ref: https://github.com/Homebrew/homebrew-core/pull/230515
Ref: 6c8e3eed12

Closes #17973
2025-07-20 00:02:19 +02:00
Viktor Szakats
319d27e568
GHA/macos: add Rustls, aws-lc jobs
Bind them to the (arbitrary choice of) Apple clang and gcc compilers,
respectively.

Also:
- bind existing mbedTLS job to the llvm compiler, to keep the number of
  jobs the same as before this patch.
- move OpenLDAP from mbedTLS over to LibreSSL to keep testing it with
  all 3 compilers.
- simplify exclusions for clang-tidy and torture jobs.
- tag clang-tidy and torture jobs via `install_steps`.
  To avoid keeping around special bool fields.

Closes #17963
2025-07-19 22:37:17 +02:00
Viktor Szakats
8675c9e3f3
GHA: move iOS jobs from non-native to macos workflow
No instant benefit, but it brings all Apple jobs into the same workflow.

Then perhaps allows intergrating iOS jobs with macOS ones, and in
the meantime keep them more tightly aligned.

Closes #17912
2025-07-14 14:01:46 +02:00
Viktor Szakats
7735cf8199
GHA/macos: adapt to macos-15 for upcoming macos-latest bump
macos-latest is soon to switch to macos-15. Make sure it keeps working.

Closes #17873
2025-07-09 13:52:18 +02:00
Dan Fandrich
ab667c73a4 CI: skip CI builds that just touch Dockerfile
This file sees regular updates but isn't used by any CI build.
2025-07-02 18:28:24 -07:00
Viktor Szakats
73840836a5
tests: move GSS-API dynamic stub into debug-mode libcurl
Replace the `libstubgss.so`-based overload solution with one built into
libcurl at compile-time.

The previous, `LD_PRELOAD`-based, solution was non-portable, allowlisted
for Linux, BSD and Solaris. It also required non-debug builds, which
turned out to be an accidental condition:
7d342c723c. It also required a curl tool
built against a shared libcurl. Detecting this condition wasn't always
accurate, e.g. with certain cmake configurations.

The overload solution also didn't work on macOS, though it theoretically
should have:
- #17653
- #2394

Experiments on making the overload solution work in more envs:
- #17759
  That revealed that it also did not work on NetBSD, in CI.

The replacement solution is overloading the necessary GSS-API functions
for test 2056 and 2057 at compile time. It requires a debug-enabled curl
build (due to its insecure nature).

This makes these tests run on all platforms. Including most GSS jobs in
CI, that are running tests. (the exception is old-linux, non-debug jobs,
where it felt overkill to enable debug for this.)

The refactored GSS stub code needs to overload less than before because
it's free to use the official GSS API. (This didn't work with
the overload solution on Alpine for example). It can also use libcurl
functions, allowing to replace `snprintf()` with `msnprintf()`.

OS/400 is also overloading GSS API functions. I haven't tested how this
works after this PR. In theory it should, because this PR doesn't rely
on preprocessor overrides.

Note that for future GSS tests, it may be necessary to stub these GSS
API functions: `gss_inquire_context()`, `gss_unwrap()`, `gss_wrap()`.
They are on codepaths not (yet) touched by tests.

Also:
- stub-gss: check for token buffer overrun.
- stub-gss: replace size macros with `sizeof()`.
- GHA: enable debug for some jobs with GSS.
- GHA/linux: ignore results for 2056 and 2057 in the valgrind job.
  They leak the same way as seen with 2077 and 2078.
  Ref: 7020ba7979 #17462
  Ref: 146759716c #14430
- GHA/linux: fix to ignore `gss_import_name()` leaks in valgrind builds.
  only.
- lib/vauth/krb5_gssapi: reduce variable scope.
- lib/vauth/spnego_gssapi: reduce variable scope.
- tests/libtest: drop code and build logic dealing with `libstubgss`.
- runtests:
  - drop `ld_preload` feature.
  - drop special handling of `LD_PRELOAD` env in tests.
  - drop logic dealing with shared curl tool detection.
  - drop `LD_PRELOAD` envs from tests.

Follow-up to 56d949d31a #1687

Closes #17752
2025-07-01 00:17:15 +02:00
Viktor Szakats
48d3407d7c
GHA: fix zizmor 1.10.0 warnings, update names
Job `name:` now mandatory in zizmor.

Also:
- enclose `name:` values in single-quotes, for uniformity.
- drop `name: checkout` where set, for uniformity.
- dist: also install with cmake.
- dist: replace `make` with `cmake --build` for cmake.
  (to make this make-tool agnostic)
- appveyor-status: double-quote shell arguments.
- tweak existing names to be shorter, to sync terms and style across
  jobs and steps.

Ref: https://github.com/zizmorcore/zizmor/releases/tag/v1.10.0

Closes #17773
2025-06-30 18:38:56 +02:00
Viktor Szakats
171b623759
cmake: build stubgss library for libtests to match autotools
Used by test 2056 and 2057, in a way that's Linux- & autotools-specific.
This patch builds it for all Unix, syncing cmake with autotools.

Adapt the two tests to find the library in CMake builds as well.

Tested OK on Linux. (CI does not test this. The corresponding jobs build
in debug mode, while the `LD_PRELOAD` feature is locked to non-debug.)

On macOS it didn't load without building everything for aarch64e arch:
"../bld/tests/libtest/libstubgss.dylib' (mach-o file, but is
an incompatible architecture (have 'arm64', need 'arm64e'))"
With that fixed it still did not load correctly and/or the tests did not
pass. So, for macOS these tests remain disabled.

Also:
- GHA/macos: build for aarch64e. (recognized by Apple clang as of this
  patch. llvm and gcc fall back to aarch64.)

Follow-up to 56d949d31a #1687

Closes #17653
2025-06-19 20:41:54 +02:00
Viktor Szakats
2c27a67daa
tests: always make bundles, adapt build and tests
Make test bundles the default. Drop non-bundle build mode.
Also do all the optimizations and tidy-ups this allows, simpler builds,
less bundle exceptions, streamlined build mechanics.

Also rework the init/deinit macro magic for unit tests. The new method
allows using unique init/deinit function names, and calling them with
arguments. This is in turn makes it possible to reduce the use of global
variables.

Note this drop existing build options `-DCURL_TEST_BUNDLES=` from cmake
and `--enable-test-bundles` / `--disable-test-bundles` from autotools.

Also:
- rename test entry functions to have unique names: `test_<testname>`
  This removes the last exception that was handled in the generator.
- fix `make dist` to not miss test sources with test bundles enabled.
- sync and merge `tests/mk-bundle.pl` into `scripts/mk-unity.pl`.
- mk-unity.pl: add `--embed` option and use it when `CURL_CLANG_TIDY=ON`
  to ensure that `clang-tidy` does not miss external test C sources.
  (because `clang-tidy` ignores code that's #included.)
- tests/unit: drop no-op setup/stop functions.
- tests: reduce symbol scopes, global macros, other fixes and tidy-ups.
- tool1621: fix to run, also fix it to pass.
- sockfilt: fix Windows compiler warning in certain unity include order,
  by explicitly including `warnless.h`.

Follow-up to 6897aeb105 #17468

Closes #17590
2025-06-14 21:08:23 +02:00
Viktor Szakats
3934431421
tidy-up: drop BearSSL and Secure Transport reference remains
Follow-up to 08a3e8e19a #16677

Closes #17582
2025-06-11 11:14:29 +02:00
Daniel Stenberg
08a3e8e19a
TLS: remove support for Secure Transport and BearSSL
These libraries do not support TLS 1.3 and have been marked for removal
for over a year. We want to help users select a TLS dependency that is
future-proof and reliable, and not supporting TLS 1.3 in 2025 does not
infer confidence. Users who build libcurl are likely to be served better
and get something more future-proof with a TLS library that supports
1.3.

Closes #16677
2025-06-11 07:54:19 +02:00
Viktor Szakats
128c5ddc6b
GHA: enclose if expressions in ${{ }} where missing
Enclosing expressions in `${{ }}` is optional, but sometimes required.
It seems more straighforward to use it always rather than decide on
a case-by-case basis. Before this patch 71 `if`s were enclosed and 49
were not. Enclosing these makes it easy to grep and recognize these
whereever used.

https://docs.github.com/actions/writing-workflows/choosing-when-your-workflow-runs/using-conditions-to-control-job-execution

Closes #17550
2025-06-08 13:01:24 +02:00
Viktor Szakats
17a669426f
CI: fix zizmor 1.9.0 warnings, shellcheck verify CI shell code, fix fallouts
zizmor 1.9.0 effectively bans using GHA macros within shell script
snippets. Rework them to pass GHA macro values via envs and use those
envs in shell snippets. `${{ env.* }}` macros could be converted
to native env references after making their names shell-compatible.

Envs and shell commands can no longer be used in GHA macro values. Most
cases could be fixed by using literals. Passing quoted values with
spaces combined with other args also doesn't work anymore. This was
replaced by passing them separately.

Despite the initial complications, avoiding GHA macros in scripts does
seems to make the CI code reasonable cleaner. It also makes it possible
to analyze the scripts with shellcheck, finding subtle issues that went
unnoticed till now.

Also:
- un-ignore and fix three existing zizmor `template-injection` issues.
- add script to extract and shellcheck all shell code stored within GHA
  and Circle CI YAML files.
- add CI job to run this script.
- fix shellcheck issues discovered.
- fix minor differences between cmake and autotools FreeBSD jobs.
- merge cmake/autotools FreeBSD jobs to avoid developing unwanted
  differences again.
- fix/sync quote use across shell code.
- replace `$HOME` with `~` or literal where it made sense.
- replace most `brew --prefix` with literals.
- move all curl install tests to the `curl-install*` prefix.
- add missing curl install tests to cygwin/msys/mingw/*bsd.
- pipe to `tar` instead of storing downloads on disk.
- drop unnecessary `PKG_CONFIG_PATH` when building nghttp3.

Ref: https://github.com/curl/curl/actions/runs/15461461371/job/43523546041
Ref: https://github.com/zizmorcore/zizmor/releases/tag/v1.9.0

Follow-up to e522f47986 #17278

Closes #17537
2025-06-06 21:30:43 +02:00
Viktor Szakats
7020ba7979
tests: re-enable 1510, unignore 2027 2051 in GHA/macos, document heimdal memleak
- test1510: it was marked flaky, then disabled. Keep the flaky flag
  and allow it to run again.
  Ref: ef02da3156
  Ref: 0110cced2a

- GHA/linux: extend information about the heimdal valgrind failures.

- GHA/macos: unignore 2037 2041 in torture tests. They no longer
  appear flaky.
  Follow-up to 4b791dca37 #14667

Closes #17462
2025-06-02 00:21:51 +02:00
Viktor Szakats
e785e898a6
checksrc: avoid extra runs in CI, enable more check locally, fix fallouts
To avoid redundant work in CI and to avoid a single checksrc issue make
all autotools jobs fail. After this patch checksrc issues make fail
the checksrc job, the `dist / verify-out-of-tree-autotools-debug`,
`dist / maketgz-and-verify-in-tree`  jobs and the fuzzer job (if run).
Of these, the `dist` jobs replicate local builds, also testing the build
logic.

Also add a script to check the complete local repository, optionally
with the build tree to verify generated C files.

Also:
- automatically run checksrc in subdirectories having a `checksrc`
  target. (examples, OS400, tests http/client, unit and tunit)
- tests/libtest: make sure to run `checksrc` on generated `lib1521.c`.
  (requires in-tree autotools build.)
- tests: run `checksrc` on targets also for non-`DEBUGBUILD`
  builds. It ensures to check `lib1521.c` in CI via job
  `dist / maketgz-and-verify-in-tree`.
- src: drop redundant `$(builddir)` in autotools builds.
- scripts: add `checksrc-all.sh` script to check all C sources and
  the build directory as an option.
- use the above from CI, also make it verify all generated sources.
- silence `checksrc` issues in generated C sources.
- checksrc: add `-v` option to enable verbose mode.
- checksrc: make verbose mode show checked filename and fix to only
  return error on failure.
- make sure that generated C files pass `checksrc`.

Assisted-by: Daniel Stenberg

Closes #17376
2025-05-19 18:59:50 +02:00
Viktor Szakats
42331cb48a
pytest: enable in 5 GHA jobs, fix fallouts
They take about 4.5 minutes of CI time in GHA/macos.

Also:
- autotools: improve `caddy`, `vsftpd` detection.
  Bringing it closer to cmake.
- autotools: fix `--with-test-caddy=no`, `--with-test-vsftps=no`,
  `--with-test-nghttpx=no` options.
- cmake: sync `nghttpx` default with autotools.
- pytest: disable failing mbedTLS tests on macOS CI.
- pytest: disable failing earlydata tests on macOS CI.
- GHA/macos: keep vsftpd pytests disabled due to lengthy run times.
- pytest: fix test_05_04 for LibreSSL. Ref: #17367
  Authored-by: Stefan Eissing

Remaining issues:
- some unidentified tests taking a long time with mbedTLS:
  `================= 462 passed, 278 skipped in 347.93s (0:05:47) =================`
  Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374999041#step:17:1536
  Workaround: not enabling pytest for mbedTLS jobs
- 17 FTP tests taking a long time (affecting all TLS backends):
  without vsftpd:
  `====================== 496 passed, 244 skipped in 56.15s =======================`
  Ref: https://github.com/curl/curl/actions/runs/15073354301/job/42374998230#step:17:1536
  with vsftpd:
  `================= 513 passed, 227 skipped in 409.37s (0:06:49) =================`
  Ref: https://github.com/curl/curl/actions/runs/15073678568/job/42376039672?pr=17362#step:17:1537
  Workaround: force-disable vsftpd.
- 100 tests failing with SecureTransport. Let's ignore that due to imminent deprecation.
  Ref: https://github.com/curl/curl/actions/runs/15055652333/job/42320873732#step:17:15362

Follow-up to 30ef79ed93 #17295
Follow-up to 9147903366 #16518

Closes #17362
2025-05-19 15:38:05 +02:00
Stefan Eissing
30ef79ed93
pytest-xdist: pytest in parallel
Require now pytest-xdist from tests/http/requirements.txt and
run pytest in 'auto' parallel mode (counts cpu cores).

For CI runs, set the worker count to 4, overriding the
core count of 2 exposed in the images.

- use Filelock to generate allocated ports at start for all
  workers and have subsequent workers just read the file and
  take the ports for their slot
- make httpd config clearing a function fixture so every test
  starts with a clean httpd config
- have fixture `configures_httpd` as parameter of test cases
  that configure httpd anyway, saving one reload
- add pytest-xdist and filelock to required pyhton modules
- add installs to ruff CI
- give live checks waiting for a server to start up longer time
- add fixtures to tests that rely on a server
- do not stop servers unnecessarily. failures may not start them
  properly again, leading to unexpected fails in whatever follows
- add a https: port to httpd that is *not* back by QUIC to allow
  failover tests without stopping the QUIC server

Closes #17295
2025-05-15 21:44:31 +02:00
Viktor Szakats
e522f47986
GHA/checksrc: check GHA rules with zizmor
The pedantic level is experimental. If it causes issues, we may just
disable it alongside the ignore comments.

Also:
- silence error:
  ```
   INFO audit: zizmor: completed label.yml
  error[dangerous-triggers]: use of fundamentally insecure workflow trigger
    --> label.yml:13:1
     |
  13 | 'on': [pull_request_target]
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ pull_request_target is almost always used insecurely
     |
     = note: audit confidence -> Medium
  ```
- fix pedantic warning:
  ```
   INFO audit: zizmor: completed label.yml
  warning[excessive-permissions]: overly broad permissions
    --> label.yml:1:1
  ...  |
  24 | |         with:
  25 | |           repo-token: '${{ secrets.GITHUB_TOKEN }}'
     | |____________________________________________________- default permissions used due to no permissions: block
     |
     = note: audit confidence -> Medium
  ```
- silence `template-injection` false positives like:
  ```
  - note: ${{ matrix.build.torture && 'test-torture' || 'test-ci' }} may expand into attacker-controllable code
  - note: ${{ contains(matrix.build.install_steps, 'pytest') && 'caddy httpd vsftpd' || '' }} may expand into attacker-controllable code
  ```
  It doesn't seem like these could be controlled by an attacker.
  Let me know if I'm missing something.

Closes #17278
2025-05-07 23:11:09 +02:00
Viktor Szakats
d58ff6e4b5
GHA/macos: fix typo in comment [ci skip] 2025-04-25 21:10:04 +02:00
Viktor Szakats
a366552243
GHA: always use Ninja on macos runners
It's installed by default on the macos runners now.

Closes #17115
2025-04-22 16:50:12 +02:00
Viktor Szakats
4646976f3b
GHA/macos: re-generate gcc-13 hacklayer on macos-15
Add workaround for an issue related to the gcc "hacklayer" after the
GitHub macos-15-arm64 runner bumped to 20250408.1231.

Fixes:
```
configure:5175: gcc-13 -o conftest  --sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk  -w conftest.c  >&5
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:71,
                 from /opt/homebrew/Cellar/gcc@13/13.3.0/lib/gcc/13/gcc/aarch64-apple-darwin24/13/include-fixed/stdio.h:75,
                 from conftest.c:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h: In function 'fmemopen':
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/_stdio.h:457:107: error: expected declaration specifiers before '__API_AVAILABLE_GET_MACRO_93585900'
  457 | FILE *fmemopen(void * __restrict __buf _LIBC_SIZE(__size), size_t __size, const char * __restrict __mode) __API_AVAILABLE(macos(10.13), ios(11.0), tvos(11.0), watchos(4.0));
      |                                                                                                           ^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/14378524390/job/40316589059?pr=17012#step:7:169

Assisted-by: Bo Anderson
Bug: https://github.com/curl/curl/pull/17012#issuecomment-2792572344
Bug: https://github.com/Homebrew/homebrew-core/issues/194778#issuecomment-2792601570

Closes #17017
2025-04-10 22:00:17 +02:00
Viktor Szakats
c8b0f0c9ad
cmake: add shell completion support
Add the last (*) missing bit for feature parity with autotools.

Also test in CI. Add a new `cmake install` step to GHA/macos.

(*) AFAIK. Let us know if there's something else missing.

Closes #16833
2025-03-28 13:51:38 +01:00
Viktor Szakats
44341e736a
runtests: generate certs dynamically, bump to EC-256, tidy up
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.

Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.

As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.

Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.

Details:

- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
  separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
  confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
  auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
  noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
  ```
  + openssl genrsa -out test-ca.key -passout fd:0 2048
  Invalid password argument, starting with "fd:"
  ```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
  (`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
  the Visual Studio generator miss to create the `clean-certs`,
  `build-certs` targets. No target depend on them, so they don't execute
  implicitly anyway. Fixes:
  ```
  MSBUILD : error MSB1009: Project file does not exist.
  Switch: clean-certs.vcxproj
  ```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.

Follow-up to 556f722fe3 #16593
Follow-up to fa461b4eff #14486

Closes #16824
2025-03-27 10:21:57 +01:00
Viktor Szakats
e7944fb3da
GHA: extend clang-tidy jobs with more build options, add Windows job
- linux: wolfssl, wolfssh (replacing libssh2), ech, kerberos/GSSAPI,
  ssls-export (libssh2 remains tested on macos.)

- macos: brotli, zstd, c-ares, gnutls, mbedtls, gsasl, rtmp, ssls-export

- windows: new job with schannel, sspi, winidn, winldap, ssls-export

- unit3205: fix/silence remaining NULL dereferences.

Commits fixing the issues found:
cbbccb8b3a #16766
554e4c14be #16777

Closes #16764
2025-03-24 12:37:43 +01:00
Viktor Szakats
77861ace49
GHA/macos: verbose cmake in test steps
To sync with other workflows, and autotools. To see build details,
including clang-tidy invocations.

Closes #16757
2025-03-18 04:13:50 +01:00
Viktor Szakats
d4f9788593
GHA: fix configure disable options
Linux AM openssl https-only:
```
configure: WARNING: unrecognized options: --disable-rtmp, --disable-scp, --disable-sftp
```
Ref: https://github.com/curl/curl/actions/runs/13823209634/job/38673119106#step:31:34

macOS AM clang !ssl HTTP-only:
```
configure: WARNING: unrecognized options: --disable-rtmp, --disable-scp, --disable-sftp, --without-ntlm-auth
```
Ref: https://github.com/curl/curl/actions/runs/13823209638/job/38673115560#step:7:54

Closes #16701
2025-03-13 11:37:30 +01:00
Viktor Szakats
51c9238c76
GHA: enable OpenSSL QUIC in a macOS and MinGW job
Closes #16665
2025-03-11 08:50:08 +01:00
Viktor Szakats
8b1b5cd4d2
DISABLED: add 313 for sectransp (move from GHA/macos)
Closes #16660
2025-03-10 20:58:06 +01:00
Viktor Szakats
9147903366
GHA/macos: add initial pytest support
Add support for running pytest in GHA/macos jobs.

Experimental, with caveats:
- slow.
- `httpd` often fails to start.
- 10-15 tests (depending on C compiler) fail consistently:
  02_20, 02_33, 02_34, 03_01, 03_03, 05_04, 07_42.
- Homebrew build of vsftpd misses TLS support.
- `nghttpx` temporarily disabled for pytest.

You can test pytest by adding `install_steps: pytest` to a job.

Closes #16518
2025-03-07 02:19:28 +01:00
Viktor Szakats
2d94439eaa
sectransp: add support for HTTP/2 in gcc builds
Before this patch `--http2` did not work in gcc builds with Secure
Transport, because ALPN relied on a compiler supporting the
`HAVE_BUILTIN_AVAILABLE` aka `__builtin_available()` feature. This
is clang-specific and missing from gcc (as of gcc v14).

Add support for ALPN and HTTP/2 when this compiler feature is missing.

Also drop test exceptions from GHA/macos in CI.

Follow-up to 092f6815c8
Ref: c349bd668c #14097 (issue 15.)
Ref: #4314

Closes #16581
2025-03-06 20:33:43 +01:00
Viktor Szakats
8fbd372e0d
tests: mark tests 1631, 1632 flaky
We already marked them flaky in GHA/macos CI. They are also flaky in
other CI jobs, in other OSes, with multiple TLS backends:
- MSVC/LibreSSL: https://github.com/curl/curl/actions/runs/13683996410/job/38262956317
- MSVC/wolfSSL: https://github.com/curl/curl/actions/runs/13680682695/job/38252047077
- FreeBSD/OpenSSL3: https://github.com/curl/curl/actions/runs/13690910863/job/38283867721#step:3:1

Ref: fa461b4eff #14486

Closes #16584
2025-03-06 13:17:15 +01:00
Viktor Szakats
cfbd46d8b3
GHA/macos: stop ignoring test 2100 with gcc
It runs fine now. Tested in all gcc-12 jobs after temporarly enabling
debug in them all (test 2100 requires debug-enabled).

Ref: c349bd668c #14097 (issue 15.)

Closes #16582
2025-03-06 13:17:14 +01:00
Viktor Szakats
c38f9089f3
lib1156: pass longs to curl_easy_setopt()
To fix this test on macOS with the gcc compiler.

Also unignore test 1156 in GHA/macos.

Ref: c349bd668c #14097 (issue 15.)
Ref: 7b0240c077 #16539
Ref: 2ec00372a1 #16482

Closes #16579
2025-03-06 01:35:30 +01:00
Viktor Szakats
fd54bed51a
curl.h: switch CURL_HTTP_VERSION* enums to long constants
It fixes tests 1539, and 2402, 2404 (for non-Secure Transport), on macOS
with the gcc compiler.

Also unignore these tests in GHA/macos for non-secure transport.

Ref: c349bd668c #14097 (issue 15.)
Ref: 7b0240c077 #16539
Ref: 2ec00372a1 #16482

Closes #16580
2025-03-06 01:01:24 +01:00
Viktor Szakats
05c0a6e270
GHA/macos: restore nghttp2 package for runtests HTTP/2 tests
Regression from e7751571eb #16517
2025-03-05 22:57:49 +01:00
Viktor Szakats
0e0db0e385
GHA: brew --no-lock an invalid option now, drop
```
Error: invalid option: --no-lock
```
Ref: https://github.com/curl/curl/actions/runs/13659344413/job/38186577572#step:2:655

Closes #16561
2025-03-04 19:33:50 +01:00
Viktor Szakats
e7751571eb
GHA/macos: use quictls in some jobs, other small improvements
- enable quictls in autotools and cmake jobs. autotools requires
  a workaround due to wrong libpath in the quictls pkg-config.
  nghttp3 is offered by Homebrew, but not ngtcp2, to enable H3.

- install `libnghttp2` rather than `nghttp2`.
  `libnghttp2` is preinstalled and smaller. It also avoids detecting
  `nghttpx`, which confuses `pytest`.

- limit `brew unlink openssl` to libressl/quictls jobs.

Closes #16517
2025-02-28 16:58:12 +01:00
Viktor Szakats
08a29e7f18
GHA: tidy up parallel options, improve performance for some jobs
- replace `--parallel <n>` and `-j<n>` for individual commands with
  `MAKEFLAGS`, for jobs not yet doing it.
  This enables parallel builds in distcheck / maketgz-and-verify-in-tree,
  where `-j` option was missing.
- add `--parallel` for iOS Xcode job for improved performance.
- drop redundant `-j5` for Android jobs.
- drop stray `cmake --config` options from single-target jobs (cygwin,
  msys/mingw, dl-mingw, non-native). Drop redundant
  `CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` settings too.
- GHA/windows: add timeout for package install steps where missing.
- GHA/non-native: specify target type explicitly for iOS cmake jobs.
  Xcode default was already Debug, single-target default was generic,
  now it's Release, with unity batch to keep it fast.

`MAKEFLAGS` is necessary for autotools jobs and CMake jobs using
the default (GNU Make) generator. It's ignored by Ninja and other tools.
`cmake --parallel` is still necessary for jobs with Visual Studio or
Xcode generators. Parallelism is 5 for GHA Linux and Windows runners,
4 for macOS, 3 for VMs, 2 for AppVeyor.

Closes #16502
2025-02-27 12:11:07 +01:00
Daniel Stenberg
71f190cbbc
GHA: enable the RTSP tests on macOS gcc-12 2025-02-26 07:58:54 +01:00
Viktor Szakats
67559aa8ce
CI: dump non-pre-fill configure log on pre-fill check fail
To help debugging builds where the actual feature check is broken.

Follow-up to e7adf3e837 #15841
Closes #16369
2025-02-17 16:59:56 +01:00