To allow building test binaries without test certs, replacing
`_CURL_SKIP_BUILD_CERTS` internal option with a build target that is
similar to `testdeps`, but without building the test certificates.
To make building test binaries a little bit faster, with less noisy
output, and without having to reconfigure the build.
Closes#20708
Instead of globally disabling unity for all targets when clang-tidy is
enabled.
After this patch `CMAKE_UNITY_BUILD=ON` is honored for:
- static libcurl when building both static and shared separately.
- libcurlu and libcurltool internal libraries when building the test
target.
While keeping unity disabled for the libcurl build pass running
clang-tidy, and the curl tool, also running clang-tidy.
To make clang-tidy-enabled builds finish faster when unity mode is
enabled, yet performs the same clang-tidy checks as before this patch.
Effect on:
- GHA/macos: core build: same, buils tests 5-12 seconds faster,
with steps going down from 259 to 25.
52s: https://github.com/curl/curl/actions/runs/22279958340/job/64448913325 ->
47s: https://github.com/curl/curl/actions/runs/22279873606/job/64448710743
- GHA/windows (not enabled): it'd save about 1 minute, bringing total
time barely below 10m, still one of the slowest jobs overall.
(#20667 is trying a way for 4x speed-up (with a drawback)).
5m21s: https://github.com/curl/curl/actions/runs/22222907068/job/64284556852 ->
4m26s: https://github.com/curl/curl/actions/runs/22281033369/job/64451601548Closes#20670
To simplify setting BoringSSL version, using:
`-DBORINGSSL_VERSION=0.20260211.0`
or
`-DBORINGSSL_VERSION=${boringssl_version}`
Previously it could be set via C flags, using complicated shell quotes:
`-DCMAKE_C_FLAGS="-DCURL_BORINGSSL_VERSION=\\\"${boringssl_version}\\\""`
(the C flags method remains, also for autotools)
It'd be nice if BoringSSL published its version not just via
`MODULE.bazel` in its source tree, but from its public headers, to make
these workarounds unnecessary.
Also:
- GHA/http3-linux: test both options.
Closes#20571
Via options:
- `BROTLI_USE_STATIC_LIBS`
- `CARES_USE_STATIC_LIBS`
- `LIBSSH_USE_STATIC_LIBS`
- `LIBSSH2_USE_STATIC_LIBS`
- `MBEDTLS_USE_STATIC_LIBS`
- `NGHTTP2_USE_STATIC_LIBS`
- `NGHTTP3_USE_STATIC_LIBS`
- `NGTCP2_USE_STATIC_LIBS`
- `ZSTD_USE_STATIC_LIBS`
When enabled, make a "best effort" finding static libs first and set
the "build static" macro (on Windows) as required by the dependency.
When doing `pkg-config`-based detections, make curl select the static
configuration, which shall set the "build static" macro also.
These options resemble CMake's `OPENSSL_USE_STATIC_LIBS` and
`ZLIB_USE_STATIC_LIBS` (the latter does not support `pkg-config` as of
CMake v4.2.2).
Shared/static library selection based on loose filename conventions is
fragile and prone to break if the non-static-suffixed library is found
and happens to be a shared library, or, if the linker decides to pick up
a shared copy (e.g. `.a.dll`) that shadows the static one. It may help
to provide either static or shared, but not both, on the disk, and match
that with this setting.
Experimental.
Ref: #20013Closes#20015
To build all targets in a single go, meaning curl and libcurl as normal,
and tests and examples in addition. To build-test everything without
running multiple cmake commands.
Enable with:
- `-DCURL_BUILD_EVERYTHING=ON`
Special values: `QUICK` to build examples quickly for build test,
`NOEXAMPLES` to not build examples.
A well-equipped build takes 2.8s of configure time, and 1.7s to build
everything (shared, unity, ninja, prefill), 1.4s with `QUICK`. Without
this option it takes <1s to build curl/libcurl.
Also: streamline `CURL_LINT` internal logic.
Closes#20429
To enable known linker options dropping unused, dead, code and data from
the executables built.
Useful to reduce binary sizes for curl, libcurl shared lib and apps
linking static libcurl. It's effective on both "unity" and non-unity
builds. Aligning "unity" build sizes with default, non-unity ones.
Supported platforms: Apple, MSVC, llvm/clang and GCC on all tested
platforms: Linux, BSDs, Windows, MSYS2/Cygwin, Android, MS-DOS.
Notes:
- Static libraries grow 20-30% with non-Apple toolchains.
This effect is controlled by separate, optional compiler flags on
non-Apple. This patch enables them automatically for public binaries
(libcurl and curl tool), and leaves them off for internal/test ones.
- MSVC enables this option by default for 'Release' configurations.
The curl build option has no effect on it.
- Observed effect on VS2010 is negligible. VS2012+ is recommended.
- Works with LTO, Fil-C.
- No observed/conclusive effect on build speed.
- On Windows with clang/gcc (mingw-w64/MSYS2/Cygwin) it also enables
`-fno-asynchronous-unwind-tables` as a workaround to make
the toolchain options actually work.
Ref: https://sourceware.org/bugzilla/show_bug.cgi?id=11539
Thanks-to: Andarwinux
Also:
- GHA: enable in Linux and MinGW jobs to test it. Size changes:
- linux aws-lc H3:
curl: 2000000 -> 1937152, libcurl.a: 2065724 -> 2716532 bytes
- macos clang HTTP-only:
curl: 1364376 -> 128799 bytes, libcurl.a: unchanged
- macos llvm MultiSSL:
curl: 410056 -> 405720, libcurl.dylib: 1350336 -> 1348480 bytes
- mingw schannel c-ares U:
curl: 1588736 -> 1507328, libcurl-d.a: 3322040 -> 3884746 bytes
bld: 34 -> 35MB
- GHA: enable in MSVC and Apple jobs to reduce disk footprint, with no
obvious downside. Size changes:
- AppVeyor CI VS2019:
curl: 2339840 -> 1295872, libcurl-d.dll: 3155968 -> 1900544 bytes
bld: 161 -> 97MB
- AppVeyor CI VS2022 clang-cl:
curl: 2933248 -> 2332160, libcurl-d.lib: 4762688 -> 5511330 bytes
bld: 133 -> 121MB
- AppVeyor CI VS2022 HTTP-only:
curl: 3514368 -> 2177024, libcurl-d.lib: 2538420 -> 3151740 bytes
bld: 137 -> 83MB
- GHA intel:
curl: 2629120 -> 2023424, libcurl-d.lib: 4366652 -> 5350670 bytes
bld: 86 -> 69MB
- GHA arm64:
curl: 2832896 -> 2063872, libcurl-d.lib: 4690616 -> 5597250 bytes
bld: 82 -> 66MB
Refs:
https://maskray.me/blog/2021-02-28-linker-garbage-collectionhttps://web.archive.org/web/20110811230637/msdn.microsoft.com/en-us/library/bxwfs976.aspx (VS2010)
https://learn.microsoft.com/cpp/build/reference/opt-optimizationshttps://learn.microsoft.com/cpp/build/reference/gy-enable-function-level-linkingCloses#20357
Drop separate `TrackMemory` (aka `CURLDEBUG`) debug feature.
After recent changes (thread-safety,
193cb00ce9, and updates leading up to
it), `TrackMemory` is unlikely to cause build or runtime issues.
To simplify builds and debug options, enable `TrackMemory`
unconditionally for debug-enabled (aka `DEBUGBUILD`) builds. Before
this patch, this was already the default, with an option to disable
it, or enable it in non-debug-enabled builds.
Note, in practice these two debug options already went hand in hand. It
was not possible to toggle them separately for a long time due to bugs,
before 59dc9f7e69 (2024-05-28) fixed it.
This patch also removes/deprecates separate knobs and feature flags for
`TrackMemory`:
- autotools: `--enable-curldebug`/`--disable-curldebug`
- cmake: `-DENABLE_CURLDEBUG=ON`/`OFF`
- C macro: `CURLDEBUG`
- libcurl: `CURL_VERSION_CURLDEBUG` symbol deprecated in favor
of `CURL_VERSION_DEBUG`. They always return the same value after this
patch.
Also:
- drop `TrackMemory` from `curl -V` output.
- rename internal `CURLDEBUG` macro to `CURL_MEMDEBUG` internally.
To avoid confusion with `DEBUGBUILD`, but to keep guarding
`TrackMemory`-related internals for readability.
- runtests: bind `TrackMemory` to debug feature. Keep it a separate
test feature requirement, for clarity.
- CI: drop test builds for combinations of the two options.
- GHA/linux: no longer disable TrackMemory in the TSAN job.
Ref: https://github.com/curl/curl/pull/20328#issuecomment-3754528407Closes#20331
- It is slower and uses more memory than the alternatives and is only
experimental in curl.
- We disable a few tests for OpenSSL-QUIC because of flakiness
- It gets little attention from OpenSSL and we have no expectation of the
major flaws getting corrected anytime soon.
- No one has spoken up for keeping it
- curl users building with vanilla OpenSSL can still use QUIC through the
means of ngtcp2
Closes#20226
With the same semantics as Apple SecTrust, in both libcurl and the curl
tool, when using non-Schannel TLS backends. In practice it means that
it makes TLS work without manually or implicitly configuring a CA bundle
`.crt` file, such as `curl-ca-bundle.crt`.
To enable:
- autotools: `--enable-ca-native`
- cmake: `-DCURL_CA_NATIVE=ON`
- CPPFLAGS: `-DCURL_CA_NATIVE`
When enabled:
- enables `CURLSSLOPT_NATIVE_CA` (libcurl) / `--ca-native`
and `--proxy-ca-native` (curl tool) options by default.
- unsafe search for an on-disk CA bundle gets disabled by default.
Equivalent to `--disable-ca-search` with autotools,
`-DCURL_DISABLE_CA_SEARCH=ON` with CMake.
- build-time detection of CA bundle and CA path gets disabled. As with
Apple SecTrust. This was already the default for Windows.
- native CA can be disabled at run-time with the `--no-ca-native`
and/or `--no-proxy-ca-native` command-line options.
Rationale: This build option:
- has a repeat and active interest from packagers and users.
- helps integrating curl with Windows for those who need this.
- it also applies to macOS: #17525
Shipped in curl 8.17.0.
- makes it trivial to use custom certs configured on the OS.
- frees applications/packagers/users from the task of securely
distributing, and keeping up-to-date, a CA bundle.
- frees potentially many curl tool from configuring a CA bundle manually
to access HTTPS (and other TLS) URLs. This is traditionally difficult
on Windows because there is no concept of a universal, protected,
non-world-writable, location on the file system to securely store
a CA bundle.
- allows using modern features regardless of Windows version. Some of
these features are not supported with Schannel (e.g. HTTP/3, ECH) on
any Windows version.
- is necessary for HTTP/3 builds, where bootstrapping a CA bundle is not
possible with Schannel, because MultiSSL is not an option, and HTTP/3
is not supported with Schannel.
Ref: #16181 (previous attempt)
Ref: https://github.com/curl/curl/discussions/9348
Ref: https://github.com/curl/curl/issues/9350
Ref: https://github.com/curl/curl/pull/13111
Ref: https://github.com/microsoft/vcpkg/pull/46459#issuecomment-3162068701
Ref: 22652a5a4c#14582
Ref: eefd03c572#18703Closes#18279
To simplify the directory layout.
- OS400 and vms support move from `packages` to `projects`.
- Windows README and `generate.bat` files move from `projects`
to `projects/Windows`.
Closes#20271
To run checksrc and spacecheck on the source tree. Also for cmake
to sync up with autotools' `checksrc` target.
- cmake: `curl-lint`
With `-DCURL_LINT=ON`, checks run automatically for all targets.
- autotools: `lint`
Closes#20175
- apply more clang-format.
- lib/version: use `CURL_ARRAYSIZE()`.
- INSTALL-CMAKE.md: sync-up an option description with others.
- examples: delete unused main args.
- examples/ftpgetinfo: document `_CRT_SECURE_NO_WARNINGS` symbol.
- delete remaining stray duplicate lines.
- acinclude.m4: drop an unnecessary x-hack.
- vtls/mbedtls: join a URL split into two lines.
- src/tool_cb_see: add parentheses around macro expressions.
- src/tool_operate: move literals to the right side of comparisons.
- libtests: sync up fopen/fstat error messages between tests.
- curl_setup.h: replace `if ! defined __LP64` with `ifndef __LP64`.
I assume it makes no difference on Tandem systems, as the latter form
is already used in `include/curl/system.h`.
Closes#20018
- cmake: automatically disable typecheck when running clang-tidy,
to avoid possible interference, and to improve performance.
- INSTALL-CMAKE: document both this, and unity=off for clang-tidy.
- GHA/linux: disable for some static analyzers CI jobs to avoid possible
interference.
- GHA/linux: disable in Fil-C job to improve build performance.
Follow-up to 9e6f1c5efb#19637
Follow-up to fd2ca2399e#17955Closes#19941
With either /usr/sbin/sshd found or configured via --with-test-sshd=path
add tests for SCP down- and uploads, insecure, with known hosts or not,
with authorized user key or unauthorized one.
Working now with libssh and libssh2, using a hashed known_hosts file.
Closes#19934
Also:
- memdebug: fix symbol collision in unity builds.
- memdebug: fix compiler warning by making a variable static.
Follow-up to c77bed81a2#19657Closes#19666
Usage:
- autotools: `--disable-typecheck` (or `--enable-typecheck` (default))
- cmake: `-DCURL_DISABLE_TYPECHECK=ON`.
To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.
Other than these cases, we recommend leaving type checking enabled.
Ref: fdacf34aae#19632
Also:
- GHA/codeql: use it.
- test1165: check in `include/curl`.
- lib1912: delete stray todo comment.
- spelling and comment nits.
Closes#19637
If CMake reports the target platform not supporting shared libs, turn
`BUILD_SHARED_LIBS` off by default. CMake 3.30+ fails with an error
when trying to create a `SHARED` target for such platforms. Earlier
versions used a workaround that may or may not have worked in practice.
Ref: https://cmake.org/cmake/help/v3.30/policy/CMP0164.html
Seen this with a build setting `-DCMAKE_SYSTEM_NAME=Generic`, e.g.
AmigaOS.
Note this may introduce incompatibility for "Generic" targets, which
support shared libs. If that's the case, set `BUILD_SHARED_LIBS=ON`
manually.
Also drop AmigaOS-specific logic handled automatically after this patch.
Ref: https://cmake.org/cmake/help/v3.7/command/get_property.html
Ref: https://cmake.org/cmake/help/v3.7/prop_gbl/TARGET_SUPPORTS_SHARED_LIBS.htmlCloses#19420
Make autotools and cmake detect DES support in OpenSSL and mbedTLS.
Forward feature macros to C and omit NTLM from the feature preview list.
Use the feature macros in source. This ensure that `-V` output matches
the preview.
OpenSSL doesn't support DES when built with `no-des` or `no-deprecated`.
mbedTLS 4.x no longer supports it, and it's possible to disable it in
<4 with `scripts/config.py unset MBEDTLS_DES_C`.
Before this patch this worked for
mbedTLS 4 only, and with a regression for pending PR #16973.
Also:
- drop NTLM feature check from `curl_setup.h` in favour of autotools/
cmake feature macros. This makes `curl_setup.h` no longer need
to include an mbedTLS header, which in turn makes tests/server build
without depending on mbedTLS.
Fixing, in #16973:
```
In file included from tests/server/first.h:40,
from bld/tests/server/servers.c:3:
lib/curl_setup.h:741:10: fatal error: mbedtls/version.h: No such file or directory
741 | #include <mbedtls/version.h>
| ^~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/18689537893/job/53291322012?pr=16973
Ref: #19181 (initial fix idea)
Follow-up to 3a305831d1#19077
- move back mbedTLS header include and version check from
`curl_setup.h` to each source which consumes mbedTLS.
- GHA/http3-linux: drop workaround that disabled NTLM for
`no-deprecated` OpenSSL builds.
Follow-up to 006977859d#12384
- curl_ntlm_core: drop pointless macro `CURL_NTLM_NOT_SUPPORTED`.
Follow-up to 006977859d#12384Closes#19206
To not have to guess. Also to sync with autotools, which already uses
this wording.
Also:
- replace the stray term 'folder' with 'directory' for consistency.
- store help text in a temp variable to avoid overly long strings
(mandatory in CMake <4.2.0 and can't be trivially split), also
to avoid repeating this string 4 times.
Ref: https://cmake.org/cmake/help/v4.2/command/set.htmlCloses#19169
Replacing a combination of custom logic in the main script and relying
on CMake's built-in Find module, with code and behavior used for
the rest of dependencies.
Also to:
- add version detection in the non-pkg-config path.
- make `GNUTLS_INCLUDE_DIR` and `GNUTLS_LIBRARY` take precedence over
pkg-config. As with other dependencies.
- document the above two configuration options.
- prepare for #16973, which originally introduced this local Find
module.
The local module is doing largely the same as CMake's built-in
FindGnuTLS. Differences:
- honors `CURL_USE_PKGCONFIG`.
- returns GnuTLS version for non-pkg-config detection.
- consistently returns `GNUTLS_VERSION`.
(CMake's built-in uses s different name in <3.16.)
- CMake 3.16+ returns an imported target. curl supports 3.7,
therefore we may only use it conditionally, which isn't worth it.
Cherry-picked from #16973Closes#19163
curl requires 1.2.4 or newer.
Also:
- vms: stop defining `gss_nt_service_name`. Added in
f9cf3de70b, symbol not used in curl code
since 355bf01c82.
Closes#18978
The kerberos5 library Heimdal is one of three GSS libraries curl support.
It has a memory leak triggered by the new test in #18917 and the project
seems mostly abandoned.
Drop support and steer users to the MIT krb5 or GNU GSS libraries.
Co-authored-by: Daniel Stenberg
Ref: #18928Closes#18928Closes#18932
- configure/cmake support for enabling the option
- supported in OpenSSL and GnuTLS backends
- when configured, Apple SecTrust is the default trust store
for peer verification. When one of the CURLOPT_* for adding
certificates is used, that default does not apply.
- add documentation of build options and SSL use
Closes#18703
The implementation was incomplete and lesser than the other backends. No
one ever reported a bug or requested enhancements for this, indicating
that this backend was never used.
Closes#18700
- add startup check for 'danted' to avoid fails on low cpu
- rename 'sockd' to 'danted' everywhere to clarify what we use
- add proper defaults for 'danted' for debian
- install 'dante-server' in pytest ci runs
Closes#18075
Configure curl with `--with-test-sockd=<path to sockd>` for a locally
installed dante sockd server and new `test_40_*` will verify that
down- and uploads work via SOCKS.
Invoke scorecard.py with `--socks4` or `--socks5` to run performance
tests with SOCKS. Note that SOCKS is not supported for HTTP/3.
Ref: #17969Closes#17986
I added it just in case when removing enabled-by-default memdebug
from test servers. Apparently it broke after recent changes. It's
probably not a widely used feature and does not seem to be worth fixing.
It creates odd dependencies as the error message indicates:
```
[28/54] Building C object tests/server/CMakeFiles/servers.dir/__/__/lib/memdebug.c.obj
FAILED: tests/server/CMakeFiles/servers.dir/__/__/lib/memdebug.c.obj
[...]
lib/memdebug.c: In function 'curl_dbg_log':
lib/memdebug.c:465:12: error: implicit declaration of function 'mvsnprintf'; did you mean 'vsnprintf'? [-Wimplicit-function-declaration]
465 | nchars = mvsnprintf(buf, sizeof(buf), format, ap);
| ^~~~~~~~~~
| vsnprintf
lib/memdebug.c:465:12: warning: nested extern declaration of 'mvsnprintf' [-Wnested-externs]
```
This patch is dropping these build options:
- cmake: `ENABLE_SERVER_DEBUG`
- autotools: `--enable-server-debug` / `--disable-server-debug`
Follow-up to a16485a42e#16705Closes#17629
Make test bundles the default. Drop non-bundle build mode.
Also do all the optimizations and tidy-ups this allows, simpler builds,
less bundle exceptions, streamlined build mechanics.
Also rework the init/deinit macro magic for unit tests. The new method
allows using unique init/deinit function names, and calling them with
arguments. This is in turn makes it possible to reduce the use of global
variables.
Note this drop existing build options `-DCURL_TEST_BUNDLES=` from cmake
and `--enable-test-bundles` / `--disable-test-bundles` from autotools.
Also:
- rename test entry functions to have unique names: `test_<testname>`
This removes the last exception that was handled in the generator.
- fix `make dist` to not miss test sources with test bundles enabled.
- sync and merge `tests/mk-bundle.pl` into `scripts/mk-unity.pl`.
- mk-unity.pl: add `--embed` option and use it when `CURL_CLANG_TIDY=ON`
to ensure that `clang-tidy` does not miss external test C sources.
(because `clang-tidy` ignores code that's #included.)
- tests/unit: drop no-op setup/stop functions.
- tests: reduce symbol scopes, global macros, other fixes and tidy-ups.
- tool1621: fix to run, also fix it to pass.
- sockfilt: fix Windows compiler warning in certain unity include order,
by explicitly including `warnless.h`.
Follow-up to 6897aeb105#17468Closes#17590
