Try to make the wording more clear. It is the addresses in the resolver
result that are affected, not anything regarding *how* resolving is
done.
Closes#20585
Simplify the language expaining the --variable option, reducing
repetition. Also fix some minor grammar issues and makes language for
examples more consistent.
Closes#20636
Fix bigger and smaller kinks in how clang-tidy is configured and used.
Sync behavior more between autotools and cmake, lib/src and tests. Bump
clang-tidy minimum version and prepare logic to allow using clang-tidy
to a fuller extent.
- move clang-tidy settings from builds to a new `.clang-tidy.yml`.
To make it easy to see and edit checks at one place. Also to allow
using the `--checks=` option internally to silence tests-specific
checks. (clang-tidy does not support multiple `--check=` options via
the command-line.)
Use explicit `--config-file=` option to point to the configuration.
- .clang-tidy.yml: link to documentation.
- suppress `clang-diagnostic-nullability-extension` due to a false
positive in libtests with `CURL_WERROR=ON` and `PICKY_COMPILER=OFF`.
- .clang-tidy.yml: enable `portability-*`, `misc-const-correctness`.
- drop `--quiet` clang-tidy option by default to make its working a bit
more transparent. The extra output is minimial.
- consistently use double-dashes in clang-tidy command-line options.
Supported by clang-tidy 9.0.0+ (2019-09-19). Before this patch single
and double were used arbitrarily.
- src/tool_parsecfg: silence false positive `clang-analyzer-unix.Stream`.
Seen with clang 18 + clang-tidy 19 and 20 (only with autotools.)
- INTERNALS: require clang-tidy 14.0.0+. For the `--config-file` option.
- INTERNALS: recommend clang-tidy 19.1.0+, to avoid bogus
`clang-analyzer-valist.Uninitialized` warnings. (bug details below)
autotools:
- allow configuring the clang-tidy tool via `CLANG_TIDY` env.
Also to use in GHA to point to a suffixed clang-tody tool.
- fix to pass CFLAGS to lib, src sources.
(keep omitting them when using a non-clang compiler.)
- fix to pass `--warnings-as-errors=*` in quotes to avoid globbing.
cmake:
- fix to not pass an empty `-I` to clang-tidy.
- fix to pass CFLAGS (picky warnings) to clang-tidy for test sources.
(keep omitting them when using a non-clang compiler.)
- fix to disable `clang-diagnostic-unused-function` for test sources.
(tests have static entry points, which trigger this check when
checking them as individidual sources.)
- fix forwarding `CURL_CLANG_TIDYFLAGS` to clang-tidy.
- force disable picky warnings when running clang-tidy with a non-clang
compiler. To not pass these flags when checking lib and src.
CI:
- GHA/linux: avoid clang-tidy bug by upgrading to v19, and drop the
workaround.
- GHA/linux: switch to clang from gcc in the clang-tidy job. Using gcc
doesn't allow passing CFLAGS to clang-tidy, making it less effective.
(My guess this was one factor contributing to this job often missing
to find certain issues compared to GHA/macos.)
I recomment using clang-tidy with a clang compiler, preferably the same
version or one that's compatible. Other cases are best effort, and may
fail if a C flag is passed to clang-tidy that it does not understand.
Picky warnings are mostly omitted when using a non-clang compiler,
reducing its usefulness.
Details and reproducer for the v18 (and earlier) clang-tidy bug,
previously affecting the GHA/linux job:
clang-tidy <=18 emits false warnings way when passing multiple C sources
at once (as done with autotools):
```sh
cat > src1.c <<EOF
#include <string.h>
static void dummy(void *p) { memcmp(p, p, 0); }
EOF
cat > src2.c <<EOF
#include <stdarg.h>
void vafunc(int option, ...)
{
va_list param;
va_start(param, option);
if(option)
(void)va_arg(param, int);
va_end(param);
}
EOF
/opt/homebrew/opt/llvm@18/bin/clang-tidy --checks=clang-analyzer-valist.Uninitialized src1.c src2.c
# src2.c:7:11: warning: va_arg() is called on an uninitialized va_list [clang-analyzer-valist.Uninitialized]
```
Follow-up to e86542038d#17047Closes#20605
- fix internal macro `AN_APPLE_OS` reused between sources without
resetting it. It may potentially have left the system sha256
function unused.
- fix to define `WOLFSSL_OPTIONS_IGNORE_SYS` so that it always applies
to wolfSSL headers, also during feature detection.
- md4, md5, sha256: simplify fallback logic.
- delete 20+ unused macros.
- scope or move macros to avoid `-Wunused-macros` warnings.
- examples: delete unused code.
The warning detects macros defined but not used within the same C
source. It does not warn for macros defined in headers. It also works
with unity builds, but to a lesser extent.
Closes#20593
Any failf() that fill the errorbuf need to be forgotten once happy
eyeballing finds a succssful winner. Because the errorbuf, once set, is
not overwritten with future error information.
Adds test_05_05 to verify.
Reported-by: Tim Friedrich Brüggemann
Fixes#20608Closes#20613
Replacing API calls deprecated by OpenSSL 3, and also missing
from OpenSSL 3 no-deprecated builds, fixing builds with the latter:
`PEM_read_bio_RSAPrivateKey()`, `RSA_free()`,
`SSL_CTX_use_RSAPrivateKey()`
Also: rename callback to match its `cacertinmem.c` sibling.
Fixes#20595Closes#20596
To simplify setting BoringSSL version, using:
`-DBORINGSSL_VERSION=0.20260211.0`
or
`-DBORINGSSL_VERSION=${boringssl_version}`
Previously it could be set via C flags, using complicated shell quotes:
`-DCMAKE_C_FLAGS="-DCURL_BORINGSSL_VERSION=\\\"${boringssl_version}\\\""`
(the C flags method remains, also for autotools)
It'd be nice if BoringSSL published its version not just via
`MODULE.bazel` in its source tree, but from its public headers, to make
these workarounds unnecessary.
Also:
- GHA/http3-linux: test both options.
Closes#20571
To avoid redefining the `fstat` system symbol, and to clarify
`struct_stat` is a curl symbol.
- introduce `curlx_fstat()` macro and use it.
- rename `struct_stat` to `curl_struct_stat`.
Also:
- tests: replace direct `curlx_win32_stat()` call with `curlx_stat()`.
- checksrc: disallow direct `_fstati64` and `fstat()` calls, except in
examples.
Closes#20496
- stop redefining system symbol `lseek`, by introducing `curl_lseek()`.
- handle AmigaOS quirk within the macro mapping.
- add missing parenthesis to `LSEEK_ERROR` values.
- tool_util: use curl `lseek` macros in `tool_ftruncate64()`.
- move `LSEEK_ERROR` to right-hand side of if expressions.
- checksrc: disallow direct uses of `_lseeki64`, `llseek`, `lseek`.
Closes#20488
... between curl_multi_perform and curl_multi_wait/poll documentation
The `curl_multi_perform` documentation uses integer contextual
conversion to `bool` to test the resulting `CURLMcode`, while other
functions like `curl_multi_wait` and `curl_multi_poll` test against
`CURLM_OK`. (I was initially confused by this as it looked like some
docs call curl_multi_wait/poll on error, while some called only on
success. But that was my misread, not a docs problem.)
Also fixed the example to print which function call failed; previously
an error reported by `curl_multi_perform` was printed as a failure of
`curl_multi_wait`.
Closes#20444
The -J / --remote-header-name logic now records the file name part used
in the redirects so that it can use the last one as a name if no
Content-Disposition header arrives.
Add tests to verify:
1641: -J with a redirect and extract the CD contents in the second
response
1642: -J with a redirect but no Content-Disposition, use the name from
the Location: header
1643: -J with two redirects, using the last file name and also use
queries and fragments to verify them stripped off
Closes#20430
Via options:
- `BROTLI_USE_STATIC_LIBS`
- `CARES_USE_STATIC_LIBS`
- `LIBSSH_USE_STATIC_LIBS`
- `LIBSSH2_USE_STATIC_LIBS`
- `MBEDTLS_USE_STATIC_LIBS`
- `NGHTTP2_USE_STATIC_LIBS`
- `NGHTTP3_USE_STATIC_LIBS`
- `NGTCP2_USE_STATIC_LIBS`
- `ZSTD_USE_STATIC_LIBS`
When enabled, make a "best effort" finding static libs first and set
the "build static" macro (on Windows) as required by the dependency.
When doing `pkg-config`-based detections, make curl select the static
configuration, which shall set the "build static" macro also.
These options resemble CMake's `OPENSSL_USE_STATIC_LIBS` and
`ZLIB_USE_STATIC_LIBS` (the latter does not support `pkg-config` as of
CMake v4.2.2).
Shared/static library selection based on loose filename conventions is
fragile and prone to break if the non-static-suffixed library is found
and happens to be a shared library, or, if the linker decides to pick up
a shared copy (e.g. `.a.dll`) that shadows the static one. It may help
to provide either static or shared, but not both, on the disk, and match
that with this setting.
Experimental.
Ref: #20013Closes#20015
To build all targets in a single go, meaning curl and libcurl as normal,
and tests and examples in addition. To build-test everything without
running multiple cmake commands.
Enable with:
- `-DCURL_BUILD_EVERYTHING=ON`
Special values: `QUICK` to build examples quickly for build test,
`NOEXAMPLES` to not build examples.
A well-equipped build takes 2.8s of configure time, and 1.7s to build
everything (shared, unity, ninja, prefill), 1.4s with `QUICK`. Without
this option it takes <1s to build curl/libcurl.
Also: streamline `CURL_LINT` internal logic.
Closes#20429
- drop leading indent from Markdown.
- switch to Markdown section markers where missing.
- move `&&` and `||` to the end of the line (C, Perl).
- openssl: add parenthesis to an if sub-expression.
- misc clang-format nits.
- unfold Markdown links.
- SSL-PROBLEMS.md: drop stray half code-fence.
Closes#20402
To enable known linker options dropping unused, dead, code and data from
the executables built.
Useful to reduce binary sizes for curl, libcurl shared lib and apps
linking static libcurl. It's effective on both "unity" and non-unity
builds. Aligning "unity" build sizes with default, non-unity ones.
Supported platforms: Apple, MSVC, llvm/clang and GCC on all tested
platforms: Linux, BSDs, Windows, MSYS2/Cygwin, Android, MS-DOS.
Notes:
- Static libraries grow 20-30% with non-Apple toolchains.
This effect is controlled by separate, optional compiler flags on
non-Apple. This patch enables them automatically for public binaries
(libcurl and curl tool), and leaves them off for internal/test ones.
- MSVC enables this option by default for 'Release' configurations.
The curl build option has no effect on it.
- Observed effect on VS2010 is negligible. VS2012+ is recommended.
- Works with LTO, Fil-C.
- No observed/conclusive effect on build speed.
- On Windows with clang/gcc (mingw-w64/MSYS2/Cygwin) it also enables
`-fno-asynchronous-unwind-tables` as a workaround to make
the toolchain options actually work.
Ref: https://sourceware.org/bugzilla/show_bug.cgi?id=11539
Thanks-to: Andarwinux
Also:
- GHA: enable in Linux and MinGW jobs to test it. Size changes:
- linux aws-lc H3:
curl: 2000000 -> 1937152, libcurl.a: 2065724 -> 2716532 bytes
- macos clang HTTP-only:
curl: 1364376 -> 128799 bytes, libcurl.a: unchanged
- macos llvm MultiSSL:
curl: 410056 -> 405720, libcurl.dylib: 1350336 -> 1348480 bytes
- mingw schannel c-ares U:
curl: 1588736 -> 1507328, libcurl-d.a: 3322040 -> 3884746 bytes
bld: 34 -> 35MB
- GHA: enable in MSVC and Apple jobs to reduce disk footprint, with no
obvious downside. Size changes:
- AppVeyor CI VS2019:
curl: 2339840 -> 1295872, libcurl-d.dll: 3155968 -> 1900544 bytes
bld: 161 -> 97MB
- AppVeyor CI VS2022 clang-cl:
curl: 2933248 -> 2332160, libcurl-d.lib: 4762688 -> 5511330 bytes
bld: 133 -> 121MB
- AppVeyor CI VS2022 HTTP-only:
curl: 3514368 -> 2177024, libcurl-d.lib: 2538420 -> 3151740 bytes
bld: 137 -> 83MB
- GHA intel:
curl: 2629120 -> 2023424, libcurl-d.lib: 4366652 -> 5350670 bytes
bld: 86 -> 69MB
- GHA arm64:
curl: 2832896 -> 2063872, libcurl-d.lib: 4690616 -> 5597250 bytes
bld: 82 -> 66MB
Refs:
https://maskray.me/blog/2021-02-28-linker-garbage-collectionhttps://web.archive.org/web/20110811230637/msdn.microsoft.com/en-us/library/bxwfs976.aspx (VS2010)
https://learn.microsoft.com/cpp/build/reference/opt-optimizationshttps://learn.microsoft.com/cpp/build/reference/gy-enable-function-level-linkingCloses#20357
Use non-deprecated CRT function variants on Windows.
- introduce `curlx_fdopen()`, `curlx_close()` and use them. Map them to
non-deprecated, underscored, CRT functions on Windows.
- replace `close()` uses with either `sclose()` (for sockets) or
`curlx_close()` (for files).
- map `fileno`, `unlink`, `isatty` to their non-deprecated, underscored,
versions on Windows.
- tool_dirhie: map `mkdir` to `_mkdir` on Windows.
- easy: use `_strdup()` on Windows, regardless of how `HAVE_STRDUP` is
set.
- cmake: assume `HAVE_STRDUP` on Windows. To allow dropping a detection
hack using `_CRT_NONSTDC_NO_DEPRECATE` with MSVC. Windows always has
`_strdup()` which the code uses, but also needs `HAVE_STRDUP` defined
to disable curl's own `strdup()` implementation.
- curl_setup.h: drop `_CRT_NONSTDC_NO_DEPRECATE` as no longer necessary.
Closes#20212
New define USE_SO_NOSIGPIPE in curl_setup.h, for now set whenever
SO_NOSIGPIPE is defined. Maybe overridden in the future on systems where
this does not work.
With USE_SO_NOSIGPIPE defined, set SO_NOSIGPIPE on all sockets created
by libcurl and fail the creation when setsockopt() fails.
Closes#20370
Reduce mentions of libcurl versions like "since 7.xx ..." in option
descriptions to reduce clutter and make the texts easier to read. Keep
them in, or move them to, the HISTORY or DEPRECATED sections
The last version 7 release (7.88.1) shipped on Februrary 20, 2023.
Closes#20369
Drop separate `TrackMemory` (aka `CURLDEBUG`) debug feature.
After recent changes (thread-safety,
193cb00ce9, and updates leading up to
it), `TrackMemory` is unlikely to cause build or runtime issues.
To simplify builds and debug options, enable `TrackMemory`
unconditionally for debug-enabled (aka `DEBUGBUILD`) builds. Before
this patch, this was already the default, with an option to disable
it, or enable it in non-debug-enabled builds.
Note, in practice these two debug options already went hand in hand. It
was not possible to toggle them separately for a long time due to bugs,
before 59dc9f7e69 (2024-05-28) fixed it.
This patch also removes/deprecates separate knobs and feature flags for
`TrackMemory`:
- autotools: `--enable-curldebug`/`--disable-curldebug`
- cmake: `-DENABLE_CURLDEBUG=ON`/`OFF`
- C macro: `CURLDEBUG`
- libcurl: `CURL_VERSION_CURLDEBUG` symbol deprecated in favor
of `CURL_VERSION_DEBUG`. They always return the same value after this
patch.
Also:
- drop `TrackMemory` from `curl -V` output.
- rename internal `CURLDEBUG` macro to `CURL_MEMDEBUG` internally.
To avoid confusion with `DEBUGBUILD`, but to keep guarding
`TrackMemory`-related internals for readability.
- runtests: bind `TrackMemory` to debug feature. Keep it a separate
test feature requirement, for clarity.
- CI: drop test builds for combinations of the two options.
- GHA/linux: no longer disable TrackMemory in the TSAN job.
Ref: https://github.com/curl/curl/pull/20328#issuecomment-3754528407Closes#20331
For reducing binary size. Also to remove (or greatly mitigate)
the side-effect of using "unity" builds. Similar to `-Wl,--gc-sections`
on non-Apple platforms.
For example with curl-for-win builds, macOS arm+intel:
curl (unity): 7.7MB -> 6.8MB
libcurl.dylib (unity): 7.2MB -> 6.4MB
trurl /w static libcurl (!unity): 535KB -> 251KB (same size with unity)
Ref: c4008d658aCloses#20346
- It is slower and uses more memory than the alternatives and is only
experimental in curl.
- We disable a few tests for OpenSSL-QUIC because of flakiness
- It gets little attention from OpenSSL and we have no expectation of the
major flaws getting corrected anytime soon.
- No one has spoken up for keeping it
- curl users building with vanilla OpenSSL can still use QUIC through the
means of ngtcp2
Closes#20226
With the same semantics as Apple SecTrust, in both libcurl and the curl
tool, when using non-Schannel TLS backends. In practice it means that
it makes TLS work without manually or implicitly configuring a CA bundle
`.crt` file, such as `curl-ca-bundle.crt`.
To enable:
- autotools: `--enable-ca-native`
- cmake: `-DCURL_CA_NATIVE=ON`
- CPPFLAGS: `-DCURL_CA_NATIVE`
When enabled:
- enables `CURLSSLOPT_NATIVE_CA` (libcurl) / `--ca-native`
and `--proxy-ca-native` (curl tool) options by default.
- unsafe search for an on-disk CA bundle gets disabled by default.
Equivalent to `--disable-ca-search` with autotools,
`-DCURL_DISABLE_CA_SEARCH=ON` with CMake.
- build-time detection of CA bundle and CA path gets disabled. As with
Apple SecTrust. This was already the default for Windows.
- native CA can be disabled at run-time with the `--no-ca-native`
and/or `--no-proxy-ca-native` command-line options.
Rationale: This build option:
- has a repeat and active interest from packagers and users.
- helps integrating curl with Windows for those who need this.
- it also applies to macOS: #17525
Shipped in curl 8.17.0.
- makes it trivial to use custom certs configured on the OS.
- frees applications/packagers/users from the task of securely
distributing, and keeping up-to-date, a CA bundle.
- frees potentially many curl tool from configuring a CA bundle manually
to access HTTPS (and other TLS) URLs. This is traditionally difficult
on Windows because there is no concept of a universal, protected,
non-world-writable, location on the file system to securely store
a CA bundle.
- allows using modern features regardless of Windows version. Some of
these features are not supported with Schannel (e.g. HTTP/3, ECH) on
any Windows version.
- is necessary for HTTP/3 builds, where bootstrapping a CA bundle is not
possible with Schannel, because MultiSSL is not an option, and HTTP/3
is not supported with Schannel.
Ref: #16181 (previous attempt)
Ref: https://github.com/curl/curl/discussions/9348
Ref: https://github.com/curl/curl/issues/9350
Ref: https://github.com/curl/curl/pull/13111
Ref: https://github.com/microsoft/vcpkg/pull/46459#issuecomment-3162068701
Ref: 22652a5a4c#14582
Ref: eefd03c572#18703Closes#18279
After this patch curl requires targeting Vista or newer, and a toolchain
with Vista support.
Supported MSVC compilers (VS2010+) all support Vista:
- VS2012+ target Win8 (or later) by default.
- VS2010 targets Win7 by default.
Supported mingw-w64 versions (v3+) all support Vista:
- mingw-w64 v9+ target Win10 by default.
- mingw-w64 v8 and older target Server 2003 (~XP) by default.
After this patch it may be necessary to override the default Windows
target version to Vista (or newer) via:
autotools: `CPPFLAGS=-D_WIN32_WINNT=0x0600`
cmake: `-DCURL_TARGET_WINDOWS_VERSION=0x0600`
- mingw-w64 v6+ allow changing the default at toolchain build-time.
Notes:
- For non-MSVC, non-mingw-w64 toolchains, `if_nametoindex` needs to be
allowlisted in `curl_setup.h`, if they do support it.
Fixes#17985 (discussion)
Closes#18009
- reorder functions to not need forward declarations.
- sync `ephiperfifo.c` and `evhiperfifo.c`.
- drop redundant casts for `calloc()` return value.
- ephiperfifo: silence unused variable warning.
- fix indent and apply clang-format more.
Closes#20296
To simplify the directory layout.
- OS400 and vms support move from `packages` to `projects`.
- Windows README and `generate.bat` files move from `projects`
to `projects/Windows`.
Closes#20271
To run checksrc and spacecheck on the source tree. Also for cmake
to sync up with autotools' `checksrc` target.
- cmake: `curl-lint`
With `-DCURL_LINT=ON`, checks run automatically for all targets.
- autotools: `lint`
Closes#20175
- asyn-thrdd.c: scope an include.
- apply more clang-format suggestions.
- tidy-up PP guard comments.
- delete empty line from the top of headers.
- add empty line after `curl_setup.h` include where missing.
- fix indent.
- CODE_STYLE.md: add `strcpy`.
Follow-up to 8636ad55df#20088
- lib1901.c: drop unnecessary line.
Follow-up to 436e67f65b#20076Closes#20070
Also:
- examples/hsts-preload: apply the same change as it's based on lib1915
in tests. Make a local clone of `curlx_strcopy()`. Then drop the
`_CRT_SECURE_NO_WARNINGS` hack, that's no longer necessary.
- curl_setup.h: delete `strcpy()` from the `_CRT_SECURE_NO_WARNINGS`
list.
Closes#20076
1. With `MBEDTLS_SSL_PROTO_TLS1_2` not enabled, the mbedTLS code was not
able to connect to any server due to broken logic in curl's
`mbed_set_ssl_version_min_max()`. Now it correctly sets the minimum
supported TLS version based on what is compiled in the library.
2. If debugging is enabled, move the debugging enabling earlier in the
`mbed_connect_step1()` so that verbose errors are actually displayed if
failures happen (see the previous point -- it would've made debugging
that issue easier).
3. Remove the constant `mbedtls_x509_crt_profile_fr` and instead use
mbedTLS-included profile `mbedtls_x509_crt_profile_next` with
`mbedtls_ssl_conf_cert_profile()`. This will follow the latest standards
as new mbedTLS versions are released (rather than being stuck-in-time
until someone comes along to fix what was hard-coded here). This has the
immediate benefit of no longer supporting SHA1 certs and insecure RSA
key-lengths (1024). This fix immediately prevents previously possible
MITM attacks (SHA1 hashes and RSA-1024 keys can be forged relatively
easily by nation-state actors and criminal organizations with
deep-pockets).
4. Added [predictive
resistance](https://mbed-tls.readthedocs.io/en/latest/kb/how-to/add-a-random-generator/#enabling-prediction-resistance)
to the random number generator (adding more entropy to the RNG).
5. Split the random number generator into initialization, the actual
random generation, and the "freeing" of the resources. This
significantly reduces the overhead of using the RNG.
6. Removed the separate RNG function in the TLS connect stage (instead
use the "main" one) and remove the ad-hoc threading support. Instead
properly document how to enable threading in mbedTLS. As it was, other
internals of mbedTLS could have race conditions (in the RSA module in
particular) if `MBEDTLS_THREADING_C` was *not* enabled. And if it is
enabled, then these race-conditions cannot happen. And also, if
MBEDTLS_THREADING_C is enabled then the RNG functions [are fully
thread-safe](https://mbed-tls.readthedocs.io/en/latest/kb/development/thread-safety-and-multi-threading/).
So, the previous ad-hoc threading support was both partial and broken.
7. Enable support for disabling `MBEDTLS_PEM_PARSE_C`.
8. Add support for `CURLOPT_SSLCERTTYPE` so user can specify `PEM` or
`DER` and get faster execution.
Closes#19983
Move existing `Curl_rename()` `rename()` wrapper from lib to
curlx/fopen, and make it a curlx macro/function. To allow using
the local worker function to fixup long filenames on Windows.
Then fix the Windows-specific rename implementation to support long
filenames. This operation may happen when using a cookie jar, HSTS cache
or alt-svc cache, via libcurl or the curl tool.
Before this patch, when passing a long filename to the above options,
a `<random>.tmp` file was left on the disk without renaming it to the
filename passed to curl. There was also 1 second delay for each
attempted rename operation.
Also:
- checksrc: ban raw `rename()` and `MoveFileEx*()` functions.
- Note: `Curl_rename()` returned 1 on failure before this patch, while
`curlx_rename()` returns -1 after, to match POSIX `rename()`.
Refs:
https://learn.microsoft.com/windows/win32/api/winbase/nf-winbase-movefileexahttps://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation
Ref: #20040Closes#20042
It makes them work in Schannel's CA bundle loader, and curl tool's
set/get file timestamp operations (e.g. in `-R`/`--remote-time`). Also
to match file open operations, that already support long filenames.
E.g. when using `--remote-time`, fixing:
```
Warning: Failed to set filetime 1741363917 on outfile: CreateFile failed:
Warning: GetLastError 0x00000003
```
The special long filename logic is necessary to support Windows releases
prior to Windows 10 v1607. With the latter, it's possible to opt-in to
this behavior via a manifest setting. Note that Windows itself also needs
to opt-in to support this. Finally note that curl itself needs passing
`--globoff` to let long filenames through, pending #20044 and #20046.
Refs:
https://learn.microsoft.com/windows/win32/api/fileapi/nf-fileapi-createfileahttps://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation
Ref: #8361
Inspired by: #19286
Inspired-by: Mathesh V
Closes#19286Closes#20040
Also:
- vquic-tls.h: do not include unused headers for non-H3 builds.
- autotools: stop looking for `openssl/x509.h` header.
- cmp-config.pl: delete exception for `openssl/x509.h`.
- examples: format/comment sync between the two touched files.
- openssl: drop unused `curlx/wait.h` include.
Closes#20049
Always use curlx_now() when calling Curl_pgrs_now(data). Tests with the
"manual" updates to now proved differ more then 100ms in parallel testing.
Add `curlx_nowp()` to set current time into a struct curltime.
Add `curlx_ptimediff_ms() and friends, passing pointers.
Update documentation.
Closes#19998
- apply more clang-format.
- lib/version: use `CURL_ARRAYSIZE()`.
- INSTALL-CMAKE.md: sync-up an option description with others.
- examples: delete unused main args.
- examples/ftpgetinfo: document `_CRT_SECURE_NO_WARNINGS` symbol.
- delete remaining stray duplicate lines.
- acinclude.m4: drop an unnecessary x-hack.
- vtls/mbedtls: join a URL split into two lines.
- src/tool_cb_see: add parentheses around macro expressions.
- src/tool_operate: move literals to the right side of comparisons.
- libtests: sync up fopen/fstat error messages between tests.
- curl_setup.h: replace `if ! defined __LP64` with `ifndef __LP64`.
I assume it makes no difference on Tandem systems, as the latter form
is already used in `include/curl/system.h`.
Closes#20018
After applying a recent fix made to `threaded.c` (formerly
`multithread.c`) to `threaded-ssl.c`, syncing and updating comments,
the two examples turned out to be identical except their test URLs.
Delete one of them to avoid duplication.
Also:
- examples/threaded: scope a variable.
- examples/threaded: merge comments from its deleted sibling.
Follow-up to 61273f5812#20001
Follow-up to 971e8d661c#19526#19524Closes#20002
Use `data->progress.now` as the timestamp of proecssing a transfer.
Update it on significant events and refrain from calling `curlx_now()`
in many places.
The problem this addresses is
a) calling curlx_now() has costs, depending on platform. Calling it
every time results in 25% increase `./runtest` duration on macOS.
b) we used to pass a `struct curltime *` around to save on calls, but
when some method directly use `curx_now()` and some use the passed
pointer, the transfer experienes non-linear time. This results in
timeline checks to report events in the wrong order.
By keeping a timestamp in the easy handle and updating it there, no
longer invoking `curlx_now()` in the "lower" methods, the transfer
can observer a steady clock progression.
Add documentation in docs/internals/TIME-KEEPING.md
Reported-by: Viktor Szakats
Fixes#19935Closes#19961
To really verify the presence of the XML prolog, also in CI.
- move the prolog check from `loadtest` to `checktest`.
(load did a soft error, silently skipping the test instead of failing)
- runtests: enable `-w` functionality permanently for all test targets,
drop the option. It has no measurable performance impact.
- test 798, 1665: add XML prolog.
Follow-up to f0d277cb0e
Follow-up to b5ea0736bb#19946
Follow-up to 904e7ecb66#19347Closes#19970
- cmake: automatically disable typecheck when running clang-tidy,
to avoid possible interference, and to improve performance.
- INSTALL-CMAKE: document both this, and unity=off for clang-tidy.
- GHA/linux: disable for some static analyzers CI jobs to avoid possible
interference.
- GHA/linux: disable in Fil-C job to improve build performance.
Follow-up to 9e6f1c5efb#19637
Follow-up to fd2ca2399e#17955Closes#19941
To detect mistakes made in the runtests framework that reduce
the number of test runs. Before this patch it could go undetected with
a green CI.
The minimum thresholds will need light maintenance going forward (either
bumping them periodically, or adjust if some may fell below minimums for
justified reasons). We may also make minimums tighter or looser, or more
job-specific.
Latest number of test runs for each job can be seen at Test Clutch:
https://testclutch.curl.se/static/reports/feature-matrix.html
Also:
- GHA: set minimums.
Assisted-by: Dan Fandrich
Follow-up to f2a75a14dd
Follow-up to bb1391f943#19510Closes#19942
With either /usr/sbin/sshd found or configured via --with-test-sshd=path
add tests for SCP down- and uploads, insecure, with known hosts or not,
with authorized user key or unauthorized one.
Working now with libssh and libssh2, using a hashed known_hosts file.
Closes#19934
- drop stray duplicate empty lines in docs, scripts, test data, include,
examples, tests.
- drop duplicate PP parenthesis.
- curl-functions.m4: move literals to the right side in if expressions,
to match rest of the source code.
- FAQ.md: delete language designator from an URL.
- packages: apply clang-format (OS400, VMS).
- scripts/schemetable.c: apply clang-format.
- data320: delete duplicate empty line that doesn't change the outcome.
- spacecheck: extend to check for duplicate empty lines
(with exceptions.)
- fix whitespace nits
Closes#19936
CMake 3.18 was released on 2020-07-15.
It enables using (and/or dropping workarounds) for these features:
LTO support, better performance and pkg-config support, `OBJECT` target,
`-S`, `-B`, `--verbose`, `--install` on the command-line, lib directory
support in interface targets, target_link_options(), LINK_OPTIONS,
FetchContent, `list(PREPEND ...)`, unity, Ninja, fixed imported global
issues.
Ref: https://github.com/curl/curl/discussions/18704Closes#19902
- TCP Fast Open support on Windows
TFO doesn't really work on the internet and isn't really used anywhere.
We use QUIC now.
- get rid of PATH_MAX
Not a priority and when using 3rd party libraries not really up to us.
- auto-detect proxy
A dream we can just stop having.
- config file parsing
Let's not do that in our library.
- add asynch getaddrinfo support
Let's not add a limited glibc specific backend with bad API
- FTP HOST
If we managed this far without name based vhost FTP, let's not.
Closes#19890
To make the test files XML-compliant, and the expected results
possibly easier to manage by keeping them in `.c` files.
Non-XML-compliant files are down to 36 after this patch.
Also:
- make all macro expansions apply to `%includetext` contents.
Closes#19799
- to avoid dupes.
- missing slashes.
- drop `.git` suffix from GitHub git repo URLs for a few outliers.
- use short YouTube URL like curl-www does.
- sync two RFC doc URLs with others.
Closes#19879
- drop redundant parentheses from macro definitions.
- apply clang-format in some places missed earlier.
- wolfssl: fix a macro guard comment.
- curl_setup.h: drop empty lines
- FAQ: fix C formatting.
Closes#19854
The example code does not use curl_multi_assign(), but its callback
function used socketp (called sockp in the function) to get the struct
priv pointer instead of the correct clientp (cbp).
Reported-by: Greg Hudson
Fixes#19840Closes#19841
To make it clearer to readers of the code that the resulting dup also
has a null terminator. Something a "normal" memdup() does not provide.
Closes#19833
Function Curl_bufref_ptr() now returns a const char *.
New function Curl_bufref_uptr() returns a const unsigned char *.
Usage and doc updated.
Closes#19827
To allow replacing `&` characters in `tests/data/test*` files for
XML-compliance.
Also:
- document `%GT`, `%LT`
Follow-up to de49cc89ab#19470Closes#19824
A fix for the tests that took the longest:
- test_05: make the server close the HTTP/1.1 connection when
simulating an error during a download. This eliminates waiting
for a keepalive timeout
- test_02: pause tests with slightly smaller documents, eliminate
special setup for HTTP/2. We test stream window handling now
elsewhere already
- cli_hx_download: run look in 500ms steps instead of 1sec, resuming
paused tranfers earlier.
Closes#19809
Remove connection member `waitfor` and keep it in the SSH connection
meta. Add `ssh` to supported tracing features, convert many DEBUGF
printgs to traces.
Closes#19745
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Make `port` member in these struct of type `uint16_t`.
add `uint8_t transport` to `struct ip_quadruple
Define TRNSPRT_NONE as 0. By assigning a valid transport only on a
successful connection, it is clear when the ip_quadruple members are
valid. Also, for transports not involving ports, the getinfos for
`CURLINFO_PRIMARY_PORT` and `CURLINFO_LOCAL_PORT` will now always return
-1.
Make all `transport` members and parameters of type `uint8_t`.
Document the return value of `CURLINFO_LOCAL_PORT` and
`CURLINFO_PRIMARY_PORT` in this regard. Add tests that writeout stats
report ports correctly.
Closes#19708
