- It is slower and uses more memory than the alternatives and is only
experimental in curl.
- We disable a few tests for OpenSSL-QUIC because of flakiness
- It gets little attention from OpenSSL and we have no expectation of the
major flaws getting corrected anytime soon.
- No one has spoken up for keeping it
- curl users building with vanilla OpenSSL can still use QUIC through the
means of ngtcp2
Closes#20226
With the same semantics as Apple SecTrust, in both libcurl and the curl
tool, when using non-Schannel TLS backends. In practice it means that
it makes TLS work without manually or implicitly configuring a CA bundle
`.crt` file, such as `curl-ca-bundle.crt`.
To enable:
- autotools: `--enable-ca-native`
- cmake: `-DCURL_CA_NATIVE=ON`
- CPPFLAGS: `-DCURL_CA_NATIVE`
When enabled:
- enables `CURLSSLOPT_NATIVE_CA` (libcurl) / `--ca-native`
and `--proxy-ca-native` (curl tool) options by default.
- unsafe search for an on-disk CA bundle gets disabled by default.
Equivalent to `--disable-ca-search` with autotools,
`-DCURL_DISABLE_CA_SEARCH=ON` with CMake.
- build-time detection of CA bundle and CA path gets disabled. As with
Apple SecTrust. This was already the default for Windows.
- native CA can be disabled at run-time with the `--no-ca-native`
and/or `--no-proxy-ca-native` command-line options.
Rationale: This build option:
- has a repeat and active interest from packagers and users.
- helps integrating curl with Windows for those who need this.
- it also applies to macOS: #17525
Shipped in curl 8.17.0.
- makes it trivial to use custom certs configured on the OS.
- frees applications/packagers/users from the task of securely
distributing, and keeping up-to-date, a CA bundle.
- frees potentially many curl tool from configuring a CA bundle manually
to access HTTPS (and other TLS) URLs. This is traditionally difficult
on Windows because there is no concept of a universal, protected,
non-world-writable, location on the file system to securely store
a CA bundle.
- allows using modern features regardless of Windows version. Some of
these features are not supported with Schannel (e.g. HTTP/3, ECH) on
any Windows version.
- is necessary for HTTP/3 builds, where bootstrapping a CA bundle is not
possible with Schannel, because MultiSSL is not an option, and HTTP/3
is not supported with Schannel.
Ref: #16181 (previous attempt)
Ref: https://github.com/curl/curl/discussions/9348
Ref: https://github.com/curl/curl/issues/9350
Ref: https://github.com/curl/curl/pull/13111
Ref: https://github.com/microsoft/vcpkg/pull/46459#issuecomment-3162068701
Ref: 22652a5a4c#14582
Ref: eefd03c572#18703Closes#18279
After this patch curl requires targeting Vista or newer, and a toolchain
with Vista support.
Supported MSVC compilers (VS2010+) all support Vista:
- VS2012+ target Win8 (or later) by default.
- VS2010 targets Win7 by default.
Supported mingw-w64 versions (v3+) all support Vista:
- mingw-w64 v9+ target Win10 by default.
- mingw-w64 v8 and older target Server 2003 (~XP) by default.
After this patch it may be necessary to override the default Windows
target version to Vista (or newer) via:
autotools: `CPPFLAGS=-D_WIN32_WINNT=0x0600`
cmake: `-DCURL_TARGET_WINDOWS_VERSION=0x0600`
- mingw-w64 v6+ allow changing the default at toolchain build-time.
Notes:
- For non-MSVC, non-mingw-w64 toolchains, `if_nametoindex` needs to be
allowlisted in `curl_setup.h`, if they do support it.
Fixes#17985 (discussion)
Closes#18009
- reorder functions to not need forward declarations.
- sync `ephiperfifo.c` and `evhiperfifo.c`.
- drop redundant casts for `calloc()` return value.
- ephiperfifo: silence unused variable warning.
- fix indent and apply clang-format more.
Closes#20296
To simplify the directory layout.
- OS400 and vms support move from `packages` to `projects`.
- Windows README and `generate.bat` files move from `projects`
to `projects/Windows`.
Closes#20271
To run checksrc and spacecheck on the source tree. Also for cmake
to sync up with autotools' `checksrc` target.
- cmake: `curl-lint`
With `-DCURL_LINT=ON`, checks run automatically for all targets.
- autotools: `lint`
Closes#20175
- asyn-thrdd.c: scope an include.
- apply more clang-format suggestions.
- tidy-up PP guard comments.
- delete empty line from the top of headers.
- add empty line after `curl_setup.h` include where missing.
- fix indent.
- CODE_STYLE.md: add `strcpy`.
Follow-up to 8636ad55df#20088
- lib1901.c: drop unnecessary line.
Follow-up to 436e67f65b#20076Closes#20070
Also:
- examples/hsts-preload: apply the same change as it's based on lib1915
in tests. Make a local clone of `curlx_strcopy()`. Then drop the
`_CRT_SECURE_NO_WARNINGS` hack, that's no longer necessary.
- curl_setup.h: delete `strcpy()` from the `_CRT_SECURE_NO_WARNINGS`
list.
Closes#20076
1. With `MBEDTLS_SSL_PROTO_TLS1_2` not enabled, the mbedTLS code was not
able to connect to any server due to broken logic in curl's
`mbed_set_ssl_version_min_max()`. Now it correctly sets the minimum
supported TLS version based on what is compiled in the library.
2. If debugging is enabled, move the debugging enabling earlier in the
`mbed_connect_step1()` so that verbose errors are actually displayed if
failures happen (see the previous point -- it would've made debugging
that issue easier).
3. Remove the constant `mbedtls_x509_crt_profile_fr` and instead use
mbedTLS-included profile `mbedtls_x509_crt_profile_next` with
`mbedtls_ssl_conf_cert_profile()`. This will follow the latest standards
as new mbedTLS versions are released (rather than being stuck-in-time
until someone comes along to fix what was hard-coded here). This has the
immediate benefit of no longer supporting SHA1 certs and insecure RSA
key-lengths (1024). This fix immediately prevents previously possible
MITM attacks (SHA1 hashes and RSA-1024 keys can be forged relatively
easily by nation-state actors and criminal organizations with
deep-pockets).
4. Added [predictive
resistance](https://mbed-tls.readthedocs.io/en/latest/kb/how-to/add-a-random-generator/#enabling-prediction-resistance)
to the random number generator (adding more entropy to the RNG).
5. Split the random number generator into initialization, the actual
random generation, and the "freeing" of the resources. This
significantly reduces the overhead of using the RNG.
6. Removed the separate RNG function in the TLS connect stage (instead
use the "main" one) and remove the ad-hoc threading support. Instead
properly document how to enable threading in mbedTLS. As it was, other
internals of mbedTLS could have race conditions (in the RSA module in
particular) if `MBEDTLS_THREADING_C` was *not* enabled. And if it is
enabled, then these race-conditions cannot happen. And also, if
MBEDTLS_THREADING_C is enabled then the RNG functions [are fully
thread-safe](https://mbed-tls.readthedocs.io/en/latest/kb/development/thread-safety-and-multi-threading/).
So, the previous ad-hoc threading support was both partial and broken.
7. Enable support for disabling `MBEDTLS_PEM_PARSE_C`.
8. Add support for `CURLOPT_SSLCERTTYPE` so user can specify `PEM` or
`DER` and get faster execution.
Closes#19983
Move existing `Curl_rename()` `rename()` wrapper from lib to
curlx/fopen, and make it a curlx macro/function. To allow using
the local worker function to fixup long filenames on Windows.
Then fix the Windows-specific rename implementation to support long
filenames. This operation may happen when using a cookie jar, HSTS cache
or alt-svc cache, via libcurl or the curl tool.
Before this patch, when passing a long filename to the above options,
a `<random>.tmp` file was left on the disk without renaming it to the
filename passed to curl. There was also 1 second delay for each
attempted rename operation.
Also:
- checksrc: ban raw `rename()` and `MoveFileEx*()` functions.
- Note: `Curl_rename()` returned 1 on failure before this patch, while
`curlx_rename()` returns -1 after, to match POSIX `rename()`.
Refs:
https://learn.microsoft.com/windows/win32/api/winbase/nf-winbase-movefileexahttps://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation
Ref: #20040Closes#20042
It makes them work in Schannel's CA bundle loader, and curl tool's
set/get file timestamp operations (e.g. in `-R`/`--remote-time`). Also
to match file open operations, that already support long filenames.
E.g. when using `--remote-time`, fixing:
```
Warning: Failed to set filetime 1741363917 on outfile: CreateFile failed:
Warning: GetLastError 0x00000003
```
The special long filename logic is necessary to support Windows releases
prior to Windows 10 v1607. With the latter, it's possible to opt-in to
this behavior via a manifest setting. Note that Windows itself also needs
to opt-in to support this. Finally note that curl itself needs passing
`--globoff` to let long filenames through, pending #20044 and #20046.
Refs:
https://learn.microsoft.com/windows/win32/api/fileapi/nf-fileapi-createfileahttps://learn.microsoft.com/windows/win32/fileio/maximum-file-path-limitation
Ref: #8361
Inspired by: #19286
Inspired-by: Mathesh V
Closes#19286Closes#20040
Also:
- vquic-tls.h: do not include unused headers for non-H3 builds.
- autotools: stop looking for `openssl/x509.h` header.
- cmp-config.pl: delete exception for `openssl/x509.h`.
- examples: format/comment sync between the two touched files.
- openssl: drop unused `curlx/wait.h` include.
Closes#20049
Always use curlx_now() when calling Curl_pgrs_now(data). Tests with the
"manual" updates to now proved differ more then 100ms in parallel testing.
Add `curlx_nowp()` to set current time into a struct curltime.
Add `curlx_ptimediff_ms() and friends, passing pointers.
Update documentation.
Closes#19998
- apply more clang-format.
- lib/version: use `CURL_ARRAYSIZE()`.
- INSTALL-CMAKE.md: sync-up an option description with others.
- examples: delete unused main args.
- examples/ftpgetinfo: document `_CRT_SECURE_NO_WARNINGS` symbol.
- delete remaining stray duplicate lines.
- acinclude.m4: drop an unnecessary x-hack.
- vtls/mbedtls: join a URL split into two lines.
- src/tool_cb_see: add parentheses around macro expressions.
- src/tool_operate: move literals to the right side of comparisons.
- libtests: sync up fopen/fstat error messages between tests.
- curl_setup.h: replace `if ! defined __LP64` with `ifndef __LP64`.
I assume it makes no difference on Tandem systems, as the latter form
is already used in `include/curl/system.h`.
Closes#20018
After applying a recent fix made to `threaded.c` (formerly
`multithread.c`) to `threaded-ssl.c`, syncing and updating comments,
the two examples turned out to be identical except their test URLs.
Delete one of them to avoid duplication.
Also:
- examples/threaded: scope a variable.
- examples/threaded: merge comments from its deleted sibling.
Follow-up to 61273f5812#20001
Follow-up to 971e8d661c#19526#19524Closes#20002
Use `data->progress.now` as the timestamp of proecssing a transfer.
Update it on significant events and refrain from calling `curlx_now()`
in many places.
The problem this addresses is
a) calling curlx_now() has costs, depending on platform. Calling it
every time results in 25% increase `./runtest` duration on macOS.
b) we used to pass a `struct curltime *` around to save on calls, but
when some method directly use `curx_now()` and some use the passed
pointer, the transfer experienes non-linear time. This results in
timeline checks to report events in the wrong order.
By keeping a timestamp in the easy handle and updating it there, no
longer invoking `curlx_now()` in the "lower" methods, the transfer
can observer a steady clock progression.
Add documentation in docs/internals/TIME-KEEPING.md
Reported-by: Viktor Szakats
Fixes#19935Closes#19961
To really verify the presence of the XML prolog, also in CI.
- move the prolog check from `loadtest` to `checktest`.
(load did a soft error, silently skipping the test instead of failing)
- runtests: enable `-w` functionality permanently for all test targets,
drop the option. It has no measurable performance impact.
- test 798, 1665: add XML prolog.
Follow-up to f0d277cb0e
Follow-up to b5ea0736bb#19946
Follow-up to 904e7ecb66#19347Closes#19970
- cmake: automatically disable typecheck when running clang-tidy,
to avoid possible interference, and to improve performance.
- INSTALL-CMAKE: document both this, and unity=off for clang-tidy.
- GHA/linux: disable for some static analyzers CI jobs to avoid possible
interference.
- GHA/linux: disable in Fil-C job to improve build performance.
Follow-up to 9e6f1c5efb#19637
Follow-up to fd2ca2399e#17955Closes#19941
To detect mistakes made in the runtests framework that reduce
the number of test runs. Before this patch it could go undetected with
a green CI.
The minimum thresholds will need light maintenance going forward (either
bumping them periodically, or adjust if some may fell below minimums for
justified reasons). We may also make minimums tighter or looser, or more
job-specific.
Latest number of test runs for each job can be seen at Test Clutch:
https://testclutch.curl.se/static/reports/feature-matrix.html
Also:
- GHA: set minimums.
Assisted-by: Dan Fandrich
Follow-up to f2a75a14dd
Follow-up to bb1391f943#19510Closes#19942
With either /usr/sbin/sshd found or configured via --with-test-sshd=path
add tests for SCP down- and uploads, insecure, with known hosts or not,
with authorized user key or unauthorized one.
Working now with libssh and libssh2, using a hashed known_hosts file.
Closes#19934
- drop stray duplicate empty lines in docs, scripts, test data, include,
examples, tests.
- drop duplicate PP parenthesis.
- curl-functions.m4: move literals to the right side in if expressions,
to match rest of the source code.
- FAQ.md: delete language designator from an URL.
- packages: apply clang-format (OS400, VMS).
- scripts/schemetable.c: apply clang-format.
- data320: delete duplicate empty line that doesn't change the outcome.
- spacecheck: extend to check for duplicate empty lines
(with exceptions.)
- fix whitespace nits
Closes#19936
CMake 3.18 was released on 2020-07-15.
It enables using (and/or dropping workarounds) for these features:
LTO support, better performance and pkg-config support, `OBJECT` target,
`-S`, `-B`, `--verbose`, `--install` on the command-line, lib directory
support in interface targets, target_link_options(), LINK_OPTIONS,
FetchContent, `list(PREPEND ...)`, unity, Ninja, fixed imported global
issues.
Ref: https://github.com/curl/curl/discussions/18704Closes#19902
- TCP Fast Open support on Windows
TFO doesn't really work on the internet and isn't really used anywhere.
We use QUIC now.
- get rid of PATH_MAX
Not a priority and when using 3rd party libraries not really up to us.
- auto-detect proxy
A dream we can just stop having.
- config file parsing
Let's not do that in our library.
- add asynch getaddrinfo support
Let's not add a limited glibc specific backend with bad API
- FTP HOST
If we managed this far without name based vhost FTP, let's not.
Closes#19890
To make the test files XML-compliant, and the expected results
possibly easier to manage by keeping them in `.c` files.
Non-XML-compliant files are down to 36 after this patch.
Also:
- make all macro expansions apply to `%includetext` contents.
Closes#19799
- to avoid dupes.
- missing slashes.
- drop `.git` suffix from GitHub git repo URLs for a few outliers.
- use short YouTube URL like curl-www does.
- sync two RFC doc URLs with others.
Closes#19879
- drop redundant parentheses from macro definitions.
- apply clang-format in some places missed earlier.
- wolfssl: fix a macro guard comment.
- curl_setup.h: drop empty lines
- FAQ: fix C formatting.
Closes#19854
The example code does not use curl_multi_assign(), but its callback
function used socketp (called sockp in the function) to get the struct
priv pointer instead of the correct clientp (cbp).
Reported-by: Greg Hudson
Fixes#19840Closes#19841
To make it clearer to readers of the code that the resulting dup also
has a null terminator. Something a "normal" memdup() does not provide.
Closes#19833
Function Curl_bufref_ptr() now returns a const char *.
New function Curl_bufref_uptr() returns a const unsigned char *.
Usage and doc updated.
Closes#19827
To allow replacing `&` characters in `tests/data/test*` files for
XML-compliance.
Also:
- document `%GT`, `%LT`
Follow-up to de49cc89ab#19470Closes#19824
A fix for the tests that took the longest:
- test_05: make the server close the HTTP/1.1 connection when
simulating an error during a download. This eliminates waiting
for a keepalive timeout
- test_02: pause tests with slightly smaller documents, eliminate
special setup for HTTP/2. We test stream window handling now
elsewhere already
- cli_hx_download: run look in 500ms steps instead of 1sec, resuming
paused tranfers earlier.
Closes#19809
Remove connection member `waitfor` and keep it in the SSH connection
meta. Add `ssh` to supported tracing features, convert many DEBUGF
printgs to traces.
Closes#19745
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Make `port` member in these struct of type `uint16_t`.
add `uint8_t transport` to `struct ip_quadruple
Define TRNSPRT_NONE as 0. By assigning a valid transport only on a
successful connection, it is clear when the ip_quadruple members are
valid. Also, for transports not involving ports, the getinfos for
`CURLINFO_PRIMARY_PORT` and `CURLINFO_LOCAL_PORT` will now always return
-1.
Make all `transport` members and parameters of type `uint8_t`.
Document the return value of `CURLINFO_LOCAL_PORT` and
`CURLINFO_PRIMARY_PORT` in this regard. Add tests that writeout stats
report ports correctly.
Closes#19708
The on_uv_timeout callback was trying to access req->data as
a curl_context pointer, but uv.timeout.data was never initialized,
making it always NULL. This rendered the code inside the if(context)
block unreachable.
Fixes#19462Closes#19538
Some public prototypes do not declare return values or out parameters as
const where they should be. Avoid changing the public interface, but
document those values as read-only.
Closes#19692
Description of how this works in `docs/internal/RATELIMITS.ms`.
Notable implementation changes:
- KEEP_SEND_PAUSE/KEEP_SEND_HOLD and KEEP_RECV_PAUSE/KEEP_RECV_HOLD
no longer exist. Pausing is down via blocked the new rlimits.
- KEEP_SEND_TIMED no longer exists. Pausing "100-continue" transfers
is done in the new `Curl_http_perform_pollset()` method.
- HTTP/2 rate limiting implemented via window updates. When
transfer initiaiting connection has a ratelimit, adjust the
initial window size
- HTTP/3 ngtcp2 rate limitin implemnented via ack updates
- HTTP/3 quiche does not seem to support this via its API
- the default progress-meter has been improved for accuracy
in "current speed" results.
pytest speed tests have been improved.
Closes#19384
Also:
- memdebug: fix symbol collision in unity builds.
- memdebug: fix compiler warning by making a variable static.
Follow-up to c77bed81a2#19657Closes#19666
Usage:
- autotools: `--disable-typecheck` (or `--enable-typecheck` (default))
- cmake: `-DCURL_DISABLE_TYPECHECK=ON`.
To disable `curl_easy_setopt()`/`curl_easy_getinfo()` type checking with
supported (new) gcc and clang compilers. It is useful to improve build
performance for the `tests/libtest` target. In particular the CodeQL
analyzer may take above an hour to compile with type checking enabled,
and disabling it brings it down to seconds. On local machines it may
also cut build times in half when build testdeps, depending on platform
and compiler.
Other than these cases, we recommend leaving type checking enabled.
Ref: fdacf34aae#19632
Also:
- GHA/codeql: use it.
- test1165: check in `include/curl`.
- lib1912: delete stray todo comment.
- spelling and comment nits.
Closes#19637
Based on existing code and commit history it appears
`CURL_DISABLE_INSTALL` means to prevent calling `install()`;
`CURL_ENABLE_EXPORT_TARGET` means to prevent calling `export()` and
`install()`s with `EXPORT` in them.
Fix them to also apply to the lib and src directories in that vain:
- lib: honor `CURL_DISABLE_INSTALL`
- src: honor `CURL_DISABLE_INSTALL`
- src: honor `CURL_ENABLE_EXPORT_TARGET`
https://cmake.org/cmake/help/v4.2/command/install.htmlhttps://cmake.org/cmake/help/v4.2/command/export.html
- `CURL_DISABLE_INSTALL` follow-up to:
aace27b096#12287
- `CURL_ENABLE_EXPORT_TARGET` follow-up to:
8698825106#9638643ec29645#7060Closes#19144
To complement the existing `curlx_fopen()` internal API.
It's used by the curl's `--stderr` option.
`curlx_freopen()` adds two features to the bare `freopen()`:
- tracing for debug-enabled builds.
- Unicode and long-filename support for Windows builds.
In effect this adds long-filename and enables Unicode support for
the `--stderr <filename>` curl command-line option on Windows.
Also add to checksrc.
Follow-up to 2f17a9b654#10673Closes#19598
To override the curl default of 5 minutes (300000 ms).
Sometimes a simple test data change can result in a stuck test, this
option makes it exit with an error early. Possible future use in CI
or fast machines to prevent a single test taking 5 minutes and failing
the whole job.
Example hangers:
tests/data/test65:
```diff
-<data1000 crlf="yes">
+<data1000 crlf="headers">
```
tests/data/tests993:
```diff
-%repeat[1000 x 95 328485%0d%0a]%</data>
+%repeat[1000 x 95 328485%0d%0a]%
+</data>
```
Closes#19319
There are 58 non-compliant files. Mark them with the `notxml` keyword.
Also include the compliant ones in the GHA/checksrc xmllint CI job.
Also:
- delete XML prolog from the 3 test data files that had them.
- FILEFORMAT.md: document the `notxml` keyword.
- FILEFORMAT.md: fix a typo.
Follow-up to de49cc89ab#19470
Follow-up to f3095f0dbd#19528
Follow-up to 87ba80a6dfCloses#19595
- curl_setup.h: replace `_CRT_SECURE_NO_DEPRECATE` with
`_CRT_SECURE_NO_WARNINGS`, which seems to be the preferred,
more recent macro for this. Also syncing with libssh2.
They are equivalent for curl sources with the supported compilers.
- cmake: stop setting `_CRT_SECURE_NO_DEPRECATE` globally for examples.
- examples: suppress CRT deprecation warnings on a per-file basis.
To make it work when compiling examples out of curl's build systems.
Use `_CRT_SECURE_NO_WARNINGS`.
- examples: document the functions requiring `_CRT_SECURE_NO_WARNINGS`.
- examples/block_ip: delete superfluous `_CRT_SECURE_NO_WARNINGS`.
- examples/block_ip: limit `_CRT_NONSTDC_NO_DEPRECATE` to MSVC.
- examples/log_failed_transfers: fix to set `_CRT_SECURE_NO_WARNINGS`
before headers and limit to MSVC.
- curl_setup.h: document which SDKs support `_CRT_NONSTDC_NO_DEPRECATE`.
Closes#19175
test 1459 "SFTP with corrupted known_hosts" was seen failing in the past.
To fix it, the test was automatically disabled when detecting libssh
0.9.3 or older, as in the curl CircleCI job, running on Ubuntu 20.04.
This work for a long time, until bumping the CircleCI runner to Ubuntu
22.04 (to have OpenSSL 3), where the test was running again, and failing
with the isssue seen in the past.
- Test skipped with Ubuntu 20.04 (libssh 0.9.3):
https://app.circleci.com/pipelines/github/curl/curl/16445/workflows/7f198763-e0b0-4037-9245-4c4b40ab8726/jobs/155164
- Failure seen with Ubuntu 22.04 (libssh 0.9.6):
https://app.circleci.com/pipelines/github/curl/curl/16452/workflows/b817a808-0fd4-40b0-8eb0-d064926efe12/jobs/155206?invite=true#step-107-211709_45
- Failure seen with Ubuntu 24.04 (libssh 0.10.6):
https://app.circleci.com/pipelines/github/curl/curl/16455/workflows/86c631f1-3c5f-4438-b398-3df2bdab5d20/jobs/155218
Turns out the issue issue isn't libssh 0.9.3 itself, but
a CircleCI-specific default configuration in `/etc/ssh/ssh_config`:
```
# BEGIN ANSIBLE MANAGED BLOCK
Host *
StrictHostKeyChecking no <------ this particular line
HashKnownHosts no
SendEnv LANG LC_*
# END ANSIBLE MANAGED BLOCK
```
libssh will consult configuration files on hard-coded default system
locations and alter its behavior based on settings found in them.
This libssh behavior is present in all supported versions:
5a2abd34cehttps://gitlab.com/libssh/libssh-mirror/-/tags/libssh-0.9.0
It means the existing disable logic based on libssh version worked by
coincidence, and what needs to be checked is these configurations
to decide if it's safe to run the test. Another, simpler option is
to also accept the result code 67, though in that case the test
wouldn't actually test what we want, but would pass anyway.
With the old `oldlibssh` workaround deleted, and the problematic setting
manually overridden (`StrictHostKeyChecking yes`):
- CircleCI Ubuntu 20.04 passes with 1459 enabled:
https://app.circleci.com/pipelines/github/curl/curl/16483/workflows/87a9f389-76a2-4a32-acde-c0b411a4c842/jobs/155302
- CircleCI Ubuntu 22.04 does too:
https://app.circleci.com/pipelines/github/curl/curl/16483/workflows/87a9f389-76a2-4a32-acde-c0b411a4c842/jobs/155303
To fix, replace the `runtests` `oldlibssh` detection logic to parse
libssh config files (instead of checking for libssh version) and disable
test 1459 based on that. Notice the detection is making a light attempt
to parse these files, and does not implement most config file features
(such as includes, quoted values and `=` operator.)
The new runtests workaround tests OK with the:
- default CircleCI configuration, disabling 1459 automatically.
- a sudoless configuration fix, with 1459 run successfully.
Also keep setting this option in CircleCI jobs.
- a sudo configuration fix, with 1459 run successfully.
Ref: https://app.circleci.com/pipelines/github/curl/curl/16492/workflows/56f39335-97ba-412c-9a9b-3d662694375a
GHA jobs are not affected and they work fine, with 1459 running successfully
before and after this patch.
It's possible the libssh API offers ways to control config file use
and/or set the strict host checking option programatically. Maybe
to enable in debug mode (albeit CircleCI job are not debug-enabled),
or offer an option for them. It may be something for a future patch.
Follow-up to 23540923e1#8622
Follow-up to 4b01a57c95#8548
Follow-up to bdc664a640#8490
Follow-up to 7c140f6b2d#8444
Ref: 6d9c5c91b9#19549Closes#19557
It also means that all supported OpenSSL versions and forks support
TLSv1.3 after this patch.
It reduces `openssl.c` size by more than 10%, or 400 LOC.
Ref: #18822Closes#18330
Windows CE support was limited to successful builds with ming32ce
(a toolchain that hasn't seen an update since 2009, using an ancient gcc
version and "old mingw"-style SDK headers, that curl deprecated earlier).
Builds with MSVC were broken for a long time. mingw32ce builds were never
actually tested and runtime and unlikely to work due to missing stubs.
Windows CE toolchains also miss to comply with C89. Paired with lack of
demand and support for the platform, curl deprecated it earlier.
This patch removes support from the codebase to ease maintaining Windows
codepaths.
Follow-up to f98c0ba834#17924
Follow-up to 8491e6574c#17379
Follow-up to 2a292c3984#15975Closes#17927
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Sync outliers with the rest of the code.
Also:
- return error in some failed init cases, instead of `CURLE_OK`:
1908, 1910, 1913, 2082, 3010
- lib1541: delete unused struct member.
Closes#19515
Special meanings of URLs became outdated, and it's also no longer
necessary to pass the test number via the URL or domain anymore.
Delete the text.
Follow-up to c6f1b0ff49#19429Closes#19503
Some curl command-lines are long, often repetitive, and difficult
to read or write:
Before this patch (1 test == 1 line):
- <=78 characters: 1099 tests
- 79-132 characters: 500 tests
- 133+ characters: 217 tests: patch meant to help with some of these.
After this patch:
- <=78 characters: 1288 lines
- 79-132 characters: 526 lines
- 133+ characters: 190 lines
After this patch it's possible to fold long lines into multiple ones.
Folding can reduce greppability, thus this is primarily useful for cases
when the options are repetitive, e.g. a list of form options, headers,
mail parameters and the like.
Closes#19500
Also:
- replace `manpage` with `man page`, add to `badwords.txt`.
- badwords.pl: import `-w` feature from curl-www, syncing the two
scripts fully.
- badwords.txt: import missing items from curl-www, syncing the two
files fully.
- pyspelling.words: drop `cURL` allowed word.
Closes#19468
Fix SC2046: "Quote this to prevent word splitting."
Ref: https://www.shellcheck.net/wiki/SC2046
Also:
- shellcheck.sh: add `set -eu`.
- shellcheck.sh, yamlcheck.sh: always run from repo root.
- pass `--` before passing the list of files, where missing.
- badwords.pl, cleancmd.pl: rework to accept `git ls-files` arguments.
Requires Perl 5.22+ (2015-Jun-01) on Windows.
Ref: https://perldoc.perl.org/functions/open
- INTERNALS.md: require Perl 5.22 on Windows.
- spacecheck.pl: formatting.
- GHA/http3-linux: rework command to avoid SC2046.
- stop suppressing SC2046 warnings.
The yamlcheck.sh issue reported-by: Stanislav Fort (Aisle Research)
Ref: 20251109163515_6eb31da3-deb2-4f4d-8327-935904f27da5
Closes#19432
If CMake reports the target platform not supporting shared libs, turn
`BUILD_SHARED_LIBS` off by default. CMake 3.30+ fails with an error
when trying to create a `SHARED` target for such platforms. Earlier
versions used a workaround that may or may not have worked in practice.
Ref: https://cmake.org/cmake/help/v3.30/policy/CMP0164.html
Seen this with a build setting `-DCMAKE_SYSTEM_NAME=Generic`, e.g.
AmigaOS.
Note this may introduce incompatibility for "Generic" targets, which
support shared libs. If that's the case, set `BUILD_SHARED_LIBS=ON`
manually.
Also drop AmigaOS-specific logic handled automatically after this patch.
Ref: https://cmake.org/cmake/help/v3.7/command/get_property.html
Ref: https://cmake.org/cmake/help/v3.7/prop_gbl/TARGET_SUPPORTS_SHARED_LIBS.htmlCloses#19420
The short descriptions describe the data each info retrieves. The info
itself does not 'get' the data.
This simplifies and shortens the descriptions and make them more
consistent.
Closes#19406
There is no more mixed-newline file in the repository after this patch.
Except for`.bat` and `.sln` files (4 in total), all files use LF
newlines.
Also:
- `spacecheck.pl`: drop mixed-EOL exception for test data.
- runtests: add option `-w` to check if test data has stray CR bytes in
them.
- build: enable the option above in test targets, except the CI-specific
one where `spacecheck.pl` does this job already.
- tested OK (with expected failures) in CI with stray CRs added.
- cmake: enable option `-a` for the `tests` target. To continue testing
after a failed test.
Follow-up to 63e9721b63#19313
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19347
```
docs/libcurl/opts/CURLOPT_SSL_CTX_DATA.md:86:16
docs/libcurl/opts/CURLOPT_SSL_CTX_FUNCTION.md:139:16
```
Also sync `CURL *` and result variable names with rest of docs.
Follow-up to 6d7e924e80#19375Closes#19379
- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
`verify/proxy`:
- make `crlf="yes"` force CRLF to all lines, instead of just applying
to HTTP protocol headers.
- add support for `crlf="headers"` that only converts HTTP protocol
header lines to CRLF. (previously done via `crlf="yes"`.)
- use `crlf="headers"` where possible.
- `reply/connect*`:
- add support for `crlf="yes"` and `crlf="headers"`.
- use them where possible.
- `client/file*`, `client/stdin`:
- add support for `crlf="yes"`.
- use it where possible.
- `reply/data*`, `verify/protocol`:
- replace existing uses of `crlf="yes"` with `crlf="headers`" where it
does not change the result.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)
After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19313
- fix regex to not catch CR (from CRLF), in `PORT`, `EPRT`
commands, allowing to use `crlf="yes"` more.
- add `crlf="headers"` mode for `protocol` sections.
To call `subnewlines()` without its force option.
This is the mode used in `data` sections when `crlf="yes"`.
(This confusion may be subject to a future commit.)
- subnewlines: apply CRLF to `HEAD` and `CONNECT` HTTP requests.
- subnewlines: apply CRLF to RTSP requests.
- delete remaining empty `protocol` sections.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 11325 to 10295. (119984 lines total)
- files with mixed newlines from 707 to 656. (1890 files total)
Follow-up to 4d2a05d3fe#19284Closes#19318
To make them explicit, visible, avoid being accidentally trimmed.
Also prevents Git warnings, e.g. on `git am`.
Also:
- runtests: add support for `%spc%` and `%tab%` macros.
- test59: delete non-significant line-ending space.
- spacecheck.pl: drop line-ending whitespace exception for tests.
Closes#19300
- 'CURL *' handles are called 'curl'
- 'CURLM *' handles are called 'multi'
- write callbacks are called 'write_cb'
- read callbacs are called 'read_cb'
- CURLcode variables are called 'res'
It makes the examples look and feel more consistent. It allows for
easier copy and pasting between examples.
Closes#19299
Also:
- delete dead code.
- sync `http2-download.c` and `http2-upload.c` sources.
- simplessl: fix constant expression.
- simplessl: avoid `expression is constant` VS2010 warning, drop pragma.
- replace large stack buffers with dynamic allocation.
- http2-download: fix to fill transfer number.
Some of these were pointed out by TIOBE scanner via Coverity 2025.3.0.
Closes#19292
Also:
- tests/libtest/cli_h2_serverpush: re-sync formatting.
Previously fixed in tests based on a local clang-tidy v20 report.
Pointed out by TIOBE scanner via Coverity 2025.3.0.
Follow-up to 83a8818cfe#17706Closes#19291
Make autotools and cmake detect DES support in OpenSSL and mbedTLS.
Forward feature macros to C and omit NTLM from the feature preview list.
Use the feature macros in source. This ensure that `-V` output matches
the preview.
OpenSSL doesn't support DES when built with `no-des` or `no-deprecated`.
mbedTLS 4.x no longer supports it, and it's possible to disable it in
<4 with `scripts/config.py unset MBEDTLS_DES_C`.
Before this patch this worked for
mbedTLS 4 only, and with a regression for pending PR #16973.
Also:
- drop NTLM feature check from `curl_setup.h` in favour of autotools/
cmake feature macros. This makes `curl_setup.h` no longer need
to include an mbedTLS header, which in turn makes tests/server build
without depending on mbedTLS.
Fixing, in #16973:
```
In file included from tests/server/first.h:40,
from bld/tests/server/servers.c:3:
lib/curl_setup.h:741:10: fatal error: mbedtls/version.h: No such file or directory
741 | #include <mbedtls/version.h>
| ^~~~~~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/18689537893/job/53291322012?pr=16973
Ref: #19181 (initial fix idea)
Follow-up to 3a305831d1#19077
- move back mbedTLS header include and version check from
`curl_setup.h` to each source which consumes mbedTLS.
- GHA/http3-linux: drop workaround that disabled NTLM for
`no-deprecated` OpenSSL builds.
Follow-up to 006977859d#12384
- curl_ntlm_core: drop pointless macro `CURL_NTLM_NOT_SUPPORTED`.
Follow-up to 006977859d#12384Closes#19206
To not have to guess. Also to sync with autotools, which already uses
this wording.
Also:
- replace the stray term 'folder' with 'directory' for consistency.
- store help text in a temp variable to avoid overly long strings
(mandatory in CMake <4.2.0 and can't be trivially split), also
to avoid repeating this string 4 times.
Ref: https://cmake.org/cmake/help/v4.2/command/set.htmlCloses#19169
Replacing a combination of custom logic in the main script and relying
on CMake's built-in Find module, with code and behavior used for
the rest of dependencies.
Also to:
- add version detection in the non-pkg-config path.
- make `GNUTLS_INCLUDE_DIR` and `GNUTLS_LIBRARY` take precedence over
pkg-config. As with other dependencies.
- document the above two configuration options.
- prepare for #16973, which originally introduced this local Find
module.
The local module is doing largely the same as CMake's built-in
FindGnuTLS. Differences:
- honors `CURL_USE_PKGCONFIG`.
- returns GnuTLS version for non-pkg-config detection.
- consistently returns `GNUTLS_VERSION`.
(CMake's built-in uses s different name in <3.16.)
- CMake 3.16+ returns an imported target. curl supports 3.7,
therefore we may only use it conditionally, which isn't worth it.
Cherry-picked from #16973Closes#19163
It was mistakenly removed in 8dab7465a5 (shipped in 8.9.0)
Also fix test 1139 which should have detected this but didn't due to a
bad regex check.
Reported-by: Jonathan Cardoso
Fixes#19151Closes#119152
RFC 3617 defines two specific modes, "netascii" and "octet". This code
now checks only for those trailing ones - and not in the hostname since
they can't be there anymore.
Assisted-by: Jay Satiro
Closes#19070
- Treat HTTP response codes 522 and 524 as a transient error since
Cloudflare may use them instead of 504 to signal timeout.
For example here is a 522 error message from Cloudflare:
"The initial connection between Cloudflare's network and the origin web
server timed out. As a result, the web page can not be displayed."
Prior to this change the curl tool did not retry on HTTP response codes
522 and 524 when --retry was used.
Fixes https://github.com/curl/curl/discussions/16143
Closes https://github.com/curl/curl/pull/19011
