By introducing wrappers for them in the curlx namespace:
`curlx_fopen()`, `curlx_fdopen()`, `curlx_fclose()`.
The undefine/redefine/`(function)()` methods broke on systems
implementing these functions as macros. E.g. AIX 32-bit's `fopen()`.
Also:
- rename `lib/fopen.*` to `lib/curl_fopen.*` (for `Curl_fopen()`)
to make room for the newly added `curlx/fopen.h`.
- curlx: move file-related functions from `multibyte.c` to `fopen.c`.
- tests/server: stop using the curl-specific `fopen()` implementation
on Windows. Unicode isn't used by runtests, and it isn't critical to
run tests on longs path. It can be re-enabled if this becomes
necessary, or if the wrapper receives a feature that's critical for
test servers.
Reported-by: Andrew Kirillov
Bug: https://github.com/curl/curl/issues/18510#issuecomment-3274393640
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#18634
Turns out the signal handler on Windows still wasn't signal safe after
the previous round of fix. There is an `open()` call made from there,
and `open` happens to be unconditionally overridden via `curl_setup.h`
on Windows, to its local implementation (`curlx_win32_open()`), which
does memory allocations and potentially other things that are not signal
safe.
This is a temporary fix, till avoiding the override of system symbols
`open` and `stat` on Windows.
FTR this did not fix the CI 2304 errors, diskspace fail or job hangs due
to 0xC0000142 fork failure (it's rare all three occurs in the same run):
https://github.com/curl/curl/actions/runs/18110523584?pr=18774
Ref: #18634
Follow-up e95f509c66#16852Closes#18774
Add script to run all Perl sources through `perl -c` to ensure no
issues, and run this script via GHA/checksrc in CI.
Fallouts:
- fix two repeated declarations.
- move `shell_quote()` from `testutil.pm` to `pathhelp.pm`, to
avoid circular dependency in `globalconfig.pm`.
Closes#18745
When there is more than one user-supplied 'Connection: ' header, add
values that curl needs internally to the first one and emit all
subsequent ones thereafter.
Fixes#18662
Reported-by: Evgeny Grin (Karlson2k)
Closes#18686
Fail on errors from SSL_handle_events().
Force quit Caddy test instance that is left hanging longer with
openssl-quic tests for unknown reasons.
Reported in Joshua's sarif data
Closes#18720
Restructured the code in the following ways:
* add terminal states SUCCESS and FAILED
* split SOCK4 and SOCK5 states to be more clear
* use `bufq` for send/recv of SOCK messages
* reduce SOCKS4 states, more speaking names
* for most states, move code into static function
* reduce SOCKS5 states, more speaking names
* add helpers for traversing to FAILED state
* add helper to flush bufq
* add hepler to read minimum amount into bufq
Closes#18401
- requirements.txt: shorten copyright headers.
- requirements.txt: pin packages to versions.
- GHA/windows: use `tests/requirements.txt`.
Pick a `cryptography` package version that satifies both `impacket`
and pytests dependencies.
- GHA/checksrc: move pip deps into a new `requirements.txt`.
To make Dependabot detect and bump them.
- GHA/checksrc: replace apt packages for python test deps with pip
install `tests/**/requirements.txt` to a venv.
- GHA/checksrc: use venv and drop `--break-system-packages`.
- GHA/linux: fix to actually activate venvs.
Follow-up to 2638570241#15578
- GHA/linux: fixup (did not cause an issue)
Follow-up to d75785c7de#18660
- GHA: create venvs later, simplify commands.
- GHA: sync pip command-line options, e.g. drop progress-bar,
everywhere.
Assisted-by: Dan Fandrich
Closes#18708
The implementation was incomplete and lesser than the other backends. No
one ever reported a bug or requested enhancements for this, indicating
that this backend was never used.
Closes#18700
By appending `:all:[separator]` to the header name. The `[separator]` string
is output between each header value if there are more than one to output.
Test 764 and 765 verify
Idea-by: kapsiR on github
Ref: #18449Closes#18491
It was accidentally broken in commit 0f4c439fc7, shipped since
8.8.0 (May 2024) and yet not a single person has noticed or reported,
indicating that we might as well drop support for FTP Kerberos.
Krb5 support was added in 54967d2a3a (July 2007), and we have
been carrying the extra license information around since then for this
code. This commit removes the last traces of that code and thus we can
remove the extra copyright notices along with it.
Reported-by: Joshua Rogers
Closes#18577
Before this patch `accept4()`, `socket()`, `socketpair()`, `send()` and
`recv()` system symbols were remapped via macros, using the same name,
to local curl debug wrappers. This patch replaces these overrides by
introducing curl-namespaced macros that map either to the system symbols
or to their curl debug wrappers in `CURLDEBUG` (TrackMemory) builds.
This follows a patch that implemented the same for `accept()`.
The old method required tricks to make these redefines work in unity
builds, and avoid them interfering with system headers. These tricks
did not work for system symbols implemented as macros.
The new method allows to setup these mappings once, without interfering
with system headers, upstream macros, or unity builds. It makes builds
more robust.
Also:
- checksrc: ban all mapped functions.
- docs/examples: tidy up checksrc rules.
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#18503
- update Microsoft documentation links.
(also drop language designator where present.)
- checksrc: allow longer than 78 character lines if they
contain a https URL. To make these links easier to use and parse.
- merge links that were split into two lines.
Closes#18626
`-Weverything` is not enabled by curl, and not recommended by LLVM,
because it may enable experimental options, and will result in new
fallouts after toolchain upgrades. This patch aims to fix/silence as much
as possible as found with llvm/clang 21.1.0. It also permanently enables
warnings that were fixed in source and deemed manageable in the future.
`-Wformat` warnings are addressed separately via #18343.
Fix/silence warnings in the source:
- typecheck-gcc.h: fix `-Wreserved-identifier`.
- lib: silence `-Wcast-function-type-strict`.
For llvm 16+ or Apple clang 16+.
- asyn-ares: limit `HAPPY_EYEBALLS_DNS_TIMEOUT` to old c-ares versions.
- curl_trc: fix `-Wc++-hidden-decl`.
- doh: fix `-Wc++-keyword`.
- ftp: fix `-Wreserved-identifier`.
- ldap: fix `-Wreserved-identifier`.
- mqtt: comment unused macro to avoid warning.
- multi_ev: drop unused macros to avoid warnings.
- setopt: fix useless `break;` after `return;`.
- gtls, mbedtls, rustls: silence `-Wconditional-uninitialized`.
- socks_sspi, schannel, x509asn1: fix `-Wimplicit-int-enum-cast`.
- x509asn1: fix `-Wc++-keyword`.
- openssl: scope `OSSL_UI_METHOD_CAST` to avoid unused macro warning.
- libssh2, wolfssl: drop unused macros.
- curl_ngtcp2, curl_quiche, httpsrr, urlapi: drop/limit unused macros.
- tool_getparam: fix useless `break;` after `return;` or `break;`.
Not normally enabled because it doesn't work with unity.
https://github.com/llvm/llvm-project/issues/71046
- tool_operate: fix `-Wc++-keyword`.
- curlinfo: fix a `-Wunsafe-buffer-usage`.
- tests: silence `-Wformat-non-iso`.
- lib557: fix `-Wreserved-identifier`.
- lib1565: silence `-Wconditional-uninitialized`.
Enable the above clang warnings permanently in picky mode:
- `-Wc++-hidden-decl`
- `-Wc++-keyword` (except for Windows, where it collides with `wchar_t`)
- `-Wcast-function-type-strict`
- `-Wcast-function-type`
- `-Wconditional-uninitialized`
- `-Wformat-non-iso` (except for clang-cl)
- `-Wreserved-identifier`
- `-Wtentative-definition-compat`
Silence problematic `-Weverything` warnings globally (in picky mode):
- `-Wused-but-marked-unused` (88000+ hits) and
`-Wdisabled-macro-expansion` (2600+ hits).
Triggered by `typecheck-gcc.h` when building with clang 14+.
Maybe there exists a way to fix within that header?
Ref: https://discourse.llvm.org/t/removing-wused-but-marked-unused/55310
- `-Wunsafe-buffer-usage`. clang 16+. 7000+ hits.
May be useful in theory, but such high volume of hits makes it
impractical to review and possibly address. Meant for C++.
Ref: https://clang.llvm.org/docs/SafeBuffers.html
Ref: https://stackoverflow.com/questions/77017567/how-to-fix-code-to-avoid-warning-wunsafe-buffer-usage
Ref: https://discourse.llvm.org/t/rfc-c-buffer-hardening/65734
Ref: https://github.com/llvm/llvm-project/pull/111624
- `-Wimplicit-void-ptr-cast`. clang 21+. 1700+ hits.
C++ warning, deemed pure noise.
Ref: https://github.com/curl/curl/issues/18470#issuecomment-3253506266
- `-Wswitch-default` (180+ hits), `-Wswitch-enum` (190+ hits),
`-Wcovered-switch-default` (20+ hits).
Next to impossible to fix cleanly, esp. when the covered `case`
branches depend on compile-time options.
- `-Wdocumentation-unknown-command` (8+ hits).
Triggered in a few sources. Seems arbitrary and bogus.
- `-Wpadded` (550+ hits).
- `-Wc++-keyword` on Windows, where it collides with `wchar_t`.
(100+ hits)
Ref: https://github.com/llvm/llvm-project/issues/155988
- `-Wreserved-macro-identifier`. clang 13+. 5+ hits.
Sometimes it's necessary to set external macros that use
the reserved namespace. E.g. `_CRT_NONSTDC_NO_DEPRECATE`,
`__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__`, `__NO_NET_API`,
possibly `_REENTRANT`, and more.
It's not worth trying to silence them individually.
- `-Wnonportable-system-include-path` with `clang-cl`.
It'd be broken by doing what the warning suggests.
- `-Wformat-non-iso` for clang-cl.
CMake `PICKY_COMPILER=ON` (the default) or `./configure`
`--enable-warnings` (not the default) is required to enable these
silencing rules.
Also:
- autotools, cmake: fix Apple clang and mainline llvm version translations.
Ref: https://en.wikipedia.org/wiki/Xcode#Toolchain_versions
- autotools, cmake: enable `-Warray-compare` for clang 20+.
Follow-up to 4b7accda5a#17196
- cmake: fix to enable `-Wmissing-variable-declarations` at an earlier
clang version.
- cmake: update internal logic to handle warning options with `+` in
them.
- cmake: fix internal logic to match the whole option when looking
into `CMAKE_C_FLAGS` for custom-disabled warnings.
Follow-up to b85cb8cb4e#18485Closes#18477
We used to treat 0 as "call strlen() to get the length" for
curlx_base64_encode, but it turns out this is rather fragile as we
easily do the mistake of passing in zero when the data is actually not
there and then calling strlen() is wrong.
Force the caller to pass in the correct size. A zero length input string
now returns a zero length output and a NULL pointer.
Closes#18617
Because parts of the cookie loading happens on transfer start the
in-memory cookie jar risks being incomplete and then a save might
wrongly truncate the target file.
Added test 1902 to verify.
Reported-by: divinity76 on github
Fixes#18621Closes#18622
- When an option name is used in text, this script no longer outputs the
short plus long version in the manpage output. It makes the text much
more readable.
This always showing both verions was previously done primarily to make
sure roffit would linkify it correctly, but since roffit 0.17 it
should link both long or short names correctly.
- When managen outputs generic text about options at the end of the
description it now highlights them properly so that they too get
linkified correctly in the HTML version. For consistency.
Closes#18580
Remove use of pthread_cancel in asnyc threaded resolving. While there
are system where this works, others might leak to resource leakage
(memory, files, etc.). The popular nsswitch is one example where resolve
code can be dragged in that is not prepared.
The overall promise and mechanism of pthread_cancel() is just too
brittle and the historcal design of getaddrinfo() continues to haunt us.
Fixes#18532
Reported-by: Javier Blazquez
Closes#18540
Access the error codes of ngtcp2 when a connect attempt failes. Trace
the information for analysis. Treat errors as permanent failure by
default, trigger retrying only when the server refused without
indicating an error.
Closes#18521
Tweaks around handling of --limit-rate:
* tracing: trace outstanding timeouts by name
* multi: do not mark transfer as dirty that have
an EXPIRE_TOOFAST set
* multi: have one static function to asses speed limits
* multi: when setting EXPIRE_TOOFAST remove the transfers
from the dirty set
* progress: rename vars and comment on how speed limit
timeouts are calculated, for clarity
* transfer: when speed limiting, exit the receive loop
after a quarter of the limit has been received, not
on the first chunk received.
* cf-ip-happy.c: clear EXPIRE_HAPPY_EYEBALLS on connect
* scorecard: add --limit-rate parameter to test with
speed limits in effect
Closes#18454
Only actually needed servers should be listed and none is then implied
if no servers are listed.
Outputs a warning if "none" is still set as a server.
Closes#18466
If a string argument is expected and the first two bytes are 0xe2 ex80
and the third has the 7th bit set, that's enough for curl to warn.
Previously we tried to detect and warn only for the unicode double
quote, but users might use single quotes, other quotes or even lead the
argument with one of the "zero widths" characters. This is an attempt to
detect many of those. Without triggering for "normal" IDN hostnames.
Closes#18459
Split out adding of individual request headers into a switch. Check
the connection http version only on fresh connections, use separate
methods.
Add TE: header directly without allocation. Add bit for indicating
Connection: header has been added and custom headers should not do
that again.
Closes#18444
test_10_08, uploading larger files for a h2 proxy, sporadically fails
with a decrpytion error on received data in AWS-LC. The frequency can
be increased by simulated network receive blocks.
Not setting a 4 * TLS record sized buffer, leaving AWS-LC at its
default buffer size seems to mitigate this problem.
Closes#18434
This allows --remote-time to set dates before 1970.
Due to a minor omission in the API, it will still avoid setting the time
if it is indeed exactly epoch 0 (jan 1 1970).
Verified by test 762
Fixes#18424
Reported-by: Terence Eden
Closes#18443
- using {} with single entries makes little sense
- when using {} sets with two entry lists, there can only be 64 to reach
maximum number of URLs
Verify the max check in test 761
Write out 9-length frames to client's WRITEFUNCTION
Read 0-length frames from READFUNCTION *if* the function
started a new frame via `curl_ws_start_frame()`.
Fixes#18286Closes#18332
Reported-by: Andriy Druk
Every time we first run runtests with -j and then again *without* -j,
this message was shown:
"Warning: $runnerid: cleardir(log) failed"
Not anymore.
Closes#18404
If --trace-config is used to set a level before -v is used, don't reset
the state on first -v (to "-all") as it otherwise does. This way,
--trace-config can be used to set specific trace items before -v on the
command line and it still works.
Previously, the first -v use would otherwise reset and undo the earlier
--trace-config items.
Fixes#18346Closes#18361
- schannel: apply BoringSSL workaround to AWS-LC too.
Affects Schannel + AWS-LC MultiSSL builds. (not tested in CI)
Ref: 274940d743#2643#2634
- curl_ntlm_core: deduplicate macro defines.
- curl_ntlm_core: document version thresholds for an AWS-LC-specific
workaround.
It was necessary between v1.2.0 2022-09-01 and v1.30.1 2024-06-21.
No longer necessary since v1.31.0 2024-07-01:
ba94617d99
Follow-up to 34ef4fab22#10320
- lib758: drop redundant OpenSSL version guards.
`OPENSSL_VERSION_NUMBER > 3` automatically guards against LibreSSL,
BoringSSL and AWS-LC.
Ref: 6ddd8f2c0b
Follow-up to a5f0ab7995#18288
- dllmain, curl_sha512_256: formatting.
Closes#18387
To avoid NULL derferences with many outputs and --next.
Follow-up to 034612cd51
Add test 760 to verify
Reported-by: BobodevMm on github
Fixes#18375Closes#18376
... even if there are tests still pending. To help visualize tests that
might be hanging.
Attempts to set the limit at 10 seconds of silence.
Closes#18349
