To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.
Closes#21151
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
tried to get HTTPS-RRs is no answer for a resolve that wants
it.
- fix late arrivals of resolve answers to match the "async"
records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
same time. We may need to resolve an IP interface while the
main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
available, even if A/AAAA queries are still ongoing.
For this, the "async" infrastructure is changed:
- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
nothing about HTTPS.
Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
resolves inside one multi. This is weak, as the id may
wrap around. However it is combined with the `mid` of
the easy handle, making collisions highly unlikely.
`data->state.async` is only accessed in few places where
the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
retrieve the HTTPS-RR information from the dns connection filter.
Delay the connect if the HTTPS-RR is needed, but has not
been resolved yet.
The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.
Details:
c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.
Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.
Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.
Closes#21175
- Rename `Curl_resolv_unlink()` to `Curl_dns_entry_unlink()`.
- Change `Curl_dnscache_get()` to return CURLcode result. Returns
now `CURLE_COULDNT_RESOLVE_HOST` for "negative" cache entries.
- Add `Curl_dnscache_add_negative()` to put a "negative" entry
into the cache.
Closes#20864
- make extract-unit-protos handle multi-line prototypes - but they need
to be above the implementation
- Prototypes for static functions we use in unit tests should not be in
header files. We generate lib/unitprotos.h for this purpose
- Removed some function wrappers written for unit tests and make them
use UNITTEST function directly.
- Renamed time2str() in the tool to timebuf() since we have the same
name in lib/ and in unit tests they can both be used non-static in a
build.
This reverts commit f95fadd116.
Follow-up to #21010Closes#21014
Also:
- support per-directory and per-upper-directory whitelist entries.
- convert badlist input grep tweak into the above format.
(except for 'And' which had just a few hits.)
- fix many code exceptions, but do not enforce.
(there also remain about 350 'will' uses in lib)
- fix badwords in example code, drop exceptions.
- badwords-all: convert to Perl.
To make it usable from CMake.
- FAQ: reword to not use 'will'. Drop exception.
Closes#20886
- de-dupe lib/src strdup/memdup functions into curlx.
- introduce `CURLX_STRDUP_LOW()` for mapping `strdup()`, and to do it at
one place within the code, in `curl_setup.h`.
- tests/server: use `curlx_strdup()`. (Also to fix building without
a system `strdup()`.)
- curlx/curlx.h: shorten and tidy up.
- adjust Windows build path to not need `HAVE_STRDUP`.
- build: stop detecting `HAVE_STRDUP` on Windows.
Closes#20497
When creating a dns entry, the addrinfo is passed into the entry on
success and needed deallocation by the caller on failure.
Change the signature to have Curl_dnscache_mk_entry() *always* take
ownership of the addrinfo, even on failure. Change parameter to address
of pointer so that call always clears it.
This makes the handling of failures to Curl_dnscache_mk_entry() simpler.
Fixes#20465Closes#20468
And a few variables around.
There remain cases where the accepted pointer is const, yet the returned
pointer is written to.
Partly addressing (glibc 2.43):
```
* For ISO C23, the functions bsearch, memchr, strchr, strpbrk, strrchr,
strstr, wcschr, wcspbrk, wcsrchr, wcsstr and wmemchr that return
pointers into their input arrays now have definitions as macros that
return a pointer to a const-qualified type when the input argument is
a pointer to a const-qualified type.
```
Ref: https://lists.gnu.org/archive/html/info-gnu/2026-01/msg00005.html
Reported-by: Rudi Heitbaum
Ref: #20420Closes#20421
- drop leading indent from Markdown.
- switch to Markdown section markers where missing.
- move `&&` and `||` to the end of the line (C, Perl).
- openssl: add parenthesis to an if sub-expression.
- misc clang-format nits.
- unfold Markdown links.
- SSL-PROBLEMS.md: drop stray half code-fence.
Closes#20402
When the compiler supports C99.
- map logging functions to macro stubs when verbose logging is disabled
and the compiler is C99. Make sure these stubs silence unused variable
warnings for non-variadic arguments.
Before this patch they mapped to function stubs, the same codepath
used for C89 compiler in this configuration.
- introduce new macros to tell the compiler which code to include
when verbose code is active, or inactive:
- `CURLVERBOSE`: defined when verbose code is active.
To enclose blocks of code only used for verbose logging.
- `VERBOSE(statement);`:
compile statement when verbose code is active.
To mark code lines only used for verbose logging.
- `NOVERBOSE(statement);`:
compile statement when verbose code is inactive.
To suppress warnings for arguments passed to logging functions via
printf masks, e.g. `NOVERBOSE((void)ipaddress);`, yet keeping
the warning in verbose builds.
Note these macros are not the same as `CURL_DISABLE_VERBOSE_STRINGS`.
Verbose code is always active in C89 mode (without variadic macro
support).
- drop existing uses of `CURL_DISABLE_VERBOSE_STRINGS` where redundant,
or replace with the above macros. Ending up reducing the number of
`#ifdef`s, and also the number of lines.
Assisted-by: Daniel Stenberg
Assisted-by: Jay Satiro
Reported-by: Dan Fandrich
Fixes#20341
Refs: #12105#12167Closes#20353
When checking a transfer for being expired via `Curl_timeleft_ms()`,
eleminate the `bool connecting` parameter and have the function check
the `mstate` of the transfer instead.
Advantages:
* eleminate the caller needing awareness if the transfer is
connecting or in a later state
* fix pingpong timeout handling to check the correct timeout
during "proto_connect" phases
* avoid using "connecting" timeouts during establishing a secondary
connection (e.g. FTP) since this would use the timestamp from
the original, primary connect and thus be wrong
Reported-by: Wyuer on github
Fixes#20347Closes#20354
This allows builds know about all schemes - but only have the protocol
implementations for those actually built-in.
It further allows multiple protocols to reuse the same protocol setup
and functions for both TLS and non-TLS implementations instead of
needing two (or more) structs.
The scheme information is now in 'struct Curl_scheme' and all the
function pointers for each scheme/protocol implementation are in struct
Curl_protocol.
The URL API now always work with all known protocols.
Closes#20351
- asyn-thrdd.c: scope an include.
- apply more clang-format suggestions.
- tidy-up PP guard comments.
- delete empty line from the top of headers.
- add empty line after `curl_setup.h` include where missing.
- fix indent.
- CODE_STYLE.md: add `strcpy`.
Follow-up to 8636ad55df#20088
- lib1901.c: drop unnecessary line.
Follow-up to 436e67f65b#20076Closes#20070
- replace `sendf.h` with `curl_trc.h` where it was included just for it.
- drop unused `curl_trc.h` includes.
- easy: delete obsolete comment about `send.h` include reason.
Also:
- move out `curl_trc.h` include from `sendf.h` and include it directly
in users, where not done already. To flatten the include tree and
to less rely on indirect includes.
- stop including `sendf.h` from other headers, replace it with forward
declaration of `Curl_easy`, as done already elsewhere.
Verified with an all non-unity CI run.
Closes#20061
Use `data->progress.now` as the timestamp of proecssing a transfer.
Update it on significant events and refrain from calling `curlx_now()`
in many places.
The problem this addresses is
a) calling curlx_now() has costs, depending on platform. Calling it
every time results in 25% increase `./runtest` duration on macOS.
b) we used to pass a `struct curltime *` around to save on calls, but
when some method directly use `curx_now()` and some use the passed
pointer, the transfer experienes non-linear time. This results in
timeline checks to report events in the wrong order.
By keeping a timestamp in the easy handle and updating it there, no
longer invoking `curlx_now()` in the "lower" methods, the transfer
can observer a steady clock progression.
Add documentation in docs/internals/TIME-KEEPING.md
Reported-by: Viktor Szakats
Fixes#19935Closes#19961
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
- Curl_async_getaddrinfo() always returned NULL so it was pointless.
Return proper curlcode instead to distinguish between errors. Same for
Curl_doh().
- simplify the IP address handling
- make Curl_str2addr() function return CURLcode
Closes#19669
Windows CRTs have a `share.h`. Before this patch when trying to
`#include <share.h>` it, the compiler picked up curl's internal
`lib/share.h` instead. Rename it to avoid this issue.
CRT `share.h` has constants necessary for using safe open CRT functions.
Also rename `lib/share.c` to keep matching the header.
Ref: https://learn.microsoft.com/cpp/c-runtime-library/sharing-constants
Ref: 625f2c1644#16949#16991
Cherry-picked from #19643Closes#19676
Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
For all network related protocols there must be a non-blank hostname
used. This change adds a few asserts in some places to make debug/tests
catch mistakes if any such would slip in.
Closes#19146
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814Closes#18866
The new custom_* flags in the SSL config need to be inherited when
setting up the doh easy handle, so that defaults apply the same way as
for the original easy handle.
Closes#18831
- GHA/checkdocs: rename `spellcheck` job to `pyspelling` to say
the exact tool used.
- GHA/checkdocs: restore a comment.
- GHA/linux: add `-B .` to a cmake configure to avoid warning, and
future breakage.
- autotools: use correct casing for `Schannel`.
- doh: update RFC URL.
- drop redundant parenthesis.
- fix indentation, whitespace.
Closes#18756
`-Weverything` is not enabled by curl, and not recommended by LLVM,
because it may enable experimental options, and will result in new
fallouts after toolchain upgrades. This patch aims to fix/silence as much
as possible as found with llvm/clang 21.1.0. It also permanently enables
warnings that were fixed in source and deemed manageable in the future.
`-Wformat` warnings are addressed separately via #18343.
Fix/silence warnings in the source:
- typecheck-gcc.h: fix `-Wreserved-identifier`.
- lib: silence `-Wcast-function-type-strict`.
For llvm 16+ or Apple clang 16+.
- asyn-ares: limit `HAPPY_EYEBALLS_DNS_TIMEOUT` to old c-ares versions.
- curl_trc: fix `-Wc++-hidden-decl`.
- doh: fix `-Wc++-keyword`.
- ftp: fix `-Wreserved-identifier`.
- ldap: fix `-Wreserved-identifier`.
- mqtt: comment unused macro to avoid warning.
- multi_ev: drop unused macros to avoid warnings.
- setopt: fix useless `break;` after `return;`.
- gtls, mbedtls, rustls: silence `-Wconditional-uninitialized`.
- socks_sspi, schannel, x509asn1: fix `-Wimplicit-int-enum-cast`.
- x509asn1: fix `-Wc++-keyword`.
- openssl: scope `OSSL_UI_METHOD_CAST` to avoid unused macro warning.
- libssh2, wolfssl: drop unused macros.
- curl_ngtcp2, curl_quiche, httpsrr, urlapi: drop/limit unused macros.
- tool_getparam: fix useless `break;` after `return;` or `break;`.
Not normally enabled because it doesn't work with unity.
https://github.com/llvm/llvm-project/issues/71046
- tool_operate: fix `-Wc++-keyword`.
- curlinfo: fix a `-Wunsafe-buffer-usage`.
- tests: silence `-Wformat-non-iso`.
- lib557: fix `-Wreserved-identifier`.
- lib1565: silence `-Wconditional-uninitialized`.
Enable the above clang warnings permanently in picky mode:
- `-Wc++-hidden-decl`
- `-Wc++-keyword` (except for Windows, where it collides with `wchar_t`)
- `-Wcast-function-type-strict`
- `-Wcast-function-type`
- `-Wconditional-uninitialized`
- `-Wformat-non-iso` (except for clang-cl)
- `-Wreserved-identifier`
- `-Wtentative-definition-compat`
Silence problematic `-Weverything` warnings globally (in picky mode):
- `-Wused-but-marked-unused` (88000+ hits) and
`-Wdisabled-macro-expansion` (2600+ hits).
Triggered by `typecheck-gcc.h` when building with clang 14+.
Maybe there exists a way to fix within that header?
Ref: https://discourse.llvm.org/t/removing-wused-but-marked-unused/55310
- `-Wunsafe-buffer-usage`. clang 16+. 7000+ hits.
May be useful in theory, but such high volume of hits makes it
impractical to review and possibly address. Meant for C++.
Ref: https://clang.llvm.org/docs/SafeBuffers.html
Ref: https://stackoverflow.com/questions/77017567/how-to-fix-code-to-avoid-warning-wunsafe-buffer-usage
Ref: https://discourse.llvm.org/t/rfc-c-buffer-hardening/65734
Ref: https://github.com/llvm/llvm-project/pull/111624
- `-Wimplicit-void-ptr-cast`. clang 21+. 1700+ hits.
C++ warning, deemed pure noise.
Ref: https://github.com/curl/curl/issues/18470#issuecomment-3253506266
- `-Wswitch-default` (180+ hits), `-Wswitch-enum` (190+ hits),
`-Wcovered-switch-default` (20+ hits).
Next to impossible to fix cleanly, esp. when the covered `case`
branches depend on compile-time options.
- `-Wdocumentation-unknown-command` (8+ hits).
Triggered in a few sources. Seems arbitrary and bogus.
- `-Wpadded` (550+ hits).
- `-Wc++-keyword` on Windows, where it collides with `wchar_t`.
(100+ hits)
Ref: https://github.com/llvm/llvm-project/issues/155988
- `-Wreserved-macro-identifier`. clang 13+. 5+ hits.
Sometimes it's necessary to set external macros that use
the reserved namespace. E.g. `_CRT_NONSTDC_NO_DEPRECATE`,
`__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__`, `__NO_NET_API`,
possibly `_REENTRANT`, and more.
It's not worth trying to silence them individually.
- `-Wnonportable-system-include-path` with `clang-cl`.
It'd be broken by doing what the warning suggests.
- `-Wformat-non-iso` for clang-cl.
CMake `PICKY_COMPILER=ON` (the default) or `./configure`
`--enable-warnings` (not the default) is required to enable these
silencing rules.
Also:
- autotools, cmake: fix Apple clang and mainline llvm version translations.
Ref: https://en.wikipedia.org/wiki/Xcode#Toolchain_versions
- autotools, cmake: enable `-Warray-compare` for clang 20+.
Follow-up to 4b7accda5a#17196
- cmake: fix to enable `-Wmissing-variable-declarations` at an earlier
clang version.
- cmake: update internal logic to handle warning options with `+` in
them.
- cmake: fix internal logic to match the whole option when looking
into `CMAKE_C_FLAGS` for custom-disabled warnings.
Follow-up to b85cb8cb4e#18485Closes#18477
* Add more tracing information to c-ares errors.
* remove CURL_ASYNC_SUCCESS, rename `ares->last_status` to
`ares->ares_status`. Give trace explanation for "common"
errors
* add ares "csv" information to tracing on failure
* DoH: invoke `Curl_resolver_error()` on failure to populate
error buf
Closes#18247
This patch bumps the size of these macros from `int` to `long`, while
keeping their actual values the same. It may cause incompatibilities in
user code, requiring the bump of holder variables and/or adding casts:
- CURLPROTO_HTTP
- CURLPROTO_HTTPS
- CURLPROTO_FTP
- CURLPROTO_FTPS
- CURLPROTO_SCP
- CURLPROTO_SFTP
- CURLPROTO_TELNET
- CURLPROTO_LDAP
- CURLPROTO_LDAPS
- CURLPROTO_DICT
- CURLPROTO_FILE
- CURLPROTO_TFTP
- CURLPROTO_IMAP
- CURLPROTO_IMAPS
- CURLPROTO_POP3
- CURLPROTO_POP3S
- CURLPROTO_SMTP
- CURLPROTO_SMTPS
- CURLPROTO_RTSP
- CURLPROTO_RTMP
- CURLPROTO_RTMPT
- CURLPROTO_RTMPE
- CURLPROTO_RTMPTE
- CURLPROTO_RTMPS
- CURLPROTO_RTMPTS
- CURLPROTO_GOPHER
- CURLPROTO_SMB
- CURLPROTO_SMBS
- CURLPROTO_MQTT
- CURLPROTO_GOPHERS
- CURLPROTO_ALL
Also:
- keep existing cast within the documentation to make sure it applies
to older curl versions as well.
Closes#18136
Collision happens when building with mingw-w64 v3 or v2 while targeting
Vista or newer. `iphlpapi.h` includes `windns.h` in this case, which
defines macros named `DNS_TYPE_*`, colliding with curl doh enums.
The issue was fixed in mingw-w64 v4:
ea95d55e33
Fixes:
```
lib/doh.h:54:3: error: expected identifier before numeric constant
DNS_TYPE_A = 1,
^
```
Ref: https://github.com/curl/curl/actions/runs/16551209676/job/46806303365?pr=18009#step:10:17
This workaround seems harmless and generally good practice, but
another option is to require mingw-w64 v4.
Ref: #18009Closes#18041
Add a bitset `dirty` to the multi handle. The presence of a transfer int
he "dirty" set means: this transfer has something to do ASAP.
"dirty" is set by multiplexing protocols like HTTP/2 and 3 when
encountering response data for another transfer than the current one.
"dirty" is set by protocols that want to be called.
Implementation:
* just an additional `uint_bset` in the multi handle
* `Curl_multi_mark_dirty()` to add a transfer to the dirty set.
* `multi_runsingle()` clears the dirty bit of the transfer at
start. Without new dirty marks, this empties the set after
al dirty transfers have been run.
* `multi_timeout()` immediately gives the current time and
timeout_ms == 0 when dirty transfers are present.
* multi_event: marks all transfers tracked for a socket as dirty.
Then marks all expired transfers as dirty. Then it runs
all dirty transfers.
With this mechanism:
* Most uses of `EXPIRE_RUN_NOW` are replaced by `Curl_multi_mark_dirty()`
* `Curl_multi_mark_dirty()` is cheaper than querying if a transfer is
already dirty or set for timeout. There is no need to check, just do it.
* `data->state.select_bits` is eliminated. We need no longer to
simulate a poll event to make a transfer run.
Closes#17662
The secure transport vTLS backend was the only Curl_ssl struct instance
that populated the false_start field. Since its removed, we can now
remove that field entirely. This was a protocol feature specific to TLS
1.2 that has been replaced by the more widely adopted TLS 1.3 early data
mechanisms.
--false-start is now deprecated
Closes#17595
Used for both CURLOPT_SSL_OPTIONS and CURLOPT_PROXY_SSL_OPTIONS
Also: make the DoH code use the full original argument value instead of
each individual flag. Makes it easier to keep all of these in synk.
Closes#17429
Move curlx_ functions into its own subdir.
The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.
The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.
The strcase defines are not curlx_ functions and should not be used by
tool or server code.
dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.
When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.
Assisted-by: Jay Satiro
Closes#17253
