- vms/curlmsg_vms.h: delete unused/commented code.
- vtls/schannel_verify: sort includes.
- typecheck-gcc.h: fix indent and alignment.
- lib/config-win32.h: drop idle `#undef`.
- spacecheck: check for stray empty lines before after curly braces.
- make literals more readable: 1048576 -> 1024 * 1024
- scope variables.
- use ISO date in a comment.
- drop redundant parentheses.
- drop empty comments.
- unfold lines.
- duplicate/stray spaces in comments.
- fix indent, whitespace, minor typos.
Closes#20690
Backtrack on previous change that aimed to solve the wrong `share.h`
being included. It turns out it did not fix this issue. At the same time
it introduced relative header filenames and the need to include the same
headers differently depending on the source files' location, reducing
readability and editability.
Replace this method by re-adding curl's lib source directory to the
header path and addressing headers by the their full, relative name to
that base directory. Aligning with this method already used in src and
tests.
With these advantages:
- makes includes easier to read, recognize, grep, sort, write, and copy
between sources,
- syncs the way these headers are included across curl components,
- avoids the ambiguity between system `schannel.h`, `rustls.h` vs.
local headers using the same names in `lib/vtls`,
- silences clang-tidy `readability-duplicate-include` checker, which
detects the above issue,
Ref: https://clang.llvm.org/extra/clang-tidy/checks/readability/duplicate-include.html
- possibly silences TIOBE coding standard warnings:
`6.10.2.a: Don't use relative paths in #include statements.`
- long shot: it works well with concatenated test sources, for
clang-tidy-friendly custom unity builds. Ref: #20667
Slight downside: it's not enforced.
If there happens to be a collision between a local `lib/*.h` header and
a system one, the solution is to rename (possibly with its `.c`
counterpart) into the `curl_` namespace. This is also the method used by
curl in the past.
Also:
- curlx/inet_pton: reduce scope of an include.
- toolx/tool_time: apply this to an include, and update VS project
files accordingly. Also dropping unnecessary lib/curlx header path.
- clang-tidy: enable `readability-duplicate-include`.
Follow-up to 3887069c66#19676
Follow-up to 625f2c1644#16991#16949Closes#20623
When the compiler supports C99.
- map logging functions to macro stubs when verbose logging is disabled
and the compiler is C99. Make sure these stubs silence unused variable
warnings for non-variadic arguments.
Before this patch they mapped to function stubs, the same codepath
used for C89 compiler in this configuration.
- introduce new macros to tell the compiler which code to include
when verbose code is active, or inactive:
- `CURLVERBOSE`: defined when verbose code is active.
To enclose blocks of code only used for verbose logging.
- `VERBOSE(statement);`:
compile statement when verbose code is active.
To mark code lines only used for verbose logging.
- `NOVERBOSE(statement);`:
compile statement when verbose code is inactive.
To suppress warnings for arguments passed to logging functions via
printf masks, e.g. `NOVERBOSE((void)ipaddress);`, yet keeping
the warning in verbose builds.
Note these macros are not the same as `CURL_DISABLE_VERBOSE_STRINGS`.
Verbose code is always active in C89 mode (without variadic macro
support).
- drop existing uses of `CURL_DISABLE_VERBOSE_STRINGS` where redundant,
or replace with the above macros. Ending up reducing the number of
`#ifdef`s, and also the number of lines.
Assisted-by: Daniel Stenberg
Assisted-by: Jay Satiro
Reported-by: Dan Fandrich
Fixes#20341
Refs: #12105#12167Closes#20353
When checking a transfer for being expired via `Curl_timeleft_ms()`,
eleminate the `bool connecting` parameter and have the function check
the `mstate` of the transfer instead.
Advantages:
* eleminate the caller needing awareness if the transfer is
connecting or in a later state
* fix pingpong timeout handling to check the correct timeout
during "proto_connect" phases
* avoid using "connecting" timeouts during establishing a secondary
connection (e.g. FTP) since this would use the timestamp from
the original, primary connect and thus be wrong
Reported-by: Wyuer on github
Fixes#20347Closes#20354
This allows builds know about all schemes - but only have the protocol
implementations for those actually built-in.
It further allows multiple protocols to reuse the same protocol setup
and functions for both TLS and non-TLS implementations instead of
needing two (or more) structs.
The scheme information is now in 'struct Curl_scheme' and all the
function pointers for each scheme/protocol implementation are in struct
Curl_protocol.
The URL API now always work with all known protocols.
Closes#20351
- asyn-thrdd.c: scope an include.
- apply more clang-format suggestions.
- tidy-up PP guard comments.
- delete empty line from the top of headers.
- add empty line after `curl_setup.h` include where missing.
- fix indent.
- CODE_STYLE.md: add `strcpy`.
Follow-up to 8636ad55df#20088
- lib1901.c: drop unnecessary line.
Follow-up to 436e67f65b#20076Closes#20070
- drop unused `http.h` includes.
- drop unused `http1.h` include.
- drop unused `http2.h` includes.
- vssh/ssh.h: drop unused `vssh.h` include.
- urldata.h: drop unused protocol includes.
- url: include `smtp.h` directly.
- rtsp.h: include directly where used.
- imap, smtp: drop redundant include, move another from .h to .c.
Verified with an all non-unity CI run.
Closes#20093
- replace `sendf.h` with `curl_trc.h` where it was included just for it.
- drop unused `curl_trc.h` includes.
- easy: delete obsolete comment about `send.h` include reason.
Also:
- move out `curl_trc.h` include from `sendf.h` and include it directly
in users, where not done already. To flatten the include tree and
to less rely on indirect includes.
- stop including `sendf.h` from other headers, replace it with forward
declaration of `Curl_easy`, as done already elsewhere.
Verified with an all non-unity CI run.
Closes#20061
To make it available for all files. Drop includes from individual
sources. This header was already included from most sources and not
specific to any internal subsystem.
Also to ensure that two system symbol redefines on Windows (`read()` and
`write()`) get applied to all sources. Move them to `curl_setup.h`.
Closes#20056
- curl_range: replace `sendf.h` with direct header dependency
`curl_trc.h`.
- drop `curl/curl.h` includes from internal sourcees in favor of the
include made from `curl_setup.h`. Replace it with the latter where
it's the only include.
- include `curl_setup.h` before using macros, where missing.
- drop redundant `stdlib.h`, `string.h` includes, in favor of
`curl_setup_once.h` including them.
- drop redundant `limits.h` in favor of `curl_setup.h` including it.
- fake_addrinfo.h: fix typo in comment.
- curl_setup_once.h: drop `stdio.h` in favor of earlier include in
`curl_setup.h`.
- drop stray, unused, `stddef.h` includes.
- memdebug.h: add missing `stddef.h` include. (relying on accidental
includes via other headers before this patch.)
- stddef.h: document why it's included.
- strerr: drop `curl/mprintf.h` in favor of `curl/curl.h` including it
via `curl_setup.h`.
Closes#20027
- apply more clang-format.
- lib/version: use `CURL_ARRAYSIZE()`.
- INSTALL-CMAKE.md: sync-up an option description with others.
- examples: delete unused main args.
- examples/ftpgetinfo: document `_CRT_SECURE_NO_WARNINGS` symbol.
- delete remaining stray duplicate lines.
- acinclude.m4: drop an unnecessary x-hack.
- vtls/mbedtls: join a URL split into two lines.
- src/tool_cb_see: add parentheses around macro expressions.
- src/tool_operate: move literals to the right side of comparisons.
- libtests: sync up fopen/fstat error messages between tests.
- curl_setup.h: replace `if ! defined __LP64` with `ifndef __LP64`.
I assume it makes no difference on Tandem systems, as the latter form
is already used in `include/curl/system.h`.
Closes#20018
Use `data->progress.now` as the timestamp of proecssing a transfer.
Update it on significant events and refrain from calling `curlx_now()`
in many places.
The problem this addresses is
a) calling curlx_now() has costs, depending on platform. Calling it
every time results in 25% increase `./runtest` duration on macOS.
b) we used to pass a `struct curltime *` around to save on calls, but
when some method directly use `curx_now()` and some use the passed
pointer, the transfer experienes non-linear time. This results in
timeline checks to report events in the wrong order.
By keeping a timestamp in the easy handle and updating it there, no
longer invoking `curlx_now()` in the "lower" methods, the transfer
can observer a steady clock progression.
Add documentation in docs/internals/TIME-KEEPING.md
Reported-by: Viktor Szakats
Fixes#19935Closes#19961
To conclude changing the send buffer type from `const void *` to `const
uint8_t *`, change the top level send function and its implementations.
Closes#19743
Remove connection member `waitfor` and keep it in the SSH connection
meta. Add `ssh` to supported tracing features, convert many DEBUGF
printgs to traces.
Closes#19745
Before this patch curl used the C preprocessor to override standard
memory allocation symbols: malloc, calloc, strdup, realloc, free.
The goal of these is to replace them with curl's debug wrappers in
`CURLDEBUG` builds, another was to replace them with the wrappers
calling user-defined allocators in libcurl. This solution needed a bunch
of workarounds to avoid breaking external headers: it relied on include
order to do the overriding last. For "unity" builds it needed to reset
overrides before external includes. Also in test apps, which are always
built as single source files. It also needed the `(symbol)` trick
to avoid overrides in some places. This would still not fix cases where
the standard symbols were macros. It was also fragile and difficult
to figure out which was the actual function behind an alloc or free call
in a specific piece of code. This in turn caused bugs where the wrong
allocator was accidentally called.
To avoid these problems, this patch replaces this solution with
`curlx_`-prefixed allocator macros, and mapping them _once_ to either
the libcurl wrappers, the debug wrappers or the standard ones, matching
the rest of the code in libtests.
This concludes the long journey to avoid redefining standard functions
in the curl codebase.
Note: I did not update `packages/OS400/*.c` sources. They did not
`#include` `curl_setup.h`, `curl_memory.h` or `memdebug.h`, meaning
the overrides were never applied to them. This may or may not have been
correct. For now I suppressed the direct use of standard allocators
via a local `.checksrc`. Probably they (except for `curlcl.c`) should be
updated to include `curl_setup.h` and use the `curlx_` macros.
This patch changes mappings in two places:
- `lib/curl_threads.c` in libtests: Before this patch it mapped to
libcurl allocators. After, it maps to standard allocators, like
the rest of libtests code.
- `units`: before this patch it mapped to standard allocators. After, it
maps to libcurl allocators.
Also:
- drop all position-dependent `curl_memory.h` and `memdebug.h` includes,
and delete the now unnecessary headers.
- rename `Curl_tcsdup` macro to `curlx_tcsdup` and define like the other
allocators.
- map `curlx_strdup()` to `_strdup()` on Windows (was: `strdup()`).
To fix warnings silenced via `_CRT_NONSTDC_NO_DEPRECATE`.
- multibyte: map `curlx_convert_*()` to `_strdup()` on Windows
(was: `strdup()`).
- src: do not reuse the `strdup` name for the local replacement.
- lib509: call `_strdup()` on Windows (was: `strdup()`).
- test1132: delete test obsoleted by this patch.
- CHECKSRC.md: update text for `SNPRINTF`.
- checksrc: ban standard allocator symbols.
Follow-up to b12da22db1#18866
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814
Follow-up to 9678ff5b1b#18776
Follow-up to 10bac43b87#18774
Follow-up to 20142f5d06#18634
Follow-up to bf7375ecc5#18503
Follow-up to 9863599d69#18502
Follow-up to 3bb5e58c10#17827Closes#19626
Add protocol handler flag `PROTOPT_CONN_REUSE` to indicate that the
protocol allows reusing connections for other tranfers. Add that
to all handlers that support it.
Create connections with `conn->bits.close = FALSE` and remove all
the `connkeep()` calls in protocol handlers setup/connect implementations.
`PROTOPT_CONN_REUSE` assures that the default behaviour applies
at the end of a transfer without need to juggle the close bit.
`conn->bits.close` now serves as an additional indication that a
connection cannot be reused. Only protocol handles that allow
reuse need to set it to override the default behaviour.
Remove all `connclose()` and `connkeep()` calls from connection
filters. Filters should not modify connection flags. They are
supposed to run in eyeballing situations where a filter is just
one of many determining the outcome.
Fix http response header handling to only honour `Connection: close`
for HTTP/1.x versions.
Closes#19333
Description of how this works in `docs/internal/RATELIMITS.ms`.
Notable implementation changes:
- KEEP_SEND_PAUSE/KEEP_SEND_HOLD and KEEP_RECV_PAUSE/KEEP_RECV_HOLD
no longer exist. Pausing is down via blocked the new rlimits.
- KEEP_SEND_TIMED no longer exists. Pausing "100-continue" transfers
is done in the new `Curl_http_perform_pollset()` method.
- HTTP/2 rate limiting implemented via window updates. When
transfer initiaiting connection has a ratelimit, adjust the
initial window size
- HTTP/3 ngtcp2 rate limitin implemnented via ack updates
- HTTP/3 quiche does not seem to support this via its API
- the default progress-meter has been improved for accuracy
in "current speed" results.
pytest speed tests have been improved.
Closes#19384
- badwords.pl: add `-a` option to check all lines in source code files.
Before this patch indented lines were skipped (to avoid Markdown code
fences.)
- GHA/checksrc: use `-a` when verifying the source code.
- GHA/checksrc: disable `So` and `But` rules for source code.
- GHA/checksrc: add docs/examples to the verified sources.
- badwords.txt: delete 4 duplicates.
- badwords.txt: group and sort contractions.
- badwords.txt: allow ` url = `, `DIR`, `<file name`.
Closes#19536
Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
When asking for the last N bytes of a file, and that size was larger
than the file size, it would miss the first byte due to a logic error.
The fixed range parser is now made a common function in the file now
renamed to vssh.c (from curl_path.c) - used by both backends.
Unit test 2605 verifies the parser.
Reported-by: Stanislav Fort (Aisle Research)
Closes#19460
When a SCP/SFTP connection calls the protocol handler disconnect, it
required the connections *and* the easy handles SSH meta data to be
present. When the disconnect is called with an admin handle, the easy
meta data is not present, which prevented the shutdown to run.
The easy meta data is however not necessary to run the shutdown state
machine. Calling it with a NULL `sshp` is fine. To avoid any mixups,
check `sshp` in state operations that need it.
Fixes#19293
Reported-by: And-yW on github
Closes#19295
If there is lingering letters left on the right side after the paths
have been parsed, they are syntactically incorrect so returning error is
the safe thing to do.
Reported-by: Harry Sintonen
Closes#19030
The libssh API uses a 32 bit type for datestamp, so instead of just
force-typecast it, make sure it gets capped at UINT_MAX if the value is
larger.
Reported-by: Joshua Rogers
Closes#18989
Opening the remote file with O_APPEND while attempting to resume causes
all writes to be forced to EOF on servers/implementations where O_APPEND
semantics override a prior seek(). As a result, sftp_seek64() is ignored
and the resumed data is appended, duplicating/corrupting the file.
Fix by:
- Using O_WRONLY (without O_APPEND) when resume_from > 0.
- Skipping the seek entirely if remote_append mode is requested.
Closes#18952
After this patch, the codebase no longer overrides system printf
functions. Instead it explicitly calls either the curl printf functions
`curl_m*printf()` or the system ones using their original names.
Also:
- drop unused `curl_printf.h` includes.
- checksrc: ban system printf functions, allow where necessary.
Follow-up to db98daab05#18844
Follow-up to 4deea9396b#18814Closes#18866
