Commit Graph

2414 Commits

Author SHA1 Message Date
Stefan Eissing
66d35ee5d4 lib: avoid reusing unclean connection
When `curl_easy_cleanup()` is invoked while still being part
of a multi handle, the code will auto-remove it. But since the
connection was detached first, the code in
`curl_multi_remove_handle()` that invalidates dirty connections
did not bite.

Attach the connection *after* the possible remove from a multi
handle, so that connection reuse can be prevented.

Add test753 to reproduce and verify the fix. This required adding
the new debug env var CURL_FTP_PWD_STOP, to have a transfer return
from multi_perform() early with a connection that does not show
and pending input.

Reported-by: Brian Harris

Fixes https://github.com/curl/curl/issues/17578
Closes https://github.com/curl/curl/pull/17607
2025-06-15 03:22:25 -04:00
Viktor Szakats
924c7e15dc
tidy-up: update deprecated status of some symbols/options
`CURLOPT_SSL_FALSESTART` / `--false-start`, Secure Transport, BearSSL,
GSKit, MesaLink, NSS.

Follow-up to 1e2e808def #17595
Follow-up to 08a3e8e19a #16677

Closes #17605
2025-06-15 01:38:26 +02:00
Rod Widdowson
df1ff17f88 schannel: allow partial chains for manual peer verification
- Align --cacert behaviour with OpenSSL and LibreSSL.

This changes the default behavior of Schannel manual certificate
verification, which is used when the user provides their own CA
certificates for verification, to accept partial chains. In other words,
the user may provide an intermediate certificate without having to
provide the root CA.

Win8/Server2012 widened the PKIX chain traversal API to allow
certificate traversal to terminate at an intermediate.

This behaviour (terminate at the fist matching intermediate) is the
default for LibreSSL and OpenSSL (with OpenSSL allowing control via
CURLSSLOPT_NO_PARTIALCHAIN).

This change uses the new API if it is available, and also allows the
behaviour to revert legacy if CURLSSLOPT_NO_PARTIALCHAIN is present.

Closes https://github.com/curl/curl/pull/17418
2025-06-14 18:55:08 -04:00
Viktor Szakats
2c27a67daa
tests: always make bundles, adapt build and tests
Make test bundles the default. Drop non-bundle build mode.
Also do all the optimizations and tidy-ups this allows, simpler builds,
less bundle exceptions, streamlined build mechanics.

Also rework the init/deinit macro magic for unit tests. The new method
allows using unique init/deinit function names, and calling them with
arguments. This is in turn makes it possible to reduce the use of global
variables.

Note this drop existing build options `-DCURL_TEST_BUNDLES=` from cmake
and `--enable-test-bundles` / `--disable-test-bundles` from autotools.

Also:
- rename test entry functions to have unique names: `test_<testname>`
  This removes the last exception that was handled in the generator.
- fix `make dist` to not miss test sources with test bundles enabled.
- sync and merge `tests/mk-bundle.pl` into `scripts/mk-unity.pl`.
- mk-unity.pl: add `--embed` option and use it when `CURL_CLANG_TIDY=ON`
  to ensure that `clang-tidy` does not miss external test C sources.
  (because `clang-tidy` ignores code that's #included.)
- tests/unit: drop no-op setup/stop functions.
- tests: reduce symbol scopes, global macros, other fixes and tidy-ups.
- tool1621: fix to run, also fix it to pass.
- sockfilt: fix Windows compiler warning in certain unity include order,
  by explicitly including `warnless.h`.

Follow-up to 6897aeb105 #17468

Closes #17590
2025-06-14 21:08:23 +02:00
Daniel Stenberg
08a3e8e19a
TLS: remove support for Secure Transport and BearSSL
These libraries do not support TLS 1.3 and have been marked for removal
for over a year. We want to help users select a TLS dependency that is
future-proof and reliable, and not supporting TLS 1.3 in 2025 does not
infer confidence. Users who build libcurl are likely to be served better
and get something more future-proof with a TLS library that supports
1.3.

Closes #16677
2025-06-11 07:54:19 +02:00
Viktor Szakats
b761eb5add
cmake: replace cmakelint with cmake-lint from cmakelang, fix issues
This linter detects formatting and naming issues with minimal amount of
noise. It seems to be an improvement over the existing linter which was
only detecting line width overruns.

Also: fix to exclude `CurlTests.c` from linter.

Ref: https://cmake-format.readthedocs.io/en/latest/cmake-lint.html
Ref: https://github.com/cheshirekow/cmake_format

Closes #17576
2025-06-11 07:07:49 +02:00
Daniel Stenberg
614313f12f
system.h: remove some macros
Since curl_off_t is always 64 bit these days, we can simplify and avoid
using some macros.

Closes #17498
2025-06-05 10:56:31 +02:00
Ruocco, Calvin
d3594be653
ws: tests and fixes
This started out as regression tests for the `curl_ws_recv()` and
`curl_ws_send()` implementation and ended up with a bugfix, additional
protocol validation and minor logging improvements.

- Fix reset of fragmented message decoder state when a PING/PONG is
  received in between message fragments.

- Fix undefined behavior (applying zero offset to null pointer) in
  curl_ws_send() when the given buffer is NULL.

- Detect invalid overlong PING/PONG/CLOSE frames.
- Detect invalid fragmented PING/PONG/CLOSE frames.
- Detect invalid sequences of fragmented frames.

  - a) A continuation frame (0x80...) is received without any ongoing
    fragmented message.
  - b) A new fragmented message is started (0x81/0x01/0x82/0x02...)
    before the ongoing fragmented message has terminated.

- Made logs for invalid opcodes easier to understand.
- Moved noisy logs to the `CURL_TRC_WS` log level.
- Unified the prefixes for WebSocket log messages: `[WS] ...`

- Add env var `CURL_WS_FORCE_ZERO_MASK` in debug builds.

  - If set, it forces the bit mask applied to outgoing payloads to
    0x00000000, which effectively means the payload is not masked at
    all. This drastically simplifies defining the expected `<protocol>`
    data in test cases.

- 2700: Frame types
- 2701: Invalid opcode 0x3
- 2702: Invalid opcode 0xB
- 2703: Invalid reserved bit RSV1 _(replaces 2310)_
- 2704: Invalid reserved bit RSV2
- 2705: Invalid reserved bit RSV3
- 2706: Invalid masked server message
- 2707: Peculiar frame sizes _(part. replaces 2311)_
- 2708: Automatic PONG
- 2709: No automatic PONG _(replaces 2312)_
- 2710: Unsolicited PONG
- 2711: Empty PING/PONG/CLOSE
- 2712: Max sized PING/PONG/CLOSE
- 2713: Invalid oversized PING _(replaces 2307)_
- 2714: Invalid oversized PONG
- 2715: Invalid oversized CLOSE
- 2716: Invalid fragmented PING
- 2717: Invalid fragmented PONG
- 2718: Invalid fragmented CLOSE
- 2719: Fragmented messages _(part. replaces 2311)_
- 2720: Fragmented messages with empty fragments
- 2721: Fragmented messages with interleaved pong
- 2722: Invalid fragmented message without initial frame
- 2723: Invalid fragmented message without final frame

- 2305: curl_ws_recv() loop reading three larger frames
  - This test involuntarily sent an invalid sequence of opcodes (0x01...,0x01...,0x81...) , but neither libcurl nor the test caught this! The correct sequence was tested in 2311 (0x01...,0x00...,0x80...). See below for 2311.
  - Validation of the opcode sequence was added to libcurl and is now tested in 2723.
  - Superseded by 2719 (fragmented message) and 2707 (large frames).
- 2307: overlong PING payload
  - The tested PING payload length check was actually missing, but the test didn't catch this since it involuntarily sent an invalid opcode (0x19... instead of 0x89...) so that the expected error occurred, but for the wrong reason.
  - Superseded by 2713.
- 2310: unknown reserved bit set in frame header
  - Superseded by 2703 and extended by 2704 and 2705.
- 2311: curl_ws_recv() read fragmented message
  - Superseded by 2719 (fragmented message) and 2707 (large frames).
- 2312: WebSockets no auto ping
  - Superseded by 2709.

- No tests for `CURLOPT_WRITEFUNCTION`.
- No tests for sending of invalid frames/fragments.

Closes #17136
2025-06-02 11:15:38 +02:00
Daniel Stenberg
fe81a80ae7
spelling: call it null-terminate consistently
With a dash, using two Ls. Also for different forms of the word.

Use NULL in all uppercase if it means a zero pointer.

Follow-up to 307b7543ea

Closes #17489
2025-05-30 17:29:45 +02:00
Daniel Stenberg
bdb7d8b004
spelling: 'a' vs 'an'
Closes #17487
2025-05-30 11:38:35 +02:00
Daniel Stenberg
c0d777c319
curl_url_get.md: don't call it normalized
Because that's not what it does.

Reported-by: Alberto Leiva Popper
Fixes #16829
Closes #17464
2025-05-27 09:25:32 +02:00
Daniel Stenberg
783586b8fd
CURLOPT_FOLLOWLOCATION.md: switch to GET => no body
Clarify

URL: https://curl.se/mail/lib-2025-05/0026.html
Closes #17454
2025-05-26 13:00:38 +02:00
Daniel Stenberg
217c22f315
CURLOPT_READFUNCTION.md: mention the seek callback
Brought-up-by: Jeroen Ooms
URL: https://curl.se/mail/lib-2025-05/0030.html
Closes #17455
2025-05-26 12:59:43 +02:00
Daniel Stenberg
b4310c0fb9
docs/libcurl: mention sensitive data/headers
In the CURLOPT_DEBUGFUNCTION and CURLOPT_VERBOSE documentation.

Mentioned-by: Gordon Parke
Fixes #17353
Closes #17355
2025-05-15 19:42:32 +02:00
Viktor Szakats
828f998493
spacecheck.pl: drop more exceptions
- replace ß (scharfes S) with links.
- replace § (section sign) with links.
- replace 🙏 emoji with `🙏`.
 Supported by GitHub, Forgejo/Gitea and most likely GitLab.
- docs/libcurl/curl_mprintf.md: replace Unicode ± with `{+|-}`.
- docs/CIPHERS.md: URL encode Unicode in URLs.
- lib1560: use hex encoding in `räksmörgås.se`.
- unit1307: use hex encoding in `Lindmätarv`.
- drop LATIN SMALL LETTER A WITH ACUTE exception.
  No longer appears in tests.

This leaves the single character exception: `ö`
And file exceptions holding contributor names.

Follow-up to 9243ed59b3 #17329
Follow-up to 838dc53bb7 #17247

Closes #17335
2025-05-13 16:01:07 +02:00
Jay Satiro
b794264408 curl_version_info.md: clarify ssl_version for MultiSSL
- Clarify that the ssl_version in MultiSSL builds contains all SSL
  backend names with the inactive backend names in parentheses.

Closes https://github.com/curl/curl/pull/17308
2025-05-11 12:23:36 -04:00
Daniel Stenberg
cd7904f5a9
CURLMOPT_TIMERFUNCTION.md: correct the example
Fixes #17301
Reported-by: Dirk Feytons
Closes #17303
2025-05-09 17:04:40 +02:00
Andrei Florea
a638828c88
TLS: add CURLOPT_SSL_SIGNATURE_ALGORITHMS and --sigalgs
Fixes #12982
Closes #16964
2025-04-30 17:47:22 +02:00
Daniel Stenberg
b1eebdf46a
docs/libcurl: fix type and prototype problems in examples
Found by enabling the typechecks when compiling them with
verify-examples.pl

Closes #17231
2025-04-29 22:44:05 +02:00
Daniel Stenberg
d33b449271
CURLOPT_XFERINFOFUNCTION.md: fix the callback return type in example
Fixes #17228
Reported-by: gkarracer on github
Closes #17229
2025-04-29 22:16:40 +02:00
Daniel Stenberg
9f57c2ea95
VULN-DISCLOSURE-POLICY: use of weak algos
Not necessarily security problems.

Closes #17220
2025-04-29 13:11:07 +02:00
Daniel Stenberg
f2ce6c46b9
openssl: enable builds for *both* engines and providers
OpenSSL3 can in fact have both enabled at once. Load the provider and
key/cert appropriately. When loading a provider, the user can now also
set an associated "property string".

Work on this was sponsored by Valantic.

Closes #17165
2025-04-29 10:44:21 +02:00
Daniel Stenberg
15839450be
urlapi: redirecting to "" is considered fine
If the CURLU handle already holds a proper URL, otherwise it is an
error.

Verified by test 1560

Fixes #17188
Reported-by: zopsicle on github
Closes #17189
2025-04-25 22:38:04 +02:00
Daniel Stenberg
9c59a6ac55
libcurl-tutorial.md: fix read callback explanation
Fixes #17138
Reported-by: Thomas Klausner
Closes #17154
2025-04-23 23:23:59 +02:00
Max Eliaser
7b375b8b91
CURLOPT_ERRORBUFFER.md: the buffer is read only after curl takes ownership
Clarifies #17100
Closes #17105
2025-04-22 08:40:42 +02:00
Brian Chrzanowski
c0df01fd94
websocket: add option to disable auto-pong reply
This adds another bitflag on CURLOPT_WS_OPTIONS (CURLWS_NOAUTOPONG) that
disables the default and automatic PONG reply in the WebSocket layer.

Assisted-by: Calvin Ruocco

Closes #16744
2025-04-19 00:01:28 +02:00
Daniel Stenberg
02e9690c3e
tests/server/dnsd: basic DNS server for test suite
Currently the DNS server only responds to A and AAAA queries. It always
responds with a fixed response: the localhost address. Three times.

It should work fine over either IPv4 or IPv6, but I don't think it
matters much for curl testing.

The idea is to allow curl tests to use "normal" DNS hostnames (using the
normal name resolving code paths) and still use the local test servers.

This setup currently only works if curl is built with c-ares because
redirecting DNS requests to our test server when using getaddrinfo() is
not easy.

This should be extended to respond to HTTPS queries as well to allow
more testing there, as c-ares is always used for that.

Test 2102 is the first test using this.

Closes #17015
2025-04-17 09:13:24 +02:00
Christian Schmitz
8ad0243e1f
mqtt: send ping at upkeep interval
Closes #16975
2025-04-16 09:36:19 +02:00
Daniel Stenberg
960984263f
docs/libcurl: make examples build with picky compiler options
Found by improving verify-examples.pl:

- Operate directly on markdown files to remove the need to render nroff files
  first.

- Add -Wall as a compiler option to find more issues

Closes #17028
2025-04-11 15:23:51 +02:00
Pavel Kropachev
d9a86b6729
docs: add missing return statement in examples
Closes #17024
2025-04-11 14:41:28 +02:00
Stefan Eissing
5fbd78eb2d
http2: fix stream window size after unpausing
When pausing a HTTP/2 transfer, the stream's local window size
is reduced to 0 to prevent the server from sending further data
which curl cannot write out to the application.

When unpausing again, the stream's window size was not correctly
increased again. The attempt to trigger a window update was
ignored by nghttp2, the server never received it and the transfer
stalled.

Add a debug feature to allow use of small window sizes which
reproduces this bug in test_02_21.

Fixes #16955
Closes #16960
2025-04-05 14:54:40 +02:00
Daniel Stenberg
daa8693619
VERSIONS: list all past releases
This document now lists all previous releases.

This allows us to verify that documentation refers to actual release
versions.

Test 971 now verifies options-in-versions and all command line options
documentation individually. Fixed a few discrepancies.

Test 1488 verifies libcurl options "Added-in" to exist. Fixed a few
discrepancies there as well.

Closes #16907
2025-04-03 08:26:33 +02:00
Daniel McCarney
8836e65967
docs: add rustls --ca-native & CURLSSLOPT_NATIVE_CA
The one important caveat is that presently _only_ the native platform
verifier/CAs are consulted when this option is used w/ rustls.

Closes #16848
2025-03-27 22:54:24 +01:00
Daniel McCarney
3143efd86a
docs: include rustls-ffi in ECH docs 2025-03-27 08:48:13 +01:00
Viktor Szakats
627e9816ff
cmake: add custom command scripts as dependencies where missing
Also some formatting.

Closes #16835
2025-03-26 15:31:53 +01:00
Daniel Stenberg
179c0258f5
curl_ws_recv.md: expand a little on the fragments the API delivers
Closes #16720
2025-03-14 13:36:02 +01:00
Calvin Ruocco
3588df9478
ws: fix and extend CURLWS_CONT handling
Follow-up to fa3d1e7d43

Add test 2311 to verify

Closes #16687
2025-03-14 11:46:36 +01:00
Stefan Eissing
9bfa64f850
gnutls: set priority via --ciphers
No longer ignore the `--ciphers` argument in gnutls curl builds, but use
it to set the gnutls priority string.

When the set ciphers start with '+', '-' or '!', it is *appended* to the
curl generated priority string. Otherwise it replaces the curl one
completely.

Add test_17_18 to check various combinations.

Closes #16557
2025-03-05 13:51:56 +01:00
tiymat
6758aa722d
IMAP: add CURLOPT_UPLOAD_FLAGS and --upload-flags
Set properties on the uploaded resource.

Test 3209 and 3210 verify.

Closes #15970
2025-03-04 15:21:16 +01:00
Daniel Stenberg
fb13923dd6
lib: add CURLFOLLOW_OBEYCODE and CURLFOLLOW_FIRSTONLY
With this change, the argument passed to the CURLOPT_FOLLOWLOCATION
option is now instead a "mode" instead of just a boolean. Documentation
is extended to describe the two new modes.

Test 1571 to 1581 verify.

Closes #16473
2025-03-03 11:35:48 +01:00
Stefan Eissing
0d3b5937b3
OpenSSL/quictls: add support for TLSv1.3 early data
based on #16450

Adds support for TLSv1.3 early data for TCP and QUIC via ngtcp2.

Closes #16477
2025-03-03 09:27:04 +01:00
Daniel Stenberg
a8ad9a5758
docs: minor edits to please the new spellchecker regime 2025-02-27 13:15:21 +01:00
Viktor Szakats
dbbbf717f3
curl.h: stop defining non-curl __has_declspec_attribute
Public curl headers are best not to define 3rd-party or system macros.
Introduce `CURL_HAS_DECLSPEC_ATTRIBUTE` to cover this system macro and
use it.

Detected by test1167 after dropping the indentation:
```
test 1167...[Verify curl prefix of public symbols in header files]

/usr/bin/perl -I. -I.  returned 1, when expecting 0
 1167: exit FAILED
== Contents of files in the log/10/ dir after test 1167
=== Start of file server.cmd
 Testnum 1167
=== End of file server.cmd
=== Start of file stdout1167
 Bad symbols in public header files:
   __has_declspec_attribute(x)
=== End of file stdout1167
```
Ref: https://github.com/curl/curl/actions/runs/13533200900/job/37819784405?pr=16490#step:42:2087

Ref: https://clang.llvm.org/docs/LanguageExtensions.html#has-declspec-attribute
Follow-up to 50482b8c0a #3616
Ref: #16496 (fixing test1167)
Closes #16491
2025-02-26 13:10:31 +01:00
Daniel Stenberg
6c81f2a35c
CURLOPT_HTTPHEADER.md: add comments to the example
Ref: https://mastodon.social/@jpmens/114065709635360064
Closes #16488
2025-02-25 23:56:05 +01:00
Yedaya Katsman
a55b5b7c62
rustls: add support for CERTINFO
This allows you to use the `certs` and `num_certs` writeout variables in
the curl tool, and getting information about the server certificates
using CURLINFO_CERTINFO.

Closes #16459
2025-02-25 07:59:39 +01:00
Daniel Stenberg
b930142d12
CURLOPT_HTTPHEADER.md: rephrases
An attempt to use better language

Closes #16461
2025-02-24 23:15:26 +01:00
Stefan Eissing
edd573d980
wolfssl: tls early data support
Enable TLS Early Data for wolfSSL:

- merge WOLFSSL_CTX and WOLFSSL setup from ngtcp2 with the general
  implemenation in wolfssl.c
- enable for QUIC via ngtcp2
- give Curl_vquic_tls_init() a `struct alpn_spec` like used for the TCP
  case. Adapt gnutls and other users.
- enable pytest test cases for early data with wolfSSL

and while this messes up wolfssl.c anyway, do

- rename all struct/functions with prefix 'wolfssl_' to 'wssl_' to not
  pollute that name prefix
- rename `ctx/handle` to `ssl_ctx/ssl`, as used in openssl case

Closes #16167
2025-02-24 10:01:51 +01:00
Stefan Eissing
51f8aa79a9
CURLMOPT_SOCKETFUNCTION.md: add advice for socket callback invocation times
Explain when a registered socket callback may get invoked to make user
better aware on how to handle it.

Closes #16441
2025-02-24 00:05:56 +01:00
Stefan Eissing
cfc657a48d
multi: event based rework
Rework the event based handling of transfers and connections to
be "localized" into a single source file with clearer dependencies.

- add multi_ev.c and multi_ev.h
- add docs/internal/MULTI-EV.md to explain the overall workings
- only do event handling book keeping when the socket callback
  is set
- add handling for "connection only" event tracking, when internal
  easy handles are used that are not really tied to a connection.
  Used in connection pool.
- remove transfer member "last_poll" and connections "shutdown_poll"
  and keep all that internal to multi_ev.c
- add CURL_TRC_M() for tracing of "multi" related things, including
  event handling and connection pool operations. Add new trace
  feature "multi" for trace config.
  multi traces will show exactly what is going on in regard to
  event handling.
- multi: trace transfers "mstate" in every CURL_TRC_M() call
- make internal trace buffer 2048 bytes and end the silliness
  with +n here -m there. Adjust test 1652 expectations of resulting
  length and input edge cases.
- add trace feature "lib-ids" to perfix libcurl traces with transfer
  and connection ids. Useful for debugging libcurl applications.

Closes #16308
2025-02-22 14:47:40 +01:00
Jay Satiro
2335cbaa21 ca-native.md: sync with CURLSSLOPT_NATIVE_CA
- Add that the native CA store is used to verify certs in addition to
  the other certificate location settings.

Basically clarify that --ca-native does not override --cacert etc.

Prior to this change that behavior was only documented in
CURLSSLOPT_NATIVE_CA which is what --ca-native maps to.

Ref: https://github.com/curl/curl/pull/16181#issuecomment-2663998865

Closes https://github.com/curl/curl/pull/16373
2025-02-19 03:39:52 -05:00