Commit Graph

6991 Commits

Author SHA1 Message Date
Daniel Stenberg
6d7e924e80
checksrc.pl: detect assign followed by more than one space
And fix some code previously doing this.

Closes #19375
2025-11-05 15:18:28 +01:00
Daniel Stenberg
c12a1fdd0e
tests: remove trailing spaces in server responses
Allows us to drop lots of %spc% from test cases making them easier on
the eye.

Closes #19374
2025-11-05 15:17:29 +01:00
Daniel Stenberg
0783ef2348
tests: remove most user-agent filters
Use the %VERSION instead. The user-agent stripping was introduced at the
time before we had %VERSION (introduced in e6b21d4). The tests would
then remove the user-agent header to make them possible to be compared
in a version independent way.

Fixes #19355
Reported-by: Stefan Eissing
Closes #19356
2025-11-04 13:05:11 +01:00
Viktor Szakats
63e9721b63
tests: avoid hard-coded CRLFs in more sections
- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
  `verify/proxy`:
  - make `crlf="yes"` force CRLF to all lines, instead of just applying
    to HTTP protocol headers.
  - add support for `crlf="headers"` that only converts HTTP protocol
    header lines to CRLF. (previously done via `crlf="yes"`.)
  - use `crlf="headers"` where possible.

- `reply/connect*`:
  - add support for `crlf="yes"` and `crlf="headers"`.
  - use them where possible.

- `client/file*`, `client/stdin`:
  - add support for `crlf="yes"`.
  - use it where possible.

- `reply/data*`, `verify/protocol`:
  - replace existing uses of `crlf="yes"` with `crlf="headers`" where it
    does not change the result.

Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)

After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.

Follow-up to 6cf3d7b1b1 #19318
Follow-up to 4d2a05d3fe #19284

Closes #19313
2025-11-03 21:15:12 +01:00
Viktor Szakats
254e04b702
ftpserver.pl: fix newlines in 227/229 replies, adjust tests
The test FTP server returned LF newlines for 227/229 replies, instead of
the CRLF used for the rest.

Test data added later were explicitly made to expect an LF in these
response lines.

After this patch the FTP server returns CRLF newlines, allowing
to delete this special case in test data.

Follow-up to 3bfff57e1f
Follow-up to a7937ed49c

Closes #19330
2025-11-03 12:19:03 +01:00
Viktor Szakats
aaf9522a2c
test696: decouple from test556 data
Test 696 and 556 share the same libtest code. Make sure to issue
the `GET` request to the correct runtime test number instead of using
the hard-wired "556".

It makes the `sws` test server read the response string from `test696`
`<data>` section, instead of reading it from `test556`. To avoid this
hidden interaction between test data.

AFAICS there is no other similar hard-coded string in reused libtests.

Ref: https://github.com/curl/curl/pull/19313#issuecomment-3477448933
Follow-up to be82a3605a #16003
Closes #19329
2025-11-03 12:19:03 +01:00
dependabot[bot]
7203498c6a
GHA: bump the pip-dependencies group across 2 directories with 3 updates
Closes #19321
2025-11-02 17:10:39 +01:00
Viktor Szakats
986ef77833
runtests: fix Perl warning after recent patch
```
Use of uninitialized value $hash{"crlf"} in string eq at tests/runtests.pl line 1406.
```

Follow-up to 6cf3d7b1b1 #19318
Closes #19327
2025-11-02 13:09:16 +01:00
Viktor Szakats
6cf3d7b1b1
tests: avoid more hard-coded CRLFs in protocol sections
- fix regex to not catch CR (from CRLF), in `PORT`, `EPRT`
  commands, allowing to use `crlf="yes"` more.
- add `crlf="headers"` mode for `protocol` sections.
  To call `subnewlines()` without its force option.
  This is the mode used in `data` sections when `crlf="yes"`.
  (This confusion may be subject to a future commit.)
- subnewlines: apply CRLF to `HEAD` and `CONNECT` HTTP requests.
- subnewlines: apply CRLF to RTSP requests.
- delete remaining empty `protocol` sections.

Reducing the number of `tests/data/test*`:
- CRLF newlines from 11325 to 10295. (119984 lines total)
- files with mixed newlines from 707 to 656. (1890 files total)

Follow-up to 4d2a05d3fe #19284

Closes #19318
2025-11-01 23:52:55 +01:00
Daniel Stenberg
8d0bfe74fb
httpsrr: send HTTPS query to the right target
When the target host is on a different port than 443, the name
"_[port]._https.[name]" shall be used.

Fixes #19301
Reported-by: Gunni on github
Closes #19324
2025-11-01 23:13:30 +01:00
Viktor Szakats
6a97bc2c97
tests/data: delete stray CRLFs in markup lines
Closes #19317
2025-11-01 04:36:09 +01:00
Viktor Szakats
2e408aa5cc
tests: replace standalone significant tabs with macro
Follow-up to d29f14b9cf #19300

Closes #19315
2025-10-31 23:44:21 +01:00
Viktor Szakats
d2f0a0e796
tests/libtest: consistent variable naming for easy/multi/share handles
Follow-up to 928363f28c #19299

Closes #19311
2025-10-31 23:44:21 +01:00
Stefan Eissing
c35a87d776
scorecard: more params for upload tests
Add --upload-parallel=n for controlling upload parallelism. Make upload
processing similar to download processing.

Closes #19302
2025-10-31 23:27:59 +01:00
Viktor Szakats
9c0b239ec1
spelling: fix new finds by typos-cli 1.39.0
Closes #19312
2025-10-31 21:31:32 +01:00
Viktor Szakats
70f240b2ed
tests/libtest/cli*: fix init/deinit, leaks, and more
- add global init and deinit where missing.
- check global init success.
- improve cleaning up on error codepaths.
- drop `CLI_ERR()` macro, that could quit.
  Also make error messages tell the reason.

Closes #19309
2025-10-31 20:56:05 +01:00
Viktor Szakats
d29f14b9cf
tests: replace significant invisible spaces with macros
To make them explicit, visible, avoid being accidentally trimmed.
Also prevents Git warnings, e.g. on `git am`.

Also:
- runtests: add support for `%spc%` and `%tab%` macros.
- test59: delete non-significant line-ending space.
- spacecheck.pl: drop line-ending whitespace exception for tests.

Closes #19300
2025-10-31 17:15:33 +01:00
Viktor Szakats
26e3f00469
test446, 1034, 1160: set US-ASCII encoding in XML header
To match the ASCII-7 requirement for curl test data files.

Follow-up to 9243ed59b3 #17329
Follow-up to 87ba80a6df

Closes #19297
2025-10-31 15:01:08 +01:00
Viktor Szakats
2147de554d
test429: use %repeat[]%
Follow-up to eb22e37060 #19281
Follow-up to 55d4767876 #19279

Closes #19296
2025-10-31 15:01:08 +01:00
Viktor Szakats
4d2a05d3fe
tests: use crlf=yes attribute more
To make special newlines more explicit and visible.
Mostly in `<protocol>` sections, some in `<data*>` and `<upload>`.

Reducing the number of `tests/data/test*`:
- CRLF newlines from 21535 to 11337.
- files with mixed newlines from 1335 to 707.

Also delete empty `<protocol>` sections.

Closes #19284
2025-10-31 15:01:08 +01:00
Viktor Szakats
4b85e489a4
examples/http2-serverpush: fix file handle leaks
Also:
- tests/libtest/cli_h2_serverpush: re-sync formatting.

Previously fixed in tests based on a local clang-tidy v20 report.

Pointed out by TIOBE scanner via Coverity 2025.3.0.
Follow-up to 83a8818cfe #17706

Closes #19291
2025-10-31 13:14:12 +01:00
Viktor Szakats
d2e8acfaa6
test1100: fix missing <protocol> section
To make it actually run. Also fix the NTLM expected result, also syncing
it with other tests.

Follow-up to e6b21d422e #6037

Closes #19288
2025-10-31 13:14:06 +01:00
Viktor Szakats
eb22e37060
tests: use %repeat[] to make tests smaller (cont.)
tests: 46, 265, 304, 316, 397, 443, 551, 552, 559, 651, 742, 775, 1003,
1005, 1006, 1008, 1062, 1070, 1086, 1112, 1151, 1160, 1178, 1192, 1193,
1205, 1237, 3207.

Total `test*` size reduction: 2,395,537 -> 2,165,631 bytes.

Follow-up to 55d4767876 #19279

Closes #19281
2025-10-30 16:52:26 +01:00
Daniel Stenberg
55d4767876
tests: use %repeat[] to make tests smaller
Avoid putting huge chunks of repeated texts in test cases.

test3206, test1060, test1061 and test22

Closes #19279
2025-10-30 09:48:41 +01:00
TheBitBrine
e64c28e243
imap: fix custom FETCH commands to handle literal responses
Custom IMAP commands using -X (e.g. 'FETCH 123 BODY[1]') were only
returning the first line of responses containing literals, instead of
the full multi-line body data.

The issue was that custom commands route through imap_perform_list()
and imap_state_listsearch_resp(), which didn't detect or handle IMAP
literal syntax {size}.

This commit adds literal detection to imap_state_listsearch_resp():
- Detects literal syntax {size} in untagged responses
- Writes the response header line containing the literal marker
- Handles any literal body data already in the pingpong buffer
- Sets up transfer layer to read remaining literal data from socket
- Configures maxdownload and transfer size to include header + body
- Initializes pp->overflow to 0 when no buffered data present
- Modifies imap_done() to transition to FETCH_FINAL for custom
  commands that set up downloads

Test 841 and 3206 verify.

Fixes #18847
Reported-by: BohwaZ
Bug: https://github.com/curl/curl/issues/18847
Closes #19246
2025-10-30 08:48:28 +01:00
Stefan Eissing
446dae5bfe
ngtcp2: overwrite rate-limits defaults
In pytests test_08 with the Caddy server, the new rate-limiting in
ngtcp2 did close the connection because it found "too many" stream data
packet repeats.

It is unclear if this is some Caddy issue or if the ngtcp2 implementaton
is wrong. Or if curl can do anything here.

Reported as https://github.com/ngtcp2/ngtcp2/issues/1850

This PR overwrites the ratelimit defaults in ngtcp2 with ten times
increased values. This makes the errors disappear on macOS.

Enable test_08_04/05 in CI again to see if there are any issues
to be found there. (We had those disabled before having parallel
pytests.)

Closes #19274
2025-10-29 13:34:27 +01:00
Daniel Stenberg
fe9e0115b1
tests: reduce max allowed allocations for four tests
and completely remove the check for one

Closes #19260
2025-10-28 23:28:50 +01:00
Viktor Szakats
fa49c50ac3
tests/server/dnsd: fix potential buffer overflow
When handling incoming DNS packets.

Reported-by: Joshua Rogers

Closes #19261
2025-10-27 21:55:36 +01:00
Stefan Eissing
692c7f133e
TLS: IP address verification, extend test
Change the test certificate to carry a altname 'dns:127.0.0.1' which
should *not* match in test_17_05_bad_ip_addr.

wolfSSL: since `wolfSSL_check_domain_name()` does not differentiate
between DNS and IP names, use if only for DNS names. For IP addresses,
get the peer certificate after the handshake and check that using
wolfSSL_X509_check_ip_asc().

Unfortunately, this succeeds where it should not, as wolfSSL internally
used the same check code for both cases. So, skip the test case until
wolfSSL fixes that.

Reported-by: Joshua Rogers
Closes #19252
2025-10-27 17:22:17 +01:00
Daniel Stenberg
f9dfabb42a
test1802: test --proxy-headers and --headers setting user-agent 2025-10-27 09:52:05 +01:00
Daniel Stenberg
b602de775e
test776: set as 'flaky'
I have not figured out why, but having this test failing in CI every so often
is disturbing.

Reported-by: Viktor Szakatas
Fixes #19235
Closes #19243
2025-10-26 10:47:40 +01:00
dependabot[bot]
e0d6ecdf01
GHA: bump pips
- cryptography from 46.0.2 to 46.0.3 in /tests
- filelock from 3.19.1 to 3.20.0 in /tests
- psutil from 7.1.0 to 7.1.1 in /tests

Closes #19217
Closes #19218
Closes #19219
2025-10-24 15:02:24 +02:00
Stefan Eissing
a330117bb0
smtp: fix EOB handling
SMTP automatically appends a \n.\n to an upload if there is not already
one at the end of the input. The implementation had a bug where this did
not happen, depending on read size and buffering.

Change test 900 to reproduce the failure. The bug only happened for mail
body input of known length, where EOS was known on the last chunk read.
Change test 900 to use an input file and make it large enough.

Fixes #18798
Closes #19193
Reported-by: madoe on github
2025-10-23 13:03:10 +02:00
Stefan Eissing
0a79a599a9
transfer: fix retry for empty downloads on reuse
When a reused connection did transfer 0 bytes, it assumed the transfer
had failed and needed a retry. Add a check for data->red.done, so we can
successfully accept the transfer of a 0-length file via SFTP.

Add test case 1583 to verfiy.

Fix SFTP disconnect debug trace when there was nothing to
disconnect (like when reusing a connection).

Fixes #19165
Reported-by: Alexander Blach
Closes #19189
2025-10-23 13:01:35 +02:00
Daniel Stenberg
a7ece53e96
test776: NTLM with too long NTMLv2 ntresplen 2025-10-23 10:20:16 +02:00
Daniel Stenberg
41e6b45edd
test775: verify NTLM with too long user name set 2025-10-23 10:20:16 +02:00
Stefan Eissing
40f7cd2bdd
mime: fix unpausing of readers
When unpausing a transfer, check if the reader pause state differs
in addition to the "keepon" flags.

Reported-by: 包布丁
Fixes #18848
Closes #19178
2025-10-21 16:30:47 +02:00
Daniel Stenberg
9e198618de
tool_parsecfg: detect and error on recursive --config use
The config file parser now has a maximum level of inclusions allowed (5)
to detect and prevent recursive inclusions of itself leading to badness.

Bonus: clean up return code handling from the config parser.

Test 774 verifies
Closes #19168
2025-10-21 10:11:43 +02:00
Daniel Stenberg
9596c4a258
http: return error for a second Location: header
Unless it is identical to the previous one.

Follow-up to dbcaa00657

Adjusted test 580, added test 772 and 773

Fixes #19130
Reported-by: Jakub Stasiak
Closes #19134
2025-10-20 09:35:23 +02:00
Daniel Stenberg
0bb25cdbb7
curl_easy_setopt.md: add missing CURLOPT_POSTFIELDS
It was mistakenly removed in 8dab7465a5 (shipped in 8.9.0)

Also fix test 1139 which should have detected this but didn't due to a
bad regex check.

Reported-by: Jonathan Cardoso
Fixes #19151
Closes #119152
2025-10-20 08:50:02 +02:00
Daniel Stenberg
976333dd40
curl_path: make sure just whitespace is illegal
This function could previously accidentally return true and a NULL path
if only whitespace was provided as argument.

Also, make it stricter and do not allow CR or LF within the string.

Use more strparse parsing.

Drop the comment saying this is from OpenSSH as it has now been
rewritten since then.

Closes #19141
2025-10-19 16:26:01 +02:00
Daniel Stenberg
769ccb4d42
curl_get_line: enhance the API
To make sure callers can properly differentiate between errors and know
cleanly when EOF happens. Updated all users and unit test 3200.

Triggered by a remark by ZeroPath

Closes #19140
2025-10-19 16:25:11 +02:00
Viktor Szakats
026498df43
mod_curltest: tidy-ups and small fixes
- honor request id (`id=<number>`) in `curltest/put` and
  `curltest/sslinfo` handlers.
- do not truncate `max_upload` input parameter.
- delete unused variables.
- formatting.

Inspired by Joshua's report on tests.

Closes #19061
2025-10-19 13:46:59 +02:00
Daniel Stenberg
f847d2ed02
tool_formparse: rewrite the headers file parser
The -F option allows users to provide a file with a set of headers for a
specific formpost section. This code used old handcrafted parsing logic
that potentially could do wrong.

Rewrite to use my_get_line() and dynbuf. Supports longer lines and
should be more solid parsing code.

Gets somewhat complicated by the (unwise) feature that allows "folding"
of header lines in the file: if a line starts with a space it should be
appended to the previous.

The previous code trimmed spurious CR characters wherever they would
occur in a line but this version does not. It does not seem like
something we want or that users would expect.

Test 646 uses this feature.
Closes #19113
2025-10-18 12:49:53 +02:00
Stefan Eissing
2719aa36b5
test_16: adjust timing expectations
In MOST protocols and runs, the 'pretransfer' time is less than the
'starttransfer'. E.g. request being sent before response comes in.

However, when curl is starved of cpu a server response might start
streaming in before the multi-state transitioned to DID (and recorded
the 'pretransfer' time).

Do no longer check that 'pretransfer' is less or equal 'starttransfer'.
Check that is is less or equal to the total time instead.

Closes #19096
2025-10-17 13:35:35 +02:00
Viktor Szakats
3a305831d1
mbedtls: add support for 4.0.0
After this patch libcurl requires (as already documented)
the `curl_global_init()` call when using the `curl_formadd()` API with
mbedTLS.

Note: NTLM is not supported with mbedTLS 4+, because it lacks
the necessary crypto primitive: DES.

Also:
- lib: de-dupe mbedTLS minimum version checks into `curl_setup.h`.
- lib: initialize PSA Crypto as part of `curl_global_init()`.
  For MD5, SHA-256, `curl_formadd()`, and MultiSSL builds with mbedTLS
  but where mbedTLS isn't the default backend.
- lib1308: fix to call `curl_global_init()` (for the Form API).
- curl_ntlm_core: disable with mbedTLS 4+.
- md4: disable mbedTLS implementation when building against 4.x.
- md5: use mbedTLS PSA Crypto API when available, otherwise use
  the default local implementation.
- sha256: use mbedTLS PSA Crypto API when available, otherwise use
  the default local implementation.
- vtls/mbedtls: drop PSA Crypto initialization in favor of
  `curl_global_init()`.
- vtls/mbedtls: use PSA Crypto random API with all mbedTLS versions.
- vtls/mbedtls: do the same for the SHA-256 callback.
- autotools: detect mbedTLS 4+, and disable NTLM for 3.x.
- cmake: disable NTLM for mbedTLS 3.x.
- GHA/linux: keep building mbedTLS 3.x manually and use it in
  an existing job, while also enabling pytest in it.
- GHA/linux: bump to mbedTLS 4.0.0.
  Closes #19075
  Closes #19074

Refs:
https://github.com/Mbed-TLS/mbedtls/releases/tag/mbedtls-4.0.0
https://github.com/Mbed-TLS/mbedtls/blob/mbedtls-4.0.0/docs/4.0-migration-guide.md
https://github.com/Mbed-TLS/mbedtls/blob/mbedtls-4.0.0/tf-psa-crypto/docs/1.0-migration-guide.md [404]
https://github.com/Mbed-TLS/TF-PSA-Crypto/blob/tf-psa-crypto-1.0.0/docs/1.0-migration-guide.md
https://github.com/Mbed-TLS/TF-PSA-Crypto/blob/tf-psa-crypto-1.0.0/docs/psa-transition.md
627f727bbe/docs/4.0-migration-guide

Closes #19077
2025-10-17 11:50:48 +02:00
Daniel Stenberg
be852e39b2
tftp: check for trailing ";mode=" in URL without strstr
RFC 3617 defines two specific modes, "netascii" and "octet". This code
now checks only for those trailing ones - and not in the hostname since
they can't be there anymore.

Assisted-by: Jay Satiro
Closes #19070
2025-10-15 09:59:57 +02:00
Stefan Eissing
1ea99afdc7
scorecard: add perf support on linux
When calling scorecard with --flame to produce a flamegraph, use
"perf" on linux platforms to do the measurements. Update the scorecard
documentation about it.

Closes #19058
2025-10-14 16:24:35 +02:00
Viktor Szakats
5e74b2df34
REUSE: move copyright headers to .checksrc
To make it simpler to move them around, create and delete them without
syncing with `REUSE.toml`.

Also:
- checksrc: allow empty lines in `.checksrc`.
- comment on why curl printfs are disallowed in examples.

Closes #19024
2025-10-13 11:54:22 +02:00
Daniel Stenberg
27375ca364
tool_getparam: make --fail and --fail-with-body override each other
This allows users to put one of them in their .curlrc and still easily
use the other one at will in command lines.

The --no-* versions disable both of them.

Reported-by: Mitchell Blank Jr
Fixes #19029
Closes #19034
2025-10-13 10:38:10 +02:00