Commit Graph

7404 Commits

Author SHA1 Message Date
Dan Fandrich
17e8200733 tests: make whitespace between functions and classes consistent
Mostly, this means two blank lines between classes and functions and one
line between methods. Since these checks are currently in preview, they
are done in a separate ruff invocation to avoid turning ALL the preview
checks on at the same time.
2026-04-08 10:28:05 -07:00
Dan Fandrich
98e470b3a8 tests: alphabetize and group Python imports & add check
- De-duplicates, groups, and sorts imports based on the provided `isort` settings.
2026-04-08 10:28:05 -07:00
Dan Fandrich
4c1b6f5494 tests: enable more ruff checks
- Checks for missing explicit `return` statements at the end of functions
that can return non-`None` values.
- Checks for classes that inherit from `object`.
- Checks for useless expressions.
- Within an `except*` clause, raise exceptions with `raise ... from err`
or `raise ... from None` to distinguish them from errors in exception
handling
- Checks for variable assignments that immediately precede a `return` of the
assigned variable.
- Checks for `else` statements with a `return` statement in the preceding
`if` block.
- Checks for unnecessary parentheses on raised exceptions.

Closes: #21258
2026-04-08 10:28:05 -07:00
Daniel Stenberg
283db12e05
parsedate: bsearch the time zones
There are 69 entries, bsearch is faster than linear search for random
access.

This now also makes the matching case sensitive (zone names always in
uppercase). No docs said otherwise and all tests assumed uppercase.

Extended test 517

Closes #21266
2026-04-08 12:00:59 +02:00
Ted Lyngmo
135665036f
docs: use the correct CURLOPT_WRITEFUNCTION signature
In order to not encourage users to use incompatible function pointers,
change the callback function definitions to use `char *` instead of
`void *` for the first argument.

Triggered by https://stackoverflow.com/questions/79921871/curl-c-c-library-based-application-produces-erronious-response-for-http-post-r#comment141032037_79921871 :

"The code was mostly modified from
 [this example code](https://curl.se/libcurl/c/postinmemory.html),
 honestly I never knew this is wrong. Thanks for pointing it out."

Signed-off-by: Ted Lyngmo <ted@lyncon.se>
Closes #21265
2026-04-08 11:25:54 +02:00
Daniel Stenberg
3536730f33
parsedate: clarify time2epoch and add more variations to test 517
Polish the time2epoch function to become a little more readable.

Corrected the military time zones: they were going in the wrong
direction.

Add more curl_getdate() input varations to test 517

Closes #21251
2026-04-07 16:32:22 +02:00
Stefan Eissing
db9b6fa82e
cf-ip-happy: limit concurrent attempts
Introduce a limit on the concurrent connect attempts of 6:

- document this in CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS
- close the oldest attempt before opening a new one that would
  exceed the limit
- closing failed attempts early to avoid sockets use beyong their
  usefulness
- add tests for limits in unit2600

These changes are externally visible as file descriptors will be
reassigned where we previously kept the old one around and started a
new socket, allocating always a new descriptor.

Closes #21252
2026-04-07 16:26:25 +02:00
Viktor Szakats
acda4eae5e
runtests: allow configuring SSH host/client key algorithm
via env `CURL_TEST_SSH_KEYALGO`, `rsa` (default), `ecdsa`, `ed25519`.

To ease debugging and testing and to make these code paths more
universal.

Closes #21223
2026-04-05 11:19:42 +02:00
Viktor Szakats
c98d0a2e9a
sshserver.pl: add option to disable KEX algorithms in sshd
Cherry-picked from #21199

Closes #21220
2026-04-04 17:16:36 +02:00
Daniel Stenberg
d2110e7fcf
docs: CURLOPT_LOGIN_OPTIONS is a login property
Test case 1982 verifies this

Closes #21215
2026-04-04 12:37:39 +02:00
Samuel Henrique
f4eddde9dc
test 766: flag as timing-dependent
We have noticed this test as failing when run in parallel in Fedora and
Amazon Linux:
389f140954

Debian did not have issues but given there are other FTP tests flagged
as timing-dependent, it makes sense to flag this one too if we notice
failures in other OSes.

Closes #21155
2026-04-03 17:07:36 +02:00
Stefan Eissing
78e281bf6a
tool: fix retries in parallel mode
Verify that curl's --retry behaviour works

- test 502 in serial, works
- test 502 in parallel, hangs forver, test skipped by default
- test 401, no retries done

Fixes #20669
Reported-by: BlackFuffey on github
Closes #21206
2026-04-03 17:06:16 +02:00
Daniel Stenberg
e65ba1bd34
hsts: make the HSTS read callback handle name dupes
Now the logic for handling name duplicates and picking the longest
expiry and strictest subdomain is the same for the callback as for when
reading from file.

Also strip trailing dots from the hostname added by the callback.

A minor side-effect is that the hostname provided by the callback can
now enable subdomains by starting the name with a dot, but we discourage
using such hostnames in documentation.

Amended test 1915 to verify.

Closes #21201
2026-04-02 18:01:20 +02:00
Daniel Stenberg
b27e828b93
url: init req.no_body in DO so that it works for h2 push
req.no_body was only initialized in Curl_connect, while HTTP/2 server
push adds a duplicated handle via Curl_multi_add_perform and calls
Curl_init_do with conn==NULL, never invoking Curl_connect.

Verify it by amending test 1620

Found by Codex Security

Closes #21194
2026-04-02 10:50:23 +02:00
Daniel Stenberg
93e80c75b4
hsts: accept 10K entries in the list
Up from 1K.

Reduces the risk that someone could flush the list by tricking a user to
do many transfers to new hostnames.

Document the limit.

Follow-up to 03a792b186

Closes #21200
2026-04-02 10:22:35 +02:00
Viktor Szakats
0df6c01db3
tidy-up: rename Curl_safefree()/tool_safefree() to curlx_safefree()
To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.

Closes #21151
2026-04-01 19:53:53 +02:00
Stefan Eissing
2b3dfb4ad4
lib: make resolving HTTPS DNS records reliable:
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
  tried to get HTTPS-RRs is no answer for a resolve that wants
  it.
- fix late arrivals of resolve answers to match the "async"
  records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
  same time. We may need to resolve an IP interface while the
  main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
  available, even if A/AAAA queries are still ongoing.

For this, the "async" infrastructure is changed:

- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
  and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
  nothing about HTTPS.
  Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
  resolves inside one multi. This is weak, as the id may
  wrap around. However it is combined with the `mid` of
  the easy handle, making collisions highly unlikely.
  `data->state.async` is only accessed in few places where
  the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
  retrieve the HTTPS-RR information from the dns connection filter.
  Delay the connect if the HTTPS-RR is needed, but has not
  been resolved yet.

The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.

Details:

c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.

Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.

Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.

Closes #21175
2026-04-01 15:36:31 +02:00
Daniel Stenberg
03a792b186
HSTS: cap the list at 1,000 entries
Avoid never-ending growth.

When adding more entries, it now deletes the first entry in the list,
which is the oldest added entry still held in memory. I decided to avoid
a Least Recently Used concept as I suspect with a list with this many
entries most entries have not been used, and we don't save the timestamp
of recent use anyway.

The net effect might (no matter what) be that the removed entry might
feel a bit "random" in the eyes of the user.

Verify with test 1674

Ref #21183
Closes #21190
2026-04-01 14:38:41 +02:00
Daniel Stenberg
4f3a0ef90d
test1275.pl: ignore indented sections in markdowns
They are special and should not be checked like this.

Closes #21191
2026-04-01 14:36:10 +02:00
Daniel Stenberg
0805ec043e
altsvc: cap the list at 5,000 entries
Avoid never-ending growth.

When adding more entries, it now deletes the first entry in the list,
which is the oldest added entry still held in memory. I decided to avoid
a Least Recently Used concept as I suspect with a list with this many
entries most entries have not been used, and we don't save the timestamp
of recent use anyway.

The net effect might (no matter what) be that the removed entry might
feel a bit "random" in the eyes of the user.

Verify with test 1669.

Reported-by: Geeknik Labs
Fixes #21183
Closes #21189
2026-04-01 12:05:47 +02:00
Daniel Stenberg
11609c3d45
alt-svc: skip expired entries read from file
Extend test 1654 to verify

Closes #21187
2026-04-01 10:56:11 +02:00
Daniel Stenberg
4478a10f0d
hsts: skip expired HSTS entries read from file
Extend test 780 to verify

Closes #21186
2026-04-01 10:49:26 +02:00
Daniel Stenberg
8e8bdd3604
cookie: fix rejection when tabs in value
A previous refactor changed the TAB check so that the octet could be
accepted in the 'path', which would cause an invalid line in the saved
cookie file so not possible to read the cookie back. Not terrible
because the path cannot contain a raw tab anyway so it would never match
anyway.

Add test 1685 to verify

Reported-by: Izan on hackerone

Closes #21185
2026-04-01 10:45:05 +02:00
Daniel Stenberg
8e89646a3d
tool_operhlp: iterate through all slashes to find name
If there is no trailing file name for -O or --remote-name-all, continue
searching until there is no more to search. A URL ending with multiple
slashes would previously make it do wrong.

Add test 1639 and 1644 to verify.

Follow-up to e26eefd9ce

Reported-by: James Fuller

Closes #21165
2026-03-31 07:43:47 +02:00
Dan Fandrich
bae6237c45 docs: list more dependencies for running Python HTTP tests
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Viktor Szakats <commit@vsz.me>
Closes #21110
2026-03-30 21:54:39 -07:00
Viktor Szakats
ce6c441cf2
build: drop redundant HAVE_PTHREAD_H guard
Prior to this patch code used either `HAVE_PTHREAD_H`, or
`HAVE_THREADS_POSIX`, or both, to decide if POSIX Threads support is
present. In effect requiring both to be defined for a consistent build.

Drop detecting and guarding for `pthread.h`, and assume it present when
`HAVE_THREADS_POSIX` is set.

OS/400 had `HAVE_PTHREAD_H` set, but not `HAVE_THREADS_POSIX`, which
possibly left threading disabled in most sources.

Ref: https://pubs.opengroup.org/onlinepubs/009695399/basedefs/pthread.h.html
Ref: 930f2e8227 #21144

Closes #21158
2026-03-30 18:57:07 +02:00
Daniel Stenberg
9f7295fb23
test445: fix error code, remove SMB use
The test should now expect error 7. SMB is now opt-in, so not used in as
many builds anymore.

Follow-up to a186ecf4bf

Closes #21160
2026-03-30 18:50:44 +02:00
Viktor Szakats
930f2e8227
build: drop duplicate pthread.h includes
Already included from `curl_setup.h`.

Closes #21144
2026-03-30 13:46:14 +02:00
Viktor Szakats
3c72928471
build: drop redundant USE_QUICHE guards
When used together with `USE_OPENSSL`. `USE_QUICHE` always implies
`USE_OPENSSL`.

Also: merge two (now identical) `#if` blocks in `openssl.c`.

Closes #21135
2026-03-30 13:46:14 +02:00
Stefan Eissing
335dc0e3c5
cf-dns: connection filter for DNS queries
New connection filter `cf-dns` that manages DNS queries. If hands
out addresses and HTTPS-RR records to anyone interested. Used by
HTTPS and IP happy eyeballing.

Information may become available *before* the libcurl "dns entry"
is complete, e.g. all queries have been answered. The cf-ip-happy
filter uses this information to start connection attempts as soon
as the first address is available.

The multi MSTATE_RESOLVING was removed. A new connection always
goes to MSTATE_CONNECTING. The connectdata bit `dns_resolved`
indicates when DNS information is complete. This is used for
error reporting and starting the progress meter.

Removed dns entries `data->state.dns[i]`, as the `cf-dns` filter
now keeps the reference now.

Many minor tweaks for making this work and pass address information
around safely.

Closes #21027
2026-03-30 13:28:38 +02:00
dependabot[bot]
bb0c8cf5af
GHA: bump pip-dependencies
- update `filelock` from 3.24.3 to 3.25.2
- update `ruff` from 0.15.2 to 0.15.7
- update `cryptography` from 46.0.5 to 46.0.6 (CVE-2026-34073)

Closes #21138
2026-03-28 03:43:26 +01:00
Daniel Stenberg
860c57dffd
test1619: download into %LOGDIR
Otherwise it will end up in tests/ which is not wanted.

Follow-up to d63432d1f8

Reported-by: Stefan Eissing

Closes #21114
2026-03-27 09:33:54 +01:00
Daniel Stenberg
916faac3dc
runtests: check for Debug case sensitively
For both TrackMemory and Debug

Closes #21099
2026-03-27 08:10:39 +01:00
Daniel Stenberg
d63432d1f8
tool_cb_hdr: only truncate etags output when regular file
When sending the output to stdout it cannot truncate.

Add test1619 to verify --etag-save to stdout

Spotted by Codex Security

Closes #21103
2026-03-26 23:39:28 +01:00
Daniel Stenberg
e1fdbdd16f
hsts: when a dupe host adds subdomains, use that
Otherwise a weaker earlier entry is allowed to override a later more
restrictive one.

Add test 1638 to verify.

Closes #21108
2026-03-26 23:27:24 +01:00
Viktor Szakats
20914e3753
perl: harden external command invocations
In `adddocsref.pl`, `checksrc-all.pl`, `singleuse.pl` and tests 307, 1013,
1022, 1275, 1707, 1708, 1710.

Closes #21097
2026-03-26 14:20:07 +01:00
Stefan Eissing
a186ecf4bf
proxy: chunked response, error code
Add test1715 to check proper handling of chunked transfer
encoding in CONNECT responses. Change proxy error code from
56 (RECV_ERROR) for everything to 7 (COULDNT_CONNECT) when
the server response could be read successfully, but establishing
the connection is not possible (http status code wrong).

Adapt several test expectations from 56 to 7.

Closes #21084
2026-03-25 12:16:21 +01:00
Stefan Eissing
39036c9021
async-thrdd: use thread queue for resolving
Use a thread queue and pool for asnyc threaded DNS resolves.
Add pytest test_21_* for verification.

Add `CURLMOPT_RESOLVE_THREADS_MAX` to allow applications to
resize the thread pool used.

Add `CURLMOPT_QUICK_EXIT` to allow applications to skip thread
joins when cleaning up a multi handle. Multi handles in
`curl_easy_perform()` inherit this from `CURLOPT_QUICK_EXIT`.

Add several debug environment variables for testing.

Closes #20936
2026-03-25 12:14:21 +01:00
Viktor Szakats
62d77b12fc
spacecheck: check long lines and repeat spaces, fix fallouts
Verify if lines are not longer than 192 characters. Also verify if lines
have less than 79 repeat spaces (and fix one fallout).

To improve readability by avoiding long lines and to prevent adding
overly long lines with text that may go unnoticed in an editor or diff
viewer.

In addition to pre-existing line length limits: 79 for C, 132 for CMake
sources.

Also:
- spacecheck: fix/harden allowlist regexes.
- spacecheck: tidy-up quotes and simplify escaping.
- spacecheck: allow folding strings with repeat spaces.
- GHA: fix a suppressed shellcheck warning.
- GHA/macos: simplify by dropping brew bundle.
- test1119.pl: precompile a regex.
- FAQ.md: delete very long link to a Windows 7/2008 support article
  that's lost it relevance.

Closes #21087
2026-03-25 11:02:08 +01:00
Stefan Eissing
6f9f4b3cb7
lib: add thread pool and queue
- new source files for thread pool and queue
- test cases 3217 and 3218 for them
- internal documentation

Closes #20916
2026-03-23 23:03:58 +01:00
Daniel Stenberg
664db28d29
test2405: require wakeup
To get the correct file descriptor count

Reported-by: Marcel Raad
Fixes #21069
Closes #21074
2026-03-23 23:01:01 +01:00
Daniel Stenberg
3ef422e128
lib1560: verify more URLs
- control codes at various places in URL, even when URL encoding
  is requested

- verify zone id extraction more

Closes #21071
2026-03-23 14:35:02 +01:00
Daniel Stenberg
07d109b7c6
test1714: verify telnet download with --max-filesize
(disabled on Windows because of hangs)

Closes #21023
2026-03-23 14:32:46 +01:00
Viktor Szakats
806fd7a0e1
genserv.pl: make external calls safe
By passing command-line as separate arguments instead of using a single
string. This needs skipping the shell, so rework redirections to use
Perl `open3()`.

Also explored to use `-out` to avoid redirections, but it makes the
command-line incompatible with some OpenSSL implementations/versions
(e.g. on default macOS), and would still need a solution for
`2>/dev/null`.

Ref: https://perldoc.perl.org/IPC::Open3

Closes #20971
2026-03-23 12:39:38 +01:00
Daniel Stenberg
e0be05cbab
urlapi: fix handling of "file:///"
When the path is exactly one byte, a single slash.

Extended test 1560 to verify.

Found by Codex Security

Closes #21070
2026-03-23 10:17:59 +01:00
Daniel Stenberg
9820e5dfac
tool_operate: fix memory-leak on failed uploads
Add test case 1673 to do repeated upload failures and verify there is no
leak. This proved a previous leak and now it verifies the fix.

Reported-by: James Fuller
Closes #21062
2026-03-22 23:58:02 +01:00
Daniel Stenberg
77ed315096
progress: count amount of data "delivered" to application
... and apply the CURLOPT_MAXFILESIZE limit (if set) on that as well.
This effectively protects the user against "zip bombs".

Test case 1618 verifies using a 14 byte brotli payload that otherwise
explodes to 102400 zero bytes.

Closes #20787
2026-03-21 23:21:55 +01:00
Daniel Stenberg
fa13f373b2
netrc: skip malformed macdef lines
When skipping macdef lines inside netrc files, ignore it completely and
do not tokenize or bail out on bad quotes.

Verify in test 1672

Follow-up to 3b43a05e00

Spotted by Codex Security
Closes #21049
2026-03-21 23:20:49 +01:00
Daniel Stenberg
6026d76fdf
test1668: unit test the FTP 213 time response parser
Should make it harder to introduce regressions.

Closes #21051
2026-03-21 23:10:32 +01:00
Viktor Szakats
3025c9778a
testcurl.pl: replace shell commands with Perl rmtree()
Ref: https://perldoc.perl.org/5.8.2/File::Path
Follow-up to e992aa6a54

Closes #21053
2026-03-21 22:42:38 +01:00