Add `resolv-threaded` to curlinfo to detect use of the threaded resolver
correctly even with c-ares linked to https-rr.
Run test_21_05 exactly when threaded resolver is built.
Closes#21287
- Checks for missing explicit `return` statements at the end of functions
that can return non-`None` values.
- Checks for classes that inherit from `object`.
- Checks for useless expressions.
- Within an `except*` clause, raise exceptions with `raise ... from err`
or `raise ... from None` to distinguish them from errors in exception
handling
- Checks for variable assignments that immediately precede a `return` of the
assigned variable.
- Checks for `else` statements with a `return` statement in the preceding
`if` block.
- Checks for unnecessary parentheses on raised exceptions.
Closes: #21258
Use a thread queue and pool for asnyc threaded DNS resolves.
Add pytest test_21_* for verification.
Add `CURLMOPT_RESOLVE_THREADS_MAX` to allow applications to
resize the thread pool used.
Add `CURLMOPT_QUICK_EXIT` to allow applications to skip thread
joins when cleaning up a multi handle. Multi handles in
`curl_easy_perform()` inherit this from `CURLOPT_QUICK_EXIT`.
Add several debug environment variables for testing.
Closes#20936
- It is slower and uses more memory than the alternatives and is only
experimental in curl.
- We disable a few tests for OpenSSL-QUIC because of flakiness
- It gets little attention from OpenSSL and we have no expectation of the
major flaws getting corrected anytime soon.
- No one has spoken up for keeping it
- curl users building with vanilla OpenSSL can still use QUIC through the
means of ngtcp2
Closes#20226
When checking the reported times of a transfer, do not exptect
the 'queue' time to be in any relation to others. 'queue' uses its own
start timestamp and the reported duration is thereofore independant.
Ref: #20112Closes#20163
Add a `cert-status` feature flag to `curlinfo`, based on the conditions
used in `lib/vtls` sources.
To:
- fix disabling this test when using OpenSSL (or fork) built with
the `no-ocsp` option.
- enable this test for AWS-LC in CI.
Note:
- BoringSSL (and quiche) has OSCP disabled by default.
- MultiSSL dynamic selection continues to confuse this test.
(To fix it, support would need to be detected by querying libcurl
via curl. Probably overkill given that OCSP is on its way out.)
Follow-up to f2c765028f#20149Closes#20133
When a test server is found or configured, do not silently ignore
errors to start and disable them when checking their version.
This forces pytest to fail when a server is not operating
as it should.
Closes#19996
With either /usr/sbin/sshd found or configured via --with-test-sshd=path
add tests for SCP down- and uploads, insecure, with known hosts or not,
with authorized user key or unauthorized one.
Working now with libssh and libssh2, using a hashed known_hosts file.
Closes#19934
Let nghttpx only use http/1.1 to backend. This reproduces the bug in
quiche with higher frequency. Allow test_14_05 to now return a 400 in
addition to the 431 we get from a h2 backend to nghttpx.
Skip test_05_02 in h3 on quiche not newer than version 0.24.4 in which
its bug is fixed: https://github.com/cloudflare/quiche/pull/2278
Ref: https://github.com/cloudflare/quiche/issues/2277Closes#19770 (original Issue)
Closes#19916
When the Alt-Svc points to the same host and port, add the destination
ALPN to the `wanted` versions and set it also as the `preferred` version
in negotiations.
This allows Alt-Svc for h3 to point to h2 and have it tried first. Also,
this allows Alt-Svc to say http/1.1 is preferred and changes the ALPN
protocol ordering for the TLS handshake.
Add tests in various combination to verify this works.
Reported-by: yushicheng7788 on github
Fixes#19740Closes#19874
Address issues listed in #19770:
- allow for ngttpx to successfully shut down on last attempt that might
extend beyond the finish timestamp
- timeline checks: allos `time_starttransfer` to appear anywhere in
the timeline as a slow client might seen response data before setting
the other counters
- dump logs on test_05_02 as it was not reproduced locally
Fixes#19970Closes#19783
A fix for the tests that took the longest:
- test_05: make the server close the HTTP/1.1 connection when
simulating an error during a download. This eliminates waiting
for a keepalive timeout
- test_02: pause tests with slightly smaller documents, eliminate
special setup for HTTP/2. We test stream window handling now
elsewhere already
- cli_hx_download: run look in 500ms steps instead of 1sec, resuming
paused tranfers earlier.
Closes#19809
* move the TIMER_POSTQUEUE to the time a connection is chosen,
so that TIMER_NAMELOOKUP always happens afterwards
* client writer: do not trigger TIMER_STARTTRANSFER on CLIENTWRITE_INFO
as ftp and other pingpong protocols write that before starting anything
that is the tranfer itself
* Elimnating debug trancing of "closed stream/connection - bailing"
as confusing, as connection is not really closed on most cases.
* Setting 'data->req.upload_done` correctly, so that no "abort upload"
is happening at the end of a perfectly fine download.
* Adding test cases with up-/download of 0-length files.
* pytest: add a "timeline" of timer value checks to Resulst in curl.py,
so that this can be used in several test cases, replacing the local
stuff in test_16
* add timeline checks to ftp test cases
Closes#19269
There remain some false positives, hits in test data, and `dir` use,
around 100 issues in total.
There is no plan to enforce badwords on tests.
Also:
- badwords.txt: let a few `manpage[s]` occurrences through
(in Perl code).
Closes#19541
- fix test_17_20 flakiness: the test case did not have `nghttpx` in
its parameters, causing it to no check if a reload was necessary.
When that test ran behind one that gave nghttpx another certificate,
eg. in parallel mode, it used the wrong pinned pubkey.
- Have `env` provide lists of HTTP protocol versions available for
testing. Replace parameterized tests on a fixed protocol list with
the dynamic one from env. This makes checks for protocol availability
in the test function bodies superfluous.
refs #19489Closes#19540
- allow 02_28 to run in HTTP/1.1 without H2 support again.
Follow-up to 3752de465d#19412
- fix to skip 02_28 for all protocols for curl without compression
support (either zlib, brotli or ztsd).
Closes#19458
A regression in curl 8.17.0 led to a customer CAPATH set by the
application (or the curl command) to be ignored unless licurl was built
with a default CAPATH.
Add test cases using `--capath` on the custom pytest CA, generated with
the help of the openssl command when available.
Fixes#19401
Reported-by: Brad King
Closes#19308
Change the test certificate to carry a altname 'dns:127.0.0.1' which
should *not* match in test_17_05_bad_ip_addr.
wolfSSL: since `wolfSSL_check_domain_name()` does not differentiate
between DNS and IP names, use if only for DNS names. For IP addresses,
get the peer certificate after the handshake and check that using
wolfSSL_X509_check_ip_asc().
Unfortunately, this succeeds where it should not, as wolfSSL internally
used the same check code for both cases. So, skip the test case until
wolfSSL fixes that.
Reported-by: Joshua Rogers
Closes#19252
- honor request id (`id=<number>`) in `curltest/put` and
`curltest/sslinfo` handlers.
- do not truncate `max_upload` input parameter.
- delete unused variables.
- formatting.
Inspired by Joshua's report on tests.
Closes#19061
When calling scorecard with --flame to produce a flamegraph, use
"perf" on linux platforms to do the measurements. Update the scorecard
documentation about it.
Closes#19058
Add infrastructure to colled and dispatch notifications for transfers
and the multi handle in general. Applications can register a callback
and en-/disable notification type the are interested in.
Without a callback installed, notifications are not collected. Same when
a notification type has not been enabled.
Memory allocation failures on adding notifications lead to a general
multi failure state and result in CURLM_OUT_OF_MEMORY returned from
curl_multi_perform() and curl_multi_socket*() invocations.
Closes#18432
Detect via curlinfo if curl has verbose strings disabled, and skip
tests that require it.
Also:
- cmake: make pytests depend on curlinfo.
Cherry-picked from #18797Closes#18801
When attempts on all addresses have been started, do no longer set any
EXPIRE_HAPPY_EYEBALLS timeouts.
Fixes#18767
Reported-by: Johannes Schindelin
Closes#18768
Fail on errors from SSL_handle_events().
Force quit Caddy test instance that is left hanging longer with
openssl-quic tests for unknown reasons.
Reported in Joshua's sarif data
Closes#18720
Access the error codes of ngtcp2 when a connect attempt failes. Trace
the information for analysis. Treat errors as permanent failure by
default, trigger retrying only when the server refused without
indicating an error.
Closes#18521
Tweaks around handling of --limit-rate:
* tracing: trace outstanding timeouts by name
* multi: do not mark transfer as dirty that have
an EXPIRE_TOOFAST set
* multi: have one static function to asses speed limits
* multi: when setting EXPIRE_TOOFAST remove the transfers
from the dirty set
* progress: rename vars and comment on how speed limit
timeouts are calculated, for clarity
* transfer: when speed limiting, exit the receive loop
after a quarter of the limit has been received, not
on the first chunk received.
* cf-ip-happy.c: clear EXPIRE_HAPPY_EYEBALLS on connect
* scorecard: add --limit-rate parameter to test with
speed limits in effect
Closes#18454
Since the content varies during connection setup and while doing it
(eyeballing), remove these strcut from `connectdata` and replace use
with querying the connection filters. Those keep that information
already.
Change the info logging of established connections to also give the
local address and port.
Closes#17960
libtests and clients were built the same way after recent overhauls.
libtests are used by runtests, clients by pytests.
Merge clients into libtests, aligning their entry function signature,
dropping common utility functions, and simplifying the build.
Note: After this patch `CURLDEBUG` applies to cli tests, when enabled.
Also:
- lib552: drop local copy-paste debug callback in favor of testtrace.
- lib552: drop local copy-paste dump function in favor of testtrace.
- clients: use `long` for HTTP version, drop casts.
- clients: replace local dump function in favor of testrace clone.
- sync cli test entry function prototype with libtests'.
- h2_serverpush: replace local trace callback with testtrace.
- de-duplicate 3 websocket close, ping, ping, functions. Kept the pong
iteration from `ws_pingpong`. Note: the pong clone in `lib2304` was
returning an error when `curl_ws_recv()` returned non-zero and
the payload matched the expected one anyway. After this patch, this
case returns success, as it does in `ws_pingpong`.
`lib2304` keeps passing, but I'm not sure if the previous behavior
was intentional.
- display full value in websocket close, ping, pong, drop casts.
Closes#18079
- add startup check for 'danted' to avoid fails on low cpu
- rename 'sockd' to 'danted' everywhere to clarify what we use
- add proper defaults for 'danted' for debian
- install 'dante-server' in pytest ci runs
Closes#18075
Add a new commandline option --out-null that discards all
response bytes into the void. Replaces non-portable use of
'-o /dev/null' with more efficiency.
Feature added in 8.16.0
Closes#17800
Configure curl with `--with-test-sockd=<path to sockd>` for a locally
installed dante sockd server and new `test_40_*` will verify that
down- and uploads work via SOCKS.
Invoke scorecard.py with `--socks4` or `--socks5` to run performance
tests with SOCKS. Note that SOCKS is not supported for HTTP/3.
Ref: #17969Closes#17986
When CI is not able to give clients enough cpu, the default Apache
KeepAliveTimeout of 5 seconds may trigger and cause tests to fail.
Increase the timeout to 30 seconds for reliability.
Ref: #17963Closes#17968
Eliminating the socket readability check in the socket connection
filters for the 'data_pending' callback. Improves performance of
handling of transfers, up to ~30%, depending on parallelism and response
size.
Whatever `data_pending()` once was, its semantics are now:
"Is there anything buffered in the connection filters that needs
receive?"
Any checks of the socket's readability are done via `multi_wait()`
and friends.
Fix the one place in HTTP/1 proxy code that checked `data_pending()` and
did an early return if false. Remove that check and actually try to
receive data every time.
Closes#17785
To have all the tests binaries directly under the tests directory.
There seems to be no issue adding non-http test clients to this subdir.
Closes#17637
To make building the http client tests faster, with no duplication, by
using the build method that other test binaries already use.
The difference compared to other tests is that these don't use internal
libcurl headers or code. With the exception of `curl_config.h`, for
a feature macro.
Before this patch, these tests were built like examples.
Also:
- de-duplicate code and give unique names to colliding symbols.
- add local getopt implementation and enable all code for MSVC.
Adapted for curl via Public Domain source:
4e618ef782/getopt.h
Credits-to: Christopher Wellons
Thanks!
Closes#17627
