To include what's actually used.
Also:
- drop unused includes.
- scope includes where possible.
- drop `curlx/curlx.h` umbrella header.
- config2setopts: include `netinet/in.h` for Cygwin/MSYS2.
Previously included by chance via an unused curlx include.
Closes#20776
With default invocation, make generated file timestamps deterministic
by looking up (via the GitHub API) the last commit that modified
`certdata.txt`, along with its commit timestamp.
Also:
- show the URL used to download `certdata.txt` from.
- make `ca-bundle.crt` timestamp match `certdata.txt`'s.
Closes#20528
To allow simplifying the binutils ld hack, by chaining the original
imported target to curl's local duplicate target. Also to allow linking
to dependencies' native imported targets via their CMake Configs, which
will always be hooked up to a `CURL::` interface, and may also be
chained upstream.
Fixing (seen on Linux with simplified binutils hack via #20839):
```
Requires:
Requires.private: libzstd openssl zlib
Libs: -L${libdir} -lcurl
-Libs.private: -lcrypto -lssl -lz -lzstd
+Libs.private: -lOpenSSL::Crypto -lZLIB::ZLIB -lcrypto -lssl -lz -lzstd
Cflags: -I${includedir}
Cflags.private: -DCURL_STATICLIB
Error: Process completed with exit code
```
Ref: https://github.com/curl/curl/actions/runs/22768301699/job/66041980258?pr=20839
Note this makes it possible to run into an infinite loop because CMake
allows cyclic dependencies. It isn't added by curl's CMake script nor by
any dependencies as defined by default, but may happen in theory with
custom-created targets. In such case CMake automatically stops with
an error at 1000 iterations. I find it overkill to add custom protection
for it.
Cherry-picked from #20814
Cherry-picked from #20839Closes#20840
Fix potential inifinite loop reading file content with `Curl_get_line()`
when a filename passed via these options are pointing to a directory
entry (on non-Windows):
- `--alt-svc` / `CURLOPT_ALTSVC`
- `-b` / `--cookie` / `CURLOPT_COOKIEFILE`
- `--hsts` / `CURLOPT_HSTS`
- `--netrc-file` / `CURLOPT_NETRC_FILE`
Fix by checking for this condition and silently skipping such filename
without attempting to read content. Add test 1713 to verify.
Mention in cookie documentation as an accepted case, also show a verbose
message when a directory is detected. Extend test 46 to verify if such
failure lets the logic continue to the next cookie file.
Reported-and-based-on-patch-by: Richard Tollerton
Fixes#20823Closes#20826 (originally-based-on)
Follow-up to 769ccb4d42#19140Closes#20873
To aid debugging cases when dependency detection acts unexpectedly.
Sprung from spending days trying to figure out behavior of ngtcp2 crypto
modules and their dependencies.
You can enable by setting env `CURL_TRACE_PKG_CONFIG` to a non-empty
value. When enabled, details are logged for both successful and
unsuccessful detections. Logging of unsuccessful ones is automatically
enabled when `CURL_CI` env is set, which is the case for all CI jobs.
It works by asking for `--debug` output and grepping for lines that seem
useful for this purpose. Output is different for classic pkg-config and
pkgconf, and may depending on tool version. Also append `--print-errors`
output if any.
Examples (with pkgconf):
Fail, before:
```
checking for libngtcp2_crypto_boringssl options with pkg-config... no
configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_boringssl pkg-config file.
```
Fail, after:
```
checking for libngtcp2_crypto_boringssl options with pkg-config... no
configure: pkg-config --exists libngtcp2_crypto_boringssl trace:
---- begin
trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2_crypto_boringssl
trying path: /home/runner/ngtcp2-boringssl/build/lib/pkgconfig for libngtcp2_crypto_boringssl
trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2
trying path: /home/runner/ngtcp2-boringssl/build/lib/pkgconfig for libngtcp2
trying path: /home/runner/nghttp3/build/lib/pkgconfig for openssl
trying path: /home/runner/ngtcp2-boringssl/build/lib/pkgconfig for openssl
trying path: /home/runner/nghttp2/build/lib/pkgconfig for openssl
==== error:
Package openssl was not found in the pkg-config search path.
Perhaps you should add the directory containing `openssl.pc'
to the PKG_CONFIG_PATH environment variable
Package 'openssl', required by 'libngtcp2_crypto_boringssl', not found
---- end
configure: error: --with-ngtcp2 was specified but could not find ngtcp2_crypto_boringssl pkg-config file.
```
Success, after:
```
checking for libngtcp2_crypto_boringssl options with pkg-config... found
configure: pkg-config --exists libngtcp2_crypto_boringssl trace:
---- begin
trying path: /home/runner/awslc/build/lib/pkgconfig for libngtcp2_crypto_boringssl
trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2_crypto_boringssl
trying path: /home/runner/nghttp2/build/lib/pkgconfig for libngtcp2_crypto_boringssl
trying path: /home/runner/ngtcp2/build/lib/pkgconfig for libngtcp2_crypto_boringssl
trying path: /home/runner/awslc/build/lib/pkgconfig for libngtcp2
trying path: /home/runner/nghttp3/build/lib/pkgconfig for libngtcp2
trying path: /home/runner/nghttp2/build/lib/pkgconfig for libngtcp2
trying path: /home/runner/ngtcp2/build/lib/pkgconfig for libngtcp2
trying path: /home/runner/awslc/build/lib/pkgconfig for openssl
trying path: /home/runner/awslc/build/lib/pkgconfig for libssl
trying path: /home/runner/awslc/build/lib/pkgconfig for libcrypto
---- end
```
More examples:
https://github.com/curl/curl/pull/20926#issuecomment-4064259935
If there is an externally enablable, built-in feature like this in
classic pkg-config or pkgconf, I could not find it.
Also:
- GHA/http3-linux: set `CURL_TRACE_PKG_CONFIG` to log detection details.
H3 builds are prone to hard-to-debug dependency issues.
Ref: #20920
Follow-up to 3c64ffaff4#18415#18188
Follow-up to 99500660af#18028#18022
Cherry-picked from #20926Closes#20931
Add protocol.h and protocol.c containing all about libcurl's
known URI schemes and their protocol handlers (so they exist).
Moves the scheme definitions from the various sources files into
protocol.c. Schemes are known and used, even of the protocol
handler is not build or just not implemented at all.
Closes#20906
Improve the name, type and handling of `data->req.keepon`:
- Rename `keepon` to `io_flags`
- make `io_flags` and `uint8_t` and reposition in struct
- Rename `KEEP_*` defines to `REQ_IO_*`, move to request.h
- Replace all direct bit tests to `CURL_REQ_WANT_*` use
- Replace all direct bit manipulations with new macros
Closes#20905
Convert more `int port` to `uint16_t` port types. Reshuffle ports in
connectdata to save some bytes. Change `conn->destination` format to
- make it more readable and thus usable in tracing
- add the IPv6 scope_id only when not default (global)
and make it resemble more the textual format for IPv6
(e.g. suffix '%<scope_id>')
Closes#20918
Eliminate `conn->bits.ipv6_ip`
The bit was only correct for the first transfer using a connection. Use
`data->state.up.hostname` instead in places that need the URL hostname
in its original form.
Fix parseurlandfillconn() to not modify `data->state.up.hostname` before
copying the connection's hostname, but modify the copy instead, leaving
the URL hostname intact.
Closes#20919
The file is almost entirely made up by first-lines of previous git
commits, and we usually push it without a PR cycle, making it annoying
to trigger on typos later as they then show in independent PRs by other
people.
Closes#20917
- when scanning source code, this now only checks source code comments
and double-quote strings. No more finding bad words as part of code
- this allows the full scan to be done in a single invocation
- detects source code or markdown by file name extension
- moved the whitelist words config into the single `badwords.txt` file,
no more having them separately (see top of file for syntax)
- all whitelisted words are checked case insensitively now
- removed support for whitelisting words on a specific line number. We
did not use it and it is too fragile
Removing the actual code from getting scanned made the script take an
additional 0.5 seconds on my machine.
Scanning 1525 files now takes a little under 1.7 seconds for me.
Closes#20909
- tool_getparam: revert an unnecessary/no-op C89 warning silencer.
Follow-up to 09c9afdd71#20363
- tool_writeout: add comment saying silencing is a no-op for llvm/clang.
For `strftime()` it is a GCC-specific, as of llvm/clang v22.1.0.
Follow-up to f07a98ae11#20366
- unit1652: drop always-false `!defined(__clang__)` guard.
Pointed-out-by: Orgad Shaneh
Ref: #20902
Follow-up to 7e814c8717#16062
- unit1652: document that `-Wformat` is necessary for GCC v5 to v8.
Follow-up to 71cf0d1fca#14772Closes#20908
In a -j192 build, this output used a three-digit number for the output,
thus wrapping differently and causing it to error.
Reported-by: Carlos Henrique Lima Melara
Closes#20910
To help understanding what's happening on systems where native CA misses
to verify legitimate public websites.
Also:
- drop a superfluous, hanging, `else`.
Ref: #20897Closes#20899
As the struct is now always unconditionally memset with zeros, we
can remove two zero assigns.
Follow-up to 015f1c7de4
Pointed out by CodeSonar
Closes#20900
Follow-up to 04289c62de. Regression shipped in 8.13.0.
- a logic error made it not loop and thus only match if the searched string
was first
- it no longer matches a substring
Adjusted test 1 to use multiple values in the Connection: response
header. Adjusted test 1542 to have a "Connection: close-not" which
should not match.
Reported-by: Henrique Pereira
Closes#20894
Also:
- support per-directory and per-upper-directory whitelist entries.
- convert badlist input grep tweak into the above format.
(except for 'And' which had just a few hits.)
- fix many code exceptions, but do not enforce.
(there also remain about 350 'will' uses in lib)
- fix badwords in example code, drop exceptions.
- badwords-all: convert to Perl.
To make it usable from CMake.
- FAQ: reword to not use 'will'. Drop exception.
Closes#20886
A logic error made the function not check the last character, which thus
could make it accept invalid schemes.
Added test 1965 to verify
Reported-by: Otis Cui Lei
Closes#20893
- Move `RESP_TIMEOUT` from urldata.h to pingpong.h as
`PINGPONG_TIMEOUT_MS`.
- Rename `Curl_pp_state_timeout()` to `Curl_pp_state_timeleft_ms()` as
the function returns the time left, not the timout..
- Update implementation comments and variable names
Closes#20888
The protocol handler method `connection_check` allowed to variable
operations to trigger with variable result bits. Only the `CONNCHECK_ISDEAD`
and `CONNRESULT_DEAD` were in use. Transform the function into
`connection_is_dead` without extra parameter and a bool result.
- Remove defines for `CONNCHECK_*` and `CONNRESULT_*`
- Rename protocol function in handler comments
- Change RTSP implementation (only protocol that uses this)
Closes#20890
On Solaris this was causing intermittent issues when the private
structure member __sin6_src_id had unexpectedly some value. connect(2)
would then fail with EADDRNOTAVAIL.
Closes#20885
Previously it lacked the actual return. libssh.c uses the same function
name.
Verified by test 2007.
Reported-by: m777m0 on hackerone
Follow-up to 578706addeCloses#20883
