Commit Graph

425 Commits

Author SHA1 Message Date
Daniel Stenberg
c37f4b6ac3
inet_ntop: rename curlx_inet_ntop to Curl_inet_ntop
It is not part of the curlx club.

Closes #17313
2025-05-12 07:42:59 +02:00
Daniel Stenberg
d4dd43b20d
curlx: move curlx_inet_pton
Used by test server code.

Closes #17300
2025-05-09 13:45:24 +02:00
Daniel Stenberg
255aac56f9
curlx: move into to curlx/
Move curlx_ functions into its own subdir.

The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.

The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.

The strcase defines are not curlx_ functions and should not be used by
tool or server code.

dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.

When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.

Assisted-by: Jay Satiro

Closes #17253
2025-05-07 11:01:15 +02:00
Daniel Stenberg
f142056e01
hostip: fix build when no async resolvers are present
Fixes #17124
Closes #17125
2025-04-22 09:18:40 +02:00
Daniel Stenberg
1609570c68
hostip: remove duplicate assignment
Follow-up from 56e40ae6a5

Pointed out by CodeSonar

Closes #17080
2025-04-17 09:12:29 +02:00
Stefan Eissing
1ebd92d0fd
async: DoH improvements
Adds a "meta_hash" to each easy handle for keeping special data during
operations. All meta data set needs to add its destructor callback, so
that meta data gets destroyed properly when the easy handle is cleaned
up or reset.

Add data->master_mid for "sub" transfers that belong to a "master" easy
handle. When a "sub" transfer is done, the corresponding "master" can
add a callback to be invoked. Used in DoH name resolution.

DoH: use easy meta hash to add internal structs for DoH name resolution.
One in each in each probe easy handle. When probes are done, response
data is copied from the probe to the initiating easy.

This allows DoH using transfers and their probes to be cleaned up in any
sequence correctly.

Fold DoH cleanup into the Curl_async_shutdown() and Curl_async_destroy()
functions.

Closes #16384
2025-04-16 16:06:03 +02:00
Stefan Eissing
56e40ae6a5
asyn resolver code improvements
"asyn" is the internal name under which both c-ares and threaded
resolver operate. Make the naming more consistent. Implement the c-ares
resolver in `asyn-ares.*` and the threaded resolver in `asyn-thrdd.*`.
The common functions are in `asyn-base.c`.

When `CURLRES_ASYNCH` is defined, either of the two is used and
`data->state.async` exists. Members of that struct vary for the selected
implementation, but have the fields `hostname`, `port` and `ip_version`
always present. This are populated when the async resolving starts and
eliminate the need to pass them again when checking on the status and
processing the results of the resolving.

Add a `Curl_resolv_blocking()` to `hostip.h` that relieves FTP and SOCKS
from having to repeat the same code.

`Curl_resolv_check()` remains the function to check for status of
ongoing resolving. Now it also performs internally the check if the
needed DNS entry exists in the dnscache and if so, aborts the asnyc
operation. (libcurl right now does not check for duplicate resolve
attempts. an area for future improvements).

The number of functions in `asyn.h` has been reduced. There were subtle
difference in "cancel()" and "kill()" calls, both replaced by
`Curl_async_shutdown()` now. This changes behaviour for threaded
resolver insofar as the resolving thread is now always joined unless
`data->set.quick_exit` is set. Before this was only done on some code
paths. A future improvement would be a thread pool that keeps a limit
and also could handle joins more gracefully.

DoH, not previously tagged under "asny", has its struct `doh_probes` now
also in `data->state.async`, moved there from `data->req` because it
makes more sense. Further integration of DoH underneath the "asyn"
umbrella seems like a good idea.

Closes #16963
2025-04-16 09:34:20 +02:00
Daniel Stenberg
db3e7a24b5
hostip: show the correct name on proxy resolve error
Regression, probably from 8ded8e5f3f (#16451)

Fixes #16958
Reported-by: Jean-Christophe Amiel
Closes #16961
2025-04-04 14:23:01 +02:00
Stefan Eissing
01e76702ac
dnscache: slight refactoring
Slight refactoring around dnscache, e.g. hostcache

- eliminate `data->state.hostcache`. Always look up
  relevant dnscache at share/multi.
- unify naming to "dnscache", replacing "hostcache"
- use `struct Curl_dnscache`, even though it just
  contains a `Curl_hash` for now.
- add `Curl_dnscache_destroy()` for cleanup in
  share/multi.

Closes #16941
2025-04-03 16:40:56 +02:00
Daniel Stenberg
8f496d05b6
hostip: fix build without threaded-resolver and without DoH
Closes #16938
2025-04-03 11:52:37 +02:00
Viktor Szakats
ee73d553ed
build: replace Curl_ prefix with curlx_ for functions used in servers
Closes #16689
2025-03-13 00:03:15 +01:00
Daniel Stenberg
75acda70aa
hostip: don't use alarm() for DoH resolves
When built to use the synch resolver and DoH is used for a transfer, do
not use alarm() for timeout since DoH resolving is not blocking.

Closes #16649
2025-03-10 13:17:18 +01:00
Daniel Stenberg
3c2948e0a6
lib: clear up CURLRES_ASYNCH vs USE_CURL_ASYNC use
CURLRES_ASYNCH - is for when built to use an async name resolver; threaded or
c-ares

USE_CURL_ASYNC - is for when built to use either an async name resolver OR DoH

Reported-by: Lars Karlitski
Fixes #16645
Closes #16648
2025-03-10 13:15:20 +01:00
Stefan Eissing
df672695e5
shutdowns: split shutdown handling from connection pool
Further testing with timeouts in event based processing revealed that
our current shutdown handling in the connection pool was not clear
enough. Graceful shutdowns can only happen inside a multi handle and it
was confusing to track in the code which situation actually applies. It
seems better to split the shutdown handling off and have that code
always be part of a multi handle.

Add `cshutdn.[ch]` with its own struct to maintain connections being
shut down. A `cshutdn` always belongs to a multi handle and uses that
for socket/timeout monitoring.

The `cpool`, which can be part of a multi or share, either passes
connections to a `cshutdn` or terminates them with a one-time, best
effort.

Add an `admin` easy handle to each multi and share. This is used to
perform all maintenance operations where no "real" easy handle is
available. This solves the problem that the multi admin handle requires
some additional initialisation (e.g. timeout list).

The share needs its admin handle as it is often cleaned up when no other
transfer or multi handle exists any more. But we need a `data` in almost
every call.

Fix file:// handling of errors when adding a new connection to the pool.

Changes in `curl` itself:

- for parallel transfers, do not set a connection pool in the share,
  rely on the multi's connection pool instead. While not a requirement
  for the new `cshutdn` to work, this is

  a) helpful in testing to trigger graceful shutdowns
  b) a broader code coverage of libcurl via the curl tool

- on test_event with uv, cleanup the multi handle before returning from
  parallel_event(). The uv struct is on the stack, cleanup of the multi
  later will crash when it tries to register sockets. This is a "eat
  your own dogfood" related fix.

Closes #16508
2025-03-02 11:13:34 +01:00
Daniel Stenberg
8ded8e5f3f
urldata: remove 'hostname' from struct Curl_async
It is unnecessary duplicated information, as the host name being
resolved is already present in conn->host.

Closes #16451
2025-02-27 17:18:43 +01:00
Viktor Szakats
2a292c3984
build: add Windows CE / CeGCC support, with CI jobs
Make it possible to build curl for Windows CE using the CeGCC toolchain.
With both CMake and autotools, including tests and examples, also in CI.
The build configuration is the default one with Schannel enabled. No
3rd-party dependencies have been tested.

Also revive old code to make Schannel build with Windows CE, including
certificate verification.

Builds have been throughougly tested. But, I've made no functional tests
for this PR. Some parts (esp. file operations, like truncate and seek)
are stubbed out and likely broken as a result. Test servers build, but
they do not work on Windows CE. This patch substitutes `fstat()` calls
with `stat()`, which operate on filenames, not file handles. This may or
may not work and/or may not be secure.

About CeGCC: I used the latest available macOS binary build v0.59.1
r1397 from 2009, in native `mingw32ce` build mode. CeGCC is in effect
MinGW + GCC 4.4.0 + old/classic-mingw Windows headers. It targets
Windows CE v3.0 according to its `_WIN32_WCE` value. It means this PR
restores portions of old/classic-mingw support. It makes the Windows CE
codepath compatible with GCC 4.4.0. It also adds workaround for CMake,
which cannot identify and configure this toolchain out of the box.

Notes:
- CMake doesn't recognize CeGCC/mingw32ce, necessitating tricks as seen
  with Amiga and MS-DOS.
- CMake doesn't set `MINGW` for mingw32ce. Set it and `MINGW32CE`
  manually as a helper variable, in addition to `WINCE` which CMake sets
  based on `CMAKE_SYSTEM_NAME`.
- CMake fails to create an implib for `libcurl.dll`, due to not
  recognizing the platform as a Windowsy one. This patch adds the
  necessary workaround to make it work.
- headers shipping with CeGCC miss some things curl needs for Schannel
  support. Fixed by restoring and renovating code previously deleted
  old-mingw code.
- it's sometime non-trivial to figure out if a fallout is WinCE,
  mingw32ce, old-mingw, or GCC version-specific.
- WinCE is always Unicode. With exceptions: no `wmain`,
  `GetProcAddress()`.
- `_fileno()` is said to convert from `FILE *` to `void *` which is
  a Win32 file `HANDLE`. (This patch doesn't use this, but with further
  effort it probably could be.)
  https://stackoverflow.com/questions/3989545/how-do-i-get-the-file-handle-from-the-fopen-file-structure
- WinCE has no signals, current directory, stdio/CRT file handles, no
  `_get_osfhandle()`, no `errno`, no `errno.h`. Some of this stuff is
  standard C89, yet missing from this platform. Microsoft expects
  Windows CE apps to use Win32 file API and `FILE *` exclusively.
- revived CeGCC here (not tested for this PR):
  https://building.enlyze.com/posts/a-new-windows-ce-x86-compiler-in-2024/

On `UNDER_CE` vs. `_WIN32_WCE`: (This patch settled on `UNDER_CE`)

- A custom VS2008 WinCE toolchain does not set any of these.
  The compiler binaries don't contain these strings, and has no compiler
  option for targeting WinCE, hinting that a vanilla toolchain isn't
  setting any of them either.
- `UNDER_CE` is automatically defined by the CeGCC compiler.
  https://cegcc.sourceforge.net/docs/details.html
- `UNDER_CE` is similar to `_WIN32`, except it's not set automatically
  by all compilers. It's not supposed to have any value, like a version.
  (Though e.g. OpenSSL sets it to a version)
- `_WIN32_WCE` is the CE counterpart of the non-CE `_WIN32_WINNT` macro.
  That does return the targeted Windows CE version.
- `_WIN32_WCE` is not defined by compilers, and relies on a header
  setting it to a default, or the build to set it to the desired target
  version. This is also how `_WIN32_WINNT` works.
- `_WIN32_WCE` default is set by `windef.h` in CeGCC.
- `_WIN32_WCE` isn't set to a default by MSVC Windows CE headers (the
  ones I checked at least).
- CMake sets `_WIN32_WCE=<ver>`, `UNDER_CE`, `WINCE` for MSVC WinCE.
- `_WIN32_WCE` seems more popular in other projects, including CeGCC
  itself. `zlib` is a notable exception amongst curl dependencies,
  which uses `UNDER_CE`.
- Since `_WIN32_WCE` needs "certain" headers to have it defined, it's
  undefined depending on headers included beforehand.
- `curl/curl.h` re-uses `_WIN32_WCE`'s as a self-guard, relying on
  its not-(necessarily)-defined-by-default property:
  25b445e479/include/curl/curl.h (L77)

Toolchain downloads:
- Windows:
  https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_cygwin1.7_r1399.tar.bz2
- macOS Intel:
  https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_snowleopard_r1397.tar.bz2

Closes #15975
2025-02-21 13:56:34 +01:00
Daniel Stenberg
f0d7318193
strparse: provide access functions
To access the string and the length without having to directly use the
struct field names. Gives more freedom, flexbility and keeps
implementation specifics out of users' code.

Closes #16386
2025-02-19 12:17:32 +01:00
Stefan Eissing
1b710381ca
https-rr: implementation improvements
- fold DoH and async HTTPS-RR handling into common code.
  have common cleanups, etc. Have a CURLcode result in async
  handling to allow HTTPS RR parsing to fail.
- keep target, ipv4hints, ipv6hints, port and echconfig also
  when resolving via cares. We need to know `target` and `port`
  when evaluating possible ALPN candidates to not go astray.
- add CURL_TRC_DNS for tracing DNS operations
- replace DoH specific tracing with DNS, use doh as alias
  for dns in curl_global_tracea()

Closes #16132
2025-02-18 16:12:26 +01:00
Daniel Stenberg
2f4dc6525c
hostip: make CURLOPT_RESOLVE support replacing IPv6 addresses
This also applies to --resolve of course.

Applied strparse functions on the function.

Fixes #16357
Reported-by: rmg-x on github
Closes #16358
Assisted-by: Jay Satiro
2025-02-18 08:55:56 +01:00
Daniel Stenberg
b4538ec522
strparse: switch to curl_off_t as base data type
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
  between platforms

Extended the unit test 1664 to verify hex and octal parsing.

Closes #16336
2025-02-15 21:58:48 +01:00
Daniel Stenberg
b696fc129b
lib: use Curl_str_number() for parsing decimal numbers
Instead of strtoul() and strtol() calls.

Easier API with better integer overflow detection and built-in max check
that now comes automatic everywhere this is used.

Closes #16319
2025-02-14 10:38:56 +01:00
Daniel Stenberg
bb93536270
doh: send HTTPS RR requests for all HTTP(S) transfers
When enabled in the build.

Update test2100: verify with HTTPS RR included

Adjust runtests and server/disabled.c to include "HTTPSRR" as a feature
in the test suite.

Also, decode the ALPN list in HTTPS records straight into IDs. There's
no point in storing everything in string format. Skip ALPNs we do not
support.

Closes #16007
2025-01-15 12:55:15 +01:00
Daniel Stenberg
5d70a5c5a4
doh: cleanups and extended HTTPS RR code
In preparation for using HTTPS outside of ECH, the parser now also
extracts the port number.

Plus other minor cleanups.

Closes #16007
2025-01-15 12:55:04 +01:00
Viktor Szakats
5474d70c3e
tidy-up: drop parenthesis around return expression
Closes #15990
2025-01-14 12:11:42 +01:00
Viktor Szakats
a3585c9576
Makefile.mk: drop in favour of autotools and cmake (MS-DOS, AmigaOS3)
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.

This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.

`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.

Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.

Details:

- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
  - default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
  - add support for `WATT_ROOT`.
  - use static libcurl with MS-DOS.
  - fixup default CMake suffixes/prefixes for DJGPP.
  - disable hidden symbols for MS-DOS. Not supported on MS-DOS.
  - opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
  - default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
  `Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
  Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
  - configure: detect `CloseSocket()` when it's a macro.
  - configure: fix `IoctlSocket` detection on AmigaOS.
  - curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
  - cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
    `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
  - set necessary system libs.
  - add AmiSSL support.
  - inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
    and they did not compile with AmigaOS.
  - cmake: better sync `gethostname` detection with autotools.
    Fixes detection for AmigaOS, where `gethostname` is a macro.
  - cmake: fix `sys/utime.h` detection on AmigaOS.
  - cmake: force-disable `getaddrinfo` for AmigaOS.
  - cmake: tweak threading and static/shared default for AmigaOS.
  - cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
  Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
  `zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
  `ioctl` is also detected, but fails when used. Make the above override
  it for a successful build.
  Authored-by: Darren Banfi
  Fixes #15537
  Closes #15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
  Overriding the default 4096.
  Suggested-by: Darren Banfi
  Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
  Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.

Non-MS-DOS/AmigeOS-specific tidy-ups:

- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
  configurations/requirements.
  This allows to honor global requirements added earlier.
  Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
  No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.

Follow-up to a8861b6ccd #9764

Closes #15543
2024-12-16 23:20:55 +01:00
Jay Satiro
a4458c7ee3 hostip: don't use the resolver for FQDN localhost
- Treat `[<any>.]localhost.` (ie localhost with FQDN period terminator)
  as fixed value 127.0.0.1 and ::1 instead of querying the resolver.

Prior to this change, b5c0fe20 (precedes 7.85.0) did the same for
non-FQDN `<any>.localhost`.

Prior to this change, 1a0ebf66 (precedes 7.78.0) did the same for
non-FQDN `localhost`.

Ref: https://github.com/curl/curl/issues/15628#issuecomment-2515540315

Closes https://github.com/curl/curl/pull/15676
2024-12-05 17:16:34 -05:00
Daniel Stenberg
cd2b45201a
src/lib: remove redundant ternary operators
Closes #15435
2024-10-29 08:18:30 +01:00
Daniel Stenberg
bcec0840b0
lib: use bool/TRUE/FALSE properly
booleans should use the type 'bool' and set the value to TRUE/FALSE

non-booleans should not be 'bool' and should not set the value to
TRUE/FALSE

Closes #15123
2024-10-03 09:31:56 +02:00
Daniel Stenberg
fbf5d507ce
lib/src: white space edits to comply better with code style
... as checksrc now finds and complains about these.

Closes #14921
2024-09-19 14:59:12 +02:00
Gabriel Marin
5a263710f6
lib, src, tests: added space around ternary expressions
Closes #14912
2024-09-18 15:27:26 +02:00
Stefan Eissing
1be704e17e
cpool: rename "connection cache/conncache" to "Connection Pools/cpool"
This is a better match for what they do and the general "cpool"
var/function prefix works well.

The pool now handles very long hostnames correctly.

The following changes have been made:

* 'struct connectdata', e.g. connections, keep new members
  named `destination` and ' destination_len' that fully specifies
  interface+port+hostname of where the connection is going to.
  This is used in the pool for "bundling" of connections with
  the same destination. There is no limit on the length any more.
* Locking: all locks are done inside conncache.c when calling
  into the pool and released on return. This eliminates hazards
  of the callers keeping track.
* 'struct connectbundle' is now internal to the pool. It is no
  longer referenced by a connection.
* 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters
  no longer need to set it. Instead, the multi checks on leaving
  MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now
  multiplexed and new, e.g. not conn->bits.reuse. In that case
  the processing of pending handles is triggered.
* The pool's init is provided with a callback to invoke on all
  connections being discarded. This allows the cleanups in
  `Curl_disconnect` to run, wherever it is decided to retire
  a connection.
* Several pool operations can now be fully done with one call.
  Pruning dead connections, upkeep and checks on pool limits
  can now directly discard connections and need no longer return
  those to the caller for doing that (as we have now the callback
  described above).
* Finding a connection for reuse is now done via `Curl_cpool_find()`
  and the caller provides callbacks to evaluate the connection
  candidates.
* The 'Curl_cpool_check_limits()' now directly uses the max values
  that may be set in the transfer's multi. No need to pass them
  around. Curl_multi_max_host_connections() and
  Curl_multi_max_total_connections() are gone.
* Add method 'Curl_node_llist()' to get the llist a node is in.
  Used in cpool to verify connection are indeed in the list (or
  not in any list) as they need to.

I left the conncache.[ch] as is for now and also did not touch the
documentation. If we update that outside the feature window, we can
do this in a separate PR.

Multi-thread safety is not achieved by this PR, but since more details
on how pools operate are now "internal" it is a better starting
point to go for this in the future.

Closes #14662
2024-08-28 13:52:49 +02:00
Daniel Stenberg
c0233a35da
hash: provide asserts to verify API use
- converted the Curl_hash_count() macro to a function

- Discourage accessing struct fields directly

- Document the internal API in HASH.md

Closes #14503
2024-08-15 08:54:19 +02:00
Viktor Szakats
f81f351b9a
tidy-up: OS names
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.

Mostly OS names and a few more.

Also a couple of other minor text fixups.

Closes #14360
2024-08-04 19:17:45 +02:00
Stefan Eissing
5a9262a333
url: dns_entry related improvements
Replace Curl_resolv_unlock() with Curl_resolv_unlink():

-replace inuse member with refcount in Curl_dns_entry

- pass Curl_dns_entry ** to unlink, so it gets always cleared

- solve potential (but unlikley) UAF in FTP's handling of looked up
  Curl_dns_entry. Esp. do not use addr information after unlinking an entry.
  In reality, the unlink will not free memory, as the dns entry is still
  referenced by the hostcache. But this is not safe and relying on no other
  code pruning the cache in the meantime.

- pass permanent flag when adding a dns entry instead of fixing timestamp
  afterwards.

url.c: fold several static *resolve_* functions into one.

Closes #14195
2024-08-03 19:51:02 +02:00
CMD
c2e427cc93
hostip: skip error check for infallible function call
Closes #14147
2024-07-10 16:14:06 +02:00
Daniel Stenberg
c074ba64a8
code: language cleanup in comments
Based on the standards and guidelines we use for our documentation.

 - expand contractions (they're => they are etc)
 - host name = > hostname
 - file name => filename
 - user name = username
 - man page => manpage
 - run-time => runtime
 - set-up => setup
 - back-end => backend
 - a HTTP => an HTTP
 - Two spaces after a period => one space after period

Closes #14073
2024-07-01 22:58:55 +02:00
Viktor Szakats
72abf7c13a
lib: tidy up types and casts
Cherry-picked from #13489
Closes #13862
2024-06-05 14:02:39 +02:00
Viktor Szakats
d0728c9109
lib: fix compiler warnings (gcc)
Seen when setting `ENABLE_DEBUG=ON` and `-DDEBUGBUILD` for mingw-w64
gcc 13.2.0 CMake unity builds in 'Release' configurations.

```
curl/lib/curl_gethostname.c:71:5: error: 'strncpy' specified bound 1025 equals destination size [-Werror=stringop-truncation]
   71 |     strncpy(name, force_hostname, namelen);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:175:
In function 'hostcache_timestamp_remove',
    inlined from 'Curl_hash_clean_with_criterium' at curl/lib/hash.c:265:19,
    inlined from 'Curl_hash_clean_with_criterium' at curl/lib/hash.c:247:1,
    inlined from 'hostcache_prune' at curl/lib/hostip.c:228:3,
    inlined from 'Curl_hostcache_prune' at curl/lib/hostip.c:256:21:
curl/lib/hostip.c:205:12: error: 'now' may be used uninitialized [-Werror=maybe-uninitialized]
  205 |     time_t age = prune->now - c->timestamp;
      |            ^~~
curl/lib/hostip.c: In function 'Curl_hostcache_prune':
curl/lib/hostip.c:241:10: note: 'now' was declared here
  241 |   time_t now;
      |          ^~~
In function 'hostcache_timestamp_remove',
    inlined from 'fetch_addr' at curl/lib/hostip.c:310:8:
curl/lib/hostip.c:205:23: error: 'user.now' may be used uninitialized [-Werror=maybe-uninitialized]
  205 |     time_t age = prune->now - c->timestamp;
      |                  ~~~~~^~~~~
curl/lib/hostip.c: In function 'fetch_addr':
curl/lib/hostip.c:304:33: note: 'user' declared here
  304 |     struct hostcache_prune_data user;
      |                                 ^~~~
In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:40:
curl/lib/cf-socket.c: In function 'cf_socket_send':
curl/lib/cf-socket.c:1294:10: error: 'c' may be used uninitialized [-Werror=maybe-uninitialized]
 1294 |     if(c >= ((100-ctx->wblock_percent)*256/100)) {
      |        ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
curl/lib/cf-socket.c:1292:19: note: 'c' was declared here
 1292 |     unsigned char c;
      |                   ^
In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity_0_c.c:364:
In function 'tftp_state_timeout',
    inlined from 'tftp_multi_statemach' at curl/lib/tftp.c:1230:27:
curl/lib/tftp.c:1208:5: error: 'current' may be used uninitialized [-Werror=maybe-uninitialized]
 1208 |   if(current > state->rx_time + state->retry_time) {
      |     ^
curl/lib/tftp.c: In function 'tftp_multi_statemach':
curl/lib/tftp.c:1192:10: note: 'current' was declared here
 1192 |   time_t current;
      |          ^~~~~~~
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49792835/job/91c8dj5qb36spfe0#L112
Ref: https://github.com/curl/curl/actions/runs/9082968838/job/24960616145#step:12:62

Ref: #13592
Closes #13643
2024-05-17 12:00:19 +02:00
Daniel Stenberg
aef369867f
lib: call Curl_strntolower instead of doing crafted loops
Closes #13627
2024-05-14 08:00:19 +02:00
Viktor Szakats
a35bbe8977
lib: bump hash sizes to size_t
Follow-up to cc907e80a2 #13502
Cherry-picked from #13489
Closes #13601
2024-05-12 18:53:09 +02:00
Stephen Farrell
a362962b72
TLS: add support for ECH (Encrypted Client Hello)
An EXPERIMENTAL feature used with CURLOPT_ECH and --ech.

Closes #11922
2024-04-16 08:10:53 +02:00
Viktor Szakats
e411c98f70
build: prefer USE_IPV6 macro internally (was: ENABLE_IPV6)
Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use
the latter for consistency with other similar switches.

`-DENABLE_IPV6` remains accepted for compatibility as a synonym for
`-DUSE_IPV6`, when passed to the compiler.

`ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc`
options to control this feature.

Closes #13349
2024-04-13 08:33:26 +00:00
RainRat
2cd78f525c misc: Fix typos in docs and lib
This fixes miscellaneous typos and duplicated words in the docs, lib
and test comments and a few user facing errorstrings.

Author: RainRat on Github
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Dan Fandrich <dan@coneharvesters.com>
Closes: #13019
2024-03-01 09:59:48 +01:00
Daniel Stenberg
907eea0804
Revert "urldata: move async resolver state from easy handle to connectdata"
This reverts commit 56a4db2e4e (#12198)

We want the c-ares channel to be held in the easy handle, not per
connection - for performance.

Closes #12524
2023-12-15 12:57:35 +01:00
Daniel Stenberg
907dce2dc0
hostip: return error immediately when Curl_ip2addr() fails
Closes #12522
2023-12-14 22:57:28 +01:00
Viktor Szakats
84338c4de2
build: add more picky warnings and fix them
Enable more picky compiler warnings. I've found these options in the
nghttp3 project when implementing the CMake quick picky warning
functionality for it [1].

`-Wunused-macros` was too noisy to keep around, but fixed a few issues
it revealed while testing.

- autotools: reflect the more precisely-versioned clang warnings.
  Follow-up to 033f8e2a08 #12324
- autotools: sync between clang and gcc the way we set `no-multichar`.
- autotools: avoid setting `-Wstrict-aliasing=3` twice.
- autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2].
  It triggers in libtool-generated stub code.

- lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch.

- lib/curl_setup.h: delete duplicate declaration for `fileno`.
  Added in initial commit ae1912cb0d
  (1999-12-29). This suggests this may not be needed anymore, but if
  it does, we may restore this for those specific (non-Windows) systems.
- lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since
  c1d6fe2aaa.
- lib: delete unused macro `isxdigit_ascii` since
  f65f750742.
- lib/mqtt: delete unused macro `MQTT_HEADER_LEN`.
- lib/multi: delete unused macro `SH_READ`/`SH_WRITE`.
- lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN`
  macro.
- lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`.
- lib/rand: fix `-Wunreachable-code` and related fallouts [3].
- lib/setopt: fix `-Wunreachable-code-break`.
- lib/system_win32 and lib/timeval: fix double declarations for
  `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4].
- lib/warnless: fix double declarations in CMake UNITY mode [5].
  This was due to force-disabling the header guard of `warnless.h` to
  to reapply it to source code coming after `warnless.c` in UNITY
  builds. This reapplied declarations too, causing the warnings.
  Solved by adding a header guard for the lines that actually need
  to be reapplied.
- lib/vauth/digest: fix `-Wunreachable-code-break` [6].
- lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant
  block.
- lib/vtls/sectransp: fix `-Wunreachable-code-break` [7].
- lib/vtls/sectransp: suppress `-Wunreachable-code`.
  Detected in `else` branches of dynamic feature checks, with results
  known at compile-time, e.g.
  ```c
  if(SecCertificateCopySubjectSummary)  /* -> true */
  ```
  Likely fixable as a separate micro-project, but given SecureTransport
  is deprecated anyway, let's just silence these locally.
- src/tool_help: delete duplicate declaration for `helptext`.
- src/tool_xattr: fix `-Wunreachable-code`.
- tests: delete duplicate declaration for `unitfail` [8].
- tests: delete duplicate declaration for `strncasecompare`.
- tests/libtest: delete duplicate declaration for `gethostname`.
  Originally added in 687df5c8c3
  (2010-08-02).
  Got complicated later: c49e9683b8
  If there are still systems around with warnings, we may restore the
  prototype, but limited for those systems.
- tests/lib2305: delete duplicate declaration for
  `libtest_debug_config`.
- tests/h2-download: fix `-Wunreachable-code-break`.

[1] a70edb08e9/cmake/PickyWarningsC.cmake
[2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjauiqla5fj45?fullLog=true#L1675
[3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=12331#step:7:72
    https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=12331#step:7:109
[4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L204
[5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrriklpf1ut#L218
[6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=12331#step:7:290
[7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=12331#step:9:1193
[8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=12331#step:33:1870

Closes #12331
2023-11-21 16:35:42 +00:00
Sam James
bc8509a748
misc: fix -Walloc-size warnings
GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ for type ‘struct per_transfer’ with size ‘480’ [-Walloc-size]
  213 |   p = calloc(sizeof(struct per_transfer), 1);
      |     ^
src/var.c: In function ‘addvariable’:
src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type ‘struct var’ with size ‘32’ [-Walloc-size]
  361 |   p = calloc(sizeof(struct var), 1);
      |     ^
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
    ```

So, just swap the number of members and size arguments to match the
prototype, as we're initialising 1 struct of size `sizeof(struct
...)`. GCC then sees we're not doing anything wrong.

Closes #12292
2023-11-11 23:35:47 +01:00
Viktor Szakats
d14e11d248
hostip: silence compiler warning -Wparentheses-equality
Seen with LLVM 17.

```
hostip.c:1336:22: warning: equality comparison with extraneous parentheses [-Wparentheses-equality]
 1336 |        (a->ai_family == PF_INET)) {
      |         ~~~~~~~~~~~~~^~~~~~~~~~
hostip.c:1336:22: note: remove extraneous parentheses around the comparison to silence this warning
 1336 |        (a->ai_family == PF_INET)) {
      |        ~             ^         ~
hostip.c:1336:22: note: use '=' to turn this equality comparison into an assignment
 1336 |        (a->ai_family == PF_INET)) {
      |                      ^~
      |                      =
1 warning generated.
```

Follow-up to b651aba096 #12145

Reviewed-by: Daniel Stenberg
Closes #12215
2023-10-27 22:30:26 +00:00
Stefan Eissing
dac293cfb7
lib: apache style infof and trace macros/functions
- test for a simplified C99 variadic check
- args to infof() in --disable-verbose are no longer disregarded but
  must compile.

Closes #12167
Fixes #12083
Fixes #11880
Fixes #11891
2023-10-26 17:42:54 +02:00
Stefan Eissing
56a4db2e4e
urldata: move async resolver state from easy handle to connectdata
- resolving is done for a connection, not for every transfer
- save create/dup/free of a cares channel for each transfer
- check values of setopt calls against a local channel if no
  connection has been attached yet, when needed.

Closes #12198
2023-10-26 17:29:17 +02:00