$ curl -h all | grep -- --http
Now:
--http0.9 Allow HTTP/0.9 responses
-0, --http1.0 Use HTTP/1.0
--http1.1 Use HTTP/1.1
--http2 Use HTTP/2
--http2-prior-knowledge Use HTTP/2 without HTTP/1.1 Upgrade
--http3 Use HTTP/3
--http3-only Use HTTP/3 only
Before:
--http0.9 Allow HTTP 0.9 responses
-0, --http1.0 Use HTTP 1.0
--http1.1 Use HTTP 1.1
--http2 Use HTTP/2
--http2-prior-knowledge Use HTTP 2 without HTTP/1.1 Upgrade
--http3 Use HTTP v3
--http3-only Use HTTP v3 only
Closes#16542
Further testing with timeouts in event based processing revealed that
our current shutdown handling in the connection pool was not clear
enough. Graceful shutdowns can only happen inside a multi handle and it
was confusing to track in the code which situation actually applies. It
seems better to split the shutdown handling off and have that code
always be part of a multi handle.
Add `cshutdn.[ch]` with its own struct to maintain connections being
shut down. A `cshutdn` always belongs to a multi handle and uses that
for socket/timeout monitoring.
The `cpool`, which can be part of a multi or share, either passes
connections to a `cshutdn` or terminates them with a one-time, best
effort.
Add an `admin` easy handle to each multi and share. This is used to
perform all maintenance operations where no "real" easy handle is
available. This solves the problem that the multi admin handle requires
some additional initialisation (e.g. timeout list).
The share needs its admin handle as it is often cleaned up when no other
transfer or multi handle exists any more. But we need a `data` in almost
every call.
Fix file:// handling of errors when adding a new connection to the pool.
Changes in `curl` itself:
- for parallel transfers, do not set a connection pool in the share,
rely on the multi's connection pool instead. While not a requirement
for the new `cshutdn` to work, this is
a) helpful in testing to trigger graceful shutdowns
b) a broader code coverage of libcurl via the curl tool
- on test_event with uv, cleanup the multi handle before returning from
parallel_event(). The uv struct is on the stack, cleanup of the multi
later will crash when it tries to register sockets. This is a "eat
your own dogfood" related fix.
Closes#16508
To help applications do the right thing easier, change some enum values
into defines with L suffixes so that they get the corect type (long)
easier when used with curl_easy_setopt(). This also fixes a few of our
own libtests.
To reduce the risk that this change breaks the compile for any existing
users, the previously provided enums are still provided, but the values
to use are not defined by the enums.
This change "magically" fixes a few RTSP test failures we have had on
64-bit platforms because those options were not see using longs
properly.
Closes#16482
curl requires Windows XP since 2023. Drop version detection code using
`GetVersionEx()` aimed to support earlier Windows versions. With that
call deleted, the embedded manifest in `curl.rc` becomes unnecessary.
Delete it too, along with the enabler logic in build systems.
This allows to stop forcing `/MANIFEST:NO` for MSVC builds. Dropping it
fixes VS2008 shared builds, that require an auto-generated SxS
(side-by-side assembly) manifest to find their CRT DLLs. This was the
issue that prevented VS2008 `curl.exe` launching on AppVeyor CI:
```
src/curl.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51577006/job/eitypvwlb1rxr11d#L261
FWIW the `curl.rc` embedded manifest wasn't ever enabled for VS2008 CI
builds either, because CMake did not pass our custom macro via
`CMAKE_RC_FLAGS` to `rc.exe`. For reasons I could not figure out.
After this patch the curl build no longer inject its own manifest, and
lets the default be applied by linkers and toolchains. It fixes VS2008
shared builds. curl continues to detect the real Windows version via
`RtlVerifyVersionInfo()` from `ntdll`.
Follow-up to 960d601481#12225
Follow-up to 5044909ca2#7810
Follow-up to ebd213270a#1221
Ref: #15972
Cherry-picked from #16394Closes#16453
This now insists on using a server auth option unless --insecure is
provided. As an added bonus, it now also only checks for the knownhosts
file once (if found).
Ref: #16197Closes#16205
Verified in test 455 and 487.
If the provided string cannot be base64-decoded, it will instead use
"[64dec-fail]" (without the quotes).
Documented
Ref: #16288Closes#16330
Make it possible to build curl for Windows CE using the CeGCC toolchain.
With both CMake and autotools, including tests and examples, also in CI.
The build configuration is the default one with Schannel enabled. No
3rd-party dependencies have been tested.
Also revive old code to make Schannel build with Windows CE, including
certificate verification.
Builds have been throughougly tested. But, I've made no functional tests
for this PR. Some parts (esp. file operations, like truncate and seek)
are stubbed out and likely broken as a result. Test servers build, but
they do not work on Windows CE. This patch substitutes `fstat()` calls
with `stat()`, which operate on filenames, not file handles. This may or
may not work and/or may not be secure.
About CeGCC: I used the latest available macOS binary build v0.59.1
r1397 from 2009, in native `mingw32ce` build mode. CeGCC is in effect
MinGW + GCC 4.4.0 + old/classic-mingw Windows headers. It targets
Windows CE v3.0 according to its `_WIN32_WCE` value. It means this PR
restores portions of old/classic-mingw support. It makes the Windows CE
codepath compatible with GCC 4.4.0. It also adds workaround for CMake,
which cannot identify and configure this toolchain out of the box.
Notes:
- CMake doesn't recognize CeGCC/mingw32ce, necessitating tricks as seen
with Amiga and MS-DOS.
- CMake doesn't set `MINGW` for mingw32ce. Set it and `MINGW32CE`
manually as a helper variable, in addition to `WINCE` which CMake sets
based on `CMAKE_SYSTEM_NAME`.
- CMake fails to create an implib for `libcurl.dll`, due to not
recognizing the platform as a Windowsy one. This patch adds the
necessary workaround to make it work.
- headers shipping with CeGCC miss some things curl needs for Schannel
support. Fixed by restoring and renovating code previously deleted
old-mingw code.
- it's sometime non-trivial to figure out if a fallout is WinCE,
mingw32ce, old-mingw, or GCC version-specific.
- WinCE is always Unicode. With exceptions: no `wmain`,
`GetProcAddress()`.
- `_fileno()` is said to convert from `FILE *` to `void *` which is
a Win32 file `HANDLE`. (This patch doesn't use this, but with further
effort it probably could be.)
https://stackoverflow.com/questions/3989545/how-do-i-get-the-file-handle-from-the-fopen-file-structure
- WinCE has no signals, current directory, stdio/CRT file handles, no
`_get_osfhandle()`, no `errno`, no `errno.h`. Some of this stuff is
standard C89, yet missing from this platform. Microsoft expects
Windows CE apps to use Win32 file API and `FILE *` exclusively.
- revived CeGCC here (not tested for this PR):
https://building.enlyze.com/posts/a-new-windows-ce-x86-compiler-in-2024/
On `UNDER_CE` vs. `_WIN32_WCE`: (This patch settled on `UNDER_CE`)
- A custom VS2008 WinCE toolchain does not set any of these.
The compiler binaries don't contain these strings, and has no compiler
option for targeting WinCE, hinting that a vanilla toolchain isn't
setting any of them either.
- `UNDER_CE` is automatically defined by the CeGCC compiler.
https://cegcc.sourceforge.net/docs/details.html
- `UNDER_CE` is similar to `_WIN32`, except it's not set automatically
by all compilers. It's not supposed to have any value, like a version.
(Though e.g. OpenSSL sets it to a version)
- `_WIN32_WCE` is the CE counterpart of the non-CE `_WIN32_WINNT` macro.
That does return the targeted Windows CE version.
- `_WIN32_WCE` is not defined by compilers, and relies on a header
setting it to a default, or the build to set it to the desired target
version. This is also how `_WIN32_WINNT` works.
- `_WIN32_WCE` default is set by `windef.h` in CeGCC.
- `_WIN32_WCE` isn't set to a default by MSVC Windows CE headers (the
ones I checked at least).
- CMake sets `_WIN32_WCE=<ver>`, `UNDER_CE`, `WINCE` for MSVC WinCE.
- `_WIN32_WCE` seems more popular in other projects, including CeGCC
itself. `zlib` is a notable exception amongst curl dependencies,
which uses `UNDER_CE`.
- Since `_WIN32_WCE` needs "certain" headers to have it defined, it's
undefined depending on headers included beforehand.
- `curl/curl.h` re-uses `_WIN32_WCE`'s as a self-guard, relying on
its not-(necessarily)-defined-by-default property:
25b445e479/include/curl/curl.h (L77)
Toolchain downloads:
- Windows:
https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_cygwin1.7_r1399.tar.bz2
- macOS Intel:
https://downloads.sourceforge.net/cegcc/cegcc/0.59.1/cegcc_mingw32ce_snowleopard_r1397.tar.bz2Closes#15975
Add a 'wanted' major HTTP version bitmask next to the 'allowed' bitmask
in HTTP version negotiation. This will try connections as specified in
'wanted', but enabled Alt-Svc and HTTPS-RR to redirect to other major
HTTP versions, if those are 'allowed'.
Changes libcurl internal default to `CURL_HTTP_VERSION_NONE` and removes
the code in curl that sets `CURL_HTTP_VERSION_2TLS` if the command line
does not say anything else.
Closes#16117
curl attempts to clear some flags to hide them from snooping neighbors
(on platforms where it works). For example the credentials provided with
-u. Previously it would only do that if there was a space between the
option and the credentials as in "-u joe:s3cr3t" but not when done
without a separating space as in "-ujoe:s3cr3t".
This addresses that previous shortcoming.
Reported-by: kayrus on github
Fixes#16396Closes#16401
- replace `add_compile_options()`, `add_definitions()` with directory
properties. To harmonize this across all scripts. The new commands are
verbose, but describe better how they work. The syntax is also closer
to setting target properties, helps grepping.
- prefer `CMAKE_INSTALL_PREFIX` over `--prefix` (in tests, CI).
- tidy up cmake invocations.
- formatting.
Closes#16238
- add hex and octal parsers to the Curl_str_* family
- make curlx_strtoofft use these parsers
- remove all use of strtol() and strtoul() in library code
- generally use Curl_str_* more than strtoofft, for stricter parsing
- supports 64-bit universally, instead of 'long' which differs in size
between platforms
Extended the unit test 1664 to verify hex and octal parsing.
Closes#16336
To reduce the number `-Wunused-macro` compiler warnings:
- delete unused macros.
- comment out unused macro that are part of a set.
- move macros into the scope they are used.
This may be useful to enable by default, but there are tricky cases that
I didn't manage to fix and paused the effort. E.g. internal features
checks in `openssl.c`. There is more, once those are fixed.
Closes#16279
Before this patch curl code was redefining `getaddrinfo` and
`freeaddrinfo` system symbols to plug in its debug wrappers. This was
causing pains to avoid applying the redefinitions to system headers
defining these functions, and to the local debug wrappers. Especially
in unity builds. It also required workarounds for systems where these
symbols are already macros.
Introduce curl-namespaced macros for these functions and use them.
This allows to drop all workarounds and makes it work in all envs,
local targets and unity/bundle combinations.
Also drop GHA/windows workaround and use the same unity batch across
all jobs. Follow-up to 29e4eda631#16272
Ref: #16272
Ref: 71cf0d1fca#14772
Ref: 3efba94f77#14765
Ref: f7d5f47059#14399Closes#16274
When the total transfered amount (upload or download) for parallel
transfers was larger than 2^63/100 bytes (81 petabytes) the progress
percent counter would show wrong.
Closes#16284
Instead of picky-backing on the libcurl one using the curlx shortcut,
which is fragile since the libcurl one is not present in all builds.
Reported-by: mschroeder-fzj on github
Fixes#16201Closes#16203
Rework the way `tool_hugehelp.c` is included in builds.
After this patch, with `./configure` and CMake `tool_hugehelp.c` is only
compiled when building with manuals enabled. With manuals disabled this
source file is not used anymore. The method is similar to how
8a3740bc8e implemented `tool_ca_embed.c`.
`./configure` always generates it as before, otherwise the build fails.
- winbuild: rework to not need `buildconf.bat`, but automatically use
`tool_hugehelp.c` if present (e.g. when building from an official
source tarball) and enable `USE_MANUAL` accordingly.
- `buildconf.bat`: after dropping `tool_hugehelp.c` generation, the only
logic left was `cp Makefile.dist Makefile`. This allowed to launch
winbuild builds via GNU Make in a Git repo. Drop this option together
with the batch file.
- build `libcurltool` without `USE_MANUAL` macro to exclude the manual
and the dependence on the generator commands. Drop relying on
`UNITTESTS` for this purpose.
Follow-up to 96843f4ef7#16068
- `src/mkhelp.pl`: include `tool_hugehelp.h` before using `USE_MANUAL`
to have it set in `config-*.h` builds with source tarballs created
with manual but without zlib.
Closes#16081
CMake builds using the Xcode generator broke with an error saying it
doesn't support multiple targets depending on the same custom commands.
These custom commands are generating `tool_hugehelp.c` and
`tool_c_embed.c` for the curl tool and libcurltool.
`unit1394` and `unit1604` tests use libcurltool to test tool-specific
functions. They don't need hugehelp and ca-embed. It's thus safe to
disable and exclude them when compiling the sources for libcurltool.
Use the `UNITTESTS` macro to detect a libcurltool build within C.
After this patch these sources are solely used for building the curl
tool. Making the build compatible with the CMake Xcode generator.
Apply the change to autotools too to keep build systems synchronized.
Follow-up to 12a6de2f66#16043Closes#16068
iOS:
- add jobs with autotools, CMake, CMake Xcode generator.
The Xcode generator is >10x slower than Unix Makefiles. Keep it
because it's the one recommended by CMake and for having its own
quirks we may want to know about.
- build, cache and use LibreSSL for these jobs.
With workaround for an iOS build issue fixed in master.
- make Xcode generator work by explicitly disabling code signing.
- make tests and examples build with the Xcode generator by setting
`-DMACOSX_BUNDLE_GUI_IDENTIFIER=se.curl`, to avoid
"Bundle identifier is missing" errors.
- cmake: disable `CURL_USE_PKGCONFIG` by default for Apple device.
- cmake: add `stdc++` library for BoringSSL and AWS-LC, with
`OPENSSL_USE_STATIC_LIBS=ON` set.
- cmake: add workaround for Xcode generator issue, where it cannot
handle two targets depending on one custom command. A better fix may
be dropping `tool_hugehelp.c` and `tool_ca_embed.c` from curltool
library. For a future PR.
Android:
- add vcpkg to Android jobs, enable dependencies.
Assisted-by: Tal Regev via #16045
- make vcpkg work with autotools.
- pass `--with-brotli` to autotools to detect the vcpkg-supplied brotli.
- enable BoringSSL for Android and add a job with it.
- silence 457 CMake configure warnings about the Android NDK CMake
scripts targeting freshly deprecated CMake versions.
These were much more involved than imagined. Basically nothing works out
of the box, and when combined, everything becomes a unique edge case.
autotools builds were a much easier to make work than CMake ones.
Also:
- GHA/non-native: re-sync names to be shorter and more aligned with
other workflows.
- GHA: add `persist-credentials: false` where missing.
Unresolved issues:
- `OPENSSL_ROOT_DIR` ignored/mis-used when pointing it to LibreSSL.
CMake seems to prepend the sysroot to the passed absolute directory.
Found no workaround.
- CMake when combined with Android, both the Google-recommended method
and the built-in CMake method fail to provide a way to avoid
`pkg-config` packages at system directories. Failed to find a knob
that can remove `/usr/include` from the search path. The workaround is
to disable zstd. (I enabled it by default in this release, maybe
premature?: f2adb3b6d7#15431)
Disabling `pkg-config` doesn't work because vcpkg dependencies do not
link without it.
- CMake's Xcode generator is slow because each `try_compile()` feature
check springs a new CMake + Xcode project taking a long time to run,
just to compile single-liner C files. A known issue, with no solution.
`-DCMAKE_MACOSX_BUNDLE=OFF` did not help, limiting build types to
a single one (e.g. `Debug`) also had no effect.
make | Xcode | GHA run
:---- | :---- | :--------------------------------------------------------------------
16s | 2m57s | https://github.com/curl/curl/actions/runs/12866334102/job/35868712426
23s | 4m13s | https://github.com/curl/curl/actions/runs/12868128013/job/35874212461
16s | 3m39s | https://github.com/curl/curl/actions/runs/12859073531/job/35849041880
14s | 2m23s | https://github.com/curl/curl/actions/runs/12858298423/job/35847201313
15s | 2m36s | https://github.com/curl/curl/actions/runs/12858058492/job/35846669761
19s | 3m19s | https://github.com/curl/curl/actions/runs/12868919430/job/35876601168Closes#16043
In effect it meant `_WIN32 && !USE_WIN32_LARGE_FILES`.
Replace it with these macros.
Also:
- configure: delete tautological check for small file support.
- configure: delete stray `_MSC_VER` reference. autotools does not
support MSVC.
- drop tautological checks for WinCE in `config-win32*.h` when setting
`USE_WIN32_LARGE_FILES`.
- merge related PP logic.
- prefer `#ifdef`, fix whitespace.
Suggested-by: Marcel Raad
Report: https://github.com/curl/curl/pull/15952#issuecomment-2580092328Closes#15968
- DJGPP 1.x (including `__GO32__`) (MS-DOS)
DJGPP 2.x support remains unchanged.
- Salford C (Windows)
- Turbo C (Windows 16-bit)
- Borland C++ < 5.2 (Windows 16-bit?)
- Pelles C < 2.8 (Windows)
These targets mapped `curl_off_t` to `long`. On Windows and MS-DOS
`long` is always 32-bit.
curl requires C compilers supporting 64-bit `curl_off_t` type since
835682661c#10597 (v8.0.0).
Also: drop remaining `__GO32__` and Salford C guards.
Closes#15957
Adds the experimental feature `ssls-export` to libcurl and curl for
importing and exporting SSL sessions from/to a file.
* add functions to libcurl API
* add command line option `--ssl-sessions <filename>` to curl
* add documenation
* add support in configure
* add support in cmake
+ add pytest case
Closes#15924
