We started using codeql for static code analysis in 7183f5acc3,
June 2020.
Since then, not a single commit has been merged into the source code
repository citing codeql as source or reason. Yet, it keeps getting
updated and we get constant reminders to upgrade the pinning it to the
latest hash.
During 4.5 years with intense development and significant code churn.
While Coverity, scan-build and CodeSonar have belped us point out many
mistakes, codeql has remained silent (or had false positives).
For this little gain, I think we spend a disproportionate amount of work
on codeql maintanance.
We can try again in a future if we think it improves.
Assisted-by: Viktor Szakats
Closes#15798
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper
Closes#15120
`Makefile.mk` supported MS-DOS and Amiga, but `./configure` also
supported them in a better tested and more flexible way.
This patch also adds CMake support for MS-DOS/DJGPP and Amiga OS 3.
`Makefile.mk` was not maintained. Delete it in favour of first-tier
build methods.
Also include some non-MS-DOS/AmigaOS-specific tidy-up, see details at
the end of this message.
Details:
- fix/silence all MS-DOS/DJGPP build warnings and issues.
- add MS-DOS support to cmake.
- default to `ENABLE_THREADED_RESOLVER=OFF` for MS-DOS.
- add support for `WATT_ROOT`.
- use static libcurl with MS-DOS.
- fixup default CMake suffixes/prefixes for DJGPP.
- disable hidden symbols for MS-DOS. Not supported on MS-DOS.
- opt-in MS-DOS into `USE_UNIX_SOCKETS`.
- improve MS-DOS support in autotools.
- default to `--disable-threaded-resolver` for MS-DOS.
- make sure to use `close_s()` (from Watt-32) with autotools and cmake.
`Makefile.mk` used it before this patch.
- GHA: add DJGPP cmake (~30s) and autotools (~60s) build jobs.
Also build tests and examples with cmake.
- improve AmigaOS support in autotools:
- configure: detect `CloseSocket()` when it's a macro.
- configure: fix `IoctlSocket` detection on AmigaOS.
- curl-amissl.m4: pass AmiSSL libs to tests/servers.
- add AmigaOS3 support to cmake:
- cmake: fix `HAVE_IOCTLSOCKET_CAMEL` and
`HAVE_IOCTLSOCKET_CAMEL_FIONBIO` detections.
- set necessary system libs.
- add AmiSSL support.
- inet_ntop, inet_pton: fix using it for AmigaOS. cmake detects them,
and they did not compile with AmigaOS.
- cmake: better sync `gethostname` detection with autotools.
Fixes detection for AmigaOS, where `gethostname` is a macro.
- cmake: fix `sys/utime.h` detection on AmigaOS.
- cmake: force-disable `getaddrinfo` for AmigaOS.
- cmake: tweak threading and static/shared default for AmigaOS.
- cmake: rely on manual variable `AMIGA` to enable the platform.
- GHA: add AmigaOS cmake and autotools (~45s) jobs.
Also build tests and examples with cmake.
- INSTALL: update MS-DOS and AmigaOS build instructions.
- amigaos: fix `-Wpointer-sign` and
`zero or negative size array '_args'` in `Printf()`.
- amigaos: fix `-Wpointer-sign`
- amigaos: fix `-Wredundant-decls` `errno` and `h_errno`.
- amigaos: brute-force silence `lseek()` size warnings.
- amigaos: server/resolve: silence `-Wdiscarded-qualifiers`.
- amigaos: server/resolve: fix `-Wpointer-sign`.
- amigaos: fix `CURL_SA_FAMILY_T` type.
- nonblock: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
`ioctl` is also detected, but fails when used. Make the above override
it for a successful build.
Authored-by: Darren Banfi
Fixes#15537Closes#15603
- tftpd: prefer `HAVE_IOCTLSOCKET_CAMEL_FIONBIO` for AmigaOS.
- tftpd: tidy-up conditional code.
- curl: set stack size to 16384 for AmigaOS3/4
Overriding the default 4096.
Suggested-by: Darren Banfi
Ref: https://github.com/curl/curl/pull/15543#issuecomment-2498783123
Ref: https://wiki.amigaos.net/wiki/Controlling_Application_Stack
- functypes.h: fix `SEND_QUAL_ARG2` for AmigaOS.
- tftp: add missing cast in sendto() call for AmigaOS.
- getinfo: fix warning with AmigaOS.
- tool_operate: silence warning with AmigaOS
- amigaos: fix building libtests due to missing `RLIMIT_NOFILE`.
- curl_gethostname: silence warning for AmigaOS.
- ftp: silence `-Wtype-limits` for AmigaOS.
- libtest: fix timeval initialization for AmigaOS.
- examples: fix `timeval` initialization for AmigaOS.
- examples: silence warning for AmigaOS.
- configure: fix IPv6 detection for cross-builds.
- netrc: fix to build with AmigaOS cleanly.
- buildinfo: detect and add `DOS` tag for MS-DOS builds.
- buildinfo: add `AMIGA` to buildinfo.txt in auttools.
- build: move `USE_WATT32` macro definition to cmake/configure.
Non-MS-DOS/AmigeOS-specific tidy-ups:
- configure: sync `sa_family_t` detection with cmake.
- configure: sync `ADDRESS_FAMILY` detection signals with cmake.
- doh: use `CURL_SA_FAMILY_T`.
- lib: drop mingw-specific `CURL_SA_FAMILY_T` workaround.
- cmake: extend instead of override check-specific
configurations/requirements.
This allows to honor global requirements added earlier.
Necessary for AmigaOS for example.
- cmake: omit warning on disabled IPv6 for MS-DOS and AmigaOS.
No IPv6 support on these platforms. Also sync with autotools.
- lib1960: use libcurl `inet_pton()` wrapper.
- cmake: detect LibreSSL (to match autotools).
- cmake: say the specific OpenSSL flavour detected.
- hostip: add missing `HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID` guard.
- lib: simplify classic mac feature guards.
Follow-up to a8861b6ccd#9764Closes#15543
- cmp-config.pl: add remaining exceptions. Sort list.
- drop unused `HAVE_SYS_WAIT_H`.
Follow-up to 50def7c881#13249
- drop unused `HAVE_FCHMOD`.
Follow-up to 03cb1ff4d6#12395
- autotools: stop promoting variables to macros:
`USE_NGTCP2_CRYPTO_*`, `USE_NGTCP2_H3`, `USE_OPENSSL_H3`,
`HAVE_LIBRESSL`.
They are not used in the source.
- cmake: drop unused `HAVE_O_NONBLOCK`,
`HAVE_DISABLED_NONBLOCKING`.
- lib: drop `NEED_MALLOC_H`.
It was used in manual-build cases for Amiga/MS-DOS/Windows/WinCE, but
never by autotools/cmake, thus apparently unnecessary.
- lib: drop unused `NEED_MEMORY_H`.
- lib: simplify classic mac feature guards,
drop `HAVE_EXTRA_STRICMP_H` and `HAVE_EXTRA_STRDUP_H`.
- autotools: drop unused `HAVE_GETHOSTBYNAME` detection.
- autotools: drop unused OpenSSL feature tests:
`HAVE_ERR_H`, `HAVE_PEM_H`, `HAVE_RSA_H`
- autotools: drop unused OpenSSL feature tests:
`HAVE_X509_H`, `HAVE_CRYPTO_H`, `HAVE_SSL_H`.
They performed a fallback check when the primary check missed
`openssl/x509.h`. Though if any other prefixed headers were found,
OpenSSL is already assumed detected.
The fallback check was looking for 3 unprefixed OpenSSL headers, and
if all found, marked OpenSSL found internally, but did not promote
it to `curl_config.h` via `USE_OPENSSL`. Meaning it either didn't do
anything or may have continued with an inconsistent state.
Added in d99c20f628 (2008)
At the time, there was an extra `AC_DEFINE(USE_SSLEAY, 1 ...` logic
after this code, which kicked in in the fallback case, but that code
was deleted in 709cf76f6b (2015)
Follow-up to 709cf76f6b
- autotools: drop `AC_SUBST()` where the value is explicitly set anyway
and the macro is unused.
- autotools: replace `AC_SUBST(VAR, 1)` with local variable assigments,
where the `@VAR@` macro is unused. Also dedupe the local variable if
there was a parallel one used for the same purpose.
- autotools: drop local feature variables that were never used.
- autotools: drop unused `CURL_CHECK_OPTION_NTLM_WB`,
`CURL_CHECK_NTLM_WB`.
Also stop setting unused `NTLM_WB_ENABLED` macro for VMS.
Follow-up to 50def7c881#13249
- autotools: drop unused `PKGADD_*`.
Follow-up to bae0d473f5#3331
- autotools: drop unused `CURL_NETWORK_LIBS`.
Follow-up to 3af75e18d6#14697Closes#15577
Extend `INSTALL-CMAKE` document with the list of available options,
a short description and default values.
The list may not be 100% complete.
There are no component boundaries in CMake, so the line is blurry
between curl options, CMake options, CMake Find modules options.
I included certain CMake options that seemed useful, and/or have
dedicated use withing curl's CMake source. But, all CMake built-in
options are usable, as documented upstream in CMake.
The naming of the options has a heritage and the inconsistencies with
it, including a lack of clear namespace. This may be subject to future
updates, also after figuring out which name has special meaning within
CMake and/or CMake projects out of unwritten convention or something
more tangible.
CMake allows to initialize any internal variable via `-D`. This may be
useful to pre-initialize/override feature check results. The list
doesn't contain these, and they remain officially undocumented.
Also:
- make adjustments to keep the spellchecker happy.
- retrofit description changes to the cmake sources.
- stop documenting deprecated `Find*` variables.
Reported-by: Daniel Stenberg
Fixes https://github.com/curl/curl/discussions/14885Closes#15388
Because we cannot disable the individual warnings we do not care about,
making this tool almost unusable for our purposes. See
https://github.com/amperser/proselint/issues/1367
Instead, make 'very' a banned word (as recently that has been what
proselint most commonly points out for us).
Closes#15314
`Curl_bufq_unwrite()` used the head instead of the tail chunk to shrink
the bufq's content. Fix this and add test case that checks correct
behaviour.
Amended test 2601 accordingly.
Reported-by: Chris Stubbs
Closes#15136
This needs the version in the GHA context, so move those back
from external file `VERSIONS`.
Also move back `VERSIONS` content in it previous place to
`.circleci/config.yml`. Update renovate config.
Renovate bot should keep updating the wolfSSL version, but from now on
not in one, but two files.
Follow-up to 820afa2b7c#15030
Follow-up to 73a3602120Closes#15130
Add CI checker to compare `libcurl.pc` and `curl-config` files
generated by autotools and cmake builds.
Fix differences and apply tiny cleanups:
- curl-config: use single-quotes for literals.
- curl-config: quote all variables.
- curl-config: replace double with single quotes in a substituted value
that's always literal (`@prefix@`).
- libcurl.pc: spelling in `Description:`.
- libcurl.pc: avoid substitution in a comment.
- cmake: fill `libdir` with `${exec_prefix}` instead of a literal.
To sync with './configure'.
- configure: fix `CURL_CA_BUNDLE` value to not generate nested quotes
in `curl-config`.
- configure: add missing `LDFLAGS` to `Libs.private` in `libcurl.pc`.
To sync with CMake.
- cmake: skip adding `CMAKE_C_IMPLICIT_LINK_LIBRARIES` for MINGW and
UNIX. They added these values as seen in CI:
MINGW: `-lmingw32 -lgcc -lmoldname -lmingwex -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc -lmoldname -lmingwex`
Linux: `-lgcc -lgcc_s -lc -lgcc -lgcc_s`
- cmake: delete FIXME about enabling libssh2 by default.
`./configure` has special defaults for these packages (called: "off"):
brotli, zstd, libpsl, libssh2, libssh, wolfssl, librtmp
It looks for them, but only at system locations, which makes them
never detected e.g. on macOS. CMake doesn't offer such default mode
for now.
- GHA/macos: drop now redundant `-DCURL_DISABLE_LDAPS=ON`.
- cmake: use `CMAKE_INSTALL_INCLUDEDIR` and `CMAKE_INSTALL_LIBDIR`
instead of hardcoded `include`/`lib` when generating `libcurl.pc`.
Updates to the GHA workflow:
- move autotools out-of-tree and rename cmake out-of-tree directory
to `bld_cm` to tell it's cmake.
- disable static libcurl for `./configure` to match cmake.
- enable `pkg-config` debug output with `./configure`.
- dump list of Homebrew packages on macOS.
- dump `./configure` detailed logs.
- disable zstd and brotli for Linux, to match cmake.
There remain differences, mostly due to detection order and method. Also
some values are inherently different when using CMake and autotools,
such as `--cc`, `--configure`. autotools also generates duplicates for
`-lssl` and `-lcrypto`. macOS LDAP wants to link `-lber` while autotools
doesn't. Some build defaults are also different in autotools and cmake.
These differences are smoothened out for now by the checker script, or
via build options. Notice that lib order (a dupes) _can_ be significant
in some cases. E.g. the binutils linker is infamous for that on Windows.
Closes#14681
Both are used by `./configure` internally, not by curl C code.
CMake adds `NEED_LBER_H` to `CURL_TEST_DEFINES`, which is used by
`curl_internal_test()`, but there is no LDAP-related test made with that
call.
Thus, stop detecting and publishing these from CMake.
`NEED_LBER_H` added in 7320e53d9e.
`HAVE_LDAP_H` added in 4c5307b456 (initial CMake commit).
Closes#14690
- configure: disable pthreads by default on Windows.
- configure: disable detecting `fseeko()` on Windows.
(It exists in mingw-w64 2.0.0 and newer, but it's permanently ignored
in CMake, as this function is never necessary on Windows.)
- extend existing exceptions with their Windows variants.
- `lib/formdata.c`: prioritize `_fseeki64()` over `fseeko()`.
To reduce the difference between Windows builds, which now all use
`_fseeki64()`.
- cmake: perm-enable `HAVE_DIRENT_H` and `HAVE_OPENDIR` for mingw-w64,
to match configure.
Follow-up to bfe54b0e88#13137
This in theory could make the dir listing feature work in mingw-w64
build, but in my tests (on WINE) it failed at the preceding `open()`
call.
- cmake: perm-enable `HAVE_STRINGS_H` and `HAVE_UTIME_H` for mingw-w64,
to match configure. (They are wrappers and make no difference in the build.)
Also:
- configure: sync `USE_MANUAL` macro with cmake, by only setting it for
`src`. Drop checker exception.
- CI: use `--disable-dependency-tracking` in existing jobs.
- CI: install packages before git checkout, in existing jobs.
Closes#14678
- sync cmake macOS `OS` value manually with the autotools one.
- stop exporting/subst-ing `HAVE_OPENSSL_QUIC` from autotools.
The variable was only used internally.
- exclude a dependency detection symbol.
- allow to run when the workflow itself was updated.
- simplify cmake command.
- fix indentation.
Closes#14546
Give a more consice overview of curl's cipher options and cipher suites.
Removed long lists of ciphers that were directly copied from the SSL
backends' documentation. Instead present the user a more common aproach
acorss the SSL backends, with notes for backends that do not conform
and/or provide alternate means.
Provide a shorter list of cipher suites that is more relevant for modern
usage and should work mostly across all backends, provide a seperate
list with all cipher suites, and provide links to the SSL backends'
documentation for more information.
Also give examples with modern cipher suites.
Add docs/CIPHERS-TLS12.md for TLS 1.2 ciphers.
Closes#14460
The documented and mandated step has been to not use buildconf but to
invoke 'autoreconf -fi' for four years already.
This change only drops buildconf from the release tarball, it remains
present in git for now.
Follow-up to 85868537d6Closes#14412
Use these words and casing more consistently across text, comments and
one curl tool output:
AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode,
WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock.
Mostly OS names and a few more.
Also a couple of other minor text fixups.
Closes#14360
The only binary-looking files that are accepted in the git repository
need to match the checksums in the sha256sum file
".github/scripts/binarycheck.sums".
This is done to make sure that no one has planted any hidden (encrypted)
potentially dangerous payload in the tree.
Closes#14333
As runtests.md and testcurl.md. Very few people actually need these as
manpages anyway.
With this, we have no more nroff formatted documents in git.
Closes#14324
- prefer `.bat` extension over `.cmd` for MS-DOS batch, which also
avoids confusion with OS/400 `.cmd` files.
- cleanup `echo` quotes, drop them consistently.
- delete empty output line from one of the error branches.
- prefer lowercase commands like the rest of MS-DOS batches.
- delete a contraction.
- drop backticks from error message.
- use `nmake.exe` consistently.
- use equal/not-equal operator style consistently.
- inline a single-line `if` branch.
- delete exceptions and rules dealing with Windows `.cmd` extension.
Closes#14084
Based on the standards and guidelines we use for our documentation.
- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period
Closes#14073
Fix issues detected.
Also:
- One of the `.vc` files used LF EOLs, while the other didn't.
Make that one also use LF EOLs, as this is apparently supported by
`nmake`.
- Drop `.dsw` and `.btn` types from `.gitattributes`.
The repository doesn't use them.
- Sync section order with the rest of files in
`tests/certs/EdelCurlRoot-ca.prm`.
- Indent/align `.prm` and `.pem` files.
- Delete dummy `[something]` section from `.prm` and `.pem` files.
Mental note:
MSVC `.sln` files seem to accept spaces for indentation and also support
LF line-endings. I cannot test this and I don't know what's more
convenient when updating them, so left them as-is, with specific
exclusions.
Closes#14031
If a malicious server can trigger a NULL dereference in curl or
otherwise cause curl to crash (and nothing worse), chances are big that
we do not consider that a security problem.
Closes#13974
Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension
that enables a TCP connection to use different paths.
Multipath TCP has been used for several use cases. On smartphones, MPTCP
enables seamless handovers between cellular and Wi-Fi networks while
preserving established connections. This use-case is what pushed Apple
to use MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to automatically use the
best performing path, either IPv4 or IPv6. If one path fails, MPTCP
automatically uses the other path.
To benefit from MPTCP, both the client and the server have to support
it. Multipath TCP is a backward-compatible TCP extension that is enabled
by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...).
Multipath TCP is included in the Linux kernel since version 5.6 [3]. To
use it on Linux, an application must explicitly enable it when creating
the socket. No need to change anything else in the application.
This attached patch adds an --mptcp option which allows the creation of
an MPTCP socket instead of TCP on Linux. If Multipath TCP is not
supported on the system, an error will be reported. It is important to
note that if the end server doesn't support MPTCP, the connection will
continue after a seamless fallback to TCP.
Link: https://www.rfc-editor.org/rfc/rfc8684.html [1]
Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2]
Link: https://www.mptcp.dev [3]
Co-developed-by: Dorian Craps (@CrapsDorian) <doriancraps@gmail.com>
Co-developed-by: Olivier Bonaventure (@obonaventure) <Olivier.Bonaventure@uclouvain.be>
Co-developed-by: Matthieu Baerts (@matttbe) <matttbe@kernel.org>
Signed-off-by: Dorian Craps <dorian.craps@student.vinci.be>
Closes#13278
Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and use in checks
- fix caddy version check for slight difference under linux
- set caddy default path fitting for linux
- fix deprecation warnings in valid time checks
FTP tests:
- add '--with-test-vsftpd=path' to configure
- use vsftpd default path suitable for linux
- add test_30 with plain FTP tests
- add test_31 with --ssl-reqd FTP tests
- add vsftpd to linux GHA for pytest workflows
Closes#13661
WolfSSL/WolfSSH use a different versioning scheme;
stable builds end with `-stable`. Renovate requires
some extra configuration to extract the version
from these types of tags.
Closes#13644
Use a lookup list to set the cipher suites, allowing the
ciphers to be set by either openssl or IANA names.
To keep the binary size of the lookup list down we compress
each entry in the cipher list down to 2 + 6 bytes using the
C preprocessor.
Closes#13442
- use `$()` instead of backticks, and re-arrange double-quotes inside.
- add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu`.)
- add `-n` to a few `if`s.
- shorten redirections by using `{} >` (as shellcheck recommended).
- silence warnings where variables were detected as unused (SC2034).
- a couple misc updates to silence warnings.
- switch to bash shebang for `-ot` feature.
- split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, `$(dirname \`)
Also enable CI checks for OS/400 shell scripts.
Ref: #13307Closes#13309
Lots of organizations distribute curl packages to end users. This is a
collection of pointers to where to learn more about curl on and with
each distro.
Assisted-by: Alan Coopersmith
Assisted-by: Andrew Kaster
Assisted-by: Andy Fiddaman
Assisted-by: Arjan van de Ven
Assisted-by: Brian Clemens
Assisted-by: chrysos349 on github
Assisted-by: Dan Fandrich
Assisted-by: Dan McDonald
Assisted-by: Gaelan Steele
Assisted-by: graywolf on github
Assisted-by: Jan Macku
Assisted-by: John Marshall
Assisted-by: Jonathan Perkin
Assisted-by: Kevin Daudt
Assisted-by: Marcus Müller
Assisted-by: Michał Górny
Assisted-by: Outvi V
Assisted-by: Ross Burton
Assisted-by: Sean Molenaar
Assisted-by: Till Wegmüller
Assisted-by: Viktor Szakats
Assisted-by: Winni Neessen
Closes#13178
- Turn docs/INSTALL.cmake into a proper markdown file,
docs/INSTALL-CMAKE.md
- Move things around to divide the description into configuration,
building and installing sections
- Mention the more modern cmake options to configure, build and install,
but also retain the older variants as fallbacks
Closes#12772
This means words, phrases or things we have decided not to use - words that
are spelled right according to the dictionary but we want to avoid. In the
name of consistency and better documentation.
Closes#12764
- switch all invidual files documenting command line options into .md,
as the documentation is now markdown-looking.
- made the parser treat 4-space indents as quotes
- switch to building the curl.1 manpage using the "mainpage.idx" file,
which lists the files to include to generate it, instead of using the
previous page-footer/headers. Also, those files are now also .md
ones, using the same format. I gave them underscore prefixes to make
them sort separately:
_NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md,
_VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md,
_OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md,
_EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md
- updated test cases accordingly
Closes#12751
curldown is this new file format for libcurl man pages. It is markdown
inspired with differences:
- Each file has a set of leading headers with meta-data
- Supports a small subset of markdown
- Uses .md file extensions for editors/IDE/GitHub to treat them nicely
- Generates man pages very similar to the previous ones
- Generates man pages that still convert nicely to HTML on the website
- Detects and highlights mentions of curl symbols automatically (when
their man page section is specified)
tools:
- cd2nroff: converts from curldown to nroff man page
- nroff2cd: convert an (old) nroff man page to curldown
- cdall: convert many nroff pages to curldown versions
- cd2cd: verifies and updates a curldown to latest curldown
This setup generates .3 versions of all the curldown versions at build time.
CI:
Since the documentation is now technically markdown in the eyes of many
things, the CI runs many more tests and checks on this documentation,
including proselint, link checkers and tests that make sure we capitalize the
first letter after a period...
Closes#12730
- Check that all backslashes in EXAMPLE are properly escaped.
eg manpage must always use `\\n` never `\n`.
This is because the manpage requires we always double blackslash to show
a single backslash. Prior to this change an erroneous single backslash
would pass through and compile even though it would not show correctly
in the manpage.
Co-authored-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/12588
Closes https://github.com/curl/curl/pull/12589
- use the correct include file
- make sure they are declared as in the header file
- fix minor nroff syntax mistakes (missing .fi)
These are verified by verify-synopsis.pl, which extracts the SYNPOSIS
code and runs it through gcc.
Closes#12402
- Move documentation of libcurl environment variables used only in debug
builds from libcurl-env into a separate document libcurl-env-dbg.
- Document more debug environment variables.
Previously undocumented or missing a description:
CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL,
CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME,
CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS,
CURL_TIME.
Closes https://github.com/curl/curl/pull/11811
- ipfs://<cid>
- ipns://<cid>
This allows you tu use ipfs in curl like:
curl ipfs://<cid>
and
curl ipns://<cid>
For more information consult the readme at:
https://curl.se/docs/ipfs.htmlCloses#8805
- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add missing EOL at EOF.
- delete whitespace at EOL (except from expected test results).
- convert tabs to spaces.
- convert CRLF EOLs to LF in GHA yaml.
- text casing fixes in `./CMakeLists.txt`.
- fix a codespell typo in `packages/OS400/initscript.sh`.
Closes#11772
These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was
introduced but always lived a life in the shadows, embedded in the
CURLMOPT_PUSHFUNCTION man page. Until now.
It makes better sense and gives more visibility to document them in
their own stand-alone man pages.
Closes#11286
* Configure changes to detect AWS-LC
* CMakeLists.txt changes to detect AWS-LC
* Compile-time branches needed to support AWS-LC
* Correctly set OSSL_VERSION and report AWS-LC release number
* GitHub Actions script to build with autoconf and cmake against AWS-LC
AWS-LC is a BoringSSL/OpenSSL derivative
For more information see https://github.com/awslabs/aws-lc/Closes#10320
New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.
HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.
HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue
HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.
GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.
Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.
HTTP/3 documentation on how https eyeballing works.
TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).
Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy
pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.
Closes#10349
- adding '--with-test-httpd=<path>' to configure non-standard apache2
install
- python env and base classes for running httpd
- basic tests for connectivity with h1/h2/h3
- adding test cases for truncated responses in http versions.
- adding goaway test for HTTP/3.
- adding "stuttering" tests with parallel downloads in chunks with
varying delays between chunks.
- adding a curltest module to the httpd server, adding GOAWAY test.
- mod_curltest now installs 2 handlers
- 'echo': writing as response body what came as request body
- 'tweak': with query parameters to tweak response behaviour
- marked known fails as skip for now
Closes#10175
- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING
checksrc is updated to allow non-year using copyright statements
Closes#10205
Update bare GNU Make `Makefile.m32` to:
- Move objects into a subdirectory.
- Add support for MS-DOS. Tested with DJGPP.
- Add support for Watt-32 (on MS-DOS).
- Add support for AmigaOS.
- Rename `Makefile.m32` to `Makefile.mk`
- Replace `ARCH` with `TRIPLET`.
- Build `tool_hugehelp.c` proper (when tools are available).
- Drop MS-DOS compatibility macro `USE_ZLIB` (replaced by `HAVE_LIBZ`)
- Add support for `ZLIB_LIBS` to override `-lz`.
- Omit object files when building examples.
- Default `CC` to `gcc` once again, for convenience. (Caveat: compiler
name `cc` cannot be set now.)
- Set `-DCURL_NO_OLDIES` for examples, like autotools does.
- Delete `makefile.dj` files. Notice the configuration details and
defaults are not retained with the new method.
- Delete `makefile.amiga` files. A successful build needs a few custom
options. We're also not retaining all build details from the existing
Amiga make files.
- Rename `Makefile.m32` to `Makefile.mk` to reflect that they are not
Windows/MinGW32-specific anymore.
- Add support for new `CFG` options: `-map`, `-debug`, `-trackmem`
- Set `-DNDEBUG` by default.
- Allow using `-DOS=...` in all `lib/config-*.h` headers, syncing this
with `config-win32.h`.
- Look for zlib parts in `ZLIB_PATH/include` and `ZLIB_PATH/lib`
instead of bare `ZLIB_PATH`.
Note that existing build configurations for MS-DOS and AmigaOS likely
become incompatible with this change.
Example AmigaOS configuration:
```
export CROSSPREFIX=/opt/amiga/bin/m68k-amigaos-
export CC=gcc
export CPPFLAGS='-DHAVE_PROTO_BSDSOCKET_H'
export CFLAGS='-mcrt=clib2'
export LDFLAGS="${CFLAGS}"
export LIBS='-lnet -lm'
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```
Example MS-DOS configuration:
```
export CROSSPREFIX=/opt/djgpp/bin/i586-pc-msdosdjgpp-
export WATT_PATH=/opt/djgpp/net/watt
export ZLIB_PATH=/opt/djgpp
export OPENSSL_PATH=/opt/djgpp
export OPENSSL_LIBS='-lssl -lcrypt'
export CFG=-zlib-ssl
make -C lib -f Makefile.mk
make -C src -f Makefile.mk
```
Closes#9764
This spellchecker checks markdown files. For this reason this job
converts all man pages in the repository to markdown with pandoc before
the check runs.
The perl script 'cleanspell' filters out details from the man page in
the process, to avoid the spellchecker trying to spellcheck things it
can't. Like curl specific symbols and the SYNOPSIS and EXAMPLE sections
of libcurl man pages.
The spell checker does not check words in sections that are within pre,
strong and em tags.
'spellcheck.words' is a custom word list with additional accepted words.
Closes#9523
