To formalize they are now XML-compliant (with some asterisks.)
Also to help syntax highlighters work on them to make their content more
readable.
Also:
- Delete empty comment decorations.
- GHA/checksrc: simplify XML check.
- runtests: fail to load test data with XML prolog missing.
Follow-up to bfe6eb1c06#19927
Follow-up to 87ba80a6dfCloses#19946
- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
`verify/proxy`:
- make `crlf="yes"` force CRLF to all lines, instead of just applying
to HTTP protocol headers.
- add support for `crlf="headers"` that only converts HTTP protocol
header lines to CRLF. (previously done via `crlf="yes"`.)
- use `crlf="headers"` where possible.
- `reply/connect*`:
- add support for `crlf="yes"` and `crlf="headers"`.
- use them where possible.
- `client/file*`, `client/stdin`:
- add support for `crlf="yes"`.
- use it where possible.
- `reply/data*`, `verify/protocol`:
- replace existing uses of `crlf="yes"` with `crlf="headers`" where it
does not change the result.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)
After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19313
To make special newlines more explicit and visible.
Mostly in `<protocol>` sections, some in `<data*>` and `<upload>`.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 21535 to 11337.
- files with mixed newlines from 1335 to 707.
Also delete empty `<protocol>` sections.
Closes#19284
Before this patch the curl repository and source tarball distribution
contained test certificates as binary blobs. Used by runtests.
Drop these certificates in favor of generating them dynamically as
part of the build process. Both via autotools and CMake.
As part of this, improve certificates, the generator script and process,
file layout, and fix any issue to make it work fast and smooth both in
CI and local builds.
Note, cert generator scripts require OpenSSL >=1.0.2
(or LibreSSL >=3.1.0). Generation requires POSIX shell, also with CMake.
Without a POSIX shell tests relying on TLS (and stunnel) will fail.
Details:
- build: generate certs as part of the test run process.
- build, tests: generate certs in the build directory.
- binarycheck: drop concept of known binary files with hashes.
- binarycheck: move binary check logic into spacecheck and drop this
separate checker tool.
- build: fix to clean all cert files.
- autotools: fix to not run leaf cert generators in parallel. To avoid
confusion when updating the revocation database and counter.
- scripts: drop `scripts` subdir, merge two scripts into one,
auto-generate root cert, allow generating multiple leafs at once.
- scripts: switch to EC-256 keys (was: RSA-2048). For key size and perf.
- scripts: drop `-x` echo, text dumps, most other output. To avoid log
noise and make it quicker in CI.
- scripts: make it non-RSA-specific.
- scripts: delete unused code.
- scripts: use POSIX shell shebang. Some envs don't have bash (Alpine).
- scripts: pass test pseudo-secrets via the command-line. To avoid:
```
+ openssl genrsa -out test-ca.key -passout fd:0 2048
Invalid password argument, starting with "fd:"
```
- cmake: fix to launch generator scripts via the detected POSIX shell.
- cmake: fix `build-certs` rule to not depend on `SRPFILES`
(`srp-verifier-*`).
- cmake: drop `EXCLUDE_FROM_ALL` for the cert subdir. It makes
the Visual Studio generator miss to create the `clean-certs`,
`build-certs` targets. No target depend on them, so they don't execute
implicitly anyway. Fixes:
```
MSBUILD : error MSB1009: Project file does not exist.
Switch: clean-certs.vcxproj
```
- cmake: add `VERBATIM USES_TERMINAL` to `build-certs` target.
- GHA/linux: install openssl on Alpine, for the cert generator scripts.
Follow-up to 556f722fe3#16593
Follow-up to fa461b4eff#14486Closes#16824
Keep only the generated files needed for tests. Place generated
intermediaries in `tests/certs/gen` where they are ignored by git. No
longer generated `*.dhp` files.
Have a shorter naming scheme: `test-ca` instead of `EdelCurlRoot-ca` and
`test-localhost` instead of `Server-localhost-sv`, etc.
Remove the `stunnel` certificate as it was nearly a duplicate of
`test-localhost`.
No longer copy a generated certificates to `tests/stunnel.pem`. Let test
server default to `certs/test-localhost.pem` instead.
Closes#16593
Our test servers run either on ipv4 *or* on ipv6, as requested.
A test case using 'localhost' or '*.local' in the url needs to
run with the specific version of the server started.
Otherwise, curl's "happy eyeball"ing will connect to another
server that may be running due to parallel testing or for some
other reasons.
Note that port reuse here depends on the OS strategy and it
seems netbsd is especially likely to hit this.
Closes#15060
The threee tags `<name>`, `</name>` and `<command>` were frequently used
with a leading space that this removes. The reason this habbit is so
widespread in testcases is probably that they have been copy and pasted.
Hence, fixing them all now might curb this practice from now on.
Closes#12028
Adds Schannel variants of SSLpinning tests that include the option
--ssl-revoke-best-effort to ignore certificate revocation check
failures which is required due to our custom test CA certificate.
Disable the original variants if the Schannel backend is enabled.
Also skip all IDN tests which are broken while using an msys shell.
This is a step to simplify test exclusions for Windows and MinGW.
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes#7968
Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were
previously disabled.
The Pipelining code was removed from curl in commit 2f44e94efb,
April 2019.
Closes#5921
Prior to this change tests that required NTLM feature did not require
SSL feature.
There are pending changes to cmake builds that will allow enabling NTLM
in non-SSL builds in Windows. In that case the NTLM auth strings created
are different from what is expected by the NTLM tests and they fail:
"The issue with NTLM is that previous non-SSL builds would not enable
NTLM and so the NTLM tests would be skipped."
Assisted-by: marc-groundctl@users.noreply.github.com
Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729
Closes https://github.com/curl/curl/pull/4768
Both these tests run the same underlying test code: libntlmconnect.c -
this test code made some assumptions about socket ordering when it used
curl_easy_fdset() and when we changed timing or got accidental changes
in libcurl the tests would fail.
The tests verify that the different transfers keep using the same
connections, which I now instead made sure by adding the number of bytes
each transfer gets and then verifies that they always get the same
amount as when these tests worked.
Closes#1576
According to RFC 2616 and RFC 2326 individual protocol elements, like
headers and except the actual content, are terminated by using CRLF.
Therefore the test data files for these protocols need to contain
mixed line-endings if the actual protocol elements use CRLF while
the file uses LF.
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.
Two new options for limiting the number of connections:
CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.
CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.
Several new options for pipelining:
CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).
CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.
CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.
CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.
CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.
See the curl_multi_setopt() man page for details.
