Rename `Curl_timeleft()` to `Curl_timeleft_ms()` to make the units in
the returned `timediff_t` clear. (We used to always have ms there, but
with QUIC started to sometimes calc ns as well).
Rename some assigned vars without `_ms` suffix for clarity as well.
Closes#19486
Overhaul of the internal cookie APIs and an attempt to better return
errors for OOM and similar critical problems, separate from ordinary and
benign parsing problems.
Closes#19493
Some curl command-lines are long, often repetitive, and difficult
to read or write:
Before this patch (1 test == 1 line):
- <=78 characters: 1099 tests
- 79-132 characters: 500 tests
- 133+ characters: 217 tests: patch meant to help with some of these.
After this patch:
- <=78 characters: 1288 lines
- 79-132 characters: 526 lines
- 133+ characters: 190 lines
After this patch it's possible to fold long lines into multiple ones.
Folding can reduce greppability, thus this is primarily useful for cases
when the options are repetitive, e.g. a list of form options, headers,
mail parameters and the like.
Closes#19500
To untangle the different curl/server requirements of these tests.
Also to make this test run for non-H2 builds. Searching the cause of
the flakiness documented in #19481.
Also:
- fix the callback function prototype. Detected by ASAN with this patch,
though the issue was pre-existing.
```
lib/cw-out.c:211:14: runtime error: call to function emptyWriteFunc
through pointer to incorrect function type
'unsigned long (*)(char *, unsigned long, unsigned long, void *)'
tests/libtest/lib2405.c:72: note: emptyWriteFunc defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior lib/cw-out.c:211:14
```
Ref: https://github.com/curl/curl/actions/runs/19296686908/job/55180334364?pr=19487#step:44:3768
Follow-up to 96a5ce5a82#19481Closes#19487
To better reject junk and detect overflows. There were already
additional precautions and protections in place, but this is cleaner.
Extended the 1614 unit tests with some more bad syntax cases.
Closes#19475
Also:
- replace `manpage` with `man page`, add to `badwords.txt`.
- badwords.pl: import `-w` feature from curl-www, syncing the two
scripts fully.
- badwords.txt: import missing items from curl-www, syncing the two
files fully.
- pyspelling.words: drop `cURL` allowed word.
Closes#19468
When asking for the last N bytes of a file, and that size was larger
than the file size, it would miss the first byte due to a logic error.
The fixed range parser is now made a common function in the file now
renamed to vssh.c (from curl_path.c) - used by both backends.
Unit test 2605 verifies the parser.
Reported-by: Stanislav Fort (Aisle Research)
Closes#19460
Data files no longer depend on mixed newline styles. Before this
patch the harness still assumed data files to use LF newlines,
ensured by `.gitattribute` and distributing sources with LF newlines.
To allow using platform native newlines (CRLF on Windows typically),
update the test harness to support data files with any newline style
on disk. And delete `.gitattributes`.
Fix to:
- load original data files (from test/data) so that their newline-style
doesn't matter on the checked out source repo, meaning it works
when its CRLF on Windows, just like any other file.
(if a BOM slips in, it's caught by `spacecheck.pl` as binary content.)
- do the same in `util.py` used by `smbserver.py` (for test 1451).
- also fix `util.py` to use us-ascii encoding for data files, replacing utf-8.
Also:
- runtests: rework the stray CR checker to allow full CRLF data files,
and keep warning for mixed newlines.
Follow-up to 904e7ecb66#19347Closes#19398
Before this patch servers were loading the original data source file
(from `tests/data/test*`) if they failed to open the preprocessed data
file.
It was causing issues in many (most?) tests, because original data files
are not preprocessed, thus may be incomplete and/or come with wrong
newline characters. It's also causing difficult to diagnose issues when
a test accidentally references another test's data, which by chance
makes the test pass initially, until either that or the executed test
data gets an update, and breaking it, as seen in #19329.
Historically, the fallback existed first, then the preprocessed copy.
The fallback is no longer used by tests (except by stray accidents).
Fix it by dropping the fallback logic and relying on the preprocessed
data file saved there by the runtests framework.
Also fix two remaining test data cross-references:
- test1565: reference own server input data instead of test1's.
- test3014: reference own server input data instead of test1439's.
Ref: #19398
Follow-up to aaf9522a2c#19329Closes#19429
curl is for transfers so disabling all protocols has to be a mistake.
Previously it would allow this to get set (even if curl_easy_setopt()
returns an error for it) and then let libcurl return error instead.
Updated 1474 accordingly.
Closes#19388
There is no more mixed-newline file in the repository after this patch.
Except for`.bat` and `.sln` files (4 in total), all files use LF
newlines.
Also:
- `spacecheck.pl`: drop mixed-EOL exception for test data.
- runtests: add option `-w` to check if test data has stray CR bytes in
them.
- build: enable the option above in test targets, except the CI-specific
one where `spacecheck.pl` does this job already.
- tested OK (with expected failures) in CI with stray CRs added.
- cmake: enable option `-a` for the `tests` target. To continue testing
after a failed test.
Follow-up to 63e9721b63#19313
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19347
Use the %VERSION instead. The user-agent stripping was introduced at the
time before we had %VERSION (introduced in e6b21d4). The tests would
then remove the user-agent header to make them possible to be compared
in a version independent way.
Fixes#19355
Reported-by: Stefan Eissing
Closes#19356
- `reply/data*`, `verify/stdout`, `verify/stderr`, `verify/file*`,
`verify/proxy`:
- make `crlf="yes"` force CRLF to all lines, instead of just applying
to HTTP protocol headers.
- add support for `crlf="headers"` that only converts HTTP protocol
header lines to CRLF. (previously done via `crlf="yes"`.)
- use `crlf="headers"` where possible.
- `reply/connect*`:
- add support for `crlf="yes"` and `crlf="headers"`.
- use them where possible.
- `client/file*`, `client/stdin`:
- add support for `crlf="yes"`.
- use it where possible.
- `reply/data*`, `verify/protocol`:
- replace existing uses of `crlf="yes"` with `crlf="headers`" where it
does not change the result.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 10295 to 1985. (119985 lines total)
- files with mixed newlines from 656 to 113. (1890 files total)
After this patch there remain 141 sections with mixed newlines, where
the mixing is not split between headers/non-headers. There is no obvious
pattern here. Some of the CRLF uses might be accidental, or
non-significant. They will be tackled in a future patch.
Follow-up to 6cf3d7b1b1#19318
Follow-up to 4d2a05d3fe#19284Closes#19313
The test FTP server returned LF newlines for 227/229 replies, instead of
the CRLF used for the rest.
Test data added later were explicitly made to expect an LF in these
response lines.
After this patch the FTP server returns CRLF newlines, allowing
to delete this special case in test data.
Follow-up to 3bfff57e1f
Follow-up to a7937ed49cCloses#19330
Test 696 and 556 share the same libtest code. Make sure to issue
the `GET` request to the correct runtime test number instead of using
the hard-wired "556".
It makes the `sws` test server read the response string from `test696`
`<data>` section, instead of reading it from `test556`. To avoid this
hidden interaction between test data.
AFAICS there is no other similar hard-coded string in reused libtests.
Ref: https://github.com/curl/curl/pull/19313#issuecomment-3477448933
Follow-up to be82a3605a#16003Closes#19329
- fix regex to not catch CR (from CRLF), in `PORT`, `EPRT`
commands, allowing to use `crlf="yes"` more.
- add `crlf="headers"` mode for `protocol` sections.
To call `subnewlines()` without its force option.
This is the mode used in `data` sections when `crlf="yes"`.
(This confusion may be subject to a future commit.)
- subnewlines: apply CRLF to `HEAD` and `CONNECT` HTTP requests.
- subnewlines: apply CRLF to RTSP requests.
- delete remaining empty `protocol` sections.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 11325 to 10295. (119984 lines total)
- files with mixed newlines from 707 to 656. (1890 files total)
Follow-up to 4d2a05d3fe#19284Closes#19318
When the target host is on a different port than 443, the name
"_[port]._https.[name]" shall be used.
Fixes#19301
Reported-by: Gunni on github
Closes#19324
To make them explicit, visible, avoid being accidentally trimmed.
Also prevents Git warnings, e.g. on `git am`.
Also:
- runtests: add support for `%spc%` and `%tab%` macros.
- test59: delete non-significant line-ending space.
- spacecheck.pl: drop line-ending whitespace exception for tests.
Closes#19300
To make special newlines more explicit and visible.
Mostly in `<protocol>` sections, some in `<data*>` and `<upload>`.
Reducing the number of `tests/data/test*`:
- CRLF newlines from 21535 to 11337.
- files with mixed newlines from 1335 to 707.
Also delete empty `<protocol>` sections.
Closes#19284
Custom IMAP commands using -X (e.g. 'FETCH 123 BODY[1]') were only
returning the first line of responses containing literals, instead of
the full multi-line body data.
The issue was that custom commands route through imap_perform_list()
and imap_state_listsearch_resp(), which didn't detect or handle IMAP
literal syntax {size}.
This commit adds literal detection to imap_state_listsearch_resp():
- Detects literal syntax {size} in untagged responses
- Writes the response header line containing the literal marker
- Handles any literal body data already in the pingpong buffer
- Sets up transfer layer to read remaining literal data from socket
- Configures maxdownload and transfer size to include header + body
- Initializes pp->overflow to 0 when no buffered data present
- Modifies imap_done() to transition to FETCH_FINAL for custom
commands that set up downloads
Test 841 and 3206 verify.
Fixes#18847
Reported-by: BohwaZ
Bug: https://github.com/curl/curl/issues/18847Closes#19246
SMTP automatically appends a \n.\n to an upload if there is not already
one at the end of the input. The implementation had a bug where this did
not happen, depending on read size and buffering.
Change test 900 to reproduce the failure. The bug only happened for mail
body input of known length, where EOS was known on the last chunk read.
Change test 900 to use an input file and make it large enough.
Fixes#18798Closes#19193
Reported-by: madoe on github
When a reused connection did transfer 0 bytes, it assumed the transfer
had failed and needed a retry. Add a check for data->red.done, so we can
successfully accept the transfer of a 0-length file via SFTP.
Add test case 1583 to verfiy.
Fix SFTP disconnect debug trace when there was nothing to
disconnect (like when reusing a connection).
Fixes#19165
Reported-by: Alexander Blach
Closes#19189
