Commit Graph

39 Commits

Author SHA1 Message Date
Stefan Eissing
1ea99afdc7
scorecard: add perf support on linux
When calling scorecard with --flame to produce a flamegraph, use
"perf" on linux platforms to do the measurements. Update the scorecard
documentation about it.

Closes #19058
2025-10-14 16:24:35 +02:00
Viktor Szakats
45438c8d6f
checksrc: reduce directory-specific exceptions
By making them defaults, then fixing and/or reshuffling remaining
exceptions as necessary.

- checksrc: ban by default: `snprintf`, `vsnprintf`, `sscanf`, `strtol`.
- examples: replace `strtol` with `atoi` to avoid a checksrc exception.
- tests/libtest: replace `strtol` with `atol`.
- tests/server: replace most `strtol` with `atol`.
- tests/server: replace most `strtoul` with `atol`/`atoi`.
- tests/server: drop no longer used `util_ultous`.
- fix typo in checksrc rules: `vsnprint` -> `vsnprintf`.
- update local exceptions.

Also:
- examples: ban curl printf functions. They're discouraged in user code.
- examples: replace curl printf with system printf.
  Add `snprintf` workaround for <VS2015.
- examples/synctime: fix `-Wfloat-equal`.
- examples/synctime: exclude for non-Windows and non-UWP Windows.
- examples/synctime: build by default.

Closes #18823
2025-10-04 00:48:58 +02:00
Viktor Szakats
20142f5d06
build: avoid overriding system symbols for fopen functions
By introducing wrappers for them in the curlx namespace:
`curlx_fopen()`, `curlx_fdopen()`, `curlx_fclose()`.

The undefine/redefine/`(function)()` methods broke on systems
implementing these functions as macros. E.g. AIX 32-bit's `fopen()`.

Also:
- rename `lib/fopen.*` to `lib/curl_fopen.*` (for `Curl_fopen()`)
  to make room for the newly added `curlx/fopen.h`.
- curlx: move file-related functions from `multibyte.c` to `fopen.c`.
- tests/server: stop using the curl-specific `fopen()` implementation
  on Windows. Unicode isn't used by runtests, and it isn't critical to
  run tests on longs path. It can be re-enabled if this becomes
  necessary, or if the wrapper receives a feature that's critical for
  test servers.

Reported-by: Andrew Kirillov
Bug: https://github.com/curl/curl/issues/18510#issuecomment-3274393640

Follow-up to bf7375ecc5 #18503
Follow-up to 9863599d69 #18502
Follow-up to 3bb5e58c10 #17827

Closes #18634
2025-09-30 01:10:36 +02:00
Viktor Szakats
ad26a6cb99
tidy-up: avoid using the reserved macro namespace
To avoid hitting `-Wreserved-macro-identifier` where possible.

- amigaos: introduce local macro instead of reusing `__request()`.
- easy_lock: avoid redefining `__has_builtin()`.
  Follow-up to 33fd57b8ff #9062
- rand: drop interim macro `_random()`.
- windows: rename local macro `_tcsdup()` to `Curl_tcsdup()`.
  To avoid using the reserved macro namespace and to avoid
  colliding with `_tcsdup()` as defined by Windows headers.
- checksrc: ban `_tcsdup()` in favor of `Curl_tcsdup()`.
- tool_doswin: avoid redefining `_use_lfn()` (MS-DOS).
- tool_findfile: limit `__NO_NET_API` hack to AmigaOS.
  Syncing this pattern with `lib/netrc.c`.
  Follow-up to 784a8ec2c1 #16279
- examples/http2-upload: avoid reserved namespace for local macro.

More cases will be removed when dropping WinCE support via #17927.

Cases remain when defining external macros out of curl's control.

Ref: #18477
Closes #18482
2025-09-20 02:27:06 +02:00
Viktor Szakats
3298a43133
spelling: file system
Closes #18348
2025-08-22 15:53:41 +02:00
Stefan Eissing
6b70e8a838
pytest: use dante-server in CI
- add startup check for 'danted' to avoid fails on low cpu
- rename 'sockd' to 'danted' everywhere to clarify what we use
- add proper defaults for 'danted' for debian
- install 'dante-server' in pytest ci runs

Closes #18075
2025-07-29 15:02:30 +02:00
Viktor Szakats
b2bccdc257
tidy-up: move literal to the right side of comparisons
Closes #17876
2025-07-27 22:35:17 +02:00
Viktor Szakats
63a8167f3e
CODE_STYLE: sync with recent checksrc.pl updates
Follow-up to ef2ccf862f #17840
Follow-up to f9656445ba #17764

Closes #18015
2025-07-25 03:10:01 +02:00
Stefan Eissing
ab5e0bfddc
pytest: add SOCKS tests and scoring
Configure curl with `--with-test-sockd=<path to sockd>` for a locally
installed dante sockd server and new `test_40_*` will verify that
down- and uploads work via SOCKS.

Invoke scorecard.py with `--socks4` or `--socks5` to run performance
tests with SOCKS. Note that SOCKS is not supported for HTTP/3.

Ref: #17969
Closes #17986
2025-07-23 18:12:31 +02:00
Stefan Eissing
51f933801e
scorecard: flame graphs and documentation
Add `--flame` option to scorecard.py for generating flame graphs.
Add documentation in docs/internal/SCORECARD.md on how to use this.

Closes #17792
2025-07-07 09:04:22 +02:00
Daniel Stenberg
65f8253336
bufq: remove the unused Curl_bufq_unwrite function
Follow-up to 02edae54e8

Closes #17730
2025-06-24 10:56:54 +02:00
Daniel Stenberg
fe81a80ae7
spelling: call it null-terminate consistently
With a dash, using two Ls. Also for different forms of the word.

Use NULL in all uppercase if it means a zero pointer.

Follow-up to 307b7543ea

Closes #17489
2025-05-30 17:29:45 +02:00
Daniel Stenberg
bdb7d8b004
spelling: 'a' vs 'an'
Closes #17487
2025-05-30 11:38:35 +02:00
Daniel Stenberg
65444f1dc3
curlx: move the docs to docs/internals/
Closes #17333
2025-05-13 10:30:02 +02:00
Daniel Stenberg
255aac56f9
curlx: move into to curlx/
Move curlx_ functions into its own subdir.

The idea is to use the curlx_ prefix proper on these functions, and use
these same function names both in tool, lib and test suite source code.
Stop the previous special #define setup for curlx_ names.

The printf defines are now done for the library alone. Tests no longer
use the printf defines. The tool code sets its own defines. The printf
functions are not curlx, they are publicly available.

The strcase defines are not curlx_ functions and should not be used by
tool or server code.

dynbuf, warnless, base64, strparse, timeval, timediff are now proper
curlx functions.

When libcurl is built statically, the functions from the library can be
used as-is. The key is then that the functions must work as-is, without
having to be recompiled for use in tool/tests. This avoids symbol
collisions - when libcurl is built statically, we use those functions
directly when building the tool/tests. When libcurl is shared, we
build/link them separately for the tool/tests.

Assisted-by: Jay Satiro

Closes #17253
2025-05-07 11:01:15 +02:00
Stefan Eissing
657aae79c0
lib: add meta_hash to connection, eliminate hash_offt
With a meta_hash at each connection (similar to easy handle, let
multi_ev.c store its pollsets as meta data, no longer needing its own
hashes.

This eliminates the last use of Curl_hash_offt. Remove it.

Closes #17095
2025-04-22 15:57:18 +02:00
Brian Chrzanowski
c0df01fd94
websocket: add option to disable auto-pong reply
This adds another bitflag on CURLOPT_WS_OPTIONS (CURLWS_NOAUTOPONG) that
disables the default and automatic PONG reply in the WebSocket layer.

Assisted-by: Calvin Ruocco

Closes #16744
2025-04-19 00:01:28 +02:00
Stefan Eissing
909af1a43b
multi: do transfer book keeping using mid
Change multi's book keeping of transfers to no longer use lists, but a
special table and bitsets for unsigned int values.

`multi-xfers` is the `uint_tbl` where `multi_add_handle()` inserts a new
transfer which assigns it a unique identifier `mid`. Use bitsets to keep
track of transfers that are in state "process" or "pending" or
"msgsent".

Use sparse bitsets to replace `conn->easyq` and event handlings tracking
of transfers per socket. Instead of pointers, keep the mids involved.

Provide base data structures and document them in docs/internal:
* `uint_tbl`: a table of transfers with `mid` as lookup key,
   handing out a mid for adds between 0 - capacity.
* `uint_bset`: a bitset keeping unsigned ints from 0 - capacity.
* `uint_spbset`: a sparse bitset for keeping a small number of
  unsigned int values
* `uint_hash`: for associating `mid`s with a pointer.

This makes the `mid` the recommended way to refer to transfers inside
the same multi without risk of running into a UAF.

Modifying table and bitsets is safe while iterating over them. Overall
memory requirements are lower as with the double linked list apprach.

Closes #16761
2025-04-17 17:28:38 +02:00
Christian Schmitz
8ad0243e1f
mqtt: send ping at upkeep interval
Closes #16975
2025-04-16 09:36:19 +02:00
Daniel Stenberg
c4863e9d3b
docs/internals/PORTING.md: new document
The basic approach I use when porting libcurl to another OS when the
existing configure or cmake build setups are not suitable.

Closes #16860
2025-03-29 14:10:10 +01:00
Daniel Stenberg
06ae0eceb9
STRPARSE: amend with recently added functions
Closes #16647
2025-03-10 12:18:48 +01:00
tiymat
6758aa722d
IMAP: add CURLOPT_UPLOAD_FLAGS and --upload-flags
Set properties on the uploaded resource.

Test 3209 and 3210 verify.

Closes #15970
2025-03-04 15:21:16 +01:00
Daniel Stenberg
4c5099868e
http: make the RTSP version check stricter
- make it only accept version 1.0, as that is the version curl supports
- convert the parser to use strparse
- the status code max is now 999, but it does allow != 3 digits

Closes #16435
2025-02-22 15:07:31 +01:00
Stefan Eissing
cfc657a48d
multi: event based rework
Rework the event based handling of transfers and connections to
be "localized" into a single source file with clearer dependencies.

- add multi_ev.c and multi_ev.h
- add docs/internal/MULTI-EV.md to explain the overall workings
- only do event handling book keeping when the socket callback
  is set
- add handling for "connection only" event tracking, when internal
  easy handles are used that are not really tied to a connection.
  Used in connection pool.
- remove transfer member "last_poll" and connections "shutdown_poll"
  and keep all that internal to multi_ev.c
- add CURL_TRC_M() for tracing of "multi" related things, including
  event handling and connection pool operations. Add new trace
  feature "multi" for trace config.
  multi traces will show exactly what is going on in regard to
  event handling.
- multi: trace transfers "mstate" in every CURL_TRC_M() call
- make internal trace buffer 2048 bytes and end the silliness
  with +n here -m there. Adjust test 1652 expectations of resulting
  length and input edge cases.
- add trace feature "lib-ids" to perfix libcurl traces with transfer
  and connection ids. Useful for debugging libcurl applications.

Closes #16308
2025-02-22 14:47:40 +01:00
Daniel Stenberg
f0d7318193
strparse: provide access functions
To access the string and the length without having to directly use the
struct field names. Gives more freedom, flexbility and keeps
implementation specifics out of users' code.

Closes #16386
2025-02-19 12:17:32 +01:00
Daniel Stenberg
bd15d8beb3
STRPARSE.md: sync with recent changes
New functions and Curl_str_number() changed number return type.

Closes #16365
2025-02-17 14:40:52 +01:00
Daniel Stenberg
37128035e5
CODE_STYLE: readability and banned functions
Closes #16349
2025-02-16 16:11:13 +01:00
Yedaya Katsman
38ad3902bf
docs/TLS-SESSIONS: fix typo, the->they
Closes #16057
2025-01-21 09:23:26 +01:00
Daniel Stenberg
3eb57d6ba7
docs: use lowercase curl and libcurl
Adjusted badwords to find them.

Plus: make badwords run on all markdown files in the repo and update
markdowns previously unchecked

Closes #15898
2025-01-02 17:15:54 +01:00
Daniel Stenberg
fc3e1cbc50
hyper: drop support
lib : remove all hyper code
configure: stop detecting hyper
docs: no more mention of hyper
tests: mo more special-handling of hyper builds
CI: no jobs using hyper

Closes #15120
2024-12-21 11:33:05 +01:00
Stefan Eissing
fa0ccd9f1f
lib: TLS session ticket caching reworked
Described in detail in internal doc TLS-SESSIONS.md

Main points:
- use a new `ssl_peer_key` for cache lookups by connection filters
- recognize differences between TLSv1.3 and other tickets
  * TLSv1.3 tickets are single-use, cache can hold several of them for a peer
  * TLSv1.2 are reused, keep only a single one per peer
- differentiate between ticket BLOB to store (that could be persisted) and object instances
- use put/take/return pattern for cache access
- remember TLS version, ALPN protocol, time received and lifetime of ticket
- auto-expire tickets after their lifetime

Closes #15774
2024-12-20 14:59:23 +01:00
Daniel Stenberg
d5c738c608
strparse: string parsing helper functions
Designed to aid converting off from sscanf parsers. sscanf is hard to
use right, easy to mess up and often makes for sloppy error checking.

The new parsers allow more exact and pedandic parsing.

This new set of functions should be possible to use (and extend) and
switch over other libcurl parser code to use going forward.

Adapts the following to use the new functions:

- altsvc.c
- hsts.c
- http_aws_sigv4.c

Bonus: fewer memory copies, fewer stack buffers.

Test: Unit test1664

Docs: docs/internals/STRPARSE.md

Closes #15692
2024-12-12 16:00:52 +01:00
Viktor Szakats
9acecc923d
tidy-up: whitespace, fix CI spacecheck for docs
Also: fixup CI spacecheck to apply to docs again.

Closes #15423
2024-10-30 23:15:28 +01:00
Daniel Stenberg
c0d2b9bee7
MQTT: remove trailing newline 2024-10-30 14:04:08 +01:00
Marwan Yassini
e4aa07b526
mqtt: fix mqtt.md wording and add clearer explanation
Some of the wording in the mqtt.md confused me as to how the commands
were used and what they did, so I cleared up some of the wording to
better explain what each command does.

Closes #15451
2024-10-30 08:44:05 +01:00
Viktor Szakats
3040971d12
GHA: silence proselint warnings and an error
Fix new issues found by `proselint`.

Also:
- silence this technical warning:
  ```
  :0: DeprecationWarning: /home/runner/.proselintrc was found instead of a JSON file. Rename to /home/runner/.proselintrc.json.
  ```
- fix an input filename.
  `proselints` fails now if an input file is missing.

Reported-by: Jay Satiro
Bug: https://github.com/curl/curl/pull/15291#issuecomment-2410505100
Closes #15293
2024-10-15 16:44:17 +02:00
Stefan Eissing
2400a6c6b2
bufq: unwrite fix
`Curl_bufq_unwrite()` used the head instead of the tail chunk to shrink
the bufq's content. Fix this and add test case that checks correct
behaviour.

Amended test 2601 accordingly.

Reported-by: Chris Stubbs
Closes #15136
2024-10-03 13:27:36 +02:00
Daniel Stenberg
f9f2eaaec3
internals/SPLAY.md: internal API documentation
Closes #14563
2024-08-16 17:13:37 +02:00
Daniel Stenberg
20aa8d8f31
docs/internals: new subdirectory
For libcurl internal APIs and concepts.

Closes #14552
2024-08-16 08:59:25 +02:00