Commit Graph

60 Commits

Author SHA1 Message Date
Viktor Szakats
3b43b9080d
badwords: dedupe invocation into a runnable script
Also:
- GHA/checksrc: merge two badwords CI jobs into a single one.
- re-add the more common bitness entries.

Closes #20874
2026-03-10 17:44:15 +01:00
Daniel Stenberg
713287188e
badwords: move into ./scripts, speed up
- 'badwords' is now a target in Makefile.am

- change badwords.txt to specify plain "words" instead of regexes so the
  script can build single regexes when scanning, which makes the script
  perform much faster (~6 times faster)

Closes #20869
2026-03-09 22:47:07 +01:00
dependabot[bot]
47734f3244
GHA: bump GHA and pip dependencies
- update `actions/checkout` from 6.0.1 to 6.0.2
- update `ruff` from 0.14.11 to 0.14.14
- update `cryptography` from 46.0.3 to 46.0.4
- update `psutil` from 7.2.1 to 7.2.2
- update `websockets` from 15.0.1 to 16.0

Closes #20490
Closes #20491
2026-02-01 16:40:37 +01:00
Viktor Szakats
86190dccb3
GHA: migrate 3 linter jobs to arm64
Also to make them finish as fast or overall faster.

checkdocs/proselint:
before: https://github.com/curl/curl/actions/runs/21255607528/job/61169136666 22s
after: https://github.com/curl/curl/actions/runs/21266680535/job/61207470652?pr=20403 19s

checksrc/spellcheck, linters, REUSE:
before: https://github.com/curl/curl/actions/runs/21255607522/job/61169137076 3m11s
after: https://github.com/curl/curl/actions/runs/21266680282/job/61207475282?pr=20403 2m27s
typos: 13s -> 28s
pytype: 2m24s -> 1m24s

checksrc/misc checks:
before: https://github.com/curl/curl/actions/runs/21255607522/job/61169137176 2m29s
after: https://github.com/curl/curl/actions/runs/21266680282/job/61207475248?pr=20403 2m5s
shellcheck: seems larger and slower on arm64. 33 -> 45MB, 1s -> 3s
badwords: 2m1s -> 1m35s

Closes #20403
2026-01-22 23:44:47 +01:00
Viktor Szakats
3ee1d3b573
tidy-up: merge root packages directory into projects
To simplify the directory layout.

- OS400 and vms support move from `packages` to `projects`.

- Windows README and `generate.bat` files move from `projects`
  to `projects/Windows`.

Closes #20271
2026-01-12 23:49:35 +01:00
Viktor Szakats
9dc2552ed1
GHA/checkdocs: drop duplicate spacecheck job
No longer necessary after making GHA/checksrc also run on `*.md` file
changes.

Reported-by: Daniel Stenberg
Bug: https://github.com/curl/curl/pull/20266#issuecomment-3738955165
Follow-up to 3800a26582 #18935
Follow-up to 9acecc923d #15423
Closes #20268
2026-01-12 16:31:23 +01:00
Viktor Szakats
1b24428d62
GHA: delete disable-man-db hack, runners doing it by default now
Refs:
1f107542ae
https://github.com/actions/runner-images/pull/13268
https://github.com/actions/runner-images/issues/13213

Follow-up to 5acba8bc36 #17181

Closes #20254
2026-01-11 15:40:21 +01:00
Viktor Szakats
a6087d69a9
GHA: delete new apt sources adding the flaky distro source
CI workflows deleted apt package source `microsoft-prod.list` due to
flakiness seen in the past with those sources. Sources are still flaky
and they are now also used from `azure-cli.sources` which is included
by default by the runner images. Add it to the delete list.

Also: remove another new, unnecessary (for curl CI) package source seen
on the `ubuntu-24.04-arm` runner, for good measure, and performance.

Fixing:
```
Reading package lists...
E: Failed to fetch https://packages.microsoft.com/repos/azure-cli/dists/noble/InRelease  403  Forbidden [IP: 13.107.246.66 443]
E: The repository 'https://packages.microsoft.com/repos/azure-cli noble InRelease' is no longer signed.
Error: Process completed with exit code 100.
```
Ref: https://github.com/curl/curl/actions/runs/20896127602/job/60034941964?pr=20142#step:2:79

Follow-up to 303bb8785c #13473

Closes #20253
2026-01-11 15:25:19 +01:00
Viktor Szakats
2b0d8dcc16
GHA: switch 12 Linux jobs to arm64
More cost-effective and seems as fast or faster than Intel.

In particular, valgrind seems to be almost 2x fast. So fast the job pair
could fit under 10 minutes if merged again (but would be the longest in
GHA/Linux.)

Installing packages is slightly slower. The package repo is Ubuntu's
which is slower than the Azure mirror used on Intel (unless Azure is
broken, which happened a lot last year).

To add to more jobs, the locally built deps also need to be migrated.

Also:
- add workaround for failing sshd server on Linux arm runners, caused by
  world-writable `HOME` directory.

Closes #20231
2026-01-09 19:35:16 +01:00
renovate[bot]
e6146418df
GHA: update actions
- update actions/checkout action to v6.0.1
- update github/codeql-action action to v4.31.8
- update msys2/setup-msys2 action to v2.30.0

Closes #19962
Closes #19967
Closes #19968
2025-12-14 09:17:31 +01:00
Viktor Szakats
46429d6f44
GHA/checkdocs: re-enable proselint, update setup, fix issues found
- update configuration and invocation.
- install via pip.
- drop a file exception.
- alpha sort proselint settings.
- FILEFORMAT: update text about XML compliance.
- CI job takes 22 seconds total.

Ref: https://github.com/amperser/proselint/releases/tag/v0.16.0

Follow-up to 38bfe1c2aa #15314

Closes #19931
2025-12-11 11:42:28 +01:00
Daniel Stenberg
073b85cefe
GHA: make links get checked daily
Assisted-by: Viktor Szakats
Closes #19898
2025-12-09 16:28:23 +01:00
Daniel Stenberg
3dd1ffdeb0
FAQ/TODO/KNOWN_BUGS: convert to markdown
- convert to markdown
- auto-generate the TOCs on the website, remove them from the docs
- cleanups
- spellchecked
- updated links

Closes #19875
2025-12-09 10:52:56 +01:00
Daniel Stenberg
51587f6f14
mdlinkcheck: detect and check "raw" links
- URLs specified outside of the markdown []() are now extracted and
  checked

- also check TODO, FAQ and KNOWN_BUGS

- more aggressive avoiding to check github.com/curl/curl, all uses of
  example domains and some more established URLs on the curl.se site

- list all errors in the end to make them easier to spot in CI logs

Closes #19848
2025-12-05 23:41:41 +01:00
Viktor Szakats
7a1e99eefa
badwords: check FAQ with allowlisted 'will', fix a typo
Also:
- badwords.pl: add support for filename:word exceptions.
- badwords.pl: handle `-w` file open errors.

Ref: https://github.com/curl/curl/pull/19817#issuecomment-3612386568
Closes #19837
2025-12-04 19:30:08 +01:00
renovate[bot]
c5ef882e90
GHA: update actions/checkout action to v6 from v5
Closes #19628
2025-11-20 22:27:00 +01:00
Viktor Szakats
231e8a71e1
docs: fix checksrc warning, fix checkdocs CI filter
Also:
- GHA/checkdocs: fix CI filters to catch it early.
  Follow-up to 28dd14aafe #15797

Closes #19533
2025-11-14 17:32:04 +01:00
Viktor Szakats
b39c158e4a
scripts: fix shellcheck SC2046 warnings
Fix SC2046: "Quote this to prevent word splitting."
Ref: https://www.shellcheck.net/wiki/SC2046

Also:
- shellcheck.sh: add `set -eu`.
- shellcheck.sh, yamlcheck.sh: always run from repo root.
- pass `--` before passing the list of files, where missing.
- badwords.pl, cleancmd.pl: rework to accept `git ls-files` arguments.
  Requires Perl 5.22+ (2015-Jun-01) on Windows.
  Ref: https://perldoc.perl.org/functions/open
- INTERNALS.md: require Perl 5.22 on Windows.
- spacecheck.pl: formatting.
- GHA/http3-linux: rework command to avoid SC2046.
- stop suppressing SC2046 warnings.

The yamlcheck.sh issue reported-by: Stanislav Fort (Aisle Research)
Ref: 20251109163515_6eb31da3-deb2-4f4d-8327-935904f27da5

Closes #19432
2025-11-10 14:21:35 +01:00
Viktor Szakats
c8aaa5d2f2
scripts: pass -- before passing xargs
Also:
- GHA/checkdocs: escape `.` in -E regex expression.

Closes #19076
2025-10-16 20:45:28 +02:00
Viktor Szakats
801ebf1e1a
GHA: rename config files to match pyspelling
To make it more obvious what needs to be looked at when pyspelling is
reporting an issue.

Follow-up to 95e50ad694 #18756
Closes #18974
2025-10-09 14:17:32 +02:00
Viktor Szakats
e73759f1a9
GHA: show full versions next to pinned actions
Also quotes to a configuration entry.

Follow-up to 2e5993ab08 #18827

Closes #18832
2025-10-03 16:34:44 +02:00
Viktor Szakats
95e50ad694
tidy-up: miscellaneous
- GHA/checkdocs: rename `spellcheck` job to `pyspelling` to say
  the exact tool used.
- GHA/checkdocs: restore a comment.
- GHA/linux: add `-B .` to a cmake configure to avoid warning, and
  future breakage.
- autotools: use correct casing for `Schannel`.
- doh: update RFC URL.
- drop redundant parenthesis.
- fix indentation, whitespace.

Closes #18756
2025-09-27 12:59:07 +02:00
Viktor Szakats
e48c1ea415
GHA: use pyspelling directly
To avoid depending on Docker Hub, an Docker image and a GitHub Action.
Also to simplify running this check on a local machine.

Pending question if Dependabot and Mend/Renovate will automatically pick
up `requirements-docs.txt`.

Also:
- enable parallel spellchecking. (also to win back the time lost with
  installing components directly from Debian and pip.)
- pin `pyspelling`.
- link to official `pyspelling` docs.

Closes #18736
2025-09-25 16:59:27 +02:00
Viktor Szakats
b2356a3197
GHA: tidy up actions/checkout version in comments [ci skip] 2025-09-19 14:20:14 +02:00
renovate[bot]
7417f14b93
GHA: update rojopolis/spellcheck-github-actions digest to 739a1e3
Closes #18515
2025-09-11 08:40:09 +02:00
renovate[bot]
c0e63bab6f
GHA: update actions/checkout action to v5
Closes https://github.com/curl/curl/issues/18250
2025-08-12 11:17:08 +02:00
renovate[bot]
4db91c4411
GHA: update actions/checkout digest to 08eba0b
Closes #18249
2025-08-11 14:59:35 +02:00
Viktor Szakats
48d3407d7c
GHA: fix zizmor 1.10.0 warnings, update names
Job `name:` now mandatory in zizmor.

Also:
- enclose `name:` values in single-quotes, for uniformity.
- drop `name: checkout` where set, for uniformity.
- dist: also install with cmake.
- dist: replace `make` with `cmake --build` for cmake.
  (to make this make-tool agnostic)
- appveyor-status: double-quote shell arguments.
- tweak existing names to be shorter, to sync terms and style across
  jobs and steps.

Ref: https://github.com/zizmorcore/zizmor/releases/tag/v1.10.0

Closes #17773
2025-06-30 18:38:56 +02:00
renovate[bot]
657e0240f5
GHA: update rojopolis/spellcheck-github-actions digest to 35a02ba
Closes #17686
2025-06-20 23:36:07 +02:00
renovate[bot]
260ec730c2
GHA: update rojopolis/spellcheck-github-actions digest to 63aba94
Closes #17643
2025-06-17 12:22:02 +02:00
Viktor Szakats
17a669426f
CI: fix zizmor 1.9.0 warnings, shellcheck verify CI shell code, fix fallouts
zizmor 1.9.0 effectively bans using GHA macros within shell script
snippets. Rework them to pass GHA macro values via envs and use those
envs in shell snippets. `${{ env.* }}` macros could be converted
to native env references after making their names shell-compatible.

Envs and shell commands can no longer be used in GHA macro values. Most
cases could be fixed by using literals. Passing quoted values with
spaces combined with other args also doesn't work anymore. This was
replaced by passing them separately.

Despite the initial complications, avoiding GHA macros in scripts does
seems to make the CI code reasonable cleaner. It also makes it possible
to analyze the scripts with shellcheck, finding subtle issues that went
unnoticed till now.

Also:
- un-ignore and fix three existing zizmor `template-injection` issues.
- add script to extract and shellcheck all shell code stored within GHA
  and Circle CI YAML files.
- add CI job to run this script.
- fix shellcheck issues discovered.
- fix minor differences between cmake and autotools FreeBSD jobs.
- merge cmake/autotools FreeBSD jobs to avoid developing unwanted
  differences again.
- fix/sync quote use across shell code.
- replace `$HOME` with `~` or literal where it made sense.
- replace most `brew --prefix` with literals.
- move all curl install tests to the `curl-install*` prefix.
- add missing curl install tests to cygwin/msys/mingw/*bsd.
- pipe to `tar` instead of storing downloads on disk.
- drop unnecessary `PKG_CONFIG_PATH` when building nghttp3.

Ref: https://github.com/curl/curl/actions/runs/15461461371/job/43523546041
Ref: https://github.com/zizmorcore/zizmor/releases/tag/v1.9.0

Follow-up to e522f47986 #17278

Closes #17537
2025-06-06 21:30:43 +02:00
renovate[bot]
76ab38d3b8
GHA: update rojopolis/spellcheck-github-actions digest to 584b2ae
Closes #17423
2025-05-23 11:42:35 +02:00
Viktor Szakats
5acba8bc36
GHA: skip updating man-db for faster installs (Ubuntu)
This step could take from 5 seconds to 5 minutes, sometimes making it
run out of its time slot. It affected 60 CI jobs.

Saving an estimated minimum of 5 minutes per CI run.

Also fixing:
```
Fri, 25 Apr 2025 06:19:14 GMT
Processing triggers for man-db (2.12.0-4build2) ...
Fri, 25 Apr 2025 06:23:40 GMT
Running kernel seems to be up-to-date.
[...]
Error: The action 'install packages' has timed out after 5 minutes.
```
Ref: https://github.com/curl/curl/actions/runs/14658212268/job/41136971525?pr=17180#step:2:169

Closes #17181
2025-04-25 10:17:36 +02:00
Daniel Stenberg
960984263f
docs/libcurl: make examples build with picky compiler options
Found by improving verify-examples.pl:

- Operate directly on markdown files to remove the need to render nroff files
  first.

- Add -Wall as a compiler option to find more issues

Closes #17028
2025-04-11 15:23:51 +02:00
dependabot[bot]
7e4d516bcb
GHA: bump rojopolis/spellcheck-github-actions to 0.48.0
Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.47.0 to 0.48.0.
- [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases)
- [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md)
- [Commits](ed0756273a...23dc186319)

---
updated-dependencies:
- dependency-name: rojopolis/spellcheck-github-actions
  dependency-version: 0.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Closes #16988
2025-04-08 16:58:10 +02:00
Daniel Stenberg
7007f59caa
GHA: spellcheck curl symbols better
This now makes sure to trim off exact matches for curl symbols and long
curl commanad line options instead of using pattern matching as before.
This should catch typoed names (that still follow the pattern) better.

The cleanspell.pl script is no longer used. cleancmd.pl is used for all
markdown files.

Closes #16504
2025-02-27 13:15:21 +01:00
dependabot[bot]
a74dd9f330
GHA: bump rojopolis/spellcheck-github-actions
Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 9e0a5fb25a80b89c84899657949cbd6e17eb376c to ed0756273a1658136c36d26e3d0353de35b98c8b.
- [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases)
- [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md)
- [Commits](9e0a5fb25a...ed0756273a)

---
updated-dependencies:
- dependency-name: rojopolis/spellcheck-github-actions
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Closes #16368
2025-02-17 16:47:10 +01:00
Viktor Szakats
b13e9066b3
GHA: tidy up apt commands
- drop `--quiet 2` option where used, to have uniform output.
- replace `apt` with `apt-get` in one job. sync options with rest.
- replace deprecated `apt-key` command with the alternative recommended
  by `apt-key(8)`.
- drop stray `cd /tmp`, no longer needed after migrating to GHA.
- shorten `--option Dpkg::Use-Pty=0` to `-o Dpkg::Use-Pty=0`.
- add `-o Dpkg::Use-Pty=0` to hide `apt-get` progress bars taking
  vertical log space, where missing.
- drop `-y --no-install-suggests --no-install-recommends` `apt-get`
  options. They are the default in the ubuntu-24.04 image.
- GHA/distcheck: move `name:` to top in steps where not there.
- scripts/cijobs.pl: catch `apt-get` lines with the `-o` option.

Closes #16127
2025-01-30 02:36:43 +01:00
Viktor Szakats
911f003db8
GHA: start using ARM Linux runners
- GHA/linux: allow per-job runner image override.
- GHA/linux: add an arm version of an existing job.
  Add workaround for broken `PATH` in the arm runner image.
- GHA/non-native: add CPU arch to job name where missing.
- GHA/checkdocs: switch a linter job to arm.

Performance looks a little bit better than Intel
(presumably with lower power consumption).

Test jobs, with openssl, tests, pytests, examples:
- arm:
  - https://github.com/curl/curl/actions/runs/12816430794/job/35737374521 4m7s
  - https://github.com/curl/curl/actions/runs/12816201136/job/35736615144 4m5s
- Intel:
  - https://github.com/curl/curl/actions/runs/12816430794/job/35737374118 4m32s
  - https://github.com/curl/curl/actions/runs/12816201136/job/35736614764 4m13s

Ref: https://github.blog/changelog/2025-01-16-linux-arm64-hosted-runners-now-available-for-free-in-public-repositories-public-preview/

Closes #16023
2025-01-17 00:28:12 +01:00
renovate[bot]
7cfc245a37
ci: update rojopolis/spellcheck-github-actions digest to 9e0a5fb
Closes #15933
2025-01-11 14:33:02 +01:00
Daniel Stenberg
f62be9cfb6
checkdocs: trim links off docs/ markdowns before spellcheck
So that we can use words in links without having them spellchecked.

Closes #15910
2025-01-03 18:38:04 +01:00
Daniel Stenberg
3eb57d6ba7
docs: use lowercase curl and libcurl
Adjusted badwords to find them.

Plus: make badwords run on all markdown files in the repo and update
markdowns previously unchecked

Closes #15898
2025-01-02 17:15:54 +01:00
Daniel Stenberg
28dd14aafe
scripts/mdlinkcheck: fix the ../ handling in file links
Follow-up to 62515e8e9d

Closes #15797
2024-12-21 13:57:38 +01:00
Viktor Szakats
ba9fe58d43
GHA: set persist-credentials: false
Suggested by zizmor GHA analysis tool.

Also:
- Move GH variables within single-quotes.
- Prefer single-quotes in shell code. (tidy-up)

Ref: https://github.com/actions/checkout/issues/485
Ref: https://github.com/actions/checkout/pull/1687
Ref: https://woodruffw.github.io/zizmor/

Closes #15746
2024-12-16 18:00:22 +01:00
Daniel Stenberg
62515e8e9d
scripts/mdlinkcheck: markdown link checker
This script parses all markdown files in the repository, extracts all
links and verifies that they work.

It makes sure to only check the URLs once, even if used in multiple
links. There is a whitelist for URLs we deem unnecessary to check.

It uses curl to do the checks.

As a bonus, this makes it easy to run this check locally.

Closes #15742
2024-12-14 14:22:30 +01:00
renovate[bot]
97866f1620
GHA: pin dependencies
For recently added actions in checkdocs.yml and windows.yml

Closes #15722
2024-12-13 10:52:40 +01:00
Daniel Stenberg
a8397643f3
GHA/checkdocs: change markdown link checker to linkspector
Fixes #15717
Closes #15719
2024-12-11 13:05:11 +01:00
renovate[bot]
b723f6a445
GHA: update four dependencies
- gnutls/gnutls to v3.8.8
- rojopolis/spellcheck-github-actions digest to 403efe0
- awslabs/aws-lc to v1.38.0
- github/codeql-action digest to 4f3212b

Closes #15487
Closes #15490
Closes #15516
Closes #15528
2024-11-09 14:30:53 +01:00
Viktor Szakats
cb092c0a7e
GHA: add apt update where missing
To mitigate this kind of (repeat) errors:
```
Err:14 http://azure.archive.ubuntu.com/ubuntu noble-updates/main amd64 python3-werkzeug all 3.0.1-3ubuntu0.1
  404  Not Found [IP: 40.81.13.82 80]
E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/p/python-werkzeug/python3-werkzeug_3.0.1-3ubuntu0.1_all.deb
Fetched 10.4 MB in 4s (2593 kB/s)
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
```
https://github.com/curl/curl/actions/runs/11732257460/job/32684111508#step:3:74

Follow-up to 842f88434f #15082
Closes #15519
2024-11-08 10:48:11 +01:00
Viktor Szakats
9acecc923d
tidy-up: whitespace, fix CI spacecheck for docs
Also: fixup CI spacecheck to apply to docs again.

Closes #15423
2024-10-30 23:15:28 +01:00