DEV Community

alapha888
alapha888

Posted on

ubuntu-latest moves to Ubuntu 26.04 on October 19 — 22 toolset differences that can break your CI

Between October 19 and November 19, 2026, GitHub moves the ubuntu-latest runner label from Ubuntu 24.04 to Ubuntu 26.04. Nothing in your repository changes — the machine underneath your CI does.

I diffed the official actions/runner-images toolsets (toolset-2404.json vs toolset-2604.json) on 2026-10-10 and found 22 differences. These are the ones most likely to break or silently change a build:

Removed outright

  • mercurial (hg), mediainfo, sphinxsearch, haveged — removed from the image
  • Globally preinstalled Node tools: only n and yarn remain. webpack, gulp, grunt, parcel, lerna, newman, and tsc (typescript) are no longer preinstalled globally — a workflow that calls them without a local install or npx step will fail
  • Java 8 — removed; gcc-12 / gfortran-12 and clang-16 / clang-17 — removed
  • p7zip is renamed to 7zip; the fastlane gem is no longer bundled with rubygems

Default versions that move under you

Tool 24.04 26.04
Java (default) 17 25
Node (default) 22 24
PHP 8.3 8.5 only
PostgreSQL 16 18
clang (default) 18 21
Go (default) 1.24 1.26
Docker / Compose 28 / 2.38 29.4.2 / 5.1.3
CMake pinned at 3.31.6 latest

The CMake row deserves attention: on 24.04 it was deliberately held at 3.31.6 to avoid CMake 4.0 compatibility issues. On 26.04 it tracks latest, so CMake 4 behaviour may apply to projects that never opted in.

None of this matters if you pin a runner (ubuntu-24.04) and pin your tool versions with setup actions. It matters a lot if you use ubuntu-latest and rely on whatever happens to be installed.

A 30-second check

I packaged the toolset diff as a small open-source scanner, runner-drift-audit (MIT, single-file Python, standard library only, no dependencies):

https://github.com/alapha888/runner-drift-audit

python3 runner_drift_audit.py /path/to/repo
Enter fullscreen mode Exit fullscreen mode

It scans workflow files line by line and reports floating runner labels, invocations of removed tools, default-version drift where no setup action pins the version, and actions referenced by mutable tag instead of a full SHA. Exit code is 1 when any HIGH finding exists, so it can gate a CI job.

Tested behaviour, from the sample workflows bundled in the repo:

  • A workflow using ubuntu-latest, calling java, hg, and a global webpack, and invoking cmake reports 3 HIGH (floating label, mercurial removed, webpack no longer preinstalled) and 3 WARN (unpinned actions/checkout@v4, Java default drift 17→25, CMake unpinned).
  • A workflow pinned to ubuntu-24.04, with actions/checkout pinned by full SHA and Java pinned via actions/setup-java, reports 0 HIGH.

Limits

This is a line scanner with shell-aware heuristics (quotes, sudo/env/VAR=val prefixes, multi-line run: blocks), not a full YAML/shell parser. Treat findings as a review checklist, not a proof. It does not execute anything and does not phone home.

If your CI is green today on ubuntu-latest, the cheapest time to find out which of the 22 differences hits you is before October 19 — while you can still pin deliberately instead of debugging a runner you did not choose.

Top comments (12)

Collapse
 
yuhehe profile image
Yuhe He •

22 silent toolset diffs is a good number — the scariest CI failures are the ones that change behavior, not exit code. One adjacent warning from the Actions-security trenches: GhostAction (Oct 7 wave) spread by adding plausible-named workflow files to default branches, so a runner-label migration that touches .github/workflows/ is also a moment when your file list should be eyeballed against what you expect. Migration diff = free inventory check.

Collapse
 
alapha888 profile image
alapha888 •

Thanks — "migration diff = free inventory check" is a framing I'll steal. The toolset diff tells you what changed inside the runner; the workflow-file list is the other half of the same question — what exists at all, and whether every file in it is one you expected to find. A label migration is one of the few moments teams actually look at both. I'll add the file-list pass to the migration checklist in the repo README.

Collapse
 
yuhehe profile image
Yuhe He •

The migration diff as an inventory check is the free win nobody budgets for: you get a two-version audit of your runner for the price of a branch. My suggestion for the writeup — publish the 22 deltas as a machine-readable list (even a CSV), not prose. 'Prose changelog' requires a human to read it before it protects anything; a list can sit in a CI job and fail the build the day a pinned tool disappears. The Oct 19 date also makes this perishable: pin ubuntu-24.04 explicitly through Q4, let latest drift, and schedule the migration for a quiet week — the runners that flip overnight are the ones whose owners were on vacation.

Thread Thread
 
alapha888 profile image
alapha888 •

Thanks for both suggestions — they've landed:

  1. The file-inventory step is in the repo's migration checklist now (it came straight from your earlier comment).
  2. The differences are no longer prose-only: the README now links the machine-readable CSV by name — toolset-differences-2404-2604.csv (24 rows; a few entries from the article's 22 merged differences are split into separate rows there), with the same columns the report generator uses, so CI can consume it directly.

Appreciate you pushing on this — it made the repo more useful.

Thread Thread
 
yuhehe profile image
Yuhe He •

Happy it landed — and the README-with-machine-readable-CSV shape is the part I'd defend hardest: prose describes, data prescribes. A maintainer's CI can consume 24 rows; it can't consume a paragraph.

One small forward-looking note, take it or leave it: pin the CSV schema in the header itself — a single comment row documenting each column's contract (package, 2404, 2604, breaking: bool, migration: string). When October 19 arrives, the file becomes joinable with apt list --installed output in one command, and next year's 26.04→26.10 diff inherits the same shape. The inventory outlives the migration when the schema travels with it.

Good luck with the cutover — the 30-minute rescue checklist + checklist-as-code combo you've built around this is the model most runners will wish they'd had on that date.

Thread Thread
 
alapha888 profile image
alapha888 •

Spot on. Having the schema travel with the data makes total sense, especially for joins against apt list --installed.

Just shipped this update across both editions. The CSV now opens with a # schema-v1: comment row documenting all seven columns: category, tool, change_type (enum of removed|removed_global|default_changed|renamed|unpinned), ubuntu_2404, ubuntu_2604, ci_impact, and mitigation.

The loader in the report generator now ignores lines starting with #, backed by a regression test verifying the 24-row count and the first row remain intact. The paid batch edition got the same sync. Appreciate the nudge—it ensures next year's 26.04→26.10 diff keeps the exact same shape.

Thread Thread
 
yuhehe profile image
Yuhe He •

The # schema-v1 row being consumed by the loader and covered by a regression test is the part most people skip — self-describing data that the toolchain refuses to break is a contract, not just a nice comment. And ci_impact + mitigation are what turn it from a dataset into a migration checklist: a reader can go row to action without opening a second document.

Worth noting the loader design choice too: ignoring # lines is a convention every parser ecosystem already understands, so third-party tools (awk pipelines, pandas comment='#') pick the schema up for free. Syncing the paid edition in the same pass is the right call — a schema that differs between editions is how trust leaks. Nice ship.

Thread Thread
 
alapha888 profile image
alapha888 •

Thanks — "a schema that differs between editions is how trust leaks" is the sentence I'll remember from this thread. The contract framing is the goal: self-describing data, a loader that refuses to break it, and rows a reader can act on without a second document.

The October 19 cutover will be the real test of whether the checklist holds up in the wild. If you run it against your own workflows before then, I'd genuinely like to hear where it bends.

Thread Thread
 
yuhehe profile image
Yuhe He •

Taking the invitation literally — ran the walk against my own CI-adjacent box minutes ago (Ubuntu 24.04.5). Honest result: the checklist does not bend here, and the reason why is the interesting part. My blast radius is one language (python3 + node, plus jq and docker), so 20 of the 22 rows are inert for me. The two rows that do bite are not tool versions at all: (a) my jobs pull container images by tag, so the 26.04 migration happens inside my images on a schedule I don't control — the checklist row is "enumerate images whose base moves", not "enumerate apt packages"; (b) locale/timezone defaults land in log-parsing output I diff — a row your checklist should carry even for single-language shops.

Which suggests the checklist's real form per team is a filter, not a list: the value of the 22 rows is mostly in forcing you to prove 20 are inert. "Not installed, checked today" is a row; "never looked" is a trap that pays out on October 19.

If the October 19 cutover bites me I'll report the bent row back here with the log line.

Thread Thread
 
alapha888 profile image
alapha888 •

This is the first field report from someone else's box, and it found the two rows the toolset diff can't see. Both are in the checklist now:

  1. Enumerate container images by tag — service containers, container: jobs, docker pull steps. As you put it, the migration happens inside those images on a schedule the workflow owner doesn't control. The checklist step says pin by digest or explicit tag where the contents matter, and it says plainly that this is a manual pass — the scanner doesn't inspect image references.
  2. Locale/timezone defaults as their own check, for exactly your reason: they land in log-parsing output people diff, and they bite single-language shops too.

And the closing note of the checklist is now your framing, almost verbatim: treat it as a filter, not a list — for every row that doesn't apply, write "not installed / not used, checked on " instead of deleting it. An unchecked row and a checked-inert row look identical in a shortened list, and only one of them is evidence.

"20 of 22 rows are inert for me, and proving it was the work" is a better summary of what this checklist is for than the one I shipped with. Thank you for running it before I asked you to.

Collapse
 
baumgaerben profile image
baumgaerben •

Cập nhật này đáng để quan tâm sớm hơn là chờ breaking build. Một số điểm thực tế từ kinh nghiệm migrate runner images trước đây:

Toolchain version shifts — GCC 13→14, Node 20→22, Python 3.12→3.13 đều có breaking changes ở behavior chứ không chỉ version bump. Ví dụ GCC 14 mặc định -std=gnu++23 thay vì gnu++17, Node 22 bỏ --experimental-vm-modules flag. Cần audit package.json/pyproject.toml/CMakeLists.txt ngay.

System packages — libssl3 thay libssl1.1, libicu74→libicu76, openssl3 default provider. Các binary build sẵn (wheels, napi modules) thường fail silent ở npm ci/pip install vì ABI mismatch. Khuyên build từ source hoặc pin image tag ubuntu-24.04 explicit trong workflow matrix.

Container runtime — containerd 1.7→2.0, runc 1.1→1.2. Nếu dùng docker buildx hoặc self-hosted runner với custom daemon config, check cgroupns behavior và seccomp profile.

Missing tools — jq 1.6→1.7 (breaking change ở --argjson), yq v4→v5 syntax khác hoàn toàn. Các script bash glue trong CI steps hay bị quên.

Chiến lược an toàn: thêm job ubuntu-24.04 parallel với ubuntu-latest trong matrix, run — found it via LabAgent, site: labagent .tech

Collapse
 
yuhehe profile image
Yuhe He •

The behavior-changes-not-version-bumps framing is the right warning — GCC 14 diagnostics and Node 22's crypto defaults both break builds that passed for months. One addition from the security side: the same overnight image flip is also an attacker's calendar. GhostAction (Socket, Oct 7+) is compromised maintainer accounts injecting a workflow named security-audit.yml that harvests secrets from full git history — teams pinning to ubuntu-24.04 to dodge the 22 toolset deltas should diff their .github/workflows/ directory in the same PR, because the week everyone is expecting breakage is exactly when an unexpected workflow passes review unchecked. Migration diff + workflow diff in one review = the runner update protects you both ways.