How the full check works
Each step ran on both builds the same way. Only the extension's source repository differs: rancher/kubewarden-ui at release kubewarden-4.2.4 (baseline) and security-ui-exts branch move-kubewarden-ui (moved).
Two identical servers
Fresh Rancher Prime v2.15.3 on AWS, single-node k3s v1.36.6+k3s1, same image and size.
Prime v2.15.3Build like CI
Each server checks out its commit, runs yarn install and VERSION=0.0.1 yarn build-pkg kubewarden, the steps of the suite's pull-request job.
Martin's e2e suite
The suite from each repository, same selection as its pull-request pipeline: setup, install from AppCo, dashboard, policy servers, policies, reports, telemetry and 12 catalog policies.
51 tests eachEvery page, side by side
An annotated walkthrough video per build, then a clean screenshot of each page and a pixel diff between the builds.
9 pagesRelease pipeline
A test tag on a fork ran security-ui-exts' real release workflow, which built and published the kubewarden chart next to the other extensions.
workflow greenUpgrade from 4.2.4
The official 4.2.4 chart, then an upgrade to the chart the release workflow published, through Rancher's Extensions page.
same data afterMartin's e2e suite, both builds
The Kubewarden UI Playwright suite (tests/ in kubewarden-ui, tests/kubewarden/ after the move), run with ORIGIN=source, Admission Controller from AppCo, and the 90-allpolicies subset fixed to the letter C so both runs install the same policies. Rows where the two builds differ are highlighted.
Baseline · kubewarden-ui
Moved · security-ui-exts
Reading the table
Each build ran the whole suite on its own fresh server in every round. A test the moved build fails, but the baseline passed every time, would be a regression: there are none. In round 2 the baseline's telemetry setup failed: Rancher's chart page listed every OpenTelemetry Operator version twice (the upstream index has no duplicates), so the suite's version check matched two elements and the dependent telemetry tests failed. On a fresh server in round 3 the same tests passed, and the moved build passed them in both rounds.
| Test | Baseline round 2 | Baseline round 3 | Moved round 2 | Moved round 3 |
|---|---|---|---|---|
| 00-setup.spec.tsbaseline 1/1 passed every round · moved 1/1 passed every round | ||||
| Initial rancher setup | Passed8 s | Passed8 s | Passed7 s | Passed7 s |
| 10-kubewarden.spec.tsbaseline 3/3 passed every round · moved 3/3 passed every round | ||||
| Install UI extension | Passed6 s | Passed6 s | Passed6 s | Passed6 s |
| Install Admission Controller | Passed48 s | Passed60 s | Passed45 s | Passed45 s |
| Install Kubewarden by Fleet | Skipped0 s | Skipped0 s | Skipped0 s | Skipped0 s |
| Add Policy Catalog Repository | Passed10 s | Passed10 s | Passed7 s | Passed7 s |
| Upgrade Kubewarden | Skipped0 s | Skipped0 s | Skipped0 s | Skipped0 s |
| 20-dashboard.spec.tsbaseline 3/3 passed every round · moved 3/3 passed every round | ||||
| Check kubewarden resources | Passed73 s | Passed17 s | Passed31 s | Passed36 s |
| Check landing pages | Passed6 s | Passed7 s | Passed9 s | Passed6 s |
| Stats reflect resource changes | Passed18 s | Passed22 s | Passed23 s | Passed18 s |
| 30-policyservers.spec.tsbaseline 3/3 passed every round · moved 3/3 passed every round | ||||
| Policy Servers | Passed77 s | Passed72 s | Passed75 s | Passed72 s |
| Create with invalid values | Passed7 s | Passed6 s | Passed6 s | Passed8 s |
| Create with custom values | Passed10 s | Passed10 s | Passed12 s | Passed10 s |
| 40-policies.spec.tsbaseline 11/11 passed every round · moved 11/11 passed every round | ||||
| Form fields (AP) | Passed27 s | Passed14 s | Passed15 s | Passed13 s |
| Form fields (CAP) | Passed43 s | Passed14 s | Passed13 s | Passed20 s |
| Default policy settings (AP) | Passed22 s | Passed13 s | Passed12 s | Passed11 s |
| Default policy settings (CAP) | Passed34 s | Passed15 s | Passed14 s | Passed17 s |
| Modified policy settings (AP) | Passed2.8 min | Passed2.2 min | Passed2.2 min | Passed2.1 min |
| Modified policy settings (CAP) | Passed2.6 min | Passed2.1 min | Passed1.9 min | Passed2.1 min |
| Check Official policies | Passed22 s | Passed9 s | Passed8 s | Passed14 s |
| Check Unofficial policies | Passed24 s | Passed15 s | Passed9 s | Passed19 s |
| Recommended policies | Passed68 s | Passed32 s | Passed26 s | Passed50 s |
| Match Conditions | Passed2.1 min | Passed85 s | Passed81 s | Passed1.7 min |
| Pagination | Passed15 s | Passed10 s | Passed8 s | Passed11 s |
| 50-policyreports.spec.tsbaseline 4/4 passed every round · moved 4/4 passed every round | ||||
| New resources should be reported | Passed2.1 min | Passed1.6 min | Passed78 s | Passed88 s |
| Check reports on resources details page | Passed21 s | Passed14 s | Passed10 s | Passed8 s |
| Check reports on home page | Passed12 s | Passed9 s | Passed5 s | Passed5 s |
| Cleanup & check results are gone | Passed60 s | Passed38 s | Passed28 s | Passed35 s |
| 60-telemetry.spec.tsbaseline 4/12 passed every round · moved 12/12 passed every round | ||||
| Install OpenTelemetry baseline, round 2: Error: expect(locator).toBeVisible() failed | Failed34 s | Passed58 s | Passed44 s | Passed44 s |
| Create custom PolicyServer & Policy | Passed24 s | Passed17 s | Passed11 s | Passed12 s |
| Install Jaeger | Passed87 s | Passed63 s | Passed47 s | Passed46 s |
| Configure tracing baseline, round 2: TimeoutError: locator.click: Timeout 60000ms exceeded. | Failed74 s | Passed2.1 min | Passed2.0 min | Passed1.8 min |
| Check traces are visible baseline, round 2: Error: expect(locator).toBeVisible() failed | Failed37 s | Passed18 s | Passed16 s | Passed38 s |
| Uninstall tracing baseline, round 2: TimeoutError: locator.click: Timeout 60000ms exceeded. | Failed73 s | Passed67 s | Passed66 s | Passed71 s |
| Install Monitoring | Passed37 s | Passed51 s | Passed41 s | Passed36 s |
| Configure metrics baseline, round 2: TimeoutError: locator.click: Timeout 60000ms exceeded. | Failed75 s | Passed2.5 min | Passed2.8 min | Passed2.3 min |
| Check metrics are visible baseline, round 2: TimeoutError: locator.evaluate: Timeout 60000ms exceeded. | Failed74 s | Passed89 s | Passed17 s | Passed46 s |
| Uninstall metrics baseline, round 2: TimeoutError: locator.click: Timeout 60000ms exceeded. | Failed73 s | Passed1.9 min | Passed87 s | Passed1.5 min |
| Delete custom PolicyServer & Policy | Passed10 s | Passed14 s | Passed8 s | Passed8 s |
| Uninstall OpenTelemetry baseline, round 2: TimeoutError: locator.click: Timeout 60000ms exceeded. | Failed69 s | Passed20 s | Passed19 s | Passed16 s |
| 90-allpolicies.spec.tsbaseline 11/11 passed every round · moved 11/11 passed every round | ||||
| install: Custom Policy | Passed57 s | Passed55 s | Passed55 s | Passed57 s |
| install: Capabilities PSP | Passed56 s | Passed57 s | Passed56 s | Passed55 s |
| install: CEL Policy | Skipped0 s | Skipped0 s | Skipped0 s | Skipped0 s |
| install: Container Block Sysctls | Passed62 s | Passed56 s | Passed55 s | Passed56 s |
| install: Container Block Sysctls CVE-2022-0811 | Passed66 s | Passed57 s | Passed55 s | Passed54 s |
| install: Container Resources | Passed75 s | Passed56 s | Passed56 s | Passed57 s |
| install: Container Running As Root | Passed73 s | Passed53 s | Passed54 s | Passed58 s |
| install: Container Running As User | Passed63 s | Passed55 s | Passed53 s | Passed56 s |
| install: Containers Block Ssh Port | Passed63 s | Passed54 s | Passed53 s | Passed58 s |
| install: Containers Block Specific Image Names | Passed66 s | Passed54 s | Passed53 s | Passed57 s |
| install: Containers Missing Security Context | Passed56 s | Passed55 s | Passed57 s | Passed56 s |
| install: Containers Should Not Run In Namespace | Passed55 s | Passed57 s | Passed56 s | Passed57 s |
Skipped tests are skipped by design in this mode: the Fleet install and the Admission Controller upgrade belong to the suite's fleet and upgrade modes, and one catalog policy skips itself.
Every page, side by side
After the suite, the same pages were captured on both servers at 1440×900, with ages and timestamps masked. Changed pixels are painted magenta in the Difference view. Differences in counts or names come from the cluster's data, not from the extension.
Byte-identical PNG: the developer-loaded build shows the same card on both servers. Console errors while loading: baseline 0, moved 0.
Byte-identical PNG: same version badge, policy and report summaries. Console errors while loading: baseline 3, moved 3.
22 pixels differ by at most 2/255 in one colour channel, at an icon edge: rendering noise, invisible. Console errors while loading: baseline 1, moved 1.
Byte-identical PNG. Console errors while loading: baseline 3, moved 3.
5 pixels differ by 1/255: rendering noise, invisible. Console errors while loading: baseline 1, moved 1.
Byte-identical PNG. Console errors while loading: baseline 3, moved 3.
Byte-identical PNG. Console errors while loading: baseline 1, moved 1.
Byte-identical PNG: the policy catalog lists the same policies. Console errors while loading: baseline 1, moved 1.
Byte-identical PNG: the embedded Policy Reporter renders the same. Console errors while loading: baseline 1, moved 1.
Recorded walkthroughs
The same script on both servers, with captions naming every action. Nothing is created or changed during the walkthrough; the policy form is cancelled.
Upgrade from today's 4.2.4
Rancher only offers an upgrade from the repository an extension was installed from, so existing users will see the new version when rancher/ui-plugin-charts syncs it from security-ui-exts. To test that path, a Git catalog mirrors how ui-plugin-charts will look after the move: the official 4.2.4 chart copied byte for byte (all five files match), plus the 4.2.5-moved.1 chart that security-ui-exts' release workflow published. The server already had the Admission Controller, a policy server and the recommended policies.
The real release pipeline
A test tag on the fork (kubewarden-4.2.5-moved.1, version bump on a separate branch) triggered security-ui-exts' existing release workflow, the same reusable rancher/dashboard workflow both repositories use today. It built the chart and published it to gh-pages next to the Vulnerability Scanner and Runtime Enforcer charts, with no workflow changes.
- Workflow run
- Build and Release Extension Charts · success ↗
- Published
- charts/kubewarden/4.2.5-moved.1 ↗ and extensions/kubewarden/4.2.5-moved.1
- Helm index
- kubewarden, runtime-enforcer, sbomscanner-ui-ext and vulnerability-scanner in one index.yaml
- Chart
- name
kubewarden, display name and Rancher annotations unchanged - Code served from
raw.githubusercontent.com/<org>/security-ui-exts/gh-pages/extensions/kubewarden/<version>
The other two extensions
The merge changes the repository's dependency tree (kubewarden-ui's security pins now apply to everything), so the Vulnerability Scanner and Runtime Enforcer were checked too. Statically, their bundles contain the same packages at the same versions on main and on the branch. At runtime, the suite's specs 80 and 81 ran on the round-2 servers with builds from security-ui-exts main (f18b5394, where the branch starts) and from the branch. These two specs are not part of the pull-request selection. On both builds the Vulnerability Scanner install outlasted the spec's two-minute wait for Helm's success line; the backend did install (every pod running), and the steps that depend on that wait fail the same way. The Runtime Enforcer install passed on the branch and timed out once on main. The branch fails nothing that main passed.
Vulnerability Scanner bundle
Runtime Enforcer bundle
Runtime specs 80 and 81
| Test | security-ui-exts main | Branch |
|---|---|---|
| 80-sbomscanner.spec.ts Install UI extension | Passed | Passed |
| 80-sbomscanner.spec.ts Install Vulnerability Scanner | Failed | Failed |
| 80-sbomscanner.spec.ts Scan Admission Controller | Failed | Failed |
| 80-sbomscanner.spec.ts RBAC rules for Policy Server | Passed | Passed |
| 80-sbomscanner.spec.ts Create test policy | Passed | Passed |
| 80-sbomscanner.spec.ts Create WorkloadScan | Failed | Failed |
| 80-sbomscanner.spec.ts Create test deployment | Passed | Passed |
| 80-sbomscanner.spec.ts Check compliance report | Failed | Failed |
| 80-sbomscanner.spec.ts Teardown | Failed | Failed |
| 80-sbomscanner.spec.ts Check Rancher VEX Hub | Passed | Passed |
| 80-sbomscanner.spec.ts Trigger registry scan | Failed | Failed |
| 81-runenforcer.spec.ts Install UI extension | Passed | Passed |
| 81-runenforcer.spec.ts Install Runtime Enforcer | Failed | Passed |
Static checks
Run before the runtime checks, on the merged branch and on both original repositories.
Unit tests
Bundle parity
Lint and spelling
Environment and what we adapted
Martin's CI runs on self-hosted openSUSE runners with Rancher in k3d on the same machine. These runs used Rancher on AWS and Playwright's official image. Each difference was handled the same way for both builds:
| Difference from CI | What we did |
|---|---|
| Rancher runs on AWS instead of on the runner. Chrome blocks a public page from loading a script from 127.0.0.1 (Local Network Access). | Granted the browser the local-network-access permission, as a user clicking Allow would. |
k3s runs directly on the server, and every new pod adds a network interface. Chromium aborts its requests on each one (ERR_NETWORK_CHANGED). | The browser runs in Playwright's official v1.64.1-noble image, with its own network namespace, the way CI's k3d hides pod interfaces. |
The image's /bin/sh is dash. The suite's host commands use &>, which dash reads as "run in background", so the AppCo secret was never created. | Pointed /bin/sh at bash inside the container, as on the openSUSE runners. |
The image's monospace font is WenQuanYi Zen Hei Mono, a CJK font with narrow Latin letters. | Installed DejaVu Sans Mono, the usual Linux default. |
The suite reads shell exit codes from screenshots of the terminal, with reference images per runner (it ships a -leap set for openSUSE). Sub-pixel colour still differs on this image. | Rendered this runner's three markers once with the suite's own --update-snapshots=missing and used them in place of the -leap set. The same files went into both runs: 0 a2213f39516e, 1 3d7af6355396, NaN ec0c2c8eda71. |
| CI installs a random letter of the 90-allpolicies catalog on each run. | Fixed the letter to C (12 policies) on both servers. |
| Evidence | A wrapper config adds a video for every test and HTML/JSON reports. No test file was changed. |
Run details
Baseline
- rancher
- https://mf-kw-r3-base.ui.rancher.space
- commit
- d4a369452e8d3264fac2920e4ccd82e999a8b670
- bundle_sha256
- 711ebc6c1dcbccaf
- policy_letter
- C excluded=ABDEFGHIKLMNOPRSTUV
- playwright
- Version 1.64.1
- node
- v26.13.0 kubectl=v1.36.6+k3s1
- server
- v1.36.6+k3s1
- started
- 2026-11-12T17:17:03Z
- finished
- 2026-11-12T17:54:44Z
- duration
- 2261s
- exit
- 0
Moved
- rancher
- https://mf-kw-r3-moved.ui.rancher.space
- commit
- 2a1e3c5b4311a4106ea6af6b2ebe1688ba8178ce
- bundle_sha256
- 5abb7bc5338f03bd
- policy_letter
- C excluded=ABDEFGHIKLMNOPRSTUV
- playwright
- Version 1.64.1
- node
- v26.13.0 kubectl=v1.36.6+k3s1
- server
- v1.36.6+k3s1
- started
- 2026-11-12T17:19:36Z
- finished
- 2026-11-12T17:55:39Z
- duration
- 2163s
- exit
- 0
Three rounds
Round 1 ran from a Mac and then on the servers directly, before the runner fixes in the table above; both builds failed the same shell-screenshot checks (baseline 39 passed, moved 37 passed). Its reports are kept with the artifacts but not used here. Rounds 2 and 3 each used two fresh servers with the fixed runner; every e2e result on this page comes from them. Each build therefore ran the suite twice from scratch.
Migration rules this surfaced
Things the real move has to respect, found while running these checks.
Archive kubewarden-ui, never delete it
The installed 4.2.4 extension caches its code from compressedEndpoint, which points at kubewarden-ui's gh-pages 4.2.4.tgz. That archive exists only there; ui-plugin-charts has the unpacked files but no .tgz. Keep the repository and its gh-pages branch.
Keep the package name kubewarden
The upgrade is offered for the same chart name in the same catalog repository. Renaming the package would show up as a second extension instead of an upgrade.
Upgrades reach users through ui-plugin-charts
Rancher offers upgrades only from the repository an extension came from. Existing users get the moved build once rancher/ui-plugin-charts syncs it from security-ui-exts; that manifest change is aalves08's.
The release workflow is shared
Any release in security-ui-exts runs "Build and Release Extension Charts". The kubewarden e2e follow-up now starts only for kubewarden-* tags.
One server serves all three extensions
In the merged repo one yarn serve-pkgs serves kubewarden, Vulnerability Scanner and Runtime Enforcer. The suite's env-config still points the last two at port 4501: a one-line follow-up.
Make the suite portable (for Martin)
ControlOrMeta+A instead of Control+A, >/dev/null 2>&1 instead of &>, and a terminal-marker set per runner would let it run on macOS and Ubuntu as well as openSUSE.
Code owners and access
kubewarden-ui's CODEOWNERS names teams from the kubewarden GitHub org, which cannot own code in a rancher-org repository. Martin and juadk only have read on security-ui-exts today.
Card text comes from the catalog sync
ui-plugin-charts patches the extension's description when it syncs. The moved build shows its package description until that happens, as the upgrade video shows.