Open
Conversation
The system 'podman-compose' package for some Linux distros is too old for use with ASU. Include it as part of our python environment so that we get an up-to-date version. Fixes: openwrt#1562 Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.1.4 to 3.1.5. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.4...3.1.5) --- updated-dependencies: - dependency-name: werkzeug dependency-version: 3.1.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.2 to 2.6.3. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.6.2...2.6.3) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.6.3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.21 to 0.0.22. - [Release notes](https://github.com/Kludex/python-multipart/releases) - [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md) - [Commits](Kludex/python-multipart@0.0.21...0.0.22) --- updated-dependencies: - dependency-name: python-multipart dependency-version: 0.0.22 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Preliminary error statistics show that over 80% of all build errors
are due to package errors. So instead of just reporting "Impossible
package selection", attempt to show whether packages are missing or
conflicting and list all packages named in the stderr from the build.
Packaging errors might arise from various causes:
1) User specifies conflicting packages: nftables-json and
nftables-nojson
2) User specifies or has installed a package not from the OpenWrt
feeds
3) The upstream build of a package has failed, so it's appearing
to be missing
This new log information will give us some insight as to whether
these errors are caused by the user, the ASU client or upstream
builds, so we know where to focus mitigations.
Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
This kmod no longer exists, removed upstream, so delete it from the package lists when present. While we're here, improve handling of removals and rewrite package replacement of 'auc' with 'owut' to be cleaner. Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
Plug a potential security hole by removing job hashes from any internal server errors that include them. See included test case for an example taken from an actual error. Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
Add an opkg case for the check_data_file_clashes error, which is reported when two packages attempt to install the same file. Fix up apk conflicts message parsing, which had a bug the allowed a package name to include a newline. Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
Add REPO_NAME_PATTERN (^[\w.-]+$) to reject newlines, spaces, and slashes in repository names, preventing injection of extra lines into repositories.conf (e.g. 'evil\nsrc/gz pwned http://x.com'). Add REPO_URL_PATTERN (^https?://\S+$) to ensure repository URLs use http(s) scheme only. Includes 11 regression tests in tests/test_security.py. Signed-off-by: Paul Spooren <mail@aparcar.org>
The previous check used str.startswith() to validate repository URLs against the allow list, which is bypassable via: - Subdomain: downloads.openwrt.org.evil.com - Userinfo: downloads.openwrt.org@evil.com Replace with is_repo_allowed() that uses urllib.parse.urlparse() to compare scheme, hostname, and path prefix independently. Includes 6 unit tests for is_repo_allowed() in tests/test_util.py. Signed-off-by: Paul Spooren <mail@aparcar.org>
A malicious container could craft tar archives with path traversal entries (e.g. ../../../etc/crontab) to write files outside the intended extraction directory. Python 3.12+ supports filter='data' which rejects absolute paths, parent directory references, and other dangerous tar member attributes. This is the recommended mitigation per PEP 706. Signed-off-by: Paul Spooren <mail@aparcar.org>
We're still seeing older upgrades that include the no-longer-existing
English translations.
Impossible package selection: missing (luci-i18n-attendedsysupgrade-en,
luci-i18n-ddns-en, luci-i18n-openvpn-en, luci-i18n-p910nd-en, ...
Explicitly remove them from the package list.
Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
The Lantiq xRx200/xRx300/xRx330 switch driver is not compiled into the kernel image any more. Add this switch driver to the generated images if it is missing. This was changed in OpenWrt in: openwrt/openwrt@62bf028 Related: openwrt#1556 Fixes: openwrt/openwrt#21334
Bumps [pygments](https://github.com/pygments/pygments) from 2.19.2 to 2.20.0. - [Release notes](https://github.com/pygments/pygments/releases) - [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES) - [Commits](pygments/pygments@2.19.2...2.20.0) --- updated-dependencies: - dependency-name: pygments dependency-version: 2.20.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [requests](https://github.com/psf/requests) from 2.32.5 to 2.33.0. - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.32.5...v2.33.0) --- updated-dependencies: - dependency-name: requests dependency-version: 2.33.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Add [build-system] table so the package is properly installable with uv sync. Add boto3 for S3-compatible storage support.
Add store abstraction layer with LocalStore and S3Store backends. Workers can upload built images to S3-compatible storage (AWS S3, Cloudflare R2, MinIO) instead of requiring a shared filesystem. The /store/ endpoint serves files locally or redirects to the S3 public URL via 302 depending on the configured backend.
- Wrap container lifecycle in try/finally to ensure cleanup on all error paths (previously leaked running containers on report_error) - Use container.remove(v=True, force=True) instead of auto_remove to also clean up anonymous volumes
- Replace manual tempfile.mkdtemp with pytest tmp_path fixture to prevent leftover test directories on failures - Force store_backend=local in test fixture - Add unit tests for LocalStore (upload_file, upload_dir, exists, get_url)
Store firmware builds for longer, 7d for builds including package versions, 1d for firmware without specific versions. Signed-off-by: Paul Spooren <mail@aparcar.org>
Move to stats page and add package statistics Signed-off-by: Paul Spooren <mail@aparcar.org>
Workers read S3 credentials from their own .env and upload directly. No shared filesystem required — only needs Redis and Podman socket.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Add container_network_mode setting (default: pasta) so the network mode used for build containers can be configured via environment. Signed-off-by: Paul Spooren <mail@aparcar.org>
Add repositories_mode field to BuildRequest (append or replace). Extract repository logic into asu/repositories.py with a single merge_repositories() function handling both opkg and apk formats. Detect apk vs opkg from the running container instead of guessing from version strings. Validate repositories against the allow list at the API level with a 400 error before starting any build. Signed-off-by: Paul Spooren <mail@aparcar.org>
Add slow integration tests for external opkg and apk repositories using the LibreMesh feed. Tests verify repository merging, key injection, and builds with external packages for both package managers. Also test wrong-key rejection for apk builds. Signed-off-by: Paul Spooren <mail@aparcar.org>
Add slow integration tests for external opkg and apk repositories using the Freifunk Weimarnetz feed. Rename existing tests to include the project name (libremesh/freifunk). Signed-off-by: Paul Spooren <mail@aparcar.org>
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.1.5 to 3.1.6. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.5...3.1.6) --- updated-dependencies: - dependency-name: werkzeug dependency-version: 3.1.6 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Improve apk conflicts message parsing to handle case where conflicting file ownership is detected. Signed-off-by: Eric Fahlgren <ericfahlgren@gmail.com>
Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](pytest-dev/pytest@8.4.2...9.0.3) --- updated-dependencies: - dependency-name: pytest dependency-version: 9.0.3 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com>
Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Commits](boto/boto3@1.35.0...1.42.88) --- updated-dependencies: - dependency-name: boto3 dependency-version: 1.42.88 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
Updates the requirements on [isort](https://github.com/PyCQA/isort) to permit the latest version. - [Release notes](https://github.com/PyCQA/isort/releases) - [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md) - [Commits](PyCQA/isort@7.0.0...8.0.1) --- updated-dependencies: - dependency-name: isort dependency-version: 8.0.1 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com>
Load settings from asu.toml in addition to environment variables and .env files. Environment variables take precedence over TOML. Add asu.example.toml with all available settings documented. Signed-off-by: Paul Spooren <mail@aparcar.org>
Replace the file-based RotatingFileHandler error log with a Redis list. Errors are stored via LPUSH and capped at 5000 entries with LTRIM. This removes the need to mount a shared log directory for remote workers. Signed-off-by: Paul Spooren <mail@aparcar.org>
Hardcode podman socket path to /var/podman.sock. Workers mount ./podman.sock to that path. Users just symlink their podman socket: ln -sf /run/user/$(id -u)/podman/podman.sock podman.sock Removes CONTAINER_SOCKET_PATH setting entirely. Signed-off-by: Paul Spooren <mail@aparcar.org>
Uses prefers-color-scheme media query with colors inspired by git.openwrt.org and OpenWrt's navy blue palette. Signed-off-by: Paul Spooren <mail@aparcar.org>
Update CI matrix to match requires-python >= 3.13. Signed-off-by: Paul Spooren <mail@aparcar.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Add DebugTestClient that logs response bodies for error responses, making test failures easier to diagnose. Update slow integration tests from 23.05.5 to 25.12.2 and set real upstream URL so builds can resolve packages over the asu-build network. Signed-off-by: Paul Spooren <mail@aparcar.org>
Add a second podman-compose CI job that starts the nginx caching proxy on the asu-build network and verifies builds succeed through the cache. Extract cache service into podman-compose.cache.yml overlay file.
Slow tests override settings.upstream_url and repository_allow_list on the singleton, which leaked into subsequent tests causing them to hit the real upstream instead of the mock server. Save and restore these values in the app fixture teardown. Also replace dnsmasq conflict test with missing package test since dnsmasq/dnsmasq-full no longer conflict on 25.12 (apk). Signed-off-by: Paul Spooren <mail@aparcar.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
Podman requires explicit bridge mode when connecting to named networks. Without it, rootless Podman may default to slirp4netns/pasta, causing: "networks and static ip/mac address can only be used with Bridge mode" Also update CI integration tests from 24.10.0 to 25.12.2. https://claude.ai/code/session_019PBX4RdievwtP3TtnRwAf7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The system 'podman-compose' package for some Linux distros is too
old for use with ASU. Include it as part of our python environment
so that we get an up-to-date version.
Fixes: openwrt#1562
Signed-off-by: Eric Fahlgren ericfahlgren@gmail.com