Skip to content

Dev #582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
henryborchers merged 2 commits into from
Mar 31, 2026
Merged

Dev #582

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1c6085e
ci: extend cache lifespan
henryborchers Mar 31, 2026
13c57e8
ci: remove references to unsafe-best-match
henryborchers Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/tox_matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ jobs:
run: uv run --only-group=tox-uv tox --runner uv-venv-lock-runner -e ${{ steps.tox-env.outputs.result }}
env:
UV_EXTRA_INDEX_URL: ${{vars.PIP_EXTRA_INDEX_URL}}
UV_INDEX_STRATEGY: "unsafe-best-match"
QT_QPA_PLATFORM: "offscreen"
DISPLAY: ':99.0'

28 changes: 28 additions & 0 deletions ci/jenkins/scripts/create_uv_config.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#! /usr/bin/env bash
# This script creates a global configuration file for Jenkins to make the UV cache metadata for longer than standard

set -e

MAX_AGE_API=6000
MAX_AGE_FILES=365000000

GLOBAL_CONFIG_FILE=uv.toml

echo "# This file is auto-generated by $(basename $0)" > $GLOBAL_CONFIG_FILE

echo '' >> $GLOBAL_CONFIG_FILE
echo '[[index]]' >> $GLOBAL_CONFIG_FILE
echo "url = \"https://pypi.org/simple\"" >> $GLOBAL_CONFIG_FILE
echo 'default = true' >> $GLOBAL_CONFIG_FILE
echo "cache-control = { api = \"max-age=$MAX_AGE_API\", files = \"max-age=$MAX_AGE_FILES, immutable\" }" >> $GLOBAL_CONFIG_FILE

index=1
for arg in "$@"; do
echo '' >> $GLOBAL_CONFIG_FILE
echo '[[index]]' >> $GLOBAL_CONFIG_FILE
echo "name = \"extra-index-$index\"" >> $GLOBAL_CONFIG_FILE
echo "url = \"$arg\"" >> $GLOBAL_CONFIG_FILE
echo "cache-control = { api = \"max-age=$MAX_AGE_API\", files = \"max-age=$MAX_AGE_FILES, immutable\" }" >> $GLOBAL_CONFIG_FILE
done

echo "$(realpath $GLOBAL_CONFIG_FILE)"
27 changes: 27 additions & 0 deletions ci/jenkins/scripts/new-uv-global-config.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
$GLOBAL_CONFIG_FILE = "uv.toml"
$MAX_AGE_API="6000"
$MAX_AGE_FILES="365000000"

$scriptName = $MyInvocation.MyCommand.Name
"# This file is auto-generated by $scriptName" | Set-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8

'' | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
'[[index]]' | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
"url = `"https://pypi.org/simple`"" | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
'default = true' | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
"cache-control = { api = `"max-age=$MAX_AGE_API`", files = `"max-age=$MAX_AGE_FILES, immutable`" }" | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8


$index = 0
foreach ($arg in $args) {
if ([string]::IsNullOrWhiteSpace($arg)) { continue }

'' | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
'[[index]]' | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
"name = `"extra-index-$index`"" | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
"url = `"$arg`"" | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
'default = true' | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
"cache-control = { api = `"max-age=$MAX_AGE_API`", files = `"max-age=$MAX_AGE_FILES, immutable`" }" | Add-Content -Path $GLOBAL_CONFIG_FILE -Encoding UTF8
}

Write-Output (Get-ChildItem -Path $GLOBAL_CONFIG_FILE).FullName
106 changes: 69 additions & 37 deletions vars/runJenkinsPipeline.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,27 @@ def installMSVCRuntime(cacheLocation){
}


def createWindowUVConfig(){
def scriptFile = "ci\\jenkins\\scripts\\new-uv-global-config.ps1"
if(! fileExists(scriptFile)){
checkout scm
}
return powershell(
label: 'Setting up uv.toml config file',
script: "& ${scriptFile} \$env:UV_INDEX_URL \$env:UV_EXTRA_INDEX_URL",
returnStdout: true
).trim()
}

def createUnixUvConfig(){

def scriptFile = 'ci/jenkins/scripts/create_uv_config.sh'
if(! fileExists(scriptFile)){
checkout scm
}
return sh(label: 'Setting up uv.toml config file', script: "sh ${scriptFile} " + '$UV_INDEX_URL $UV_EXTRA_INDEX_URL', returnStdout: true).trim()
}


def hasSonarCreds(credentialsId){
try{
Expand Down Expand Up @@ -137,7 +158,6 @@ def call(){
}
environment{
PIP_CACHE_DIR = '/tmp/pipcache'
UV_INDEX_STRATEGY = 'unsafe-best-match'
UV_TOOL_DIR = '/tmp/uvtools'
UV_PYTHON_CACHE_DIR = '/tmp/uvpython'
UV_CACHE_DIR = '/tmp/uvcache'
Expand Down Expand Up @@ -204,12 +224,12 @@ def call(){
}
environment{
PIP_CACHE_DIR='/tmp/pipcache'
UV_INDEX_STRATEGY='unsafe-best-match'
UV_TOOL_DIR='/tmp/uvtools'
UV_PYTHON_CACHE_DIR='/tmp/uvpython'
UV_CACHE_DIR='/tmp/uvcache'
UV_PYTHON='3.11'
QT_QPA_PLATFORM='offscreen'
UV_CONFIG_FILE=createUnixUvConfig()
UV_FROZEN='1'
}
options {
Expand Down Expand Up @@ -452,7 +472,6 @@ def call(){
}
environment{
PIP_CACHE_DIR='/tmp/pipcache'
UV_INDEX_STRATEGY='unsafe-best-match'
UV_TOOL_DIR='/tmp/uvtools'
UV_PYTHON_CACHE_DIR='/tmp/uvpython'
UV_CACHE_DIR='/tmp/uvcache'
Expand All @@ -464,11 +483,13 @@ def call(){
checkout scm
docker.image('ghcr.io/astral-sh/uv:debian').inside('--mount source=python-tmp-speedwagon,target=/tmp'){
try{
envs = sh(
label: 'Get tox environments',
script: 'uv run --isolated --only-group=tox --frozen --quiet tox list -d --no-desc',
returnStdout: true,
).trim().split('\n')
withEnv(["UV_CONFIG_FILE=${createUnixUvConfig()}"]){
envs = sh(
label: 'Get tox environments',
script: 'uv run --isolated --only-group=tox --frozen --quiet tox list -d --no-desc',
returnStdout: true,
).trim().split('\n')
}
} finally{
cleanWs(
patterns: [
Expand All @@ -492,11 +513,13 @@ def call(){
def image = docker.build(UUID.randomUUID().toString(), '-f ci/docker/python/linux/jenkins/Dockerfile .')
try{
image.inside('--mount source=python-tmp-speedwagon,target=/tmp --tmpfs /.local/share:exec --tmpfs /.local/bin:exec'){
sh( label: 'Running Tox',
script: """uv python install cpython-${version}
uv run -p ${version} --frozen --only-group=tox-uv --isolated tox run --runner uv-venv-lock-runner -e ${toxEnv}
"""
)
withEnv(["UV_CONFIG_FILE=${createUnixUvConfig()}"]){
sh( label: 'Running Tox',
script: """uv python install cpython-${version}
uv run -p ${version} --frozen --only-group=tox-uv --isolated tox run --runner uv-venv-lock-runner -e ${toxEnv}
"""
)
}
}
} finally {
sh "docker image rm --force ${image.imageName()}"
Expand All @@ -521,7 +544,6 @@ def call(){
expression {return nodesByLabel('windows && docker && x86').size() > 0}
}
environment{
UV_INDEX_STRATEGY='unsafe-best-match'
PIP_CACHE_DIR='C:\\Users\\ContainerUser\\Documents\\pipcache'
UV_TOOL_DIR='C:\\Users\\ContainerUser\\Documents\\uvtools'
UV_PYTHON_CACHE_DIR='C:\\Users\\ContainerUser\\Documents\\uvpython'
Expand All @@ -533,14 +555,16 @@ def call(){
def envs = []
node('docker && windows'){
try{
checkout scm
docker.image(env.DEFAULT_PYTHON_DOCKER_IMAGE ? env.DEFAULT_PYTHON_DOCKER_IMAGE: 'python').inside("--mount source=uv_python_cache_dir,target=${env.UV_PYTHON_CACHE_DIR}"){
checkout scm
bat(script: 'python -m venv venv && venv\\Scripts\\pip install --disable-pip-version-check uv')
envs = bat(
label: 'Get tox environments',
script: '@.\\venv\\Scripts\\uv run --isolated --only-group tox --frozen --quiet tox list -d --no-desc',
returnStdout: true,
).trim().split('\r\n')
withEnv(["UV_CONFIG_FILE=${createWindowUVConfig()}"]){
bat(script: 'python -m venv venv && venv\\Scripts\\pip install --disable-pip-version-check uv')
envs = bat(
label: 'Get tox environments',
script: '@.\\venv\\Scripts\\uv run --isolated --only-group tox --frozen --quiet tox list -d --no-desc',
returnStdout: true,
).trim().split('\r\n')
}
}
} finally{
bat "${tool(name: 'Default', type: 'git')} clean -dfx"
Expand All @@ -562,7 +586,10 @@ def call(){
){
retry(3){
try{
withEnv(["TOX_UV_PATH=${env.WORKSPACE}\\venv\\Scripts\\uv.exe"]){
withEnv([
"TOX_UV_PATH=${env.WORKSPACE}\\venv\\Scripts\\uv.exe",
"UV_CONFIG_FILE=${createWindowUVConfig()}"
]){
bat(label: 'Running Tox',
script: """python -m venv venv && venv\\Scripts\\pip install --disable-pip-version-check uv
venv\\Scripts\\uv python install cpython-${version}
Expand Down Expand Up @@ -614,7 +641,6 @@ def call(){
}
environment{
PIP_CACHE_DIR='/tmp/pipcache'
UV_INDEX_STRATEGY='unsafe-best-match'
UV_CACHE_DIR='/tmp/uvcache'
}
options {
Expand Down Expand Up @@ -648,9 +674,6 @@ def call(){
when{
equals expected: true, actual: params.TEST_PACKAGES
}
environment{
UV_INDEX_STRATEGY='unsafe-best-match'
}
stages{
stage('Twine Check'){
agent{
Expand Down Expand Up @@ -731,6 +754,7 @@ def call(){
'UV_TOOL_DIR=/tmp/uvtools',
'UV_PYTHON_CACHE_DIR=/tmp/uvpython',
'UV_CACHE_DIR=/tmp/uvcache',
"UV_CONFIG_FILE=${createUnixUvConfig()}"
]){
sh(
label: 'Testing with tox',
Expand All @@ -746,7 +770,8 @@ def call(){
'UV_PYTHON_CACHE_DIR=c:\\Users\\ContainerUser\\Documents\\cache\\uvpython',
'UV_CACHE_DIR=c:\\Users\\ContainerUser\\Documents\\cache\\uvcache',
'UV_LINK_MODE=copy',
"TOX_UV_PATH=${env.WORKSPACE}\\venv\\Scripts\\uv.exe"
"TOX_UV_PATH=${env.WORKSPACE}\\venv\\Scripts\\uv.exe",
"UV_CONFIG_FILE=${createWindowUVConfig()}"
]){
installMSVCRuntime('c:\\msvc_runtime\\')
bat(
Expand All @@ -762,7 +787,10 @@ def call(){
}
} else {
if(isUnix()){
withEnv(["TOX_UV_PATH=${env.WORKSPACE}/venv/bin/uv"]){
withEnv([
"TOX_UV_PATH=${env.WORKSPACE}/venv/bin/uv",
"UV_CONFIG_FILE=${createUnixUvConfig()}"
]){
sh(
label: 'Testing with tox',
script: """python3 -m venv venv
Expand All @@ -772,14 +800,19 @@ def call(){
)
}
} else {
bat(
label: 'Testing with tox',
script: """python -m venv venv
.\\venv\\Scripts\\pip install --disable-pip-version-check uv
.\\venv\\Scripts\\uv python install cpython-${entry.PYTHON_VERSION}
.\\venv\\Scripts\\uv run --only-group=tox-uv --isolated --frozen tox --installpkg ${findFiles(glob: entry.PACKAGE_TYPE == 'wheel' ? 'dist/*.whl' : 'dist/*.tar.gz')[0].path} -e py${entry.PYTHON_VERSION.replace('.', '')}
"""
)
withEnv([
"TOX_UV_PATH=${env.WORKSPACE}\\venv\\Scripts\\uv.exe",
"UV_CONFIG_FILE=${createWindowUVConfig()}"
]){
bat(
label: 'Testing with tox',
script: """python -m venv venv
.\\venv\\Scripts\\pip install --disable-pip-version-check uv
.\\venv\\Scripts\\uv python install cpython-${entry.PYTHON_VERSION}
.\\venv\\Scripts\\uv run --only-group=tox-uv --isolated --frozen tox --installpkg ${findFiles(glob: entry.PACKAGE_TYPE == 'wheel' ? 'dist/*.whl' : 'dist/*.tar.gz')[0].path} -e py${entry.PYTHON_VERSION.replace('.', '')}
"""
)
}
}
}
} finally{
Expand Down Expand Up @@ -807,7 +840,6 @@ def call(){
stage('Deploy to pypi') {
environment{
PIP_CACHE_DIR='/tmp/pipcache'
UV_INDEX_STRATEGY='unsafe-best-match'
UV_TOOL_DIR='/tmp/uvtools'
UV_PYTHON_CACHE_DIR='/tmp/uvpython'
UV_CACHE_DIR='/tmp/uvcache'
Expand Down
Loading