Skip to content

Fix/s3 output files#1401

Merged
lbeckman314 merged 3 commits intofix/serviceaccount-deletionfrom
fix/s3-output-files
Apr 29, 2026
Merged

Fix/s3 output files#1401
lbeckman314 merged 3 commits intofix/serviceaccount-deletionfrom
fix/s3-output-files

Conversation

@lbeckman314
Copy link
Copy Markdown
Contributor

@lbeckman314 lbeckman314 commented Apr 29, 2026

Overview

This is a messy PR that's more of a short-term fix for GenericS3 uploads + downloads in K8s while Funnel's default working directory behavior is revisited/updated.

…anup

Set the worker Job as the owner of all namespaced task resources
(ConfigMap, ServiceAccount, Role, RoleBinding, PVC) so Kubernetes
garbage-collects them automatically when the Job is deleted.

- CreateJob now returns the created Job so callers can read its UID
- All Create* functions accept an optional *metav1.OwnerReference
- createResources in backend.go creates the Job first, builds the
  owner ref, and threads it through subsequent resource creation
- External (user-managed) SAs via _WORKER_SA tag receive no owner ref
  since they outlive individual tasks
- PVs remain explicitly managed (cluster-scoped resources cannot be
  owned by a namespaced Job)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Assisted-by: Claude Code:claude [Claude Code]
Signed-off-by: Liam Beckman <lbeckman314@gmail.com>
@lbeckman314 lbeckman314 changed the base branch from develop to fix/serviceaccount-deletion April 29, 2026 02:24
Signed-off-by: Liam Beckman <lbeckman314@gmail.com>
@lbeckman314 lbeckman314 merged commit a7fc01a into fix/serviceaccount-deletion Apr 29, 2026
5 of 7 checks passed
@lbeckman314 lbeckman314 deleted the fix/s3-output-files branch April 29, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant