Conversation
Slice 1 of kosli evaluate trail - adds the evaluate parent command
and evaluate trail subcommand that fetches a trail from the API and
wraps the response in {"trail": ...} JSON output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds internal/evaluate with Evaluate() function that validates and evaluates Rego policies. Validates package name is 'policy', requires an 'allow' rule, collects violations on deny. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds --policy flag to evaluate trail command. Reads a .rego file, evaluates it against the trail input using OPA, exits 0 on allow and 1 on deny. Uses Rego v1 syntax. Policy must use package policy and declare an allow rule. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…e and violations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…wed text Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…lations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…violations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests: non-map passthrough, no compliance_status passthrough, empty array to empty map, single attestation keyed by name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests: multiple trail attestations, artifact-level, both levels, multiple artifacts, entries without attestation_name skipped. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…maps Wire TransformTrail into evaluateTrail after JSON parse. Tests verify trail-level and artifact-level maps, plus Rego policy access by name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…vels Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…te existing fields Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hanged Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…evel attestation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…mixed, no-match) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…trail Slice 5c complete: FilterAttestations filters trail-level (plain name) and artifact-level (dot-qualified) attestations before rehydration, saving API calls for excluded attestations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Aligns evaluate trail with the codebase convention of --output table|json (with -o shorthand, defaulting to "table") instead of --format text|json. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds `kosli evaluate trails TRAIL-NAME [TRAIL-NAME...]` command that
fetches multiple trails, wraps them in {"trails": [...]} input shape,
and evaluates against a Rego policy. Supports --policy, --output,
and --show-input flags with the same semantics as evaluate trail.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds rehydration (CollectAttestationIDs + RehydrateTrail) and --attestations filtering to evaluate trails, matching the same enrichment pipeline as evaluate trail. Each trail is independently transformed, filtered, and rehydrated before being collected into the trails array. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove the no-policy code path (raw JSON dump) since there's no use case for it without evaluation. Add "policy" to RequireFlags, remove early-return blocks, update tests accordingly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move fetchAndEnrichTrail, evaluateAndPrintResult, and printEvaluateInput into evaluateHelpers.go. Both evaluate trail and evaluate trails now delegate to these shared functions, eliminating duplicated code. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Both evaluateTrailOptions and evaluateTrailsOptions now embed commonEvaluateOptions instead of duplicating the same five fields. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace manual if/else output format branching with the output.FormattedPrint dispatcher pattern used by other commands like getTrail. Adds printEvaluateResultAsJson and printEvaluateResultAsTable format functions. Removes the manual --output validation from both run methods, letting FormattedPrint handle unsupported formats consistently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…etching Log HTTP and JSON parse errors at debug level when fetching attestation details during trail enrichment, instead of silently continuing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use assert for leaf value checks in TestTransformTrail, consistent with the other test functions in the file. Keep require only for the IsType guard that validates the type before subsequent access. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sform.go Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ailEnrichment Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes SNYK-GOLANG-GOLANGORGXNETHTTP2-15363313 — missing nil check in http2 causing server panic on malformed frames. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why: Customers are duplicating attestation types to work around the lack of evaluation logic — creating separate types per environment just to encode different pass/fail rules. Three independent customers (JB, Deutsche Bank, Norsk Tipping) have hit this, and two more (Blackstone, NatWest) are building toward it. This is the most consistent product gap in our pipeline.
Objective: Add
kosli evaluate— a CLI command that applies a Rego policy to trail data and returns a structured pass/fail decision. This separates what you collect (attestation type) from how you judge it (evaluation), and is the first step toward controls as a first-class product concept.How this was built
This PR is also a demonstration of elephant carpaccio + TDD with Claude Code. The entire feature was built as a conversation — 48 commits, each one a single red-green-refactor step. Slices were kept thin enough to review independently, and the commit list reads bottom-to-top as a narrative of how the feature grew. Later commits came from reviewing the branch against Beck's Rules of Simple Design. The branch is intentionally unrebased so you can follow the progression.
Capabilities
kosli evaluate trail <name>— single trail against a Rego policykosli evaluate trails <name>...— multiple trails in one policy call--output json|table— structured audit output or human-readable (default)--show-input— include policy input in JSON output for debugging--attestations— filter which attestations reach the policy (plain name for trail-level,artifact.namefor artifact-level)Example
Validate all PRs are approved:
Kosli Server:
CyberDojo Dashboard:
Architecture
internal/evaluate/— OPA Rego engine + trail data transforms (array-to-map, rehydration from detail API, filtering)cmd/kosli/evaluateHelpers.go— shared options, flag registration, fetch+enrich pipeline, output dispatchcmd/kosli/evaluateTrail.go/evaluateTrails.go— thin command wrappersAlso included
golang.org/x/netto v0.51.0 for CVE-2026-27141