Skip to content

OCI registry support for Bicep extension publishing#18956

Open
willdavsmith wants to merge 4 commits intoAzure:mainfrom
willdavsmith:oci-support
Open

OCI registry support for Bicep extension publishing#18956
willdavsmith wants to merge 4 commits intoAzure:mainfrom
willdavsmith:oci-support

Conversation

@willdavsmith
Copy link
Copy Markdown
Contributor

@willdavsmith willdavsmith commented Feb 5, 2026

OCI Registry Support for Bicep Extension Publishing

Description

Adds support for publishing and restoring Bicep modules and extensions to/from non-Azure OCI-compliant container registries (GHCR, Docker Hub, etc.).

Previously, publish, restore, and publish-extension only worked with Azure Container Registry (ACR) because the transport was hard-coded to the Azure SDK's ContainerRegistryContentClient. This change introduces an alternative transport built on ORAS (OCI Registry As Storage), so Bicep artifacts can be stored in any compliant registry.

Related to #4884.


User Experience

Feature flag

The feature is gated behind an experimental flag called ociEnabled. Any one of the following enables it:

  1. CLI flag — pass --oci-enabled before the subcommand:
bicep --oci-enabled publish myModule.bicep \
  --target 'br:ghcr.io/myorg/bicep/modules/my-module:v1.0'
  1. Environment variable — set BICEP_EXPERIMENTAL_OCI=1 (or true)
export BICEP_EXPERIMENTAL_OCI=1
bicep publish myModule.bicep --target 'br:ghcr.io/myorg/bicep/modules/my-module:v1.0'
  1. bicepconfig.json:
{
  "experimentalFeaturesEnabled": {
    "ociEnabled": true
  }
}

Authentication

When the flag is enabled, Bicep authenticates to non-Azure registries using Docker credentials:

  1. Docker credential helpers — reads ~/.docker/config.json and invokes the configured credsStore or per-registry credHelpers (e.g. docker-credential-desktop, docker-credential-ecr-login).
  2. Static auth entries — falls back to base64-encoded username:password entries in the auths section of ~/.docker/config.json.

ACR (*.azurecr.io, mcr.microsoft.com) continues to use the existing Azure SDK auth path.

Example: publish a module to GHCR

# Log in to GHCR (one-time, stores creds in Docker config)
echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin

# Publish
bicep --oci-enabled publish ./main.bicep \
  --target 'br:ghcr.io/myorg/bicep/modules/network:v1.0'

# Reference in another file:
#   module vnet 'br:ghcr.io/myorg/bicep/modules/network:v1.0' = { ... }

# Restore
bicep --oci-enabled restore ./consumer.bicep

Example: publish an extension

bicep --oci-enabled publish-extension \
  --target 'br:myregistry.example.com/bicep/extensions/my-ext:v1.0' \
  --index-file ./out/index.json

New diagnostic: BCP440

Referencing a non-Azure registry without the flag produces:

BCP440: Using registry "ghcr.io" requires enabling EXPERIMENTAL feature "OciEnabled".


Design

Registry providers

A new IRegistryProvider interface lets different strategies handle different registry hosts:

IRegistryProvider
├── AcrRegistryProvider        (priority 100 — *.azurecr.io, mcr.microsoft.com)
└── GenericOciRegistryProvider (priority 0   — everything else)

RegistryProviderFactory resolves the provider for a given hostname. ACR always wins for Azure hosts; the ORAS-based provider is the fallback.

Transport abstraction

AzureContainerRegistryManager is now behind IOciRegistryTransport. A second implementation, OrasOciRegistryTransport, uses oras-dotnet to talk to generic registries. IOciRegistryTransportFactory picks the right one based on hostname.

Transport Library Handles
AzureContainerRegistryManager Azure SDK ACR
OrasOciRegistryTransport oras-dotnet Everything else

Session API

When ociEnabled is active, push/pull/resolve go through IRegistrySession instead of the transport directly:

  • PushAsync — pushes config, layers, and manifest
  • PullAsync — fetches manifest and layers
  • ResolveAsync — resolves a reference to manifest metadata (used by CheckArtifactExists)

Two implementations: AcrRegistrySession (wraps AzureContainerRegistryManager) and OrasRegistrySession (ORAS library).

Credential chain

CompositeCredentialChain
└── DockerCredentialSource
    └── DockerCredentialProvider
        ├── reads ~/.docker/config.json
        ├── invokes credsStore helpers (docker-credential-*)
        ├── invokes per-registry credHelpers
        └── falls back to static auth entries

DockerCredentialProvider implements the Docker credential helper protocol — sends the registry hostname to docker-credential-<helper> get via stdin, parses the JSON response. Supports username/password and identity token auth.

New dependency

Project reference to oras-dotnet for the generic OCI transport.

Registry routing

Registries are classified as Azure (ACR/MCR) or generic based on hostname. Hosts matching *.azurecr.io/.cn/.us/.de/.gov or mcr.microsoft.com route through the existing Azure SDK path (
AzureContainerRegistryManager). All other registries route through the new ORAS-based transport with Docker credential resolution. ACR registries behind custom domains will fall through to the generic path, which works via
Docker credential helpers (e.g. docker-credential-acr-env). A future improvement could use OCI auth challenge detection to automatically identify the backing provider regardless of hostname.


What's unchanged

  • ACR workflows — still use the Azure SDK path, no behavioral changes.
  • Module reference syntax — same br:registry/path:tag format for any registry.
  • Local artifact cache — non-Azure artifacts are cached the same way as ACR artifacts.

Checklist

Microsoft Reviewers: Open in CodeFlow

@majastrz
Copy link
Copy Markdown
Member

Related to #4884

@willdavsmith willdavsmith changed the title [WIP] OCI registry support for Bicep extension publishing OCI registry support for Bicep extension publishing Mar 10, 2026
@willdavsmith willdavsmith marked this pull request as ready for review March 10, 2026 21:55
@andyleejordan
Copy link
Copy Markdown
Contributor

I'm very happy to see this coming along!

Adds ORAS-based transport for non-Azure OCI registries (GHCR, Docker Hub, etc).
New experimental feature flag 'ociEnabled' gates the new code path.
Includes priority-based registry provider routing, Docker credential support,
and session-based push/pull/resolve operations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@willdavsmith willdavsmith force-pushed the oci-support branch 4 times, most recently from 0bddcfd to 4afe9d0 Compare April 1, 2026 23:01
Adds ORAS-based transport for non-Azure OCI registries (GHCR, Docker Hub, etc).
New experimental feature flag 'ociEnabled' gates the new code path.
Includes priority-based registry provider routing, Docker credential support,
and session-based push/pull/resolve operations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@andyleejordan
Copy link
Copy Markdown
Contributor

Good news! I've successfully tested this to upload my extension to GitHub:

Publishing extension package to "br:ghcr.io/microsoft/bicep-types-dsc:latest".

...I still need to test it restoring (without credentials) and I should test it with RedHat's Podmon instead of Docker.

Signed-off-by: willdavsmith <willdavsmith@gmail.com>
@willdavsmith
Copy link
Copy Markdown
Contributor Author

Good news! I've successfully tested this to upload my extension to GitHub:

Publishing extension package to "br:ghcr.io/microsoft/bicep-types-dsc:latest".

...I still need to test it restoring (without credentials) and I should test it with RedHat's Podmon instead of Docker.

Awesome! Thanks for trying it out and I'm glad it's useful for you. Feel free to update here, and if my commits break anything, please let me know.

@andyleejordan
Copy link
Copy Markdown
Contributor

@willdavsmith successful demo with a dev build of the VS Code extension restoring the extension off GitHub for my Bicep file.

@gXkch
Copy link
Copy Markdown

gXkch commented Apr 14, 2026

@willdavsmith thank you for your work on this. this will make usage in an enterprise environment where self managed OCI registries exist finally a reality.


public IDirectoryHandle CacheRootDirectory => GetCacheRootDirectory(this.configuration.CacheRootDirectory);

public bool OciEnabled => this.configuration.ExperimentalFeaturesEnabled.OciEnabled || ReadBooleanEnvVar("BICEP_EXPERIMENTAL_OCI", defaultValue: false);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this env var at all? Generally we manage feature flags purely through bicepconfig, rather than duplicating.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it was just set for testing. Removed.

Signed-off-by: willdavsmith <willdavsmith@gmail.com>
var reference = ValidateReference(args.TargetExtensionReference);
var overwriteIfExists = args.Force;

logger.LogInformation("Preparing to publish extension \"{Reference}\" (force={Force}, indexFile={IndexFile}, binariesSpecified={BinaryCount}).",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logger here will output everythign to stdout, whereas it looks more like the intention here is to add tracing.

I'd suggest either:

  • Removing the logging
  • Using trace methods instead (e.g. Trace.WriteLine(...))

"BCP367",
$"The \"{featureName}\" feature is temporarily disabled.");

public Diagnostic NonAzureOciRegistryRequiresExperimentalFeature(string registry) => CoreError(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Could you add this to the file in ascending code order (e.g. after BCP445)?

Comment on lines +65 to +91
{
var value = Environment.GetEnvironmentVariable(envVar);
if (value is null)
{
return defaultValue;
}

if (bool.TryParse(value, out var boolValue))
{
return boolValue;
}

if (int.TryParse(value, NumberStyles.Integer, CultureInfo.InvariantCulture, out var intValue))
{
if (intValue == 1)
{
return true;
}

if (intValue == 0)
{
return false;
}
}

return defaultValue;
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change strictly necessary, now you've removed the use of env vars for feature flag enablement? If not, let's remove it.

Comment on lines +50 to +54
services.TryAddSingleton<IOciRegistryTransportFactory>(sp =>
{
var providerFactory = sp.GetRequiredService<RegistryProviderFactory>();
return new OciRegistryTransportFactory(providerFactory);
});
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary? Can't you just write:

services.TryAddSingleton<IOciRegistryTransportFactory, OciRegistryTransportFactory>();

services.TryAddSingleton<AzureContainerRegistryManager>();
services.TryAddSingleton<DockerCredentialProvider>();
services.TryAddSingleton<DockerCredentialSource>();
services.TryAddSingleton<ICredentialSource>(sp => sp.GetRequiredService<DockerCredentialSource>());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem right that requesting the generic interface ICredentialSource should always give you an instance of DockerCredentialSource. If that's intentional, should we rename ICredentialSource to IDockerCredentialSource to indicate that you are specifically requesting the Docker implementation?

Comment on lines +58 to +60
<ItemGroup>
<PackageReference Include="OrasProject.Oras" />
</ItemGroup>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Add to the section with the other PackageReference blocks

Comment on lines +24 to 27
serviceProvider.GetRequiredService<IPublicModuleMetadataProvider>(),
serviceProvider.GetRequiredService<ICredentialChain>(),
serviceProvider.GetService<ILogger<OciArtifactRegistry>>() ?? NullLogger<OciArtifactRegistry>.Instance),
new TemplateSpecModuleRegistry(templateSpecRepositoryFactory),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not follow the standard DI pattern of passing these in as constructor arguments?

{
foreach (var tag in manifestProps.Tags)
{
cancellationToken.ThrowIfCancellationRequested();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The explicit cancellation checks feel unnecessary, given that you're passing the token down to async methods correctly.


// Fall back to authenticated client.
return await DownloadManifestInternalAsync(anonymousAccess: false);
return await DownloadManifestInternalAsync(anonymousAccess: false, cancellationToken).ConfigureAwait(false);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] we haven't been requiring this pattern in this repo, given it's a .NET 10 project.

@@ -23,14 +24,38 @@ public static class TypesV1Archive
{
public static async Task<BinaryData> PackIntoBinaryData(IFileHandle typeIndexFile)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why were the changes to this file necessary?

Comment on lines +22 to +23
private static readonly ImmutableHashSet<string> AzureRegistryHosts =
ImmutableHashSet.Create(StringComparer.OrdinalIgnoreCase, "mcr.microsoft.com");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels odd to treat mcr.microsoft.com as an "Azure registry host", which I would assume means an "ACR". Is this correct?

@@ -34,6 +38,20 @@ public static IServiceCollection AddBicepCore(this IServiceCollection services)
services.TryAddSingleton<INamespaceProvider, NamespaceProvider>();
services.TryAddSingleton<IResourceTypeProviderFactory, ResourceTypeProviderFactory>();
services.TryAddSingleton<IContainerRegistryClientFactory, ContainerRegistryClientFactory>();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this interface and class are now ACR-specific - is that correct? If so, should we name them something specific to make this clearer?


namespace Bicep.Core.Registry.Sessions;

public sealed record RegistryRef(string Host, string Repository, string? Tag, string? Digest);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this overlap with the existing IOciArtifactAddressComponents interface?

Comment on lines +10 to +14
IOciRegistryTransport GetTransport(OciArtifactReference reference);

IOciRegistryTransport GetTransport(string registry);

IRegistrySession CreateSession(RegistryRef reference, RegistryProviderContext context);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there's some overlap between IRegistrySession and IOciRegistryTransport, and the naming of these 2 interfaces doesn't make it super clear to me what the 2 intended responsibilities are. Could you clarify on this?


namespace Bicep.Core.Registry.Sessions;

public sealed record RegistryRef(string Host, string Repository, string? Tag, string? Digest);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegistryRef looks structurally identical to IOciArtifactAddressComponents (same fields, just Host renamed to Registry). The only usage in OciArtifactRegistry is CreateRegistryRef, which projects an OciArtifactReference into a RegistryRef — and AcrRegistrySession then immediately wraps it back into a private SessionOciArtifactReference that re-implements IOciArtifactReference. Could we just pass IOciArtifactAddressComponents (or OciArtifactReference) directly to IRegistrySession, and avoid the round-trip?


namespace Bicep.Core.Registry.Sessions;

public sealed record RegistryArtifact(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegistryArtifact looks like OciManifest minus SchemaVersion. After a pull via IRegistrySession.PullAsync, the result is immediately converted back to OciArtifactResult via ConvertToOciArtifactResult. Could we just use OciManifest for push input and OciArtifactResult for pull output, and avoid the conversion step?


namespace Bicep.Core.Registry.Sessions;

public sealed record RegistryManifestInfo(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fields here (Digest, MediaType, ArtifactType, Annotations) are a subset of what's already available from OciManifest + OciArtifactResult.ManifestDigest. Could ResolveAsync return OciArtifactResult instead, so we don't need a separate type?


public override async Task<bool> CheckArtifactExists(ArtifactType artifactType, OciArtifactReference reference)
{
if (reference.ReferencingFile.Features.OciEnabled)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every operation (PublishModule, PublishExtension, CheckArtifactExists, TryRestoreArtifactAsync, TryGetOciAnnotations) has an if (OciEnabled) { session path } else { transport path } fork. Since AcrRegistrySession already wraps AzureContainerRegistryManager, is there a reason the ACR path can't go through IRegistrySession unconditionally? It would simplify things a lot and mean we're not carrying dead code once the feature flag is removed.


namespace Bicep.Core.Registry.Auth;

public interface ICredentialSource
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's only one ICredentialSource implementation (DockerCredentialSource), and CompositeCredentialChain just iterates a single-element list. Also, the AuthMetadata? challenge parameter is accepted by all the credential interfaces but never actually read in DockerCredentialSource. Could we simplify this — e.g. inject DockerCredentialProvider directly into OrasRegistrySession, and add back the composite pattern only if a second source is actually needed?

/// <summary>
/// Represents a strategy for handling registry interactions for a specific host or group of hosts.
/// </summary>
public interface IRegistryProvider
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The routing chain here is OciArtifactRegistryIOciRegistryTransportFactoryRegistryProviderFactoryIRegistryProvider, and both OciRegistryTransportFactory and RegistryProviderFactory do essentially the same thing (resolve by hostname, delegate). The Name/Priority/CanHandle strategy pattern on IRegistryProvider feels heavyweight for a two-case dispatch (*.azurecr.io vs everything else). Could these layers be collapsed?


public required CloudConfiguration Cloud { get; init; }

public CancellationToken CancellationToken { get; init; }
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing CancellationToken through a context object is unusual in .NET — tokens normally flow through method arguments. Looking at the usage, TryCreateSession always passes default for the token, and every session method already accepts its own token directly. Could we remove the token from the context? Similarly, ILogger and CloudConfiguration feel like they should be constructor parameters on the session implementations (via DI) rather than passed in at session-creation time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants