Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 5 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,14 @@ This version of the DevCycle SDK works with Java 11 and above.

Using the Java SDK library requires [Maven](https://maven.apache.org/) or [Gradle](https://gradle.org/) >= 7.6+ to be installed.

An x86_64 or aarch64 JDK is required for Local Bucketing with the DevCycle Java SDK.
Local bucketing runs the bucketing WebAssembly module using **wasmtime-java** (default) or **[Chicory](https://chicory.dev/)** (pure Java, using Chicory's runtime compiler). Pick at **process startup** with the environment variable **`DEVCYCLE_USE_CHICORY`**:

Currently Supported Platforms are:
- **Unset or any other value:** wasmtime-java (JNI; on Linux requires **glibc** and a supported arch for the bundled native library).
- **`1`**, **`true`**, or **`yes`** (case-insensitive): Chicory only for WASM execution (no WASM JNI; suitable for **Alpine Linux / musl**).

| OS | Arch |
|----------------|-----------|
| Linux (ELF) | x86_64 |
| Linux (ELF) | aarch64 |
| Mac OS | x86_64 |
| Mac OS | aarch64 |
| Windows | x86_64 |
Both runtimes are on the classpath; only the selected one is used for `LocalBucketing`.

In addition, the environment must support GLIBC v2.16 or higher. You can use the following command to check your GLIBC version:

```bash
ldd --version
```
Use a [supported JDK](https://adoptium.net/) for your OS and CPU architecture.

## Installation

Expand Down
14 changes: 13 additions & 1 deletion benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,21 @@ Currently Supported Platforms are:

In addition the benchmark tool requires [Maven](https://maven.apache.org/) to be installed.

## WASM / `LocalBucketing` microbenchmarks (Gradle)

From the **repository root** (not this directory), run:

```bash
./gradlew jmh
```

This runs JMH benchmarks in `src/jmh/java/.../WasmInterfaceBenchmark.java` against `LocalBucketing` (the WASM boundary): `generateBucketedConfigForUserUTF8` and `variableForUser_PB`, using `fixture_small_config.json` and `fixture_large_config.json` under `src/jmh/resources/`. Results are written under `build/results/jmh/`.

The harness uses `LocalBucketing.forWasmtime()` and `LocalBucketing.forChicory()` so **both** runtimes are measured in one JMH run (`wasmRuntime` parameter). The Chicory path uses Chicory's runtime compiler rather than the default interpreter. You do not need `DEVCYCLE_USE_CHICORY` for this benchmark.

## How Does it Work?

The benchmark itself is defined in `SDKBenchmark.java`. It is configured to test a single boolean variable evaluation repeatedly based on the config defined in `resources/fixture_large_config.json`
The Maven benchmark is defined in `SDKBenchmark.java`. It is configured to test a single boolean variable evaluation repeatedly based on the config defined in `resources/fixture_large_config.json`

A local web server (see `MockServer.java`) is created to replicate DevCycle config and event services and support the SDK client.

Expand Down
9 changes: 9 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ plugins {
id 'signing'
id "com.google.protobuf" version "0.9.4"
id "com.vanniktech.maven.publish" version "0.30.0"
id 'me.champeau.jmh' version '0.7.2'
}
import com.vanniktech.maven.publish.SonatypeHost

Expand Down Expand Up @@ -110,6 +111,7 @@ ext {
lombok_version = "1.18.30"
okhttp_version = "4.12.0"
wasmtime_version = "0.19.0"
chicory_version = "1.7.3"
junit_version = "4.13.2"
mockito_core_version = "5.6.0"
protobuf_version = "3.25.7"
Expand All @@ -132,6 +134,8 @@ dependencies {
implementation("io.swagger.core.v3:swagger-annotations:$swagger_annotations_version")

implementation("io.github.kawamuray.wasmtime:wasmtime-java:$wasmtime_version")
implementation("com.dylibso.chicory:runtime:$chicory_version")
implementation("com.dylibso.chicory:compiler:$chicory_version")

implementation("com.google.protobuf:protobuf-java:$protobuf_version")

Expand Down Expand Up @@ -179,3 +183,8 @@ task runOpenFeatureExample(type: JavaExec) {
classpath = sourceSets.examples.runtimeClasspath
mainClass = 'com.devcycle.examples.OpenFeatureExample'
}

jmh {
duplicateClassesStrategy = DuplicatesStrategy.EXCLUDE
includes = ['com\\.devcycle\\.sdk\\.server\\.local\\.bench\\..*']
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
package com.devcycle.sdk.server.local.bench;

import com.devcycle.sdk.server.common.model.DevCycleUser;
import com.devcycle.sdk.server.common.model.PlatformData;
import com.devcycle.sdk.server.local.bucketing.LocalBucketing;
import com.devcycle.sdk.server.local.protobuf.VariableForUserParams_PB;
import com.devcycle.sdk.server.local.protobuf.VariableType_PB;
import com.devcycle.sdk.server.local.utils.ProtobufUtils;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.TimeUnit;

import com.fasterxml.jackson.core.JsonProcessingException;

/**
* JMH microbenchmarks comparing {@code wasmtime-java} vs {@code Chicory} for the same {@link
* LocalBucketing} WASM calls. Run: {@code ./gradlew jmh}
*/
public class WasmInterfaceBenchmark {

private static String readClasspathResource(String name) throws IOException {
try (InputStream in =
WasmInterfaceBenchmark.class.getClassLoader().getResourceAsStream(name)) {
if (in == null) {
throw new IOException("Missing classpath resource: " + name);
}
return new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
}

private static LocalBucketing newBucketingForParam(String wasmRuntime) {
if ("chicory".equals(wasmRuntime)) {
return LocalBucketing.forChicory();
}
if ("wasmtime".equals(wasmRuntime)) {
return LocalBucketing.forWasmtime();
}
throw new IllegalArgumentException("wasmRuntime must be wasmtime or chicory: " + wasmRuntime);
}

@State(Scope.Benchmark)
public static class SmallFixtureState {
@Param({"wasmtime", "chicory"})
public String wasmRuntime;

LocalBucketing bucketing;
String sdkKey;
DevCycleUser user;
byte[] variableParamsProtobuf;

@Setup
public void setup() throws IOException, JsonProcessingException {
String config = readClasspathResource("fixture_small_config.json");
bucketing = newBucketingForParam(wasmRuntime);
bucketing.setPlatformData(PlatformData.builder().build().toString());
sdkKey = "server-jmh-small-" + wasmRuntime;
bucketing.storeConfig(sdkKey, config);
user =
DevCycleUser.builder()
.userId("jmh-user")
.email("bench@example.com")
.build();
VariableForUserParams_PB params =
VariableForUserParams_PB.newBuilder()
.setSdkKey(sdkKey)
.setUser(ProtobufUtils.createDVCUserPB(user))
.setVariableKey("a-cool-new-feature")
.setVariableType(VariableType_PB.Boolean)
.setShouldTrackEvent(false)
.build();
variableParamsProtobuf = params.toByteArray();
}
}

@State(Scope.Benchmark)
public static class LargeFixtureState {
@Param({"wasmtime", "chicory"})
public String wasmRuntime;

LocalBucketing bucketing;
String sdkKey;
DevCycleUser user;

@Setup
public void setup() throws IOException {
String config = readClasspathResource("fixture_large_config.json");
bucketing = newBucketingForParam(wasmRuntime);
bucketing.setPlatformData(PlatformData.builder().build().toString());
sdkKey = "server-jmh-large-" + wasmRuntime;
bucketing.storeConfig(sdkKey, config);
user =
DevCycleUser.builder()
.userId("jmh-user")
.email("some.user@gmail.com")
.build();
}
}

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 2, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(1)
public void generateBucketedConfig_smallFixture(SmallFixtureState state)
throws JsonProcessingException {
state.bucketing.generateBucketedConfig(state.sdkKey, state.user);
}

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 2, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(1)
public void generateBucketedConfig_largeFixture(LargeFixtureState state)
throws JsonProcessingException {
state.bucketing.generateBucketedConfig(state.sdkKey, state.user);
}

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 2, time = 1)
@Measurement(iterations = 5, time = 1)
@Fork(1)
public void variableForUserProtobuf_smallFixture(SmallFixtureState state) {
state.bucketing.getVariableForUserProtobuf(state.variableParamsProtobuf);
}
}
Loading
Loading