Skip to content

feat(android): Move flake.nix to devbox.d for version control#17

Open
abueide wants to merge 16 commits intomainfrom
feat/move-flake-to-devbox-d
Open

feat(android): Move flake.nix to devbox.d for version control#17
abueide wants to merge 16 commits intomainfrom
feat/move-flake-to-devbox-d

Conversation

@abueide
Copy link
Copy Markdown
Contributor

@abueide abueide commented Apr 20, 2026

Summary

Implements a two-stage configuration model for reproducible Android development:

  1. Stage 1: Edit env vars in devbox.json (easy to change)
  2. Stage 2: Run devbox run android:sync to generate lock files (commit to git)

This PR moves Android SDK configuration to version-controlled lock files for team-wide reproducibility while keeping env vars simple for day-to-day changes.

Key Changes

1. Flake Location - Moved to devbox.d/ for version control

  • flake.nix now copied to devbox.d/<plugin>/ instead of ephemeral .devbox/virtenv/
  • flake.lock pins nixpkgs version (committed)
  • Lives alongside device configs in committed directory

2. android.lock - New lock file for Android SDK config

  • Generated from env vars by android:sync command
  • Pins: build tools version, compile SDK, target SDK, system image tag, NDK/CMake settings
  • Committed to git for reproducibility
  • Makes SDK changes reviewable in PRs

3. Unified Sync Command - devbox run android:sync

  • Generates android.lock from env vars
  • Regenerates devices.lock from device JSON files
  • Syncs AVDs to match device definitions
  • One command to sync all configuration

4. Drift Detection - Automatic warnings on shell init

⚠️  WARNING: Android configuration has changed but lock file is outdated.

Environment variables don't match android.lock:
  ANDROID_BUILD_TOOLS_VERSION: "36.1.0" (env) vs "35.0.0" (lock)

To apply changes:
  devbox run android:sync

5. Comprehensive Documentation

  • Complete guide on env var → lock file model
  • Step-by-step update workflows
  • Separates Android SDK updates from nixpkgs updates
  • Explains why reproducibility matters

Configuration Model

devbox.d/segment-integrations.mobile-devtools.android/
├── flake.nix          # Nix template (from plugin, committed)
├── flake.lock         # Pins nixpkgs version (committed)
├── android.lock       # Pins Android SDK config (committed) ✨ NEW
└── devices/
    ├── devices.lock   # Pins device definitions (committed)
    ├── min.json       # Device configs (committed)
    └── max.json

Why lock files?

  • flake.lock → Ensures same nixpkgs (same Android package versions)
  • android.lock → Makes SDK changes reviewable
  • devices.lock → Pins which devices/APIs are used

Example Workflow

# 1. Edit env vars in devbox.json
vim devbox.json  # Change ANDROID_BUILD_TOOLS_VERSION: "36.1.0"

# 2. Sync to generate lock files
devbox run android:sync

# 3. Review and commit
git diff devbox.d/
git add devbox.json devbox.d/
git commit -m "chore: update Android SDK to 36.1.0"

Benefits

Reproducible: Lock files ensure identical builds across team
Reviewable: SDK changes visible in PRs
Explicit: Must run sync to apply changes (no accidents)
Detectable: Warns if env vars drift from lock file
Simple: Still just edit env vars, then run one command

Migration

Existing projects automatically get the new structure on next devbox shell:

  • flake.nix and flake.lock appear in devbox.d/
  • Run devbox run android:sync to generate android.lock
  • Commit the new lock files

Test Plan

  • Test with analytics-react-native E2E examples
  • Verify android.lock generated correctly
  • Verify drift detection works
  • Test sync command updates all locks
  • Verify Android SDK builds successfully

🤖 Generated with Claude Code

abueide and others added 2 commits April 20, 2026 12:28
Moves the Android SDK flake.nix from the ephemeral .devbox/virtenv
directory to devbox.d/<plugin-name>/ so the flake and its lock file
can be committed and version controlled per-project.

**Changes:**
- Copy flake.nix to `{{ .DevboxDir }}` instead of `{{ .Virtenv }}`
- Update core.sh to look for flake in ANDROID_CONFIG_DIR first
- Update README to clarify flake.lock location and purpose
- Add react-native example devbox.d configs to demonstrate structure

**Benefits:**
- Projects can version control their flake.lock for reproducible builds
- Each project can have different Android SDK versions/configurations
- Lock file updates via `devbox run devices.sh sync` or `nix flake update`
- Consistent with where device configs live (devbox.d/)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements a two-stage configuration model for reproducible Android builds:
- **Stage 1**: Edit env vars in devbox.json (easy to change)
- **Stage 2**: Run `android:sync` to generate lock files (commit to git)

**Changes:**

1. **android.lock file** - Pins Android SDK configuration
   - Generated from env vars by `android:sync` command
   - Committed to git for team-wide reproducibility
   - Makes SDK changes reviewable in PRs

2. **Unified sync command** - `devbox run android:sync`
   - Generates android.lock from env vars
   - Regenerates devices.lock from device JSONs
   - Syncs AVDs to match device definitions
   - One command to sync all configuration

3. **Drift detection** - Warns on shell init if config is out of sync
   - Compares env vars with android.lock
   - Shows which values don't match
   - Provides clear instructions to fix

4. **Comprehensive documentation**
   - Explains env var → lock file model
   - Step-by-step update guide
   - Separates Android SDK updates from nixpkgs updates
   - Clarifies why reproducibility matters

**Benefits:**
- Reproducible: Lock files ensure identical builds across team
- Reviewable: SDK changes visible in PRs
- Explicit: Must run sync to apply changes (no accidents)
- Detectable: Warns if env vars drift from lock file

**Example workflow:**
```sh
devbox run android:sync
git add devbox.json devbox.d/ && git commit
```

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@abueide abueide force-pushed the feat/move-flake-to-devbox-d branch from b07e540 to b7468ce Compare April 20, 2026 17:29
abueide and others added 3 commits April 20, 2026 12:33
Adds lightweight health checks that run automatically on shell init for
both Android and iOS environments. Shows a simple ✓ checkmark if everything
is good, or detailed warnings if issues are detected.

**Changes:**

1. **doctor-init.sh scripts** - Lightweight checks for shell init
   - Android: Checks SDK, tools, and config drift
   - iOS: Checks Xcode, simctl, and device lock
   - Silent output if healthy: just "✓ Android" or "✓ iOS"
   - Verbose output if issues: lists problems with fix instructions

2. **Integrated drift detection** - Moved from setup.sh to doctor
   - SDK drift warning now part of doctor-init check
   - Shows config mismatches between env vars and android.lock
   - Provides clear fix instructions

3. **Improved full doctor command** - Comprehensive diagnostics
   - Android: New doctor.sh script with structured output
   - iOS: Enhanced doctor output in plugin.json
   - Both show categorized checks with clear ✓/✗/⚠ indicators

4. **Better UX** - Immediate feedback on environment health
   - No more silent failures or hidden misconfigurations
   - Quick visual confirmation that environment is ready
   - Detailed diagnostics available via `devbox run doctor`

**Example output (healthy):**
```
✅ [OK] Android setup complete
✓ Android
✅ [OK] iOS setup complete
✓ iOS
```

**Example output (with issues):**
```
✅ [OK] Android setup complete
⚠️  Android issues detected:
  - Config drift: env vars don't match android.lock

  Config differences:
    ANDROID_BUILD_TOOLS_VERSION: "36.1.0" (env) vs "35.0.0" (lock)
  Fix: devbox run android:sync
  Run 'devbox run doctor' for more details
```

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Moves automatic doctor checks from user/ to init/ layer for better
separation of concerns.

**Structure:**
- init/doctor.sh - Automatic health check (runs on shell init)
- user/doctor.sh - Full diagnostic report (user-invoked command)

**Rationale:**
- user/ should only contain user-invokable commands
- init/ contains automatic initialization logic
- Clearer layering between public API (user/) and internal (init/)

No functional changes, just better organization.
…nd cleanup

Fixes bugs and code smells identified in PR review:

Security Fixes:
- Fix printf format string vulnerability in both doctor scripts
- Changed from 'printf "$var"' to 'printf '%s' "$var"' to prevent
  format string interpretation if env vars contain % characters

Code Quality Improvements:
- Extract drift detection to shared function (drift.sh)
  - Eliminates ~25 lines of duplication between init/doctor.sh and user/doctor.sh
  - Single source of truth for env var vs android.lock comparison
- Refactor sync command into modular helper functions
  - android_generate_android_lock() - Generate android.lock from env vars
  - android_regenerate_devices_lock() - Regenerate devices.lock
  - android_sync_avds() - Sync AVDs with device definitions
  - User-facing 'android:sync' remains simple (3 function calls)

Backwards Compatibility Removal (pre-1.0 cleanup):
- Remove all legacy fallback paths from core.sh:
  - ANDROID_RUNTIME_DIR fallback (deprecated)
  - DEVBOX_PROJECT_ROOT fallback (deprecated)
  - DEVBOX_PROJECT_DIR fallback (deprecated)
  - DEVBOX_WD fallback (deprecated)
  - Relative path fallback (deprecated)
- Remove legacy state file fallbacks from android.sh:
  - Legacy emulator-serial.txt location
  - Legacy app-id.txt location
- Remove legacy fallback from emulator.sh:
  - Legacy emulator-serial.txt location
- Now requires ANDROID_CONFIG_DIR (fails cleanly if not set)

Net Impact:
- 7 files changed: +172 insertions, -205 deletions (net -33 lines)
- 0 legacy fallbacks remaining
- Code is cleaner, safer, and ready for 1.0 release

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@abueide abueide force-pushed the feat/move-flake-to-devbox-d branch from f46e873 to 0db92e9 Compare April 20, 2026 18:34
abueide and others added 11 commits April 20, 2026 14:17
React Native 0.83 does NOT require NDK to be pre-installed:
- React Native ships precompiled (since 0.71)
- No native C++ code compilation in standard RN apps
- New Architecture disabled (newArchEnabled=false)
- Gradle can download NDK if truly needed

Previous defaults:
- ANDROID_INCLUDE_NDK: true (unnecessary)
- ANDROID_NDK_VERSION: 29.0.14206865 (broken on aarch64-darwin)
- ANDROID_INCLUDE_CMAKE: true (unnecessary)

New defaults (matching Android plugin):
- ANDROID_INCLUDE_NDK: false
- ANDROID_NDK_VERSION: 27.0.12077973 (more stable)
- ANDROID_INCLUDE_CMAKE: false

This fixes Nix build failures on platforms where NDK 29 has limited
support (aarch64-darwin, aarch64-linux) per nixpkgs PR #379534.

Benefits:
- Faster devbox shell initialization (no NDK download)
- Avoids nixpkgs platform support issues
- Smaller SDK footprint
- Still works perfectly for standard React Native development

See notes/NDK_NOT_NEEDED.md for detailed analysis.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add detection for hash mismatch issues in Android SDK builds, which occur
when Google updates files on their servers without changing version numbers.

Changes:
- Detect "hash mismatch" and dependency failure patterns in nix build output
- Show user-friendly error message with 3 workaround options:
  1. Use local Android Studio SDK (ANDROID_LOCAL_SDK=1)
  2. Update nixpkgs to get latest hashes (nix flake update)
  3. Run on Linux x86_64 where builds are more reliable
- Add --show-trace flag to nix build for better debugging
- Link to nixpkgs issues for reference

This is a known recurring issue with Android SDK in nixpkgs due to Google's
practice of updating files at stable URLs without version changes.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add tooling to automatically detect and fix Android SDK hash mismatches
that occur when Google updates files on their servers without changing
version numbers.

New Features:
- hash-fix.sh script that:
  - Detects hash mismatches from nix build errors
  - Downloads files and computes correct hashes
  - Updates android.json with hash overrides
- devbox run android:hash-fix command for easy fixing
- Enhanced error messages with fix guidance
- Preserve error logs for analysis

Changes:
- plugins/android/virtenv/scripts/domain/hash-fix.sh (NEW)
  - auto: automatic detection and fix
  - detect: parse hash mismatch from nix stderr
  - compute: download and compute SHA1 hash
  - update: update android.json with override

- plugins/android/virtenv/scripts/platform/core.sh
  - Keep nix stderr logs instead of deleting them
  - Detect hash mismatch patterns in errors
  - Show "devbox run android:hash-fix" suggestion
  - Save error log path for hash-fix script

- plugins/android/virtenv/flake.nix
  - Add hash_overrides support in android.json
  - Prepared for future overlay-based hash patching

- plugins/android/plugin.json
  - Add hash-fix.sh to deployed scripts
  - Add android:hash-fix devbox command

This addresses the recurring nixpkgs issue where Google updates Android SDK
files at stable URLs, breaking Nix's content-addressable builds.

Related: NixOS/nixpkgs#511856 (our upstream PR to fix platform-tools hash)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Improve UX by automatically running hash-fix when a mismatch is detected,
eliminating the need for users to remember a separate command.

Changes:
- core.sh: Automatically run hash-fix.sh when hash mismatch detected
- hash-fix.sh: Add quiet mode (verbose only when run manually)
- plugin.json: Manual command uses verbose mode for details

User Experience:
BEFORE:
  $ devbox shell
  [error message suggesting devbox run android:hash-fix]
  $ devbox run android:hash-fix
  [fix happens]
  $ devbox shell
  [works]

AFTER:
  $ devbox shell
  [detects + fixes automatically]
  $ devbox shell
  [works]

The fix is automatic - users just need to run 'devbox shell' twice.
We can't make it fully silent because the Nix build fails before we can
restart it, but this is the best reasonable UX.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
BREAKING CHANGE: Hash overrides now stored in committed location to ensure
reproducibility across the team.

Problem:
- Hash overrides were stored in .devbox/virtenv/android.json (gitignored)
- Each developer had to fix hash mismatches individually
- CI/CD would fail without the fix
- No reproducibility across team

Solution:
- Store overrides in devbox.d/plugin-name/hash-overrides.json (committed)
- init-hook.sh merges overrides into android.json during shell initialization
- Developers commit the file → whole team gets the fix
- Reproducibility preserved ✅

Changes:
- hash-fix.sh: Write to hash-overrides.json in committed location
- init-hook.sh: Merge hash-overrides.json into android.json on init
- core.sh: Instruct users to commit hash-overrides.json
- config/README.md: Document hash-overrides.json purpose and workflow
- config/hash-overrides.json.example: Show file format

User Workflow:
1. devbox shell → auto-fix creates/updates hash-overrides.json
2. devbox shell → works with fixed hash
3. git add devbox.d/*/hash-overrides.json && git commit
4. Team pulls → everyone gets the fix

The file should be committed and is safe to keep (stale overrides are harmless).
Remove entries when they're no longer needed (after nixpkgs updates).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fix missing jq availability check in drift detection
- Fix race condition in stderr file creation (use mktemp)
- Add ANDROID_CONFIG_DIR validation in hash-fix
- Add trap for guaranteed temp file cleanup
- Fix boolean normalization consistency
- Extract config vars to readonly array
- Refactor hash-fix into smaller, focused functions
- Add logging utility functions for cleaner code

Resolves high-priority bugs and code smells from PR review.
Keeps iOS and Android scripts separate as they are independent plugins.
The previous attempt to disable NDK (commit 036cbb1) assumed Gradle
could auto-download NDK at build time. However, this fails because:

1. Nix-provided Android SDK path is read-only
2. Gradle cannot install NDK to /nix/store/.../androidsdk/
3. Build fails with: 'SDK directory is not writable'

Error from CI:
  Failed to install the following SDK components:
      ndk;27.0.12077973 NDK (Side by side) 27.0.12077973
  The SDK directory is not writable (/nix/store/.../androidsdk/libexec/android-sdk)

Solution:
  Re-enable ANDROID_INCLUDE_NDK and ANDROID_INCLUDE_CMAKE in the
  react-native plugin to ensure NDK is provided by Nix upfront.

This uses NDK 27.0.12077973 (more stable than 29.x) which has better
platform support while avoiding the aarch64-darwin issues.

Fixes: android-max E2E test failure in CI
The previous fix used 'trap ... EXIT' which persists across the entire
script. When the function returned, temp_dir went out of scope but the
trap remained active, causing 'unbound variable' errors.

Error in CI:
  devices.sh: line 1: temp_dir: unbound variable

Solution: Use 'trap ... RETURN' for function-scope cleanup. The trap
is automatically removed when the function returns.

Fixes: Android E2E max test bash error
API 21 (Android 5.0) system images are not available in Nix SDK,
causing E2E tests to fail with missing system image errors:

  ⚠ System image not available (API 21, tag google_apis)
  ERROR: 1 device(s) skipped due to missing system images (strict mode)

API 24 is more appropriate as:
- React Native examples already use minSdkVersion = 24
- API 24 (Android 7.0) is widely supported
- System images are available in Nix

This aligns the min device configuration with actual app requirements.

Fixes: Android E2E max test missing system image error
Problem:
- Hash overrides were defined in android.json but never used
- flake.nix read hashOverrides but didn't apply them to derivations
- platform-tools_r37.0.0-darwin.zip hash mismatch on macOS

Solution:
1. **Implement overlay in flake.nix**: Apply hashOverrides by wrapping
   fetchurl to use custom sha256 when URL matches
2. **Add platform-tools hash override**: Include correct hash for
   platform-tools r37.0.0 darwin in config/hash-overrides.json

Changes:
- plugins/android/virtenv/flake.nix:
  * Add pkgsWithOverrides that applies fetchurl overlay when hashOverrides exist
  * Use pkgsWithOverrides for androidenv instead of plain pkgs
- plugins/android/config/hash-overrides.json:
  * New file with platform-tools r37.0.0 darwin hash

Technical Details:
- Uses Nix overlay to intercept fetchurl calls
- Checks if fetched URL matches any in hashOverrides
- Substitutes sha256 when match found
- Only applies overlay if hashOverrides non-empty (zero overhead otherwise)

Impact:
- Fixes Android SDK build failures on macOS
- Projects can now override hashes via devbox.d/android/hash-overrides.json
- Hash override mechanism now fully functional

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The hash override implementation introduced a typo on line 125:
- Used: config.includeCmake (lowercase 'c')
- Should be: config.includeCMake (uppercase 'C')

This caused Nix evaluation to fail with:
  error: attribute 'includeCmake' missing

Fixed by correcting the case to match the config definition.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant