Lumen

Dynamic instrumentation and automated API reverse engineering for web and Android apps. Lumen runs the target in an isolated environment, drives the UI with a vision-language agent (Midscene.js), intercepts traffic through a Mockttp proxy, and uses an LLM pipeline with retrieval over past requests to build a living RAML 1.0 specification you can watch update in real time.

Why Lumen?

• Proxies log traffic, not intent. Tools like Burp or mitmproxy give you raw requests but not the UI action that triggered each call. Lumen tags flows with the current screen and workflow so the analyzer can reason about purpose and parameters of each request.
• Sandboxes don't explore. Lumen uses an agent to plan and execute realistic workflows, including "negative" paths where useful (e.g. bad login), instead of a fixed shallow click script.
• Instrumentation kits need you. Frida-style tooling is powerful but manual per target. Lumen wires environment + exploration + capture + semantic analysis into one loop.

How it works

1. Explore — Screenshots go to a VLM-backed planner, Midscene for the web or Android through Appium. The Blackbox orchestrator walks the app with a depth-limited strategy, checkpoints, and snapshots for branches.
2. Observe — All HTTP(S) traffic passes through the local proxy. Quick filters drop noise (method, MIME, domain classes, blocklists) before deeper work.
3. Reason — For each interesting flow, a LangGraph-based analyzer gets app context, UI tags, and similar prior routes from an in-memory SchemaStore (RAG-style retrieval + weighted scoring). Results merge into a structured route and RAML snapshot.
4. Observe (again) — The Vue dashboard streams screenshots, tasks, requests, and spec updates over HTTP + WebSocket.

IDDFS-style Exploration

Plain DFS can get stuck in one deep branch; BFS wants too many expensive state snapshots (especially on Android). Lumen uses iteratively deepening depth-first search: shallow depth limits, checkpoints for postponed branches, snapshots mainly when multiple workflows branch - broad discovery early, fewer giant saves.

Requirements

• Bun (or Node with tsx) for the backend
• Chromium (Puppeteer will fetch/manage as needed)
• LLM credentials compatible with Midscene / LangChain OpenAI-style env vars (see below)
• Android (optional): AVD, ANDROID_AVD_NAME, ADB, Frida-related assets — see assets/README.md

Quick start

1. Backend (API + proxy + blackbox)

bun install
# OpenAI-compatible API for VLMs + chat models
export OPENAI_API_KEY=...
export OPENAI_BASE_URL=...   # optional, for OpenAI-compatible gateways
export MIDSCENE_MODEL_NAME=...  # optional; Midscene reads this in device setup
bun run tsx src/index.ts serve

Default ports in src/index.ts: Mockttp proxy 3001, Express + WebSocket 3002. The sample serve configuration enables web analysis only (androidEnabled: false); enable Android in code when your environment is ready.

2. Frontend

cd app
bun install
export VITE_API_URL=http://localhost:3002
bun dev

Open the Vite URL (usually http://localhost:5173), submit a target URL or APK, and open the analysis view. Download RAML from the API browser when the run finishes.

3. CLI-style analyze (writes result.json)

bun run tsx src/index.ts analyze --target https://example.com --description "Smoke test"

Project Structure

Stack

• Bun
• TypeScript
• Express
• Vue 3
• Vite
• Midscene (@midscene/web, @midscene/android)
• Puppeteer
• LangChain / LangGraph
• Mockttp
• Zod

Architecture

Feature	Location
Web target (Chromium / Puppeteer + Midscene)	src/analysis/devices/web.ts
Android target (emulator + Midscene Android)	src/analysis/devices/android.ts
Orchestration, IDDFS-style explore, UI–request tags	src/analysis/blackbox.ts
HTTPS MITM proxy	src/proxy/index.ts
Noise rejection + domain classification	src/analysis/network/index.ts, src/analysis/network/domain-classifier.ts
Similar-route retrieval + RAML	src/analysis/network/schema-store.ts, src/analysis/network/raml.ts
REST API + WebSocket for the UI	src/web/routes.ts, src/web/websocket.ts
Analysis dashboard (network, API browser, UI graph)	app/src/views/AnalysisView.vue and app/src/components/analysis/

Evaluation

On a custom Cookie Clicker–style app with a known REST API (auth, game action, leaderboard, profile), documented experiments report most primary endpoints discovered (on the order of ~80%), strong parameter and field identification for the flows that were captured, and clear benefit from UI context. For example, labeling an opaque login request using the login screen and from retrieval of similar prior requests (e.g. extending a spec with error responses) is enhanced by context.

Known Limitations

• Long, tightly coupled workflows
• Heavy internal state
• Custom-rendered or game UIs with few discrete elements
• Advanced gestures (drag, multi-step modals)
• Analysis time and LLM cost

License

This repository implements ideas from graduate work. A thesis presentation and article about Lumen are also available. You are welcome to use and modify the code for your own purposes.