feat(web): sync staging metrics/report flow and bump convoai-api to 2.2.0 by shczhen · Pull Request #97 · AgoraIO-Community/Conversational-AI-Demo

shczhen · 2026-04-16T06:46:19Z

Summary

This PR syncs the VoiceAgent web demo with the web-next staging metrics/report flow while preserving existing local customizations.

It adds:

realtime latency collection from turn.finished
presence-based fine-grained agent state handling
local report session persistence and metrics upload
report APIs and /reports/[agentId] page
live latency metrics in subtitle UI
conversational-ai-api version bump to 2.2.0

It also includes a backward-compatibility fix for agent startup:

keep sending the new app_feature contract
also preserve legacy advanced_features flags needed by older agent servers
surface backend start errors instead of failing with a generic Zod data: null parse error

Key Changes

Added metrics/report flow:
- src/lib/latency-metrics.ts
- src/store/report.ts
- src/type/report.ts
- src/app/api/agent/metrics/[agentId]/route.ts
- src/app/api/agent/metrics/report/route.ts
- src/app/reports/[agentId]/page.tsx
- src/app/reports/[agentId]/report-page.tsx
Updated homepage/report-related UI:
- live metrics toggle in agent card
- latency badges in subtitle panel
- report entry links in settings forms
Updated agent start contract handling:
- support app_feature
- filter request payload for staging contract
- keep backward-compatible legacy advanced feature fields
Updated conversational-ai-api:
- added turn.finished handling
- added presence parsing for listening/thinking/speaking
- added AGENT_TURN_FINISHED and fine-grained agent state events
- bumped exposed API version markers to 2.2.0
Added regression tests for:
- latency metric parsing/building
- presence state mapping
- agent schema updates
- start-agent API payload shaping

Test Plan

bun test
bun run build

Commits

7b16a653 feat(web): sync staging metrics report flow
ed216e4f chore(convoai-api): bump version to 2.2.0
98519a21 fix(web): keep start-agent payload backward compatible

Copilot

Pull request overview

This PR syncs the VoiceAgent web demo with the staging metrics/report workflow by adding realtime latency collection/reporting, a reports page + APIs, and updating the embedded conversational-ai-api contract to 2.2.0 (including turn.finished + presence-based fine-grained states).

Changes:

Added latency metrics parsing/aggregation + persisted report sessions with upload + report rendering page.
Extended ConversationalAI API to emit AGENT_TURN_FINISHED and fine-grained presence events (listening/thinking/speaking), and bumped version markers to 2.2.0.
Updated start-agent payload shaping to include app_feature while keeping legacy advanced_features compatibility, plus added regression tests.

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
Web/Scenes/VoiceAgent/src/type/report.ts	Adds report-related type aliases for metrics/report payloads.
Web/Scenes/VoiceAgent/src/store/report.ts	New zustand store for report session persistence, turns, transcript cache, upload state.
Web/Scenes/VoiceAgent/src/store/index.ts	Re-exports `useReportStore`.
Web/Scenes/VoiceAgent/src/store/global.ts	Adds live-metrics UI toggle + dev overrides (custom app id).
Web/Scenes/VoiceAgent/src/store/agent.ts	Adds default `app_feature` settings.
Web/Scenes/VoiceAgent/src/services/agent.ts	Shapes start payload to include `app_feature`; adds metrics report APIs + metrics fetch.
Web/Scenes/VoiceAgent/src/lib/latency-metrics.ts	New helpers to parse `turn.finished`, build upload payload, build report row summaries, cache transcript by turn.
Web/Scenes/VoiceAgent/src/lib/latency-metrics.test.ts	Tests for parsing/building latency payloads and transcript grouping.
Web/Scenes/VoiceAgent/src/conversational-ai-api/utils/sub-render.ts	Bumps embedded API version marker to `2.2.0`.
Web/Scenes/VoiceAgent/src/conversational-ai-api/type.ts	Adds `turn.finished` message type + new emitted events and handler typings.
Web/Scenes/VoiceAgent/src/conversational-ai-api/presence.ts	New presence-state mapper producing coarse + fine-grained state changes.
Web/Scenes/VoiceAgent/src/conversational-ai-api/presence.test.ts	Tests for presence mapping behavior.
Web/Scenes/VoiceAgent/src/conversational-ai-api/index.ts	Emits new events from RTM messages/presence; parses `turn.finished`.
Web/Scenes/VoiceAgent/src/conversational-ai-api/README.md	Updates documented version to `2.2.0`.
Web/Scenes/VoiceAgent/src/constants/api/schema/agent.ts	Adds `app_feature` to start-agent request schema.
Web/Scenes/VoiceAgent/src/constants/api/schema/agent.test.ts	Regression test ensuring `app_feature` parses.
Web/Scenes/VoiceAgent/src/constants/api/index.ts	Adds local/remote endpoints for metrics report + metrics fetch.
Web/Scenes/VoiceAgent/src/constants/agent/schema.ts	Adds `app_feature` + pause-state preset support; tweaks form/start schema.
Web/Scenes/VoiceAgent/src/constants/agent/schema.test.ts	Regression tests for updated agent schema fields.
Web/Scenes/VoiceAgent/src/components/home/subtitle.tsx	Displays per-turn latency badges in subtitle UI (gated by live-metrics toggle).
Web/Scenes/VoiceAgent/src/components/home/live-metrics-toggle.tsx	Adds live metrics toggle UI component.
Web/Scenes/VoiceAgent/src/components/home/agent-setting/form-demo.tsx	Adds link to latest report session from settings form.
Web/Scenes/VoiceAgent/src/components/home/agent-setting/form-custom-private.tsx	Adds link to latest report session from settings form (custom private).
Web/Scenes/VoiceAgent/src/components/home/agent-control.tsx	Starts/updates report sessions, listens for `turn.finished`, uploads metrics report on teardown.
Web/Scenes/VoiceAgent/src/components/home/agent-card.tsx	Shows the live-metrics toggle in the agent card during calls.
Web/Scenes/VoiceAgent/src/app/reports/[agentId]/report-page.tsx	New client report page that fetches metrics and renders the report table.
Web/Scenes/VoiceAgent/src/app/reports/[agentId]/page.tsx	New route entry wiring `[agentId]` to the report page.
Web/Scenes/VoiceAgent/src/app/api/agent/route.ts	Shapes start-agent proxy payload for staging contract while preserving legacy flags.
Web/Scenes/VoiceAgent/src/app/api/agent/route.test.ts	Regression test for payload shaping + compatibility flags.
Web/Scenes/VoiceAgent/src/app/api/agent/metrics/report/route.ts	New API proxy route to upload metrics reports.
Web/Scenes/VoiceAgent/src/app/api/agent/metrics/[agentId]/route.ts	New API proxy route to fetch metrics by agent id.
Web/Scenes/VoiceAgent/messages/zh-CN.json	Adds report-related translations (toggle + settings link strings).
Web/Scenes/VoiceAgent/messages/en-US.json	Adds report-related translations (toggle + settings link strings).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-16T06:51:50Z

        'Content-Type': 'application/json',
        Authorization: `Bearer ${Cookies.get('token')}`
      },
-      body: JSON.stringify(data)
+      body: JSON.stringify(nextData)
    },
    {
      abortController


In startAgentDev, response parsing still goes straight into Zod parsing (later in the function) without the !respData?.data guard you added to startAgent. This means { data: null } / backend start failures can still surface as a generic Zod parse error in dev mode. Add the same early error check (and ideally handle non-zero code) after resp?.json() so error messages are consistent.

Copilot · 2026-04-16T06:51:50Z

+  agentId: string
+): Promise<IAgentMetrics> => {
+  const resp = await fetchWithTimeout(API_AGENT_METRICS(agentId), {
+    method: 'GET'


getAgentMetrics doesn’t send the bearer token (Authorization header). Other agent service calls include Authorization: Bearer ${Cookies.get('token')}, and without it this request will likely fail against protected APIs. Include the same auth header (and consider devMode query parity with reportAgentMetrics).

Suggested change

agentId: string

): Promise<IAgentMetrics> => {

const resp = await fetchWithTimeout(API_AGENT_METRICS(agentId), {

method: 'GET'

agentId: string,

options?: TDevModeQuery

): Promise<IAgentMetrics> => {

const query = generateDevModeQuery(options ?? {})

const url = `${API_AGENT_METRICS(agentId)}${query}`

const resp = await fetchWithTimeout(url, {

method: 'GET',

headers: {

Authorization: `Bearer ${Cookies.get('token')}`

}

Copilot · 2026-04-16T06:51:50Z

+  const { agentId } = await context.params
+
+  const url = `${agentServer}${REMOTE_CONVOAI_AGENT_METRICS(agentId)}`
+  const res = await fetch(url, {
+    method: 'GET',


This proxy route doesn’t enforce or forward Authorization to the upstream agent server. Most other /api/agent/* routes return 401 when the header is missing and pass it through; without that, metrics fetches may fail upstream and this also weakens access control consistency. Add the same authorizationHeader check and include it in the fetch headers.

Suggested change

const { agentId } = await context.params

const url = `${agentServer}${REMOTE_CONVOAI_AGENT_METRICS(agentId)}`

const res = await fetch(url, {

method: 'GET',

const { agentId } = await context.params

const authorizationHeader = request.headers.get('authorization')

if (!authorizationHeader) {

return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })

}

const url = `${agentServer}${REMOTE_CONVOAI_AGENT_METRICS(agentId)}`

const res = await fetch(url, {

method: 'GET',

headers: {

authorization: authorizationHeader

},

Copilot · 2026-04-16T06:51:51Z

+export function buildReportRows(input: {
+  metrics: AgentMetricsResponse
+  transcriptByTurnId?: TranscriptByTurnId
+}): ReportRowSet {
+  const rows = [...(input.metrics.turn_event || [])]
+    .sort((a, b) => a.turn_id - b.turn_id)
+    .map((turnEvent) => buildRow(turnEvent))
+


buildReportRows accepts transcriptByTurnId but never uses it, so the local transcript cache can’t actually influence the rendered report rows. Either remove the parameter or (preferred) merge it into userText/agentText (e.g., fallback to cached text when API transcription is missing, or prefer cached text).

Copilot · 2026-04-16T06:51:51Z

+        turn_event: (session?.turns || []).map((turn) => ({
+          turn_id: turn.turnId,
+          metrics: {
+            e2e_latency_ms: turn.e2eLatencyMs,
+            segmented_latency_ms: [
+              {
+                name: 'algorithm_processing',
+                latency: turn.segmentedLatency.algorithmProcessingMs
+              },
+              { name: 'asr_ttlw', latency: turn.segmentedLatency.asrTtlwMs },
+              { name: 'llm_ttft', latency: turn.segmentedLatency.llmTtftMs },
+              { name: 'tts_ttfb', latency: turn.segmentedLatency.ttsTtfbMs },
+              {


When building the fallback metrics object from the local session, turn_event entries omit transcription. As a result, buildReportRows will render userText/agentText as '—' even though session.transcriptByTurnId exists. Populate transcription from session.transcriptByTurnId (or pass transcriptByTurnId into buildReportRows once it’s used).

Copilot · 2026-04-16T06:51:51Z

+      toast.success('链接已复制')
+    } catch {
+      toast.error('复制失败，请手动复制当前地址')
+    }
+  }
+
+  return (
+    <main className='min-h-dvh bg-[#121212] px-4 py-6 text-white md:px-8'>
+      <div className='mx-auto flex max-w-7xl flex-col gap-6'>
+        <header className='flex flex-col gap-3 rounded-2xl border border-white/8 bg-white/4 p-5 md:flex-row md:items-center md:justify-between'>
+          <div className='flex flex-col gap-2'>
+            <div className='text-sm text-white/65'>声网对话式 AI 引擎</div>
+            <h1 className='font-semibold text-2xl'>数据报告</h1>
+          </div>
+          <Button
+            className='w-full md:w-auto'
+            variant='secondary'
+            onClick={handleCopy}
+          >
+            <CopyIcon className='size-4' />
+            复制分享链接
+          </Button>
+        </header>


This page hardcodes Chinese UI strings (header text, button text, loading/error copy, toast messages). The app is wired for next-intl (via NextIntlClientProvider in app/layout.tsx), so these should come from translations to keep en-US working and avoid mixed-locale UI.

Copilot · 2026-04-16T06:51:51Z

+function formatTimestamp(timestamp: number) {
+  return new Intl.DateTimeFormat('zh-CN', {
+    year: 'numeric',
+    month: '2-digit',
+    day: '2-digit',
+    hour: '2-digit',
+    minute: '2-digit',
+    second: '2-digit',
+    hour12: false
+  }).format(new Date(timestamp))


formatTimestamp always formats using the 'zh-CN' locale, which will be incorrect for non-Chinese locales and bypasses the app’s locale selection. Use the current locale (e.g., from useLocale() or by passing it in) when constructing Intl.DateTimeFormat.

shczhen added 3 commits April 16, 2026 12:24

feat(web): sync staging metrics report flow

7b16a65

chore(convoai-api): bump version to 2.2.0

ed216e4

fix(web): keep start-agent payload backward compatible

98519a2

Copilot AI review requested due to automatic review settings April 16, 2026 06:46

Copilot started reviewing on behalf of shczhen April 16, 2026 06:46 View session

Copilot AI reviewed Apr 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(web): sync staging metrics/report flow and bump convoai-api to 2.2.0#97

feat(web): sync staging metrics/report flow and bump convoai-api to 2.2.0#97
shczhen wants to merge 3 commits intomainfrom
codex/web-2.2.0

shczhen commented Apr 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Copilot AI Apr 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

shczhen commented Apr 16, 2026

Summary

Key Changes

Test Plan

Commits

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants