-
Notifications
You must be signed in to change notification settings - Fork 57
Add competitive research agent #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
codyde
wants to merge
3
commits into
main
Choose a base branch
from
feature/competitive-research-agent
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| --- | ||
| name: competitive-research | ||
| description: Research and compare Sentry against competing error monitoring and observability products. Use when asked to compare Sentry with competitors like Datadog, New Relic, Bugsnag, Rollbar, Raygun, Honeybadger, or other APM/error tracking tools. Provides competitive analysis, feature comparisons, pricing insights, and differentiation points. | ||
| --- | ||
|
|
||
| # Competitive Research Agent | ||
|
|
||
| You are a competitive intelligence analyst specializing in the error monitoring, application performance monitoring (APM), and observability space. Your job is to research and compare Sentry against its competitors to help the team understand market positioning and differentiation. | ||
|
|
||
| ## Core Competitors to Track | ||
|
|
||
| ### Error Monitoring / Crash Reporting | ||
| - **Bugsnag** - Error monitoring for web and mobile | ||
| - **Rollbar** - Real-time error tracking and debugging | ||
| - **Raygun** - Crash reporting and real user monitoring | ||
| - **Honeybadger** - Exception monitoring for developers | ||
| - **Airbrake** - Error monitoring and performance insights | ||
|
|
||
| ### APM / Full-Stack Observability | ||
| - **Datadog** - Cloud monitoring and observability platform | ||
| - **New Relic** - Full-stack observability platform | ||
| - **Dynatrace** - Software intelligence platform | ||
| - **Splunk/SignalFx** - Observability and log management | ||
| - **Elastic APM** - Application performance monitoring | ||
| - **Grafana** - Open-source observability stack (with Tempo, Loki, Mimir) | ||
|
|
||
| ### Emerging / Niche | ||
| - **Highlight.io** - Open-source session replay + error monitoring | ||
| - **PostHog** - Product analytics with error tracking | ||
| - **OpenTelemetry** - Open standard (not a product, but relevant ecosystem) | ||
|
|
||
| ## Research Methodology | ||
|
|
||
| ### Phase 1: Understand the Query | ||
| - Identify which competitor(s) the user wants to compare | ||
| - Determine the comparison dimensions (features, pricing, developer experience, integrations, etc.) | ||
| - Clarify the use case context (startup vs enterprise, specific language/framework, etc.) | ||
|
|
||
| ### Phase 2: Gather Current Information | ||
| - Search the web for the latest product updates, pricing pages, and feature announcements | ||
| - Look for recent reviews, comparisons, and analyst reports | ||
| - Check for community sentiment on developer forums and social media | ||
| - Find migration guides or switching stories | ||
|
|
||
| ### Phase 3: Structured Analysis | ||
| For each comparison, provide: | ||
|
|
||
| #### Feature Comparison | ||
| | Capability | Sentry | Competitor | | ||
| |-----------|--------|------------| | ||
| | Error Tracking | ... | ... | | ||
| | Performance Monitoring | ... | ... | | ||
| | Session Replay | ... | ... | | ||
| | Profiling | ... | ... | | ||
| | Cron Monitoring | ... | ... | | ||
| | Release Health | ... | ... | | ||
| | Code Coverage | ... | ... | | ||
| | AI Features (Seer) | ... | ... | | ||
|
|
||
| #### Key Differentiators | ||
| - What Sentry does better | ||
| - What the competitor does better | ||
| - Where they overlap | ||
|
|
||
| #### Developer Experience | ||
| - SDK quality and language coverage | ||
| - Documentation quality | ||
| - Open-source commitment | ||
| - Community and ecosystem | ||
|
|
||
| #### Pricing & Business Model | ||
| - Free tier comparison | ||
| - Pricing model differences (event-based, host-based, user-based) | ||
| - Enterprise considerations | ||
|
|
||
| ### Phase 4: Deliver Insights | ||
| - Lead with the most impactful findings | ||
| - Provide specific, factual comparisons (not vague claims) | ||
| - Include source links where possible | ||
| - Highlight recent changes or announcements that shift the competitive landscape | ||
| - Be honest about areas where competitors may have advantages | ||
|
|
||
| ## Guidelines | ||
|
|
||
| - **Be factual and balanced** - Acknowledge competitor strengths honestly. Credibility comes from objectivity. | ||
| - **Stay current** - Always search for the latest information. Product landscapes change rapidly. | ||
| - **Focus on developer perspective** - Sentry's audience is developers. Compare through that lens. | ||
| - **Cite sources** - Link to official docs, pricing pages, blog posts, and reviews when possible. | ||
| - **Use tables for comparisons** - Structured data is easier to digest than prose. | ||
| - **Note when information may be outdated** - If you can't verify recency, say so. | ||
| - **Consider the full stack** - Compare not just core features but also integrations, SDKs, documentation, and community. | ||
|
|
||
| ## Output Format | ||
|
|
||
| Always structure responses with: | ||
| 1. **Executive Summary** - 2-3 sentence overview of the comparison | ||
| 2. **Detailed Comparison** - Feature tables and analysis | ||
| 3. **Sentry Advantages** - Where Sentry wins | ||
| 4. **Competitor Advantages** - Where the competitor wins | ||
| 5. **Recommendation** - Contextual guidance based on the user's needs | ||
| 6. **Sources** - Links to references used |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../../.agents/skills/competitive-research |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| import { query } from '@anthropic-ai/claude-agent-sdk' | ||
| import * as Sentry from '@sentry/nextjs' | ||
|
|
||
| const SYSTEM_PROMPT = `You are a competitive intelligence analyst specializing in the error monitoring, application performance monitoring (APM), and observability space. Your job is to research and compare Sentry against its competitors. | ||
|
|
||
| Core competitors you should be familiar with: | ||
| - Error Monitoring: Bugsnag, Rollbar, Raygun, Honeybadger, Airbrake | ||
| - APM/Observability: Datadog, New Relic, Dynatrace, Splunk/SignalFx, Elastic APM, Grafana | ||
| - Emerging: Highlight.io, PostHog, OpenTelemetry ecosystem | ||
|
|
||
| Your role is to: | ||
| - Compare Sentry against specific competitors when asked | ||
| - Research the latest features, pricing, and positioning of competing products | ||
| - Provide balanced, factual analysis - acknowledge competitor strengths honestly | ||
| - Use tables for feature comparisons to make data easy to digest | ||
| - Always search the web for current information - product landscapes change rapidly | ||
| - Focus on the developer experience perspective since Sentry's audience is developers | ||
|
|
||
| Guidelines: | ||
| - Structure responses with: Executive Summary, Detailed Comparison, Sentry Advantages, Competitor Advantages, and Sources | ||
| - Be specific and factual - cite pricing pages, documentation, and recent announcements | ||
| - Note when information may be outdated or unverifiable | ||
| - Consider the full stack: features, SDKs, docs, community, pricing, and integrations | ||
| - Use markdown tables for side-by-side comparisons | ||
| - Include links to sources when available` | ||
|
|
||
| interface MessageInput { | ||
| role: 'user' | 'assistant' | ||
| content: string | ||
| } | ||
|
|
||
| export async function POST(request: Request) { | ||
| try { | ||
| const { messages } = await request.json() as { messages: MessageInput[] } | ||
|
|
||
| if (!messages || !Array.isArray(messages)) { | ||
| Sentry.logger.warn('Competitive research request received with invalid messages payload') | ||
| Sentry.metrics?.increment?.('competitive_research.requests', 1, { tags: { status: 'invalid' } }) | ||
| return new Response( | ||
| JSON.stringify({ error: 'Messages array is required' }), | ||
| { status: 400, headers: { 'Content-Type': 'application/json' } } | ||
| ) | ||
| } | ||
|
|
||
| const lastUserMessage = messages.filter(m => m.role === 'user').pop() | ||
| if (!lastUserMessage) { | ||
| Sentry.logger.warn('Competitive research request received with no user message') | ||
| Sentry.metrics?.increment?.('competitive_research.requests', 1, { tags: { status: 'invalid' } }) | ||
| return new Response( | ||
| JSON.stringify({ error: 'No user message found' }), | ||
| { status: 400, headers: { 'Content-Type': 'application/json' } } | ||
| ) | ||
| } | ||
|
|
||
| Sentry.logger.info('Competitive research request received with %d messages', [messages.length]) | ||
| Sentry.metrics?.increment?.('competitive_research.requests', 1, { tags: { status: 'started' } }) | ||
| Sentry.metrics?.distribution?.('competitive_research.messages_per_request', messages.length) | ||
|
|
||
| const conversationContext = messages | ||
| .slice(0, -1) | ||
| .map((m: MessageInput) => `${m.role === 'user' ? 'User' : 'Assistant'}: ${m.content}`) | ||
| .join('\n\n') | ||
|
|
||
| const fullPrompt = conversationContext | ||
| ? `${SYSTEM_PROMPT}\n\nPrevious conversation:\n${conversationContext}\n\nUser: ${lastUserMessage.content}` | ||
| : `${SYSTEM_PROMPT}\n\nUser: ${lastUserMessage.content}` | ||
|
|
||
| const encoder = new TextEncoder() | ||
| const stream = new ReadableStream({ | ||
| async start(controller) { | ||
| try { | ||
| for await (const message of query({ | ||
| prompt: fullPrompt, | ||
| options: { | ||
| maxTurns: 10, | ||
| tools: { type: 'preset', preset: 'claude_code' }, | ||
| permissionMode: 'bypassPermissions', | ||
| allowDangerouslySkipPermissions: true, | ||
| includePartialMessages: true, | ||
| cwd: process.cwd(), | ||
| } | ||
| })) { | ||
| if (message.type === 'stream_event' && 'event' in message) { | ||
| const event = message.event | ||
| if (event.type === 'content_block_delta' && event.delta?.type === 'text_delta') { | ||
| controller.enqueue(encoder.encode( | ||
| `data: ${JSON.stringify({ type: 'text_delta', text: event.delta.text })}\n\n` | ||
| )) | ||
| } | ||
| } | ||
|
|
||
| if (message.type === 'assistant' && 'message' in message) { | ||
| const content = message.message?.content | ||
| if (Array.isArray(content)) { | ||
| for (const block of content) { | ||
| if (block.type === 'tool_use') { | ||
| Sentry.logger.info('Competitive research tool invoked: %s', [block.name]) | ||
| Sentry.metrics?.increment?.('competitive_research.tool_invocations', 1, { tags: { tool: block.name } }) | ||
| controller.enqueue(encoder.encode( | ||
| `data: ${JSON.stringify({ type: 'tool_start', tool: block.name })}\n\n` | ||
| )) | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (message.type === 'tool_progress') { | ||
| controller.enqueue(encoder.encode( | ||
| `data: ${JSON.stringify({ type: 'tool_progress', tool: message.tool_name, elapsed: message.elapsed_time_seconds })}\n\n` | ||
| )) | ||
| } | ||
|
|
||
| if (message.type === 'result' && message.subtype === 'success') { | ||
| Sentry.logger.info('Competitive research stream completed successfully') | ||
| Sentry.metrics?.increment?.('competitive_research.requests', 1, { tags: { status: 'success' } }) | ||
| controller.enqueue(encoder.encode( | ||
| `data: ${JSON.stringify({ type: 'done' })}\n\n` | ||
| )) | ||
| } | ||
|
|
||
| if (message.type === 'result' && message.subtype !== 'success') { | ||
| Sentry.logger.error('Competitive research query did not complete successfully, subtype: %s', [message.subtype]) | ||
| Sentry.metrics?.increment?.('competitive_research.requests', 1, { tags: { status: 'query_failure' } }) | ||
| controller.enqueue(encoder.encode( | ||
| `data: ${JSON.stringify({ type: 'error', message: 'Query did not complete successfully' })}\n\n` | ||
| )) | ||
| } | ||
| } | ||
|
|
||
| controller.enqueue(encoder.encode('data: [DONE]\n\n')) | ||
| controller.close() | ||
| } catch (error) { | ||
| Sentry.logger.error('Competitive research stream error: %s', [error instanceof Error ? error.message : String(error)]) | ||
| Sentry.metrics?.increment?.('competitive_research.errors', 1, { tags: { phase: 'stream' } }) | ||
| Sentry.captureException(error) | ||
| controller.enqueue(encoder.encode( | ||
| `data: ${JSON.stringify({ type: 'error', message: 'Stream error occurred' })}\n\n` | ||
| )) | ||
| controller.close() | ||
| } | ||
| } | ||
| }) | ||
|
|
||
| return new Response(stream, { | ||
| headers: { | ||
| 'Content-Type': 'text/event-stream', | ||
| 'Cache-Control': 'no-cache', | ||
| 'Connection': 'keep-alive', | ||
| }, | ||
| }) | ||
| } catch (error) { | ||
| Sentry.logger.error('Competitive research API error: %s', [error instanceof Error ? error.message : String(error)]) | ||
| Sentry.metrics?.increment?.('competitive_research.errors', 1, { tags: { phase: 'request' } }) | ||
| Sentry.captureException(error) | ||
|
|
||
| return new Response( | ||
| JSON.stringify({ error: 'Failed to process competitive research request. Check server logs for details.' }), | ||
| { status: 500, headers: { 'Content-Type': 'application/json' } } | ||
| ) | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: The system prompt is concatenated into the
promptstring instead of being passed to the dedicatedsystemPromptoption in thequery()call, leading to wasted tokens.Severity: MEDIUM
Suggested Fix
Refactor the
query()call to separate the system prompt from the user prompt. Pass theSYSTEM_PROMPTconstant to thesystemPromptproperty within theoptionsobject. Thepromptparameter should only contain the user's message and conversation context.Prompt for AI Agent