Refactor: Generalize LLM Provider Architecture and Introduce Abstract Base Interface

[vinayakkushwah01]

PR Title

Refactor: Generalize LLM Provider Architecture and Introduce Abstract Base Interface

---

Overview

This PR removes the direct dependency on OpenAI and generalizes the LLM integration layer to support multiple providers through a unified architecture.

The refactor introduces a provider factory pattern, a shared abstract base class, standardized method naming, a normalized response schema, and environment-based configuration.

The goal is to make the system provider-agnostic, easier to extend, and more maintainable.

---

Key Changes

1. Generalized Provider Support

Implemented a factory-based approach to dynamically instantiate supported LLM providers based on configuration.

Supported providers:

• openai
• gemini
• anthropic
• groq
• aws-bedrock
• open-router

The PROVIDER environment variable must be set to one of the above values. Any other value will raise an unsupported provider error.

This removes hardcoded OpenAI logic and centralizes provider selection into a single abstraction layer.

---

2. Introduced Base Abstract LLM Interface

Added a BaseLLM abstract class using Python’s abc module.

All providers must now implement:

• generate(...)
• agenerate(...)

This ensures:

• Consistent interface across providers
• Enforced implementation contract
• Clean separation between provider logic and business logic
• Standard sync and async behavior

---

3. Standardized Method Naming

Renamed provider-specific method names to standardized LLM-neutral method names.

All generation logic now routes through:

• generate
• agenerate

This removes provider assumptions from higher-level modules and ensures consistent invocation patterns.

---

4. Defined Unified Response Schema

Introduced a normalized response schema to standardize outputs across all providers.

Benefits:

• Eliminates provider-specific response parsing in downstream layers
• Simplifies integration logic
• Enables consistent error and metadata handling

---

5. Environment-Based Configuration

Provider selection and model configuration are now driven entirely through environment variables.

Required variables:

• API_KEY
• PROVIDER (allowed values listed above)
• MODEL
• EMBEDDING_MODEL

Recommended embedding configuration:

• EMBEDDING_MODEL = "cl100k_base"

This embedding tokenizer provides reliable compatibility for most general-purpose LLM workflows.

---

Breaking Changes

• Removed direct OpenAI-specific implementation dependencies.
• Providers must now implement the BaseLLM interface.
• Legacy provider-specific method names are no longer supported.
• Environment variables are required for runtime configuration.
• PROVIDER must match one of the explicitly allowed values.

---

Benefits

• Provider-agnostic architecture
• Cleaner abstraction boundaries
• Simplified extensibility for future providers
• Reduced technical debt
• Improved testability and maintainability
• Consistent sync and async generation interface

---

Testing

Validated across all supported providers:

• Synchronous generation
• Asynchronous generation
• Response schema normalization
• Environment-driven provider switching

---

Summary

This refactor transitions the system from a single-provider implementation to a scalable multi-provider architecture. It enforces a clear abstraction layer, standardizes interfaces, and prepares the system for long-term extensibility.