Skip to content

Commit 2c393db

Browse files
docs: Update documentation with new functionality
1 parent 9356a7a commit 2c393db

File tree

2 files changed

+56
-68
lines changed

2 files changed

+56
-68
lines changed

README.md

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,14 @@ This Terraform provider allows you to manage LiteLLM resources through Infrastru
1818

1919
## Using the Provider
2020

21+
To use the LiteLLM provider in your Terraform configuration, you need to declare it in the `terraform` block:
22+
2123
```hcl
2224
terraform {
2325
required_providers {
2426
litellm = {
25-
source = "local/litellm/litellm"
26-
version = "1.0.0"
27+
source = "litellm/litellm"
28+
version = "~> 1.0.0"
2729
}
2830
}
2931
}
@@ -32,34 +34,32 @@ provider "litellm" {
3234
api_base = var.litellm_api_base
3335
api_key = var.litellm_api_key
3436
}
37+
```
38+
39+
Then, you can use the provider to manage LiteLLM resources. Here's an example of creating a model configuration:
3540

36-
# Example model configuration for AWS Bedrock
37-
resource "litellm_model" "claude_aws_bedrock" {
38-
model_name = "claude-3.5-sonnet-v2"
39-
custom_llm_provider = "bedrock"
40-
base_model = "anthropic.claude-3-5-sonnet-20241022-v2:0"
41-
tier = "paid"
42-
mode = "completion" # Specify the model mode
43-
aws_access_key_id = var.aws_access_key_id
44-
aws_secret_access_key = var.aws_secret_access_key
45-
aws_region_name = var.aws_region
46-
47-
input_cost_per_million_tokens = 4.0
48-
output_cost_per_million_tokens = 16.0
41+
```hcl
42+
resource "litellm_model" "gpt4" {
43+
model_name = "gpt-4-proxy"
44+
custom_llm_provider = "openai"
45+
model_api_key = var.openai_api_key
46+
model_api_base = "https://api.openai.com/v1"
47+
base_model = "gpt-4"
48+
tier = "paid"
49+
mode = "completion"
50+
51+
input_cost_per_million_tokens = 30.0
52+
output_cost_per_million_tokens = 60.0
4953
}
5054
```
5155

52-
### Model Mode
53-
54-
The `mode` attribute in the `litellm_model` resource allows you to specify the intended use of the model. Available options are:
56+
For full details on the `litellm_model` resource, see the [model resource documentation](docs/resources/model.md).
5557

56-
- `completion`
57-
- `embeddings`
58-
- `image_generation`
59-
- `moderation`
60-
- `audio_transcription`
58+
### Available Resources
6159

62-
This attribute is optional. If not specified, it will not be included in the API request.
60+
- `litellm_model`: Manage model configurations. [Documentation](docs/resources/model.md)
61+
- `litellm_team`: Manage teams. [Documentation](docs/resources/team.md)
62+
- `litellm_team_member`: Manage team members. [Documentation](docs/resources/team_member.md)
6363

6464
## Development
6565

docs/resources/model.md

Lines changed: 32 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,20 @@ resource "litellm_model" "gpt4" {
1111
model_api_key = "sk-your-api-key"
1212
model_api_base = "https://api.openai.com/v1"
1313
api_version = "2023-05-15"
14-
base_model = "gpt-4"
15-
tier = "paid"
16-
tpm = 100000
17-
rpm = 1000
14+
base_model = "gpt-4"
15+
tier = "paid"
16+
mode = "completion"
17+
tpm = 100000
18+
rpm = 1000
1819
1920
# Cost configuration (per million tokens)
2021
input_cost_per_million_tokens = 30.0 # $0.03 per 1k tokens = $30 per million
2122
output_cost_per_million_tokens = 60.0 # $0.06 per 1k tokens = $60 per million
23+
24+
# AWS-specific configuration (if applicable)
25+
aws_access_key_id = "your-aws-access-key-id"
26+
aws_secret_access_key = "your-aws-secret-access-key"
27+
aws_region_name = "us-west-2"
2228
}
2329
```
2430

@@ -28,61 +34,40 @@ The following arguments are supported:
2834

2935
* `model_name` - (Required) The name of the model configuration. This will be used to identify the model in API calls.
3036

31-
* `custom_llm_provider` - (Required) The LLM provider for this model. Examples include:
32-
* `openai`
33-
* `anthropic`
34-
* `azure`
35-
* `cohere`
37+
* `custom_llm_provider` - (Required) The LLM provider for this model (e.g., "openai", "anthropic", "azure", "bedrock").
3638

37-
* `model_api_key` - (Required) The API key for the underlying model provider.
39+
* `model_api_key` - (Optional) The API key for the underlying model provider.
3840

39-
* `model_api_base` - (Required) The base URL for the model provider's API.
41+
* `model_api_base` - (Optional) The base URL for the model provider's API.
4042

4143
* `api_version` - (Optional) The API version to use for the model provider.
4244

4345
* `base_model` - (Required) The actual model identifier from the provider (e.g., "gpt-4", "claude-2").
4446

45-
* `tier` - (Optional) The usage tier for this model. Valid values are:
46-
* `free`
47-
* `paid`
48-
Default is `free`.
47+
* `tier` - (Optional) The usage tier for this model. Valid values are "free" or "paid". Default is "free".
48+
49+
* `mode` - (Optional) The intended use of the model. Valid values are:
50+
* `completion`
51+
* `embeddings`
52+
* `image_generation`
53+
* `moderation`
54+
* `audio_transcription`
4955

5056
* `tpm` - (Optional) Tokens per minute limit for this model.
5157

5258
* `rpm` - (Optional) Requests per minute limit for this model.
5359

54-
* `input_cost_per_million_tokens` - (Optional) Cost per million input tokens. This will be automatically converted to the per-token cost required by the API. For example:
55-
* Set to `30.0` for a cost of $0.03 per 1k tokens ($30 per million)
56-
* Set to `3.0` for a cost of $0.003 per 1k tokens ($3 per million)
57-
58-
* `output_cost_per_million_tokens` - (Optional) Cost per million output tokens. This will be automatically converted to the per-token cost required by the API. For example:
59-
* Set to `60.0` for a cost of $0.06 per 1k tokens ($60 per million)
60-
* Set to `6.0` for a cost of $0.006 per 1k tokens ($6 per million)
61-
62-
## Additional Configuration Options
60+
* `input_cost_per_million_tokens` - (Optional) Cost per million input tokens. This will be automatically converted to the per-token cost required by the API.
6361

64-
The following optional parameters are also supported:
62+
* `output_cost_per_million_tokens` - (Optional) Cost per million output tokens. This will be automatically converted to the per-token cost required by the API.
6563

66-
* `timeout` - (Optional) Request timeout in seconds.
67-
* `stream_timeout` - (Optional) Streaming request timeout in seconds.
68-
* `max_retries` - (Optional) Maximum number of retry attempts.
69-
* `organization` - (Optional) Organization identifier for the model provider.
70-
* `region_name` - (Optional) Region name for region-specific providers.
64+
### AWS-specific Configuration
7165

72-
### Provider-Specific Options
66+
* `aws_access_key_id` - (Optional) AWS access key ID for AWS-based models.
7367

74-
#### Google Vertex AI
75-
* `vertex_project` - Project ID for Google Vertex AI.
76-
* `vertex_location` - Location for Google Vertex AI resources.
77-
* `vertex_credentials` - Credentials for Google Vertex AI authentication.
68+
* `aws_secret_access_key` - (Optional) AWS secret access key for AWS-based models.
7869

79-
#### AWS
80-
* `aws_access_key_id` - AWS access key ID.
81-
* `aws_secret_access_key` - AWS secret access key.
82-
* `aws_region_name` - AWS region name.
83-
84-
#### IBM WatsonX
85-
* `watsonx_region_name` - Region name for WatsonX services.
70+
* `aws_region_name` - (Optional) AWS region name for AWS-based models.
8671

8772
## Attribute Reference
8873

@@ -92,7 +77,10 @@ In addition to the arguments above, the following attributes are exported:
9277

9378
## Import
9479

95-
Model configurations can be imported using the model name:
80+
Model configurations can be imported using the model ID:
9681

9782
```shell
98-
terraform import litellm_model.gpt4 gpt-4-proxy
83+
terraform import litellm_model.gpt4 <model-id>
84+
```
85+
86+
Note: The model ID is generated when the model is created and is different from the `model_name`.

0 commit comments

Comments
 (0)