You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -28,61 +34,40 @@ The following arguments are supported:
28
34
29
35
*`model_name` - (Required) The name of the model configuration. This will be used to identify the model in API calls.
30
36
31
-
*`custom_llm_provider` - (Required) The LLM provider for this model. Examples include:
32
-
*`openai`
33
-
*`anthropic`
34
-
*`azure`
35
-
*`cohere`
37
+
*`custom_llm_provider` - (Required) The LLM provider for this model (e.g., "openai", "anthropic", "azure", "bedrock").
36
38
37
-
*`model_api_key` - (Required) The API key for the underlying model provider.
39
+
*`model_api_key` - (Optional) The API key for the underlying model provider.
38
40
39
-
*`model_api_base` - (Required) The base URL for the model provider's API.
41
+
*`model_api_base` - (Optional) The base URL for the model provider's API.
40
42
41
43
*`api_version` - (Optional) The API version to use for the model provider.
42
44
43
45
*`base_model` - (Required) The actual model identifier from the provider (e.g., "gpt-4", "claude-2").
44
46
45
-
*`tier` - (Optional) The usage tier for this model. Valid values are:
46
-
*`free`
47
-
*`paid`
48
-
Default is `free`.
47
+
*`tier` - (Optional) The usage tier for this model. Valid values are "free" or "paid". Default is "free".
48
+
49
+
*`mode` - (Optional) The intended use of the model. Valid values are:
50
+
*`completion`
51
+
*`embeddings`
52
+
*`image_generation`
53
+
*`moderation`
54
+
*`audio_transcription`
49
55
50
56
*`tpm` - (Optional) Tokens per minute limit for this model.
51
57
52
58
*`rpm` - (Optional) Requests per minute limit for this model.
53
59
54
-
*`input_cost_per_million_tokens` - (Optional) Cost per million input tokens. This will be automatically converted to the per-token cost required by the API. For example:
55
-
* Set to `30.0` for a cost of $0.03 per 1k tokens ($30 per million)
56
-
* Set to `3.0` for a cost of $0.003 per 1k tokens ($3 per million)
57
-
58
-
*`output_cost_per_million_tokens` - (Optional) Cost per million output tokens. This will be automatically converted to the per-token cost required by the API. For example:
59
-
* Set to `60.0` for a cost of $0.06 per 1k tokens ($60 per million)
60
-
* Set to `6.0` for a cost of $0.006 per 1k tokens ($6 per million)
61
-
62
-
## Additional Configuration Options
60
+
*`input_cost_per_million_tokens` - (Optional) Cost per million input tokens. This will be automatically converted to the per-token cost required by the API.
63
61
64
-
The following optional parameters are also supported:
62
+
*`output_cost_per_million_tokens` - (Optional) Cost per million output tokens. This will be automatically converted to the per-token cost required by the API.
65
63
66
-
*`timeout` - (Optional) Request timeout in seconds.
67
-
*`stream_timeout` - (Optional) Streaming request timeout in seconds.
68
-
*`max_retries` - (Optional) Maximum number of retry attempts.
69
-
*`organization` - (Optional) Organization identifier for the model provider.
70
-
*`region_name` - (Optional) Region name for region-specific providers.
64
+
### AWS-specific Configuration
71
65
72
-
### Provider-Specific Options
66
+
*`aws_access_key_id` - (Optional) AWS access key ID for AWS-based models.
73
67
74
-
#### Google Vertex AI
75
-
*`vertex_project` - Project ID for Google Vertex AI.
76
-
*`vertex_location` - Location for Google Vertex AI resources.
77
-
*`vertex_credentials` - Credentials for Google Vertex AI authentication.
68
+
*`aws_secret_access_key` - (Optional) AWS secret access key for AWS-based models.
78
69
79
-
#### AWS
80
-
*`aws_access_key_id` - AWS access key ID.
81
-
*`aws_secret_access_key` - AWS secret access key.
82
-
*`aws_region_name` - AWS region name.
83
-
84
-
#### IBM WatsonX
85
-
*`watsonx_region_name` - Region name for WatsonX services.
70
+
*`aws_region_name` - (Optional) AWS region name for AWS-based models.
86
71
87
72
## Attribute Reference
88
73
@@ -92,7 +77,10 @@ In addition to the arguments above, the following attributes are exported:
92
77
93
78
## Import
94
79
95
-
Model configurations can be imported using the model name:
80
+
Model configurations can be imported using the model ID:
96
81
97
82
```shell
98
-
terraform import litellm_model.gpt4 gpt-4-proxy
83
+
terraform import litellm_model.gpt4 <model-id>
84
+
```
85
+
86
+
Note: The model ID is generated when the model is created and is different from the `model_name`.
0 commit comments