Skip to main content

​
Providers

Providers are the bridge between BindAI and large language models. A provider implements the communication layer for a specific AI service, translating BindAI requests into the format expected by the underlying model and converting responses back into BindAI objects. Because every provider implements the same interface, switching models requires little or no application code changes.

​
What is a Provider?

A provider is responsible for:
  • sending prompts to an AI model
  • receiving model responses
  • streaming tokens
  • executing tool/function calling
  • reporting token usage
  • returning structured outputs
Applications interact with providers through the ModelProvider abstraction rather than provider-specific SDKs.

​
Supported Providers

BindAI is designed to support multiple providers. Current providers include:
  • OpenAI
  • Ollama
  • Anthropic (planned)
  • Google Gemini (planned)
  • Azure OpenAI (planned)
  • OpenRouter (planned)
  • Groq (planned)
  • Together AI (planned)
New providers can be added without changing the rest of the framework.

​
Creating a Provider

Most providers can be created directly.
The provider is then passed into an agent.

​
Provider Configuration

Providers are configured using a ProviderConfiguration. Typical configuration includes:
  • API key
  • model
  • base URL
  • timeout
  • organization
  • custom headers
Example:

​
Environment Variables

Most providers support loading configuration from environment variables. Example:
Then simply create the provider.
This keeps secrets out of source code.

​
Supported Features

Depending on the provider, BindAI supports: Some providers may support additional capabilities in future releases.

​
Streaming

Providers support token streaming.
Streaming allows applications to display responses immediately instead of waiting for completion.

​
Tool Calling

Providers can return tool requests generated by the language model. Example flow:
BindAI automatically converts provider-specific tool calls into portable ToolCall objects.

​
Structured Output

Providers can return structured data instead of plain text.
If supported by the provider, the response is validated and returned as a Python object.

​
Token Usage

Providers report token consumption whenever available. Example:
This information is useful for:
  • monitoring costs
  • analytics
  • optimization

​
Switching Providers

One of BindAI’s goals is provider independence. Changing providers usually requires only replacing the provider object. OpenAI:
Ollama:
The rest of the application remains unchanged.

​
Custom Providers

Any custom provider can integrate with BindAI by implementing the ModelProvider interface. A provider should implement methods such as:
  • generate()
  • stream()
  • capabilities
This allows organizations to connect proprietary models or internal AI services.

​
Capabilities

Every provider exposes its supported capabilities. Example:
Capabilities may indicate support for features such as:
  • streaming
  • structured output
  • tool calling
  • vision
  • embeddings
Applications can inspect these capabilities before using provider-specific features.

​
Provider Architecture

The provider layer sits between the execution engine and the AI service.
This separation allows the execution engine to remain provider-agnostic.

​
Best Practices

  • Store API keys in environment variables.
  • Reuse provider instances when possible.
  • Choose models appropriate for the task.
  • Enable streaming for long responses.
  • Monitor token usage in production.
  • Prefer the provider abstraction instead of calling SDKs directly.