- Updated default postgresql read operation timeout from
15sto2m
- Fixed
nilpointer error s
- Added
policyAPIs
- Started supporting storing streaming responses in bytes as part of
event
- Fixed issue with event not being recorded if
shouldLogResponseis set totruefor streaming responses
- Updated update key API to support setting
costLimitInUsdOverTime,costLimitInUsdandrateLimitOverTimeto 0
- Added querying keys by
keyIds - Increased default postgres DB read timeout to
15sand write timeout to5s
- Added setting rotation feature to key
- Added cost tracking for OpenAI audio endpoints
- Added inference cost tracking for OpenAI finetune models
- Added
userIdas a new filter option for get events API endpoint - Added option to store request and response using keys
- Added support for updating key cost limit and rate limit
- Removed validation to updating revoked key field
- Added new model "gpt-4-turbo-preview" and "gpt-4-vision-preview" to the cost map
- Added support for calculating cost for the cheaper 3.5 turbo model
- Added validation to updating revoked key field
- Added CORS support in the proxy
- Fixed custom route tokens recording issue incurred by the new architecture
- Fixed OpenAI chat completion endpoint being slow
- Drastically improved performance through event driven architecture
- Fixed API calls that exceeds cost limit not being blocked bug
- Added support for new chat completion models
- Added new querying options for metrics and events API
- Extended default proxy request timeout to 10m
- Fixed streaming response stuck at context deadline exceeded error
- Added key authentication for admin endpoints
- Changed code to string in OpenAI error response
- Fixed inability to parse OpenAI chat completion result
- Fixed problem with keys containing duplicated API credentials
- Fixed key exceeding spend limit error code issue
- Fixed a bug with route failover not working as expected
- Fixed route caching issue
- Added default values to route step timeout and cache TTL
- Started supporting caching in routes
- Added the ability to make API calls with fallbacks
- Added the support for multiple sets of provider settings per key
- Added support for Azure OpenAI embeddings and chat completions endpoints
- Added support for minute spend limits
- Added support for OpenAI audio endpoints
- Added support for OpenAI image endpoints
- Added support for monthly spend limits
- Removed deperacated fields when updating keys
- Fixed issues with inconsistent rate limit cache expiration date
- Added path access control at the API key level
- Added model access control at the provider level
- Inability to update provider settings with only
settingfield
- Added support for Anthropic completion endpoint
- Fixed streaming latency with custom provider and openai chat completion endpoints
- Fixed bug with updating a custom provider
- Added new API endpoints for custom providers
- Added validation for changing setting when updating provider setting
- Removed support for changing provider when updating provider setting
- Fixed issues with not being able to update provider setting name
- Added new filters tags and provider to getting keys API
- Added a new API endpoint for fetching an event
- Added a new API endpoint for fetching all provider settings
- Added more integration tests that cover critical paths
- Fixed issues with API key creation
- Introduced new field called
nameto provider settings
- Fixed issues with PostgreSQL schema inconsistencies
- Added support for OpenAI's embeddings API
- Fixed configuration not found inconsistency with key and provider settings
- Updated admin server to pull configurations every 5 sec by default
- Updated prod Dockerfile to have privacy mode turned on
- Added a default OpenAI proxy timeout of 180s
- Added model and keyId filters to the metrics API
- Added health check endpoints for both admin and proxy servers
- Started recording path and method for each proxy request
- Fixed Datadog tags being in the wrong format
- Added Datadog integration
- Fixed bug with proxy not fetching existing provider settings
- Added support for gpt-4-turbo models in cost estimation
- Fixed proxy logic to prevent internal errors from forwarding requests to OpenAI
- Fixed proxy logic to prevent internal errors from receiving responses from OpenAI
- Added a new API endpoint for creating and updating provider settings (e.g. OpenAI API key)
- Removed having OpenAI API key as an required env vairable for running the proxy
- Forced key creation to include a new field called settingId
- Added a new API endpoint for retrieving metrics
- Fixed open ai error response parsing
- Added new reporting endpoint for retrieving the cumulative cost of an API key
- Added support of cost estimation for fined tuned models
- Added logging for openAI error response
- Fixed incorrect logging format in production environment
- Added logging for admin and proxy web servers
- Fix PostgreSQL connection URI format issue
- Updated PostgreSQL connection URI to be more versatile
- Changed env var POSTGRESQL_SSL_ENABLED to POSTGRESQL_SSL_MODE
- Updated proxy error response to exclude empty fields
- Added a new environment varible for configurting PostgreSQL DB Name
- Fixed issue with rate limit
- Fixed issue with cost limit
- Added a web server for managing key configurations
- Added a proxy server for openai request
- Added rate limit, cost limit and ttl to API keys