Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add GCP Vertex AI platform as provider #1364

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

uddhav
Copy link

@uddhav uddhav commented Feb 24, 2025

Adds support for Claude, Gemini tool using models from the GCP Vertex AI platform

Models added in this edition

NB

I am aware of the recent contribution on this topic that got reverted from main. Builds failed due to a dependency issue that hasn't been resolved yet. Google's Auth SDK crate also seems to be an in-between state for now. That's why providers/gcpauth.rs exists.

  --- stderr
  thread 'main' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.25.1/builder/cc_builder.rs:367:13:
  ### COMPILER BUG DETECTED ###
  Your compiler (cc) is not supported due to a memcmp related bug reported in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189. We strongly recommend against using this compiler. 
  EXECUTED: true
  ERROR: 
  OUTPUT: 

Testing

  • Apart from tests, I have used it with user and service accounts that have been granted access to Vertex AI platform within their GCP project.
  • I could only hand test the GCP metadata approach + the test since I don't have access to a GCP VM or compute provisioned to access Vertex AI for now.
  • Tested all above models except Claude 3.5 Sonnet v1 - I don't have access to it but seems to be API compatible with v2.
  • Continuing to use it at work while this PR is in progress since it's my only way to access to Claude at work for now.

@uddhav uddhav marked this pull request as ready for review February 24, 2025 16:11
@uddhav
Copy link
Author

uddhav commented Feb 25, 2025

Now that Claude 3.7 is out - I can wait for the Anthropic provider to get updated. I don't see that anyone is blocked on missing Vertex AI provider.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant