How to use proxy-based proxy clients on google.generativeai

[Blues-star]

Description of the feature request:

In OpenAI's Python SDK, you can implement client-side proxying through the following operation
https://github.com/openai/openai-python#configuring-the-http-client

import httpx
from openai import OpenAI

client = OpenAI(
    # Or use the `OPENAI_BASE_URL` env var
    base_url="http://my.test.server.example.com:8083",
    http_client=httpx.Client(
        proxies="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)

I am very curious about how I should implement the above operation in google.generativeai sdk?

What problem are you trying to solve with this feature?

I want to avoid using the system-wide global proxy as much as possible to achieve the above functionality

Any other information you'd like to share?

No response

[HawkClaws]

It is for llamaindex, but I made it.
https://github.com/HawkClaws/proxy_gemini

[tianlichunhong]

没有找到prox的方法

[markmcd]

There are a couple of ways you can proxy with the SDK.

You can force the SDK to use HTTP traffic (it's gRPC by default) with genai.configure(..., transport="rest"), then the underlying requests will honour the typical environment variables - HTTP_PROXY=... or ALL_PROXY=... (note that they must be upper case).

If you have a fake endpoint or want to send the traffic to a different host directly, you can use genai.configure(client_options={'api_endpoint': 'path.to.my.domain'})

[daiaji]

If you have a fake endpoint or want to send the traffic to a different host directly, you can use genai.configure(client_options={'api_endpoint': 'path.to.my.domain'})

Are you saying that I can now use genai.configure to configure a reverse proxy for the Gemini API?

[markmcd]

Are you saying that I can now use genai.configure to configure a reverse proxy for the Gemini API?

It's not built specifically for reverse proxying, we use it for staging/preprod endpoints internally, but you do you :)

[Alexmalab]

There are a couple of ways you can proxy with the SDK.

You can force the SDK to use HTTP traffic (it's gRPC by default) with genai.configure(..., transport="rest"), then the underlying requests will honour the typical environment variables - HTTP_PROXY=... or ALL_PROXY=... (note that they must be upper case).

If you have a fake endpoint or want to send the traffic to a different host directly, you can use genai.configure(client_options={'api_endpoint': 'path.to.my.domain'})

This only works for generate_content() function, while the genai.upload_file function doesnt work because it has a builtin endpoint and . How do you solve this problem, as i have the same situation, i can't just use os.environ['HTTP_PROXY'] because the project is runing in a multiprocess and thread enviroment and i don't want it to influence other net work requests. So the reserve proxy only works for model.generate_content but doesn't work for genai.upload_file function. Is theer any one who have a resolution???

[AnoyiX]

see #610 (comment)

[FlyingStarlight]

Just give two OOTB sample of using proxy
给大家两个开箱即用的示例

import os
import google.generativeai as gai

os.environ["GRPC_PROXY"] = "http://127.0.0.1:10809" #10809 is the default proxy port of v2rayN :p, clash verge should be different 

gai.configure(api_key='YOUR GOOGLE API KEY')
flash = gai.GenerativeModel("gemini-1.5-flash")
resp = flash.generate_content("Explain Gemini to me like I am a kid")
Markdown(resp.text)

This is the most convenient one (default grpc). Another is to use additional proxy accordingly
上面的是最方便的(默认grpc)，下面用了上面所提到的额外参数

import os
import google.generativeai as gai

os.environ["ALL_PROXY"] = "http://127.0.0.1:10809" #I have tested 'HTTP_PROXY' but it doesnt work 我试了HTTP_PROXY但没效果 

gai.configure(api_key='YOUR GOOGLE API KEY')
flash = gai.GenerativeModel("gemini-1.5-flash", transport='rest')
resp = flash.generate_content("Explain Gemini to me like I am a kid")
Markdown(resp.text)