[Bug] 接入openai兼容代理，获取模型列表报错

[CoolGIS]

📦 Environment

Vercel

📌 Version

1.44.1

💻 Operating System

Windows

🌐 Browser

Chrome

🐛 Bug Description

302AI是一个AI聚合商，类似于OpenRouter，提供openai兼容的代理：https://api.302.ai/v1

• 在更新1.44.0和1.44.1之前在openai中填写代理地址，可以成功获取模型列表，更新后获取模型列表失败，报错：value too long for type character varying(10)
• 使用1.44.0之后新增的自定义服务商，选择openai兼容，获取模型列表依然失败，报错：value too long for type character varying(10)
• 浏览器控制台报错接口为：https://example.com/trpc/lambda/aiModel.batchUpdateAiModels?batch=1
• 请求cURL位于附件中：
  cURL.txt

📷 Recurrence Steps

1. 打开设置中AI服务商
2. 选择OpenAI
3. 填入API key
4. 填入https://api.302.ai/v1
5. 点击连通性检查
6. 连通性检查成功后点击获取模型列表，报错value too long for type character varying(10)

🚦 Expected Behavior

成功获取模型列表

📝 Additional Information

No response

[lobehubbot]

👀 @CoolGIS

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot]

@CoolGIS 你好，我是Dosu，一个可以帮助你解决问题、回答疑问并成为贡献者的机器人。我在这里可以帮助你，等待人类维护者的到来。

The error message value too long for type character varying(10) suggests that there is a database field with a character limit of 10 that is being exceeded. This issue started occurring after the updates to versions 1.44.0 and 1.44.1.

In version 1.44.1, a fix was made for the "model select not auto update and sort issue," which might have affected how model lists are retrieved and stored ^[1]. Additionally, version 1.44.0 introduced a "Brand new AI provider" feature, which could have changed the way models are handled or stored, potentially leading to this error if the new data exceeds the character limit ^[1].

To resolve this issue, you might need to check the database schema to ensure that the field in question can accommodate the data being stored. Alternatively, you could look into the code handling the model list retrieval to see if there are any constraints or transformations applied to the data before it is stored in the database.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[dosubot]

The error message "value too long for type character varying(10)" suggests that there is an attempt to store a string longer than 10 characters in a database column that is defined to hold a maximum of 10 characters. This issue likely arises when the system tries to store data from the API response into the database.

Here are a few steps you can take to troubleshoot and potentially resolve this issue:

1. Check Database Schema: Verify the database schema to identify which column is defined with a character varying(10) type. This will help you understand what data is being stored and why it might be exceeding the limit.

2. Review API Response: Examine the API response from the OpenAI-compatible proxy to see if any fields are longer than expected. This can be done by checking the cURL request and response details.

3. Update Database Schema: If the data being stored is valid and necessary, consider updating the database schema to increase the length of the column to accommodate the data.

4. Data Validation: Implement data validation to ensure that any data exceeding the expected length is either truncated or handled appropriately before being stored in the database.

5. Check Code Changes: Since the issue started occurring after updating to versions 1.44.0 and 1.44.1, review the changes made in these versions, especially those related to model list retrieval and storage. Version 1.44.1 included a fix for the "model select not auto update and sort issue," which might be related to how model data is processed ^[1].

By following these steps, you should be able to identify the root cause of the error and implement a solution.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[CoolGIS]

• 通过查看源代码，猜测问题应该是新增的ai_models表中列released_at长度限制为varchar(10)，接口返回的Invalid Date字符数长度超出限制了。
• 建议可以将字符数加长一些，因为很多API聚合商提供的字段并不像OpenAI那么标准，后续接入别的openai兼容接口问题也会少一些。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

• By looking at the source code, it is guessed that the problem should be that the length of the column released_at in the newly added ai_models table is limited to varchar(10), and the length of the Invalid Date returned by the interface exceeds the limit.
• It is recommended to lengthen the number of characters, because the fields provided by many API aggregators are not as standard as OpenAI, and there will be fewer problems when connecting to other OpenAI compatible interfaces.

[Yuyuyang]

遇到了一样的问题

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Encountered the same problem

[arvinxx]

通过查看源代码，猜测问题应该是新增的ai_models表中列released_at长度限制为varchar(10)，接口返回的Invalid Date字符数长度超出限制了。
建议可以将字符数加长一些，因为很多API聚合商提供的字段并不像OpenAI那么标准，后续接入别的openai兼容接口问题也会少一些

的确是服务器提供的字段不标准导致的。故意设成 varchar(10) （即 2024-03-14 这种） 也是为了强卡点不规范的日期情况。

接下来会在目前的实现做一层兼容，支持标准的timestamp/date字符串/unix时间戳，应该能满足99%的场景了，如果这样兼容后还出错的话，应该让服务商自己去改造来支持标准规范。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

By looking at the source code, we guess that the problem should be that the length of the released_at column in the newly added ai_models table is limited to varchar(10), and the length of the Invalid Date returned by the interface exceeds the limit.
It is recommended that the number of characters be lengthened, because the fields provided by many API aggregators are not as standard as OpenAI, and subsequent access to other openai compatible interfaces will have fewer problems.

It is indeed caused by the non-standard fields provided by the server. It is deliberately set to varchar 10 to avoid irregular date situations.

Next, we will make a layer of compatibility in the current implementation to support standard timestamp/date string/unix timestamp, which should be able to meet 99% of scenarios. If errors still occur after such compatibility, the service provider should modify it by themselves. Support standard specifications.

[CoolGIS]

通过查看源代码，猜测问题应该是新增的ai_models表中列released_at长度限制为varchar(10)，接口返回的Invalid Date字符数长度超出限制了。
建议可以将字符数加长一些，因为很多API聚合商提供的字段并不像OpenAI那么标准，后续接入别的openai兼容接口问题也会少一些

的确是服务器提供的字段不标准导致的。故意设成varchar 10 也是为了强卡点不规范的日期情况。

接下来会在目前的实现做一层兼容，支持标准的timestamp/date字符串/unix时间戳，应该能满足99%的场景了，如果这样兼容后还出错的话，应该让服务商自己去改造来支持标准规范。

感谢回复，我了解兼容层这个概念，但是不知道有没有正确理解您说的兼容层的作用，如果接口返回的是空字符串，或者类似null，"Invalid Date"这种不规范date值，会被拒绝入库吗？还是在兼容层做一次处理，类似直接将不能正确解析为date的值替换为一个默认的时间戳再入库之类的？

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

By looking at the source code, we guess that the problem should be that the length of the released_at column in the newly added ai_models table is limited to varchar(10), and the length of the Invalid Date returned by the interface exceeds the limit.
It is recommended that the number of characters be lengthened, because the fields provided by many API aggregators are not as standard as OpenAI, and subsequent access to other openai compatible interfaces will have fewer problems.

It is indeed caused by the non-standard fields provided by the server. It is deliberately set to varchar 10 to avoid irregular date situations.

Next, we will make a layer of compatibility in the current implementation to support standard timestamp/date string/unix timestamp, which should be able to meet 99% of scenarios. If errors still occur after this compatibility, the service provider should be allowed to modify it by themselves. to support standard specifications.

Thanks for the reply. I understand the concept of the compatibility layer, but I don’t know if I correctly understand the role of the compatibility layer you mentioned. If the interface returns an empty string, or a non-standard date value such as null or "Invalid Date", it will Rejected for storage? Or should we do a process in the compatibility layer, such as directly replacing the value that cannot be correctly parsed as date with a default timestamp and then entering it into the database?

[arvinxx]

感谢回复，我了解兼容层这个概念，但是不知道有没有正确理解您说的兼容层的作用，如果接口返回的是空字符串，或者类似null，"Invalid Date"这种不规范date值，会被拒绝入库吗？还是在兼容层做一次处理，类似直接将不能正确解析为date的值替换为一个默认的时间戳再入库之类的？

如果是空串或者null这种，会作为无效值，直接当成未定义处理，即入库的时候就没有 releasedAt 字段，展示的时候也不会显示

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Thanks for the reply. I understand the concept of the compatibility layer, but I don’t know if I correctly understand the role of the compatibility layer you mentioned. If the interface returns an empty string, or an irregular date value such as null or "Invalid Date", Will it be rejected? Or should we do a process in the compatibility layer, such as directly replacing the value that cannot be correctly parsed as date with a default timestamp and then entering it into the database?

If it is an empty string or null, it will be treated as an invalid value and directly treated as undefined. That is, there will be no releasedAt field when it is stored in the library, and it will not be displayed when displayed.