o3-mini API endpoint not supported

[tsuyoshi-otake-system-exe-jp]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Windows 11
- Continue version: 0.9.258
- IDE version: 1.96.4
- Model: o3-mini
- config.json:
  
  nothing

Description

Continue does not currently support o3-mini. It may take some time before support is added, so we will distribute a config.ts that will allow you to run o3-mini flawlessly.

config.ts for o3-mini

1. Execute the file by specifying its name: run code %USERPROFILE%\.continue\config.ts
2. Replace the contents of config.ts with the following content.
3. Enter your API key in the designated section.

export function modifyConfig(config: Config): Config {
  // Provide your OpenAI API key between the quotes.
  const apiKey = "";
  if (!apiKey) {
    console.error("OpenAI API key is not set.");
    return config;
  }

  // Create options for different load levels (high, medium, low)
  const optionsArray = [
    {
      title: "OpenAI: o3-mini (high)",
      contextLength: 200000,
      model: "o3-mini-2025-01-31",
      max_completion_tokens: 100000, // higher generation limit
      reasoning_effort: "high",        // set higher reasoning cost
      provider: "openai"
    },
    {
      title: "OpenAI: o3-mini (medium)",
      contextLength: 200000,
      model: "o3-mini-2025-01-31",
      max_completion_tokens: 100000, // higher generation limit
      reasoning_effort: "medium",      // set medium reasoning cost
      provider: "openai"
    },
    {
      title: "OpenAI: o3-mini (low)",
      contextLength: 200000,
      model: "o3-mini-2025-01-31",
      max_completion_tokens: 100000, // higher generation limit
      reasoning_effort: "low",         // set lower reasoning cost
      provider: "openai"
    }
  ];

  optionsArray.forEach((options) => {
    config.models.push({
      options: options,
      streamCompletion: async function* (
        prompt: string | string[],
        completionOptions: CompletionOptions
      ) {
        try {
          // If the prompt is not a string, join the array elements with a space to preserve all content.
          const unifiedPrompt = Array.isArray(prompt) ? prompt.join(" ") : prompt;

          // Remove internal tags (e.g., <|im_start|> and <|im_end|>).
          const cleanedPrompt = unifiedPrompt.replace(/<\|im_(start|end)\|>/g, "");

          // Send a request to the OpenAI ChatCompletion endpoint.
          const response = await fetch("https://api.openai.com/v1/chat/completions", {
            method: "POST",
            headers: {
              "Content-Type": "application/json",
              Authorization: `Bearer ${apiKey}`,
            },
            body: JSON.stringify({
              model: options.model,
              messages: [
                {
                  role: "system",
                  content: "Formatting re-enabled.",
                },
                {
                  role: "user",
                  content: cleanedPrompt,
                },
              ],
            }),
          });

          if (!response.ok) {
            const errorText = await response.text();
            throw new Error(
              `HTTP Error ${response.status}: ${response.statusText}\n${errorText}`
            );
          }

          // Parse the response as JSON.
          const data = await response.json();
          const content = data.choices?.[0]?.message.content;

          if (!content) {
            throw new Error("The response does not contain content.");
          }

          // Output the response only (exclude prompt information).
          if (Array.isArray(content)) {
            yield content.join("");
          } else {
            yield content;
          }
        } catch (error) {
          console.error(`An error occurred in ${options.title} streamCompletion:`, error);
          yield `An error occurred: ${
            error instanceof Error ? error.message : String(error)
          }`;
        }
      },
    });
  });

  return config;
}

To reproduce

No response

Log output

[okkymabruri]

I also see there's a issue when use o3-mini

400 Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

[tsuyoshi-otake-system-exe-jp]

#3914 Developers of continue, please add o3 support to the o1 handling in core/llm/llms/OpenAI.ts as soon as possible.😢

[unkn0wncode]

Seems that the model works now, but would be good to have reasoning effort added for o1/3 models too.

[okkymabruri]

yes, seems it works now
PR merged #3914

Seems that the model works now, but would be good to have reasoning effort added for o1/3 models too.

[tsuyoshi-otake-system-exe-jp]

I tried the latest version that was merge, but it only "works" and not fully operational.

[cheese-cracker]

Hi, was wondering if this is working now. Is there an example config which is working?

[IceKickr]

I'd love an example of a working config as well.

[summersonnn]

Yesterday, after many months I deposited $ to my openai account only to find out Continue does not support o3-mini.
We want an example config ASAP

[Jflick58]

@summersonnn you're being super demanding of an open source project. Anyway, just install the pre-release version and set o3-mini as your model type (following the openai examples in the docs). It works.

[tsuyoshi-otake-system-exe-jp]

By the way, regarding the issue where o3-mini can't handle long contexts when used with Continue, has the problem been resolved?