From d21f2052c4501f80d551ec72c62aeb86034e5d81 Mon Sep 17 00:00:00 2001 From: Vishal Dharmadhikari <61256217+vishal-dharm@users.noreply.github.com> Date: Mon, 10 Jun 2024 09:38:26 -0700 Subject: [PATCH] Add example notebook for error handling (#196) * Add error handling example * Fix notebook formatting --- quickstarts/Error_handling.ipynb | 312 +++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) create mode 100644 quickstarts/Error_handling.ipynb diff --git a/quickstarts/Error_handling.ipynb b/quickstarts/Error_handling.ipynb new file mode 100644 index 000000000..03413db52 --- /dev/null +++ b/quickstarts/Error_handling.ipynb @@ -0,0 +1,312 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "8968a502d25e" + }, + "source": [ + "##### Copyright 2024 Google LLC." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "id": "a73f56372655" + }, + "outputs": [], + "source": [ + "# @title Licensed under the Apache License, Version 2.0 (the \\\"License\\\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \\\"AS IS\\\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "35219116d3b3" + }, + "source": [ + "# Gemini API: Error handling" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "571fb2e6d4ba" + }, + "source": [ + "\n", + " \n", + "
\n", + " Run in Google Colab\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "b58341384860" + }, + "source": [ + "This Colab notebook demonstrates strategies for handling common errors you might encounter when working with the Gemini API:\n", + "\n", + "* **Transient Errors:** Temporary failures due to network issues, server overload, etc.\n", + "* **Rate Limits:** Restrictions on the number of requests you can make within a certain timeframe.\n", + "* **Timeouts:** When an API call takes too long to complete.\n", + "\n", + "You have two main approaches to explore:\n", + "\n", + "1. **Automatic retries:** A simple way to retry requests when they fail due to transient errors.\n", + "2. **Manual backoff and retry:** A more customizable approach that provides finer control over retry behavior.\n", + "\n", + "\n", + "**Gemini Rate Limits**\n", + "\n", + "The default rate limits for different Gemini models are outlined in the [Gemini API model documentation](https://ai.google.dev/gemini-api/docs/models/gemini#model-variations). If your application requires a higher quota, consider [requesting a rate limit increase](https://ai.google.dev/gemini-api/docs/quota)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "b2bb83e651f4" + }, + "outputs": [], + "source": [ + "!pip install -U google-generativeai" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "34bf10124280" + }, + "outputs": [], + "source": [ + "import google.generativeai as genai" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cf16b627705c" + }, + "source": [ + "### Setup your API key\n", + "\n", + "To run the following cells, store your API key in a Colab Secret named `GOOGLE_API_KEY`. If you don't have an API key or need help creating a Colab Secret, see the [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) guide." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "34e7b42a93e3" + }, + "outputs": [], + "source": [ + "from google.colab import userdata\n", + "\n", + "GOOGLE_API_KEY = userdata.get(\"GOOGLE_API_KEY\")\n", + "genai.configure(api_key=GOOGLE_API_KEY)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "89d975abf7a2" + }, + "source": [ + "### Automatic retries\n", + "\n", + "The Gemini API's client library offers built-in retry mechanisms for handling transient errors. You can enable this feature by using the `request_options` argument with API calls like `generate_content`, `generate_answer`, `embed_content`, and `generate_content_async`.\n", + "\n", + "**Advantages:**\n", + "\n", + "* **Simplicity:** Requires minimal code changes for significant reliability gains.\n", + "* **Robust:** Effectively addresses most transient errors without additional logic.\n", + "\n", + "**Customize retry behavior:**\n", + "\n", + "Use these settings in [`retry`](https://googleapis.dev/python/google-api-core/latest/retry.html) to customize retry behavior:\n", + "\n", + "* `predicate`: (callable) Determines if an exception is retryable. Default: [`if_transient_error`](https://github.com/googleapis/python-api-core/blob/main/google/api_core/retry/retry_base.py#L75C4-L75C13)\n", + "* `initial`: (float) Initial delay in seconds before the first retry. Default: `1.0`\n", + "* `maximum`: (float) Maximum delay in seconds between retries. Default: `60.0`\n", + "* `multiplier`: (float) Factor by which the delay increases after each retry. Default: `2.0`\n", + "* `timeout`: (float) Total retry duration in seconds. Default: `120.0`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "09d14986b9cf" + }, + "outputs": [], + "source": [ + "from google.api_core import retry\n", + "\n", + "model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n", + "prompt = \"Write a story about a magic backpack.\"\n", + "\n", + "model.generate_content(\n", + " prompt, request_options={\"retry\": retry.Retry(predicate=retry.if_transient_error)}\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1abafce2315c" + }, + "source": [ + "### Manually increase timeout when responses take time\n", + "\n", + "If you encounter `ReadTimeout` or `DeadlineExceeded` errors, meaning an API call exceeds the default timeout (600 seconds), you can manually adjust it by defining `timeout` in the `request_options` argument." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "5bdc7450fcd8" + }, + "outputs": [], + "source": [ + "model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n", + "prompt = \"Write a story about a magic backpack.\"\n", + "\n", + "model.generate_content(\n", + " prompt, request_options={\"timeout\": 900}\n", + ") # Increase timeout to 15 minutes" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "c27ccc71fa27" + }, + "source": [ + "**Caution:** While increasing timeouts can be helpful, be mindful of setting them too high, as this can delay error detection and potentially waste resources." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "69dd9fd7f359" + }, + "source": [ + "### Manually implement backoff and retry with error handling\n", + "\n", + "For finer control over retry behavior and error handling, you can use the [`retry`](https://googleapis.dev/python/google-api-core/latest/retry.html) library (or similar libraries like [`backoff`](https://pypi.org/project/backoff/) and [`tenacity`](https://tenacity.readthedocs.io/en/latest/)). This gives you precise control over retry strategies and allows you to handle specific types of errors differently." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "c134fc5aaecc" + }, + "outputs": [], + "source": [ + "from google.api_core import retry, exceptions\n", + "\n", + "model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n", + "\n", + "\n", + "@retry.Retry(\n", + " predicate=retry.if_transient_error,\n", + " initial=2.0,\n", + " maximum=64.0,\n", + " multiplier=2.0,\n", + " timeout=600,\n", + ")\n", + "def generate_with_retry(model, prompt):\n", + " response = model.generate_content(prompt)\n", + " return response\n", + "\n", + "\n", + "prompt = \"Write a one-liner advertisement for magic backpack.\"\n", + "\n", + "generate_with_retry(model=model, prompt=prompt)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "85fc3d0bae17" + }, + "source": [ + "### Test the error handling with retry mechanism\n", + "\n", + "To validate that your error handling and retry mechanism work as intended, define a `generate_content` function that deliberately raises a `ServiceUnavailable` error on the first call. This setup will help you ensure that the retry decorator successfully handles the transient error and retries the operation." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "981415e25158" + }, + "outputs": [], + "source": [ + "from google.api_core import retry, exceptions\n", + "\n", + "\n", + "@retry.Retry(\n", + " predicate=retry.if_transient_error,\n", + " initial=2.0,\n", + " maximum=64.0,\n", + " multiplier=2.0,\n", + " timeout=600,\n", + ")\n", + "def generate_content_first_fail(model, prompt):\n", + " if not hasattr(generate_content_first_fail, \"call_counter\"):\n", + " generate_content_first_fail.call_counter = 0\n", + "\n", + " generate_content_first_fail.call_counter += 1\n", + "\n", + " try:\n", + " if generate_content_first_fail.call_counter == 1:\n", + " raise exceptions.ServiceUnavailable(\"Service Unavailable\")\n", + "\n", + " response = model.generate_content(prompt)\n", + " return response.text\n", + " except exceptions.ServiceUnavailable as e:\n", + " print(f\"Error: {e}\")\n", + " raise\n", + "\n", + "\n", + "model = genai.GenerativeModel(\"gemini-1.5-flash-latest\")\n", + "prompt = \"Write a one-liner advertisement for magic backpack.\"\n", + "\n", + "generate_content_first_fail(model=model, prompt=prompt)" + ] + } + ], + "metadata": { + "colab": { + "name": "Error_handling.ipynb", + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}