Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
update migrated content
Browse files Browse the repository at this point in the history
abvthecity committed May 9, 2024
1 parent a727ca8 commit 7fd61d0
Showing 73 changed files with 1,680 additions and 2,083 deletions.
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ title: "Create a container and endpoint using the CLI"
## Overview

The `octoai` command-line interface (CLI) makes it easy for you create a custom endpoint for OctoAI Compute Service. The `octoai` CLI guides you through the process of creating an initial valid Python application with an example model, building it, and deploying it.
The `octoai` command-line interface (CLI) makes it easy for you create a custom endpoint for OctoAI Compute Service. The `octoai` CLI guides you through the process of creating an initial valid Python application with an example model, building it, and deploying it.

The `octoai` CLI includes some endpoint scaffolds with example models that you can deploy and try out right away. After you complete that initial workflow, you can follow the instructions in this document to modify the initial application to use the model or code of your choice on OctoAI Compute Service.

@@ -290,14 +290,14 @@ Congratulations! You have now created your first endpoint on OctoAI Compute Serv
The client code in `test_request.py` looks like this:

```Python Python
from octoai.client import Client
from octoai.client import OctoAI

inputs = {"prompt": "Hello world!"}

def main(endpoint):
"""Run inference against the endpoint."""
# create an OctoAI client
client = Client()
client = OctoAI()

# perform inference
response = client.infer(endpoint_url=f"{endpoint}/infer", inputs=inputs)
@@ -319,7 +319,7 @@ For example, using the CLI the logs look similar to this:

```Text Text
$ octoai logs --name hello-world
19s hello-world-<hash> octoai server
19s hello-world-<hash> octoai server
18s hello-world-<hash> Using service in service.HelloService.
18s hello-world-<hash> run
18s hello-world-<hash> Setting up.
@@ -346,7 +346,7 @@ This section shows you how to modify the `hello-world` endpoint implementation t

#### Step 1: Add Python Requirements

The `hello-world` endpoint implementation contains an empty `requirements.txt` file.
The `hello-world` endpoint implementation contains an empty `requirements.txt` file.

Edit this file and add the corresponding requirements for Flan-T5:

@@ -391,12 +391,12 @@ In this case, our model of interest is of the same modality as the `hello-world`

#### Step 3: Modify Sample Client Code

The `hello-world` endpoint implementation has a sample `test_request.py` that makes a request to your endpoint. In this case, our model of interest is of the same modality as the `hello-world` example, so you can just change the prompt.
The `hello-world` endpoint implementation has a sample `test_request.py` that makes a request to your endpoint. In this case, our model of interest is of the same modality as the `hello-world` example, so you can just change the prompt.

```Python Python
import argparse

from octoai.client import Client
from octoai.client import OctoAI

inputs = {"prompt": "What country is California in?"}
...
@@ -476,14 +476,13 @@ The `octoai.types` package contains helpful classes if you are customizing your

```Python Python
from octoai.service import Service
from octoai.types import Image

class MyService(Service):

def infer(self, image: Image) -> str:
image_pil = image.to_pil()
output = self.model(image_pil)

return output[0]
```

@@ -502,7 +501,7 @@ class MyService(Service):
def infer(self, audio: Audio) -> str:
audio_array, sampling_rate = audio.to_numpy()
output = self.model(audio_array, sampling_rate)

return output[0]
```

@@ -521,7 +520,7 @@ class MyService(Service):
def infer(self, video: Video) -> str:
video_frames = video.to_numpy()
output = self.model(video_frames)

return output[0]
```

@@ -598,19 +597,19 @@ The OctoAI SDK enables you to define additional routes in your endpoint. The fol
from octoai.service import Service, path

class MultipleRoutesService(Service):

def setup(self):
print("Setting up MultipleRoutesService")

# the infer() endpoint is always required
def infer(self, prompt: str) -> str:
return prompt

# this method is exposed as /new-route
@path("/new-route")
def my_new_route(self, input_text: str):
return input_text

# this method is exposed as /my-new-route2
def my_new_route2(self, input_text: str):
return input_text
@@ -642,7 +641,7 @@ endpoint_config:
env_overrides: # Environment variables to set in each replica (optional)
key1: value1
key2: value2
registry:
registry:
host: docker.io # Registry hostname (required)
path: username/yolov8 # Registry path to image (required)
tag: v1 # Tag (optional; not recommended to be set. Defaults to a generated UUID.)
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
title: "Pricing & billing"
description: Only pay for what you use.
---
At OctoAI you only pay for what you use. Upon sign up you will receive **$10 of free credit** in your account. This credit can be used until the end of your first month after sign up. That is equivalent of:
At OctoAI you only pay for what you use. Upon sign up you will receive **$10 of free credit** in your account, and these credits don't expire. That is equivalent of:
- Over 500,000 words with the largest Llama 2 70B model, and over a million words with the new Mixtral 8x7B model
- 1,000 SDXL default images
- 2+ hours of compute on our large tier hardware
@@ -84,16 +84,16 @@ Here are a few examples to illustrate how this works to assist you in applying t

We offer simple, competitive token-based pricing for text gen endpoints, with prices varying depending on parameter size and quantization level:

| Model | Per M Tokens (May 1, 2024) | Input Price | Output Price |
| ---------------------- | ---------------------------- | ----------------------- | -------------------- |
| Mixtral-8x7B models | $0.45 | $0.30 / 1M tokens | $0.50 / 1M tokens |
| Mixtral-8x22B models | $1.20 | $1.20 / 1M tokens | $1.20 / 1M tokens |
| 7B and 8B models (Mistral, Code Llama, Llama 2, Llama Guard, Llama 3) | $0.15 | $0.10 / 1M tokens | $0.25 / 1M tokens |
| 13B models (Llama 2 & Code Llama) | $0.20 | $0.20 / 1M tokens | $0.50 / 1M tokens |
| 32B models (Qwen) | $0.75 | $0.50 / 1M tokens | $1.00 / 1M tokens |
| 34B models (Code Llama)| $0.75 | $0.50 / 1M tokens | $1.00 / 1M tokens |
| 70B models (Llama 2, Llama 3) | $0.90 | $0.60 / 1M tokens | $1.90 / 1M tokens |
| GTE-large | $0.05 | | $0.05 / 1M tokens |
| Model Sizes | Per M Tokens |
| ------------------------- | ---------------------------- |
| Mixtral-8x7B models | $0.45 |
| Mixtral-8x22B models | $1.20 |
| 7B and 8B models | $0.15 |
| 13B models | $0.20 |
| 32B models | $0.75 |
| 34B models | $0.75 |
| 70B models | $0.90 |
| GTE-large | $0.05 |

If you would like to explore pricing for other models, quantization levels, or specific fine tunes, [contact us](https://octo.ai/contact-us/).

Original file line number Diff line number Diff line change
@@ -60,6 +60,8 @@ one of the greatest painters of all time, and his inventive and innovative works
influence artists and thinkers to this day. Some of his most famous works include the Mona Lisa,
The Last Supper, and Vitruvian Man.
```

### Learn with our demo apps
Get started today by following along with one of our demo apps:
- [DocTalk](https://octo.ai/demos/doctalk/)
- [Q&A app on a custom PDF](https://octo.ai/demos/q-a-on-custom-pdf/)
Original file line number Diff line number Diff line change
@@ -8,23 +8,65 @@ LlamaIndex strives to help manage the interactions between your language modles
If you are building your application and using LlamaIndex you benefit from the vast ecosystem of integrations, and top LLMs amd Embeddings models hosted by OctoAI.


## Using OctoAI's LLMs and LangChain
## Using OctoAI's LLMs and LlamaIndex
Get started reviewing more about [LlamaIndex](https://docs.llamaindex.ai/en/stable/), and [signing up for a free OctoAI account](https://identity.octoml.ai/oauth/account/sign-up?redirectUrl=https%3A%2F%2Foctoai.cloud%2Foauth%2Fcallback).
If you want to utilize models offered by OctoAI through LlamaIndex review the following code snippet to see an example of using the OctoAI embeddings with the `OctoAIEmbedding` class:

LlamaIndex has both Python and TypScript libraries, and OctoAI is available in the Python SDK.

To use OctoAI LLM endpoints with LlamaIndex start with the code below using Llama 3 8B as the LLM.

```python
from os import environ
from llama_index.llms.octoai import OctoAI

OCTOAI_API_KEY = environ.get("OCTOAI_TOKEN")

octoai = OctoAI(model="meta-llama-3-8b-instruct", token=OCTOAI_API_KEY)

# Using complete
response = octoai.complete("Octopi can not play chess because...")
print(response)

print("\n=====================\n")

# Using the chat interface
from llama_index.core.llms import ChatMessage

messages = [
ChatMessage(
role="system",
content="Below is an instruction that describes a task. Write a response that appropriately completes the request.",
),
ChatMessage(role="user", content="Write a short blog about Seattle"),
]
response = octoai.chat(messages)
print(response)
```
To use OctoAI Embedding endpoints with llamaindex
you can use the code below to get started. We’re using GTE large in the example below (default model).

```python
from os import environ
from llama_index.embeddings.octoai import OctoAIEmbedding

OCTOAI_API_KEY = environ.get("OCTOAI_TOKEN")
embed_model = OctoAIEmbedding(api_key=OCTOAI_API_KEY)

# Single embedding request
embeddings = embed_model.get_text_embedding("Once upon a time in Seattle.")
assert len(embeddings) == 1024
```
See another example using OctoAI's Llama 2 13B model:
print(embeddings[:10])

```python
from llama_index.llms.octoai import OctoAI
octoai = OctoAI(model="llama-2-13b-chat", token=OCTOAI_API_KEY)
response = octoai.complete("Octopi can not play chess because...")

# Batch embedding request
texts = [
"Once upon a time in Seattle.",
"This is a test.",
"Hello, world!"
]
embeddings = embed_model.get_text_embedding_batch(texts)
assert len(embeddings) == 3
print(embeddings[0][:10])
```

If you are using LlamaIndex you can easily switch model provider, and enjoy using models hosted and optimized for scale on OctoAI.
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@ OpenRouter provides a unified interface for using various LLMs and allows users
This let's users find the right LLM and mix of price and performance for their use case.


## Using OctoAI's LLMs and LangChain
## Using OctoAI's LLMs and OpenRouter
To access OctoAI's best in class LLMs via OpenRouter [sign into OpenRouter](https://openrouter.ai/) and create an account to obtain an `OPENROUTER_API_KEY`.

Using the code snippet below you can route your calls to OpenRouter via OpenAI's client API.
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
---
title: "Integrations"
title: "All OctoAI Integrations"
sidebarTitle: "All"
description: "Browse OctoAI's partner integrations to help you build your custom solution."
---

<CardGroup cols={3}>
<Card title="LangChain" icon="square-1">
<CardGroup cols={2}>
<Card title="LangChain" icon="square-1" href="/integrations/langchain">
LangChain provides a framework to easily construct LLM-powered apps. Langchain developers can leverage OctoAI LLM and embedding endpoints to easily access efficient compute across a wide selection of LLMs.
</Card>
<Card title="Unstructured.io" icon="square-2">
<Card title="Unstructured.io" icon="square-2" href="/integrations/unstructured">
Unstructured provides components to very easily embed text documents lke PDFs, HTML, Word Docs, and more. The OctoAIEmbedingEncoder is available, so documents parsed with Unstructured can easily be embedded with the OctoAI embeddings endpoint.
</Card>
<Card title="Pinecone (Canopy)" icon="square-3">
Pinecone provides storage and retrieval infrastructure needed ror building and running AI apps. This integration allows a developer using Canopy to choose from the best LLMs on OctoAI.
<Card title="Pinecone (Canopy)" icon="square-3" href="/integrations/pinecone">
Pinecone provides storage and retrieval infrastructure needed for building and running AI apps. This integration allows a developer using Canopy to choose from the best LLMs on OctoAI.
</Card>
<Card title="OpenRouter" icon="square-4">
<Card title="OpenRouter" icon="square-4" href="/integrations/openrouter">
OpenRouter has a unified interface for using various LLMs, allowing users to find and compare models for their needs. The OpenRouter API users can leverage OctoAI's best in class LLM endpoints.
</Card>
<Card title="LlamaIndex" icon="square-5">
<Card title="LlamaIndex" icon="square-5" href="/integrations/llamaindex">
LlamaIndex aids in the management of interactions between your LLMs and private data. A developer building AI apps can now access highly optimized LLMs and Embeddings models on OctoAI.
</Card>
</CardGroup>
Original file line number Diff line number Diff line change
@@ -33,6 +33,8 @@ chat_engine:
batch_size: 2048
```

### Learn with our demo apps

Get started today by following along with one of our demo apps:
- [DocTalk](https://octo.ai/demos/doctalk/)

Original file line number Diff line number Diff line change
@@ -6,14 +6,14 @@ description: "Custom checkpoints are fine-tuned versions of the original model a
Custom Stable Diffusion checkpoints are fine-tuned versions of the original model, trained to capture particular styles, subjects, or objects. They are designed to provide users with more control and customization options when generating images. These checkpoints can be tailored to produce images in various styles, such as realistic photography, artwork, or even specific themes like landscapes or portraits.
While checkpoints represent a significant investment in terms of storage and computational resources, they excel in maintaining the desired customizations consistently. OctoAI's Asset Library boasts a rich collection of pre-loaded custom checkpoints, offering a diverse array of styles to enhance your images. Additionally, users have the flexibility to import bespoke checkpoints from external sources, integrating them seamlessly into OctoAI's Asset Library as personalized assets.

The image results with different checkpoints, even using the same prompt, can be significantly different. Using the simple prompt `A photo of an Australian cattle dog running through a park`, you can see see the results from the SDXL base model (left) and samaritan model (right). The samaritan model represents a 3D-cartoon image style.
The image results with different checkpoints, even using the same prompt, can be significantly different. Using the simple prompt `A photo of an Australian cattle dog running through a park`, you can see see the results from the SDXL base model (left) and samaritan model (right). The samaritan model represents a 3D-cartoon image style.

<CardGroup cols={2}>
<Card title="Using SDXL base model">
![](https://www.datocms-assets.com/45680/1706909113-4d5ffd9-1.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1706909113-4d5ffd9-1.jpeg?max-w=2000&auto=compress)
</Card>
<Card title="Using samaritan checkpoint">
![](https://www.datocms-assets.com/45680/1706909150-95229ac-2.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1706909150-95229ac-2.jpeg?max-w=2000&auto=compress)
</Card>
</CardGroup>

@@ -41,12 +41,12 @@ curl -X POST "https://image.octoai.run/generate/sdxl" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.SDXL,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = client.image_gen.generate_sdxl(
prompt="A photo of an Australian cattle dog running through a park",
negative_prompt="Blurry photo, distortion, low-res, poor quality",
checkpoint="octoai:samaritan",
@@ -63,33 +63,32 @@ if __name__ == "__main__":
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSdxl({
"prompt": "A photo of an Australian cattle dog running through a park",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality",
"checkpoint": "octoai:samaritan",
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base"
};

const outputs = await client.infer<any>(endpointUrl, inputs);
"cfgScale": 12,
"useRefiner": true,
"highNoiseFrac": 0.8,
"stylePreset": "base"
})

outputs.images.forEach((output, i) => {
images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
Original file line number Diff line number Diff line change
@@ -21,14 +21,14 @@ While traditional image generation models can produce stunning visuals, they oft
```
Other than using the default controlnet checkpoints, you can also upload private ControlNet checkpoints into the OctoAI Asset Library and then use those checkpoints at generation time via the parameter `controlnet` in the API. For custom controlnet checkpoints, make sure to provide your own ControlNet mask in the `controlnet_image` parameter.

Below is an example of using a **Canny ControlNet** along with ControlNet image (left) and a simple prompt `A photo of woman wearing a (rose pink dress:1)`. Canny ControlNet is designed to detect a wide range of edges in images. Given a raw image or sketch, Canny can extract the image's contours and edges, and use them for image generation. You can see the image (right) generated from SDXL with Canny ControlNet applied.
Below is an example of using a **Canny ControlNet** along with ControlNet image (left) and a simple prompt `A photo of woman wearing a (rose pink dress:1)`. Canny ControlNet is designed to detect a wide range of edges in images. Given a raw image or sketch, Canny can extract the image's contours and edges, and use them for image generation. You can see the image (right) generated from SDXL with Canny ControlNet applied.

<CardGroup cols={2}>
<Card title="ControlNet Image">
![](https://www.datocms-assets.com/45680/1709366453-screenshot-2024-03-01-at-9-24-05-pm.png?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709366453-screenshot-2024-03-01-at-9-24-05-pm.png?max-w=2000&auto=compress)
</Card>
<Card title="SDXL with Canny ControlNet">
![](https://www.datocms-assets.com/45680/1709366467-download-9.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709366467-download-9.jpeg?max-w=2000&auto=compress)
</Card>
</CardGroup>

@@ -58,12 +58,12 @@ curl -X POST "https://image.octoai.run/generate/controlnet-sdxl" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.CONTROLNET_SDXL,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = client.image_gen.generate_controlnet_sdxl(
prompt="A photo of woman wearing a (rose pink dress:1)",
negative_prompt="Blurry photo, distortion, low-res, poor quality",
controlnet="octoai:canny_sdxl",
@@ -82,16 +82,17 @@ if __name__ == "__main__":
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/controlnet-sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateControlnetSdxl({
"prompt": "A photo of woman wearing a (rose pink dress:1)",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
"controlnet": "octoai:canny_sdxl",
@@ -106,11 +107,9 @@ const inputs = {
"style_preset": "base",
"controlnet_conditioning_scale": 1,
"controlnet_image": "<BASE64 IMAGE>"
};

const outputs = await client.infer<any>(endpointUrl, inputs);
})

outputs.images.forEach((output, i) => {
images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
@@ -122,10 +121,10 @@ Below is an example of using a **OpenPose ControlNet** along with ControlNet ima

<CardGroup cols={2}>
<Card title="ControlNet Image">
![](https://www.datocms-assets.com/45680/1709490527-screenshot-2024-03-02-at-10-33-06-am.png?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709490527-screenshot-2024-03-02-at-10-33-06-am.png?max-w=2000&auto=compress)
</Card>
<Card title="SDXL with OpenPose ControlNet">
![](https://www.datocms-assets.com/45680/1709490877-download-10.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709490877-download-10.jpeg?max-w=2000&auto=compress)
</Card>
</CardGroup>

@@ -155,12 +154,12 @@ curl -X POST "https://image.octoai.run/generate/controlnet-sdxl" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.CONTROLNET_SDXL,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = client.image_gen.generate_controlnet_sdxl(
prompt="An photo of a white man on a japanese tatami mat ",
negative_prompt="Blurry photo, distortion, low-res, poor quality, distorted legs, distorted feet, disproportionate hands and ",
controlnet="octoai:openpose_sdxl",
@@ -179,35 +178,34 @@ if __name__ == "__main__":
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/controlnet-sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateControlnetSdxl({
"prompt": "An photo of a white man on a japanese tatami mat ",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality, distorted legs, distorted feet, disproportionate hands and ",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality, distorted legs, distorted feet, disproportionate hands and ",
"controlnet": "octoai:openpose_sdxl",
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base",
"controlnet_conditioning_scale": 1,
"controlnet_image": "<BASE64 IMAGE>"
};

const outputs = await client.infer<any>(endpointUrl, inputs);

outputs.images.forEach((output, i) => {
"cfgScale": 12,
"useRefiner": true,
"highNoiseFrac": 0.8,
"stylePreset": "base",
"controlnetConditioningScale": 1,
"controlnetImage": "<BASE64 IMAGE>"
})

images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
Original file line number Diff line number Diff line change
@@ -4,22 +4,22 @@ description: "LoRAs for image or video AI models are custom weights applied to a
---
LoRAs are additional custom weights applied to a base checkpoint. Similar to checkpoints, LoRAs can represent a specific style or custom subject, but they are much smaller in size and more economical to use. You can include multiple LoRAs in a single image generation, and provide a weight for each LoRA. A greater weight value will have more influence on the generated image. Similar to checkpoints, users have the flexibility to import LoRAs from external sources, and integrate them seamlessly into OctoAI's Asset Library as personalized assets.

Below is an example of using a LoRA along with a simple prompt `Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details`. You can see the results from the SDXL base model (top left) and subsequent results with add-detail LoRA - varying weights (top right and bottom left). Add-details LoRA adds intricate details to the output image. The image generated on the botton right is a result of two LoRAs, add-details (weight:0.3) and more-art (weight:1.0). More-art LoRA adds artistic details to the output image. You can clearly see the impact of more-art LoRA over add-detail LoRA in the resulting image.
Below is an example of using a LoRA along with a simple prompt `Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details`. You can see the results from the SDXL base model (top left) and subsequent results with add-detail LoRA - varying weights (top right and bottom left). Add-details LoRA adds intricate details to the output image. The image generated on the botton right is a result of two LoRAs, add-details (weight:0.3) and more-art (weight:1.0). More-art LoRA adds artistic details to the output image. You can clearly see the impact of more-art LoRA over add-detail LoRA in the resulting image.

<CardGroup cols={2}>
<Card title="Using SDXL without LoRA">
![](https://www.datocms-assets.com/45680/1709328256-download-1.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709328256-download-1.jpeg?max-w=2000&auto=compress)
</Card>
<Card title="Using add-detail LoRA:0.5">
![](https://www.datocms-assets.com/45680/1709328968-download-2.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709328968-download-2.jpeg?max-w=2000&auto=compress)
</Card>
<Card title="Using add-detail LoRA:1.0">
![](https://www.datocms-assets.com/45680/1709328265-download.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709328265-download.jpeg?max-w=2000&auto=compress)
</Card>
<Card title="add-detail:0.3, more-art LoRA:1.0">
![](https://www.datocms-assets.com/45680/1709329510-download-3.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709329510-download-3.jpeg?max-w=2000&auto=compress)
</Card>

</CardGroup>

**Example Code:**
@@ -48,63 +48,61 @@ curl -X POST "https://image.octoai.run/generate/sdxl" \
```

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

client = OctoAI()

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.SDXL,
prompt="Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details",
negative_prompt="Blurry photo, distortion, low-res, poor quality",
loras={"octoai:add-detail":0.3,"octoai:more_art":1},
width=1024,
height=1024,
num_images=1,
sampler="DDIM",
steps=30,
cfg_scale=12,
use_refiner=True,
high_noise_frac=0.8,
style_preset="base",
)
images = image_gen_response.images
image_gen_response = client.image_gen.generate_sdxl(
prompt="Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details",
negative_prompt="Blurry photo, distortion, low-res, poor quality",
loras={"octoai:add-detail":0.3,"octoai:more_art":1},
width=1024,
height=1024,
num_images=1,
sampler="DDIM",
steps=30,
cfg_scale=12,
use_refiner=True,
high_noise_frac=0.8,
style_preset="base",
)
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
for i, image in enumerate(images):
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSdxl({
"prompt": "Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality",
"loras": {
"octoai:add-detail": 0.3,
"octoai:more_art": 1
},
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base"
};

const outputs = await client.infer<any>(endpointUrl, inputs);
"cfgScale": 12,
"useRefiner": true,
"highNoiseFrac": 0.8,
"stylePreset": "base"
})

outputs.images.forEach((output, i) => {
images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
```
</CodeGroup>

LoRAs can further customize your images, by including custom objects or styles.
LoRAs can further customize your images, by including custom objects or styles.
Original file line number Diff line number Diff line change
@@ -3,7 +3,7 @@ title: "Overview"
description: "You can tweak your images using various customizations available within OctoAI Media Gen solution including checkpoints, LoRAs, textual inversions and ControlNets"
---

With OctoAI Media Gen solution, you can effortlessly integrate Stable Diffusion’s customizable image generation features into your application. While standard pre-trained image generation assets from repositories like HuggingFace may suffice for simple tasks, customization becomes crucial for commercial uses. Customization allows precise control over generating specific subjects and environments, which is essential for most commercial needs. Stable Diffusion, offered by OctoAI, provides both basic and advanced customization options. These include adjusting prompt weights, applying style presets, and employing advanced techniques like LoRAs, checkpoints, textual inversions, and ControlNets. Additionally, with OctoAI, you can create your own LoRA asset using custom image datasets and leverage it to meet your business requirements. To learn more, review [Fine-tuning on OctoAI](/media-gen-solution/fine-tuning-stable-diffusion/fine-tuning-stable-diffusion)
With OctoAI Media Gen solution, you can effortlessly integrate Stable Diffusion’s customizable image generation features into your application. While standard pre-trained image generation assets from repositories like HuggingFace may suffice for simple tasks, customization becomes crucial for commercial uses. Customization allows precise control over generating specific subjects and environments, which is essential for most commercial needs. Stable Diffusion, offered by OctoAI, provides both basic and advanced customization options. These include adjusting prompt weights, applying style presets, and employing advanced techniques like LoRAs, checkpoints, textual inversions, and ControlNets. Additionally, with OctoAI, you can create your own LoRA asset using custom image datasets and leverage it to meet your business requirements. To learn more, review [Fine-tuning on OctoAI](/media-gen-solution/fine-tuning-stable-diffusion/fine-tuning-stable-diffusion)

<Note> Pro or Enterprise account is required to access fine-tuning. </Note>

@@ -39,12 +39,12 @@ curl -X POST "https://image.octoai.run/generate/sdxl" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.SDXL,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = client.image_gen.generate_sdxl(
prompt="Commercial photography,(snowy:0.8) ,luxury perfume bottle, angelic silver light, studio light, high resolution photography, fine details",
negative_prompt="Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
checkpoint="octoai:RealVisXL",
@@ -63,42 +63,45 @@ if __name__ == "__main__":
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSdxl({
"prompt": "Commercial photography,(snowy:0.8) ,luxury perfume bottle, angelic silver light, studio light, high resolution photography, fine details",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
"checkpoint": "octoai:RealVisXL",
"loras": {
"octoai:add-detail": 1
},
"textual_inversions": {
"textualInversions": {
"octoai:NegativeXL": "“negativeXL_D”"
},
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base"
};
"cfgScale": 12,
"useRefiner": true,
"highNoiseFrac": 0.8,
"stylePreset": "base"
})

const outputs = await client.infer<any>(endpointUrl, inputs);

outputs.images.forEach((output, i) => {
images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});


import fs from "fs";
import { Client } from "@octoai/client";
```
</CodeGroup>

Original file line number Diff line number Diff line change
@@ -3,26 +3,26 @@ title: "Textual Inversions"
description: "Customize your images on OctoAI using Textual inversions, which are embeddings that represent custom subjects."
---

Textual inversions are embeddings that represent custom subjects.They can also represent negative embeddings, which are trained on undesirable content like bad quality hands or lighting. You can use these in your _negative_ prompt to improve your images, such as avoiding bad quality hands. These are the smallest and cheapest assets we currently support.
Textual inversions are embeddings that represent custom subjects.They can also represent negative embeddings, which are trained on undesirable content like bad quality hands or lighting. You can use these in your _negative_ prompt to improve your images, such as avoiding bad quality hands. These are the smallest and cheapest assets we currently support.
The name of the textual inversion acts as a specific trigger word, which must be included in the prompt. Similar to prompt weighting, you can increase the weight of textual inversion using the format `(textual-inversion:weight)`.

Below is an example of using a NegativeXL textual inversion (trigger word: negativeXL_D).

Prompt: `Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details`

Negative prompt: `Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)`

You can see the results from the SDXL base model (left) generating an image with snowflake or flower shaped designs on the bottle and subsequent results with textual inversion (right) which ensures that the negative prompt is followed and no flower, snowflake design appears on the bottle in the output image.
You can see the results from the SDXL base model (left) generating an image with snowflake or flower shaped designs on the bottle and subsequent results with textual inversion (right) which ensures that the negative prompt is followed and no flower, snowflake design appears on the bottle in the output image.

<CardGroup cols={2}>
<Card title="Using SDXL base model">
![](https://www.datocms-assets.com/45680/1709334742-download-4.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709334742-download-4.jpeg?max-w=2000&auto=compress)
</Card>
<Card title="Using SDXL with textual inversion">
![](https://www.datocms-assets.com/45680/1709334749-download-5.jpeg?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1709334749-download-5.jpeg?max-w=2000&auto=compress)
</Card>
</CardGroup>

**Example Code:**
<CodeGroup>
```bash cURL
@@ -49,59 +49,54 @@ curl -X POST "https://image.octoai.run/generate/sdxl" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.SDXL,
prompt="Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details",
negative_prompt="Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
textual_inversions={"octoai:NegativeXL":"“negativeXL_D”"},
width=1024,
height=1024,
num_images=1,
sampler="DDIM",
steps=30,
cfg_scale=12,
use_refiner=True,
high_noise_frac=0.8,
style_preset="base",
)
images = image_gen_response.images
client = OctoAI()

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
image_gen_response = client.image_gen.generate_sdxl(
prompt="Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details",
negative_prompt="Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
textual_inversions={"octoai:NegativeXL":"“negativeXL_D”"},
width=1024,
height=1024,
num_images=1,
sampler="DDIM",
steps=30,
cfg_scale=12,
use_refiner=True,
high_noise_frac=0.8,
style_preset="base",
)
images = image_gen_response.images

for i, image in enumerate(images):
to_file(image, f"result{i}.jpg")
```

```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSdxl({
"prompt": "Commercial photography,snowy,luxury perfume bottle,angelic silver light, studio light, high resolution photography, fine details",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
"textual_inversions": {
"negativePrompt": "Blurry photo, distortion, low-res, poor quality, (flowers on bottle negativeXL_D:0.9), (snowflake on bottle negativeXL_D:0.9), (colored liquid in bottle negativeXL_D:1.0)",
"textualInversions": {
"octoai:NegativeXL": "“negativeXL_D”"
},
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base"
};

const outputs = await client.infer<any>(endpointUrl, inputs);

outputs.images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
"cfgScale": 12,
"useRefiner": true,
"highNoiseFrac": 0.8,
"stylePreset": "base"
})
```
</CodeGroup>
Original file line number Diff line number Diff line change
@@ -10,7 +10,7 @@ OctoAI lets you fine-tune Stable Diffusion to customize generated images. Fine-t
2. Run the fine-tuning job
3. Use the fine-tuned asset (a LoRA) in your image generation requests

We're using the LoRA fine-tuning method, which is an acronym for Low-Rank Adaptation. It's a fast and effective way to fine-tune Stable Diffusion. Fine-tuning is supported for Stable Diffusion v1.5 and Stable Diffusion XL. Fine-tuning is available in the OctoAI web UI or via the fine-tuning API.
We're using the LoRA fine-tuning method, which is an acronym for Low-Rank Adaptation. It's a fast and effective way to fine-tune Stable Diffusion. Fine-tuning is supported for Stable Diffusion v1.5 and Stable Diffusion XL. Fine-tuning is available in the OctoAI web UI or via the fine-tuning API.

### Web UI Guide

@@ -20,11 +20,11 @@ In the web UI, navigate to the **Tuning & Datasets** page from the **Media Gen S

Specify the name of your fine-tune, the trigger word of the subject you're fine-tuning, and the base checkpoint. The base checkpoint can be the default Stable Diffusion v1.5 checkpoint, default Stable Diffusion XL checkpoint, or any custom checkpoint.

The trigger word can be used in your inference requests to customize the images with your subject. We generally recommend using a unique trigger word, such as "sks1", that's unlikely to be associated with a different subject in Stable Diffusion. Alternatively, you can use an existing concept as the trigger word value - such as "in the style of a cartoon drawing" - to update Stable Diffusion's understanding of that concept.
The trigger word can be used in your inference requests to customize the images with your subject. We generally recommend using a unique trigger word, such as "sks1", that's unlikely to be associated with a different subject in Stable Diffusion. Alternatively, you can use an existing concept as the trigger word value - such as "in the style of a cartoon drawing" - to update Stable Diffusion's understanding of that concept.

Then, specify the number of steps to train. A range of 400 to 1,200 steps works well in most cases, and a good guideline is about 75 to 100 steps per training image. The model can underfit if the numer of training steps is too low, resulting in poor quality. If it's too high, the model can overfit and struggle to represent details that aren't represented in the training images.
Then, specify the number of steps to train. A range of 400 to 1,200 steps works well in most cases, and a good guideline is about 75 to 100 steps per training image. The model can underfit if the numer of training steps is too low, resulting in poor quality. If it's too high, the model can overfit and struggle to represent details that aren't represented in the training images.

![](https://www.datocms-assets.com/45680/1706909266-cf08950-screenshot_2023-11-20_at_5-55-26_pm.png?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1706909266-cf08950-screenshot_2023-11-20_at_5-55-26_pm.png?max-w=2000&auto=compress)

#### Upload images & tune

@@ -36,38 +36,39 @@ When you're ready, click "Start Tuning", and the fine-tune job will progress fro

#### Generating images

When complete, the fine-tuned asset is stored in your Asset Library and available for image generation. You can launch the Text to Image or Image to Image tool to start generating images with your custom asset.
When complete, the fine-tuned asset is stored in your Asset Library and available for image generation. You can launch the Text to Image or Image to Image tool to start generating images with your custom asset.

![](https://www.datocms-assets.com/45680/1703619348-new-tune-stored-in-asset-library-on-octoai.png?max-w=2000&auto=compress)
![](https://www.datocms-assets.com/45680/1703619348-new-tune-stored-in-asset-library-on-octoai.png?max-w=2000&auto=compress)

### API Guide

Complete fine-tuning API parameters are organized in our [API Reference documentation](/api-reference/fine-tuning/create-tune).
Complete fine-tuning API parameters are organized in our [API Reference documentation](/api-reference/fine-tuning/create-tune).

#### Upload images

First, upload your training images using the [AssetOrchestrator Python Client](/python-sdk/asset-orchestrator-client#creating-file-assets-from-a-folder-of-images) or [CLI](/media-gen-solution/uploading-a-custom-asset-to-the-octoai-asset-library).
First, upload your training images using the [AssetLibrary Python Client](/python-sdk/asset-orchestrator-client#creating-file-assets-from-a-folder-of-images) or [CLI](/media-gen-solution/uploading-a-custom-asset-to-the-octoai-asset-library).

**Python client**

You can easily upload individual image files, or a folder with multiple files. Here's an example uploading the `image1.jpeg` file with the name `image1` from the file path `finetuning_images`:

```Python Python
from octoai.clients.asset_orch import AssetOrchestrator, FileData, FileExtension
from octoai.client import OctoAI
from octoai.asset_library import Data_File

asset_orch = AssetOrchestrator()
client = OctoAI()

asset = asset_orch.create(
asset = client.asset_library.create_from_file(
file="finetuning_images/image1.jpeg",
data=FileData(FileExtension.JPEG),
data=Data_File(file_format="jpeg"),
name="image1",
description="Fine-tuning image",
)

print(asset)
print(asset_orch.list(name="image1"))
print(client.asset_library.list(name="image1"))

asset_orch.list()
client.asset_library.list()
```

You'll receive a response with the asset ID, name, and status:
@@ -80,11 +81,12 @@ Here's an example uploading a folder of images. This code snippet gets the files

```Python Python
import os
from octoai.clients.asset_orch import AssetOrchestrator, FileData
from octoai.client import OctoAI
from octoai.asset_library import Data_File

if __name__ == "__main__":
# OCTOAI_TOKEN set as an environment variable so do not need to pass a token.
asset_orch = AssetOrchestrator()
client = OctoAI()

dir_path = "./finetuning_images/" # Set your dir_path here to your file assets.
files = []
@@ -97,10 +99,10 @@ if __name__ == "__main__":
split_file_name = file.split(".")
asset_name = split_file_name[0]
file_format = split_file_name[1]
file_data = FileData(
file_data = Data_File(
file_format=file_format,
)
asset = asset_orch.create(
asset = client.asset_library.create_from_file(
file=dir_path + file,
data=file_data,
name=asset_name,
@@ -113,7 +115,7 @@ The final `print(asset)` will return a response with each asset ID, name, and st

**CLI**

Alternatively, you can upload images using the OctoAI CLI. Here's the CLI command using the same `image1.jpeg` example:
Alternatively, you can upload images using the OctoAI CLI. Here's the CLI command using the same `image1.jpeg` example:

```bash bash
octoai asset create \
@@ -124,45 +126,48 @@ octoai asset create \

#### Configure settings & tune

Next, create your fine-tune. In the examples in this section, we're fine-tuning Stable Diffusion XL with images of a bulldog. We specify the base checkpoint, trigger word, training steps, and fine-tune name. Also included are the individual training images and corresponding captions. We recommend including captions, describing the context of the subject, to improve fine-tuning quality. Be sure to include your trigger word within the caption.
Next, create your fine-tune. In the examples in this section, we're fine-tuning Stable Diffusion XL with images of a bulldog. We specify the base checkpoint, trigger word, training steps, and fine-tune name. Also included are the individual training images and corresponding captions. We recommend including captions, describing the context of the subject, to improve fine-tuning quality. Be sure to include your trigger word within the caption.

**Python Client**

```Python Python
from octoai.client import Client
from octoai.fine_tuning import Details_LoraTune, LoraTuneCheckpoint, LoraTuneFile
from octoai.client import OctoAI

if __name__ == "__main__":
client = Client()
client = OctoAI()

# create a fine tuning job
tune = client.tune.create(
tune = client.fine_tuning.create(
name="sks1-bulldog-01",
base_checkpoint="asset_01hdpjv7bxe1n99eazrv23ca1k",
engine="image/stable-diffusion-xl-v1-0",
files={
"asset_01hekxwwg1fzjv48sfy5s2xmnj": "sks1 bulldog playing at the beach",
"asset_01hekxwqgrev5t3ser2w2qf8bm": "sks1 bulldog playing with a bone",
"asset_01hekxwj3bekpvr52kne3c6ca7": "sks1 bulldog looking up",
"asset_01hekxwdc8eqrrahjm3ekze356": "sks1 bulldog resting in the grass",
"asset_01hekxw80kfdmrdh7ny3vyvf0h": "sks1 bulldog running at the park",
},
trigger_words="sks1",
steps=800,
details=Details_LoraTune(
files=[
LoraTuneFile(file_id="asset_01hekxwwg1fzjv48sfy5s2xmnj", caption="sks1 bulldog playing at the beach"),
LoraTuneFile(file_id="asset_01hekxwqgrev5t3ser2w2qf8bm", caption="sks1 bulldog playing with a bone"),
LoraTuneFile(file_id="asset_01hekxwj3bekpvr52kne3c6ca7", caption="sks1 bulldog looking up"),
LoraTuneFile(file_id="asset_01hekxwdc8eqrrahjm3ekze356", caption="sks1 bulldog resting in the grass"),
LoraTuneFile(file_id="asset_01hekxw80kfdmrdh7ny3vyvf0h", caption="sks1 bulldog running at the park"),
],
base_checkpoint=LoraTuneCheckpoint(
checkpoint_id="asset_01hdpjv7bxe1n99eazrv23ca1k",
engine="image/stable-diffusion-xl-v1-0",
),
trigger_words=["sks1"],
steps=800,
)
)
print(f"Tune {tune.name} status: {tune.status}")

# check the status of a fine tuning job
tune = client.tune.get(tune.id)
tune = client.fine_tuning.get(tune.id)
print(f"Tune {tune.name} status: {tune.status}")

# when the job finishes, check the asset ids of the resulted loras
# (the tune will take some time to complete)
if tune.status == "succeded":
print(f"Generated LoRAs: {tune.output_lora_ids}"

```
print(f"Generated LoRAs: {tune.output_lora_ids}")

Full API details and parameters [are available here](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#module-octoai.clients.fine%5Ftuning).
```

**REST API**

@@ -211,9 +216,9 @@ curl -X POST "https://api.octoai.cloud/v1/tune" \
}
```
Full API details and parameters [are available here](/api-reference/fine-tuning/create-tune). Using the `continue_on_rejection` boolean parameter, you can optionally continue with the fine-tune job if any of the training images are identified as NSFW.
Full API details and parameters [are available here](/api-reference/fine-tuning/create-tune). Using the `continue_on_rejection` boolean parameter, you can optionally continue with the fine-tune job if any of the training images are identified as NSFW.
You'll receive a response that includes the tune ID, which you can use to monitor the status of the job.
You'll receive a response that includes the tune ID, which you can use to monitor the status of the job.

#### Monitor fine-tuning status

@@ -226,27 +231,27 @@ curl "https://api.octoai.cloud/v1/tune/tune_01hen39pazf6s9jqpkfj05y0vx" \

#### Generating images

When complete, the fine-tuned asset is stored in your Asset Library and available for image generation. You can generate images by including the LoRA in your image generation request.
When complete, the fine-tuned asset is stored in your Asset Library and available for image generation. You can generate images by including the LoRA in your image generation request.

```bash bash
curl -X POST "<https://image.octoai.run/generate/sdxl">
-H "Content-Type: application/json"
-H "Authorization: Bearer $OCTOAI_TOKEN"
--data-raw '{
"prompt": "A photo of a sks1 bulldog running in space",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
"loras": {
"sks1-bulldog-01": 0.9
},
"width": 1024,
"height": 1024,
"num_images": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base"
curl -X POST "<https://image.octoai.run/generate/sdxl">
-H "Content-Type: application/json"
-H "Authorization: Bearer $OCTOAI_TOKEN"
--data-raw '{
"prompt": "A photo of a sks1 bulldog running in space",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
"loras": {
"sks1-bulldog-01": 0.9
},
"width": 1024,
"height": 1024,
"num_images": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12,
"use_refiner": true,
"high_noise_frac": 0.8,
"style_preset": "base"
}'
```

@@ -256,15 +261,15 @@ curl -X POST "<https://image.octoai.run/generate/sdxl">

Image captions can improve quality by providing additional context and details to the trained model. Whenever possible, we suggest using image captions - and be sure to include your trigger word in each caption.

You can also include the subject class - such as person, animal, or object - to improve quality. As an example, you could fine-tune images of a specific bulldog with an example caption `sks1 bulldog playing at the beach` where `sks1` is the trigger word and `bulldog` is the subject class.
You can also include the subject class - such as person, animal, or object - to improve quality. As an example, you could fine-tune images of a specific bulldog with an example caption `sks1 bulldog playing at the beach` where `sks1` is the trigger word and `bulldog` is the subject class.

#### Image variation

We recommend some amount of variation in your images. If every image is close-up, the fine-tuned model may be limited to representing that distance. It's also helpful to have some level of consistency among the images to ensure the model learns the intended subject. Finding the right balance between consistency and variation can require a few iterations, and we encourage you to experiment!
#### Managing fine-tunes and assets
You can separately manage a fine-tune job, and the corresponding LoRA created by a fine-tune. Deleting a fine-tune job won't automatically delete the training images nor LoRA created during fine-tuning, and vice versa. Additionally, fine-tune names must be unique. You may encounter an error if you try to create a fine-tune with a duplicate name.
You can separately manage a fine-tune job, and the corresponding LoRA created by a fine-tune. Deleting a fine-tune job won't automatically delete the training images nor LoRA created during fine-tuning, and vice versa. Additionally, fine-tune names must be unique. You may encounter an error if you try to create a fine-tune with a duplicate name.

#### Fine-tuning duration

Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ Please see [Fine-tuning Stable Diffusion](/api-reference/fine-tuning/create-tune

* Please [create an OctoAI API token](/getting-started/how-to-create-an-octoai-access-token) if you don't have one already.
* Please also verify you've completed [TypeScript SDK Installation & Setup](/typescript-sdk/installation-and-setup). Must be version >= 0.4.0.
* If you use the `OCTOAI_TOKEN` envvar for your token, you can instantiate the client with `const client = Client(process.env.OCTOAI_TOKEN)` or pass the token as a parameter to the constructor.
* If you use the `OCTOAI_TOKEN` envvar for your token, you can instantiate the client with `const client = OctoAI()process.env.OCTOAI_TOKEN)` or pass the token as a parameter to the constructor.
* An account and API token is required for all the following steps.

#### High-level steps to creating a fine-tuned LoRA
@@ -38,22 +38,30 @@ This is a snippet, however for the full example, please check at the end of this
In this example, we use multiple photos of a toy poodle named Mitchi.

```TypeScript TypeScript
import { OctoAIClient } from "@octoai/sdk";

// These constants will be the same in the rest of this doc
const NAME = "test-sks3-poodle-sd15"; // To be used for loras in infer method
const FILE_PATH = "test_assets/mitchi";
const FILE_SUFFIX = "jpg";

const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

// First, we will upload and create a number of image assets to use for fine-tuning
const assets = [];
for (let i = 0; i < 5; i++) {
const asset = await client.asset.create({
const filename = `${FILE_PATH}${i}.${FILE_SUFFIX}`;
const stream = createReadStream(filename)

const asset = await client.assetLibrary.upload(stream, {
name: `${NAME}-image-${i}`,
// test_assets/mitchi1.jpg for example.
file: `${FILE_PATH}${i}.${FILE_SUFFIX}`, // Buffers and ArrayBuffers can also be used
data: { asset_type: "file", file_format: FILE_SUFFIX },
asset_type: "file",
data: { assetType: "file", fileFormat: FILE_SUFFIX },
assetType: "file",
description: `${NAME}`,
});

assets.push(asset);
}
```
@@ -65,7 +73,7 @@ We can also use a call back for our snippet to verify that the assets are ready
let assetStatus = assets[pos].status;
while (pos < assets.length) {
await new Promise((resolve) => setTimeout(resolve, 1000));
const retrieved = await client.asset.get({ id: assets[pos].id });
const retrieved = await client.assetLibrary.get(assets[pos].id);
assets[pos] = retrieved;
assetStatus = retrieved.status;
if (assetStatus === "ready") {
@@ -84,9 +92,9 @@ Next, you'll need a checkpoint to use to tune your asset. In this example, we wi

```TypeScript TypeScript
// Let's use an OctoAI public checkpoint for tuning our LoRA
const checkpoint = await client.asset
const checkpoint = await client.assetLibrary
.list({
is_public: true,
isPublic: true,
owner: "octoai",
name: "default-sd15",
})
@@ -98,41 +106,56 @@ Next, you'll need a checkpoint to use to tune your asset. In this example, we wi
We can create a tune job most simply by passing in the checkpoint directly and the assets directly. This does come at a minor cost to quality because this will lead to the captions being set directly to the trigger word, where as for better results you'll want to set your own captions.

```TypeScript TypeScript
const createTuneRequest = {
const tune = await client.fineTuning.create({
name: NAME,
description: "sks3 poodle",
details: {
base_checkpoint: checkpoint,
baseCheckpoint: checkpoint,
files: assets,
steps: 500,
tune_type: "lora_tune",
trigger_words: ["sks3 poodle"],
tuneType: "lora_tune",
triggerWords: ["sks3 poodle"],
},
};
let tune = await client.tune.create(createTuneRequest);
});
```

For better results, you can set your own captions for your assets as follows, or use the asset\_id strings directly, then pass this to the `files` field in the above request.

```TypeScript TypeScript
files: [
{
file_id: assets[0].id, caption: "your detailed caption with sks3 poodle the trigger word in it here"
},
{
file_id: assets[1].id, caption: "another detailed caption with sk3 poodle the trigger word in it here"
}
]
const tune = await client.fineTuning.create({
name: NAME,
description: "sks3 poodle",
details: {
baseCheckpoint: checkpoint,
files: assets.map((asset) => ({
fileId: asset.id,
caption: "your detailed caption with sks3 poodle the trigger word in it here",
})),
steps: 500,
tuneType: "lora_tune",
triggerWords: ["sks3 poodle"],
},
});
```

You can also pass the details for your base\_checkpoint directly instead of looking up the asset as well if you know it.

```TypeScript TypeScript
base_checkpoint: {
checkpoint_id: "asset_01hev42y7ffc58b3aqc8wa04p4",
engine: "image/stable-diffusion-v1-5",
name: "default-sd15",
}
const tune = await client.fineTuning.create({
name: NAME,
description: "sks3 poodle",
details: {
baseCheckpoint: {
checkpointId: "asset_01hev42y7ffc58b3aqc8wa04p4",
engine: "image/stable-diffusion-v1-5",
name: "default-sd15",
},
files: assets
steps: 500,
tuneType: "lora_tune",
triggerWords: ["sks3 poodle"],
},
});
```

Similar to creating assets, we can also wait for the tune job to succeed (or fail) before we move on to running an inference.
@@ -141,7 +164,7 @@ Similar to creating assets, we can also wait for the tune job to succeed (or fai
let { status } = tune;
while (status !== "failed" && status !== "succeeded") {
await new Promise((resolve) => setTimeout(resolve, 1000));
tune = await client.tune.get(tune.id);
tune = await client.fineTuning.get(tune.id);
status = tune.status;
}
```
@@ -151,20 +174,16 @@ Similar to creating assets, we can also wait for the tune job to succeed (or fai
Next, you can run an inference with the tuned LoRA

```TypeScript TypeScript
const endpointUrl = "https://image.octoai.run/generate/sd";

const inputs = {
const { images } = await client.imageGen.generateSd({
"prompt": "A photo of sks3 poodle as a puppy",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality, extra limbs, extra tails",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality, extra limbs, extra tails",
"loras": {
"test-sks3-poodle-sd15": 0.8 // Replace this whatever you named your NAME const
},
"num_images": 1,
};

const imageOutputs = await client.infer<any>(endpointUrl, inputs);
"numImages": 1,
})

imageOutputs.images.forEach((imageOutputs: any, i: number) => {
images.forEach((imageOutputs: any, i: number) => {
const buffer = Buffer.from(imageOutputs.image_b64, "base64");
writeFileSync(`result${i}.jpg`, buffer);
});
@@ -183,37 +202,37 @@ This example will delete everything associated with the `NAME` constant used ear
If you wish to merely delete the file assets, you can filter by `asset_type: "file"`. Please refer to the [ListAssetsRequest](https://octoml.github.io/octoai-typescript-sdk/classes/ListAssetsRequest.html) reference docs for more information on parameters that might help you filter for how you'd like to use the service.

```TypeScript TypeScript
// Warning: This will delete all associated assets, including the created example LoRA.
// Please see above docs if you'd rather keep it, or you can also add additional file assets
// to tune your LoRA differently.
const tunes = await client.tune
// Warning: This will delete all associated assets, including the created example LoRA.
// Please see above docs if you'd rather keep it, or you can also add additional file assets
// to tune your LoRA differently.
const tunes = await client.fineTuning
.list({ name: NAME })
.then((r) => r.data);

for (let i = 0; i < tunes.length; i++) {
await client.tune.delete(tunes[i].id);
await client.fineTuning.delete(tunes[i].id);
}

const assets = await client.asset
.list({
name: NAME,
})
.then((r) => r.data);

for (let i = 0; i < assets.length; i++) {
await client.asset.delete(assets[i].id);
await client.assetLibrary.delete(assets[i].id);
}
});

```

#### Putting it all together: From Asset Creation to Running an Inference with Tuned LoRA

This does not include the above clean up script, but it's recommended you run that after if you'd like to clean up all related assets and tunes.

```TypeScript TypeScript
import {Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";
import { writeFileSync } from "fs";

const OCTOAI_TOKEN = process.env.OCTOAI_TOKEN;
const client = new Client(OCTOAI_TOKEN);
const client = new OctoAIClient();

// Magic Strings for example
const NAME = "test-sks3-poodle-sd15"; // To be used for loras in infer method
@@ -225,22 +244,24 @@ async function fineTuneExample() {
// First, we will upload and create a number of image assets to use for fine-tuning
const assets = [];
for (let i = 0; i < 5; i++) {
const asset = await client.asset.create({
name: `${NAME}-image-${i}`,
// test_assets/mitchi1.jpg for example.
file: `${FILE_PATH}${i}.${FILE_SUFFIX}`, // Buffers and ArrayBuffers can also be used
data: { asset_type: "file", file_format: FILE_SUFFIX },
asset_type: "file",
description: `${NAME}`,
});
assets.push(asset);
const filename = `${FILE_PATH}${i}.${FILE_SUFFIX}`;
const stream = createReadStream(filename)

const asset = await client.assetLibrary.upload(stream, {
name: `${NAME}-image-${i}`,
data: { assetType: "file", fileFormat: FILE_SUFFIX },
assetType: "file",
description: `${NAME}`,
});

assets.push(asset);
}
// Verify Assets are ready to do be used for fine-tuning
let pos = 0;
let assetStatus = assets[pos].status;
while (pos < assets.length) {
await new Promise((resolve) => setTimeout(resolve, 1000));
const retrieved = await client.asset.get({ id: assets[pos].id });
const retrieved = await client.assetLibrary.get(assets[pos].id);
assets[pos] = retrieved;
assetStatus = retrieved.status;
if (assetStatus === "ready") {
@@ -250,9 +271,9 @@ async function fineTuneExample() {

// Then let's use an OctoAI public checkpoint for tuning our LoRA
// You can also use your own checkpoints as well
const checkpoint = await client.asset
const checkpoint = await client.assetLibrary
.list({
is_public: true,
isPublic: true,
owner: "octoai",
name: "default-sd15",
})
@@ -261,46 +282,36 @@ async function fineTuneExample() {
// And finally creating a finetuning job after verifying the assets are ready
// This will set the captions to the trigger word, but setting descriptive captions will
// yield better results
const createTuneRequest = {
const tune = await client.fineTuning.create({
name: NAME,
description: "sks3 poodle",
details: {
base_checkpoint: checkpoint,
baseCheckpoint: checkpoint,
files: assets,
steps: 500,
tune_type: "lora_tune",
trigger_words: ["sks3"],
tuneType: "lora_tune",
triggerWords: ["sks3 poodle"],
},
};
let tune = await client.tune.create(createTuneRequest);
});

let { status } = tune;
while (status !== "failed" && status !== "succeeded") {
await new Promise((resolve) => setTimeout(resolve, 1000));
tune = await client.tune.get(tune.id);
tune = await client.fineTuning.get(tune.id);
status = tune.status;
}

// And once the job is finished, using that tuned lora for an image generation request
const endpointUrl = "https://image.octoai.run/generate/sd";

const inputs = {
"prompt": "A photo of sks3 poodle napping on a laptop",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
const { images } = await client.imageGen.generateSd({
"prompt": "A photo of sks3 poodle as a puppy",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality, extra limbs, extra tails",
"loras": {
"test-sks3-poodle-sd15": 0.9 // Replace this whatever you named your NAME const
"test-sks3-poodle-sd15": 0.8 // Replace this whatever you named your NAME const
},
"width": 512,
"height": 512,
"num_images": 1,
"sampler": "DDIM",
"steps": 50,
"cfg_scale": 12
};

const imageOutputs = await client.infer<any>(endpointUrl, inputs);
"numImages": 1,
})

imageOutputs.images.forEach((imageOutputs: any, i: number) => {
images.forEach((imageOutputs: any, i: number) => {
const buffer = Buffer.from(imageOutputs.image_b64, "base64");
writeFileSync(`result${i}.jpg`, buffer);
});
Original file line number Diff line number Diff line change
@@ -41,12 +41,12 @@ curl -X POST "https://image.octoai.run/generate/sdxl" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.SDXL,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = client.image_gen.generate_sdxl(
prompt="((glass orb)) with snowy christmas scene in it ",
negative_prompt="ornament, Blurry, low-res, poor quality",
checkpoint="octoai:lightning_sdxl",
@@ -63,33 +63,32 @@ if __name__ == "__main__":
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/sdxl";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSdxl({
"prompt": "((glass orb)) with snowy christmas scene in it ",
"negative_prompt": "ornament, Blurry, low-res, poor quality",
"negativePrompt": "ornament, Blurry, low-res, poor quality",
"checkpoint": "octoai:lightning_sdxl",
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 8,
"cfg_scale": 3,
"cfgScale": 3,
"seed": 3327823665,
"use_refiner": false,
"style_preset": "base"
};

const outputs = await client.infer<any>(endpointUrl, inputs);
"useRefiner": false,
"stylePreset": "base"
})

outputs.images.forEach((output, i) => {
images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
Original file line number Diff line number Diff line change
@@ -28,12 +28,12 @@ curl -X POST "https://image.octoai.run/generate/ssd" \

```Python Python
import os
from octoai.clients.image_gen import Engine, ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
image_gen = ImageGenerator(token=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = image_gen.generate(
engine=Engine.SSD,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
image_gen_response = client.image_gen.generate_ssd(
prompt="An image of a deLorean car in a city setting",
negative_prompt="Blurry photo, distortion, low-res, poor quality",
width=1024,
@@ -46,29 +46,28 @@ if __name__ == "__main__":
images = image_gen_response.images

for i, image in enumerate(images):
image.to_file(f"result{i}.jpg")
to_file(image, f"result{i}.jpg")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/ssd";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSsd({
"prompt": "An image of a deLorean car in a city setting",
"negative_prompt": "Blurry photo, distortion, low-res, poor quality",
"negativePrompt": "Blurry photo, distortion, low-res, poor quality",
"width": 1024,
"height": 1024,
"num_images": 1,
"numImages": 1,
"sampler": "DDIM",
"steps": 30,
"cfg_scale": 12
};

const outputs = await client.infer<any>(endpointUrl, inputs);
"cfgScale": 12
})

outputs.images.forEach((output, i) => {
images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, "base64");
fs.writeFileSync(`result${i}.jpg`, buffer);
});
Original file line number Diff line number Diff line change
@@ -78,32 +78,26 @@ if __name__ == "__main__":
```

```typescript Typescript
import fs from 'fs';
import { Client } from '@octoai/client';
import fs from "fs";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = 'https://image.octoai.run/generate/sdxl';
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { images } = await client.imageGen.generateSdxl({
prompt: 'A photo of a cute cat astronaut in space',
negative_prompt: 'Blurry photo, distortion, low-res, poor quality',
negativePrompt: 'Blurry photo, distortion, low-res, poor quality',
width: 1024,
height: 1024,
num_images: 1,
numImages: 1,
sampler: 'DDIM',
steps: 30,
cfg_scale: 12,
use_refiner: true,
high_noise_frac: 0.8,
style_preset: 'base',
};

const outputs = await client.infer<any>(endpointUrl, inputs);

outputs.images.forEach((output, i) => {
const buffer = Buffer.from(output.image_b64, 'base64');
fs.writeFileSync(`result${i}.jpg`, buffer);
});
cfgScale: 12,
useRefiner: true,
highNoiseFrac: 0.8,
stylePreset: 'base',
})
```

</CodeGroup>
Original file line number Diff line number Diff line change
@@ -14,10 +14,10 @@ This encompasses image-to-video conversion. Additionally, we offer support for a
**Parameters:**
* `image` (base64 encoded image, required) - Starting point image encoded in base64 string
* `height` (int; optional) - Integer representing the height of video/animation to generate- If not provided, the output height will be inferred from the input 'image', and the closest resolution supported will be chosen.
* `width` (int; optional) - Integer representing the width of video/animation to generate- If not provided, the output width will be inferred from the input 'image', and the closest resolution supported will be chosen.
* `width` (int; optional) - Integer representing the width of video/animation to generate- If not provided, the output width will be inferred from the input 'image', and the closest resolution supported will be chosen.

Supported resolutions are `(w,h): (576, 1024), (1024, 576), (768, 768)`

* `cfg_scale` (float; optional) - Floating-point number representing how closely to adhere to 'image' description- Must be a positive number no greater than 10.0.
* `fps` (int; optional) - How fast the generated frames should play back.
* `steps` (int; optional) - Integer representing how many steps of diffusion to run- Must be greater than 0 and less than or equal to 50.
@@ -51,12 +51,12 @@ curl -X POST "https://image.octoai.run/generate/svd" \

```Python Python
import os
from octoai.clients.video_gen import Engine, VideoGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
video_gen = VideoGenerator(token=os.environ.get("OCTOAI_TOKEN"))
video_gen_response = video_gen.generate(
engine=Engine.SVD,
client = OctoAI(api_key=os.environ.get("OCTOAI_TOKEN"))
video_gen_response = client.image_gen.generate_svd(
image="<BASE_64_STRING>",
steps=25,
cfg_scale=3,
@@ -68,28 +68,27 @@ if __name__ == "__main__":
videos = video_gen_response.videos

for i, image in enumerate(videos):
image.to_file(f"result{i}.mp4")
to_file(image, f"result{i}.mp4")
```
```typescript Typescript
import fs from "fs";
import { Client } from "@octoai/client";
import { OctoAIClient } from "@octoai/sdk";

const client = new Client(OCTOAI_TOKEN);
const endpointUrl = "https://image.octoai.run/generate/svd";
const client = new OctoAIClient({
apiKey: "<OCTOAI_TOKEN>",
});

const inputs = {
const { videos } = await client.imageGen.generateSvd({
"image": "<BASE_64_STRING>",
"steps": 25,
"cfg_scale": 3,
"cfgScale": 3,
"fps": 7,
"motion_scale": 0.5,
"noise_aug_strength": 0.02,
"num_videos": 1
};

const outputs = await client.infer<any>(endpointUrl, inputs);
"motionScale": 0.5,
"noiseAugStrength": 0.02,
"numVideos": 1
});

outputs.videos.forEach((output, i) => {
videos.forEach((output, i) => {
const buffer = Buffer.from(output.video, 'base64');
fs.writeFileSync(`result${i}.mp4`, buffer);
});
Original file line number Diff line number Diff line change
@@ -11,7 +11,7 @@ OctoAI empowers you to customize images by leveraging assets like checkpoints, L

This tutorial explains how to upload your own private assets to the Asset Library.

1. First download the OctoAI CLI by following the instructions in [CLI Installation.](/bring-your-own-model/cli-and-sdk-installation) Check that it is properly installed by running the following in your terminal:
1. First download the OctoAI CLI by following the instructions in [CLI Installation.](/cli/cli-and-sdk-installation) Check that it is properly installed by running the following in your terminal:

```bash bash
$ octoai asset --help
@@ -58,7 +58,6 @@ octoai asset create \
--data-type fp16 \
--type checkpoint \
--description "Dreamshaper v7"
--public false
```

You can alternatively upload the file via public URL using `upload-from-url`:
@@ -72,5 +71,4 @@ octoai asset create \
--data-type fp16 \
--type checkpoint \
--description "Dreamshaper v7"
--public false
```
Original file line number Diff line number Diff line change
@@ -36,8 +36,8 @@
"url": "api-reference"
},
{
"name": "Tutorials",
"url": "tutorials"
"name": "Integrations",
"url": "integrations"
},
{
"name": "Release Notes",
@@ -162,7 +162,7 @@
"private-deployment/octostack",
"private-deployment/secure-link"
]
},
},
{
"group": "CLI",
"pages": [
@@ -174,38 +174,25 @@
"pages": [
"python-sdk/installation-and-setup",
"python-sdk/python-sdk-inferences",
"python-sdk/python-sdk-reference"
"python-sdk/upgrading-from-old-sdk"
]
},
{
"group": "TypeScript SDK",
"pages": [
"typescript-sdk/installation-and-setup",
"typescript-sdk/typescript-sdk-inferences",
"typescript-sdk/typescript-sdk-reference"
"typescript-sdk/upgrading-from-old-sdk"
]
},
{
"group": "Tutorials",
"group": "Integrations",
"pages": [
"tutorials/image-generation-application-e2e-example",
{
"group": "LLM Q&A applications using LangChain",
"pages": [
"tutorials/llm-q-and-a-applications-using-langchain/overview",
"tutorials/llm-q-and-a-applications-using-langchain/python-wrapper-for-langchain",
"tutorials/llm-q-and-a-applications-using-langchain/environment-setup",
"tutorials/llm-q-and-a-applications-using-langchain/create-a-generic-chat-app",
"tutorials/llm-q-and-a-applications-using-langchain/create-q-and-a-app-on-custom-pdf"
]
},
{
"group": "Build a Movie Chatbot App",
"pages": [
"tutorials/building-a-movie-chatbot-app/prerequisites-and-steps-to-build-movie-chatbot-app",
"tutorials/building-a-movie-chatbot-app/building-a-movie-chatbot-app/movie-chatbot-csv-file"
]
}
"integrations/overview",
"integrations/langchain",
"integrations/pinecone",
"integrations/unstructured",
"integrations/openrouter",
"integrations/llamaindex"
]
},
{
@@ -333,7 +320,7 @@
"group": "2024",
"pages": [
"release-notes/2024/march",
"release-notes/2024/february",
"release-notes/2024/february",
"release-notes/2024/january"]
},
{
@@ -365,6 +352,30 @@
{
"source": "/tutorials/automatic1111-stable-diffusion-web-ui",
"destination": "/docs/getting-started/quickstart"
},
{
"source": "tutorials/image-generation-application-e2e-example",
"destination": "/docs/getting-started/quickstart"
},
{
"source": "tutorials/llm-q-and-a-applications-using-langchain/overview",
"destination": "/docs/getting-started/quickstart"
},
{
"source": "tutorials/llm-q-and-a-applications-using-langchain/python-wrapper-for-langchain",
"destination": "/docs/getting-started/quickstart"
},
{
"source": "tutorials/llm-q-and-a-applications-using-langchain/environment-setup",
"destination": "/docs/getting-started/quickstart"
},
{
"source": "tutorials/llm-q-and-a-applications-using-langchain/create-a-generic-chat-app",
"destination": "/docs/getting-started/quickstart"
},
{
"source": "tutorials/llm-q-and-a-applications-using-langchain/create-q-and-a-app-on-custom-pdf",
"destination": "/docs/getting-started/quickstart"
}
]
}
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ description: "Private networking with OctoAI's SecureLink."

# Overview

Keeping our users’ data private and secure is our priority. OctoAI requires token authentication for all API requests, along with TLS to enforce encryption in transit for all connections between the customer and OctoAI. We also use encryption at rest for any data written to disk.
Keeping our users’ data private and secure is our priority. OctoAI requires token authentication for all API requests, along with TLS to enforce encryption in transit for all connections between the customer and OctoAI. We also use encryption at rest for any data written to disk.

SecureLink is an additional private connectivity security measure, ensuring that network traffic between an OctoAI endpoint and the customer environment is not exposed to the public internet. SecureLink is available for Enterprise customers.

@@ -36,16 +36,16 @@ Now you’ll create the VPC Interface Endpoint in your AWS account using the VPC

![](https://www.datocms-assets.com/45680/1709312020-securelink1.png?max-w=2000&auto=compress)

Configure the Service Name value to `com.amazonaws.vpce.us-east-1.vpce-svc-0e914445c09bbe700`, then click `Verify` to ensure the service name is found and verified. Contact us for help if the service name is not found.
Configure the Service Name value to `com.amazonaws.vpce.us-east-1.vpce-svc-0e914445c09bbe700`, then click `Verify` to ensure the service name is found and verified. Contact us for help if the service name is not found.

![](https://www.datocms-assets.com/45680/1709312036-securelink2.png?max-w=2000&auto=compress)


Next, choose the VPC and subnets that should be peered with the VPC service endpoint. Make sure that Enable DNS name is checked.
Next, choose the VPC and subnets that should be peered with the VPC service endpoint. Make sure that Enable DNS name is checked.

![](https://www.datocms-assets.com/45680/1709312044-securelink3.png?max-w=2000&auto=compress)

Then, choose the security group(s) who can send traffic to the VPC endpoint. The security group must accept inbound traffic on TCP port 443 - you can verify this within the Inbound Rules page. You can now click `Create endpoint` to create the VPC endpoint. The endpoint maybe take up to 10 minutes to move from Pending to Available. Once it shows Available, it’s ready for use.
Then, choose the security group(s) who can send traffic to the VPC endpoint. The security group must accept inbound traffic on TCP port 443 - you can verify this within the Inbound Rules page. You can now click `Create endpoint` to create the VPC endpoint. The endpoint maybe take up to 10 minutes to move from Pending to Available. Once it shows Available, it’s ready for use.

**Configure OctoAI’s SDKs & CLI to use SecureLink URL**

@@ -70,22 +70,16 @@ const client = new Client(token, true);

*Python SDK*

- For text generation, fine-tuning, or asset library, configure `secure_link=True` in the client instantiation:
- For text generation, fine-tuning, or asset library, configure the `environment` parameter to use `OctoAIEnvironment.SECURE_LINK` in the client instantiation:

```
from octoai.client import Client
token=os.environ.get("OCTOAI_TOKEN")
client = Client(token, secure_link=True)
```
```python
import os

- For image generation, configure the `api_endpoint` in the client instantiation :
from octoai.client import OctoAI
from octoai.environment import OctoAIEnvironment
token=os.environ.get("OCTOAI_TOKEN")

```
client = ImageGenerator(
api_endpoint="https://image.securelink.octo.ai/",
token=os.environ.get("OCTOAI_TOKEN"),
)
client = OctoAI(api_key=token, environment=OctoAIEnvironment.SECURE_LINK)
```

This table summarizes the SecureLink equivalent to each public API URL:
@@ -95,7 +89,7 @@ This table summarizes the SecureLink equivalent to each public API URL:
| Text generation | `https://text.octoai.run` | `https://text.securelink.octo.ai` |
| Image generation | `https://image.octoai.run` | `https://image.securelink.octo.ai` |
| Asset Library & Fine-tuning | `https://api.octoai.cloud` | `https://api.securelink.octo.ai` |
| OctoAI API | `https://api.octoai.cloud` | `https://api.securelink.octo.ai` |
| OctoAI API | `https://api.octoai.cloud` | `https://api.securelink.octo.ai` |
| Async Inference | `https://async.octoai.run` | `https://async.securelink.octoai.run` |

**Configure private connection for Amazon S3 to upload assets through a private connection**
Original file line number Diff line number Diff line change
@@ -4,81 +4,81 @@ sidebarTitle: "Asset Library Python client"
description: "Manage assets using the Python SDK."
---

The AssetOrchestrator client in the Python SDK allows create, list, get, and delete actions of assets. These assets allow integration with the [ImageGenerator Client](/python-sdk/image-generator-client) to generate more customized images.
The AssetLibrary client in the Python SDK allows create, list, get, and delete actions of assets. These assets allow integration with the [ImageGenerator Client](/python-sdk/image-generator-client) to generate more customized images.

For a quick glance at features for the AssetOrchestrator Client, please see the [AssetOrchestrator reference docs](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.asset%5Forch.AssetOrchestrator). This guide will walk you through using this API to see a list of our public assets, create your own asset, and use your asset to generate an image.
This guide will walk you through using this API to see a list of our public assets, create your own asset, and use your asset to generate an image.

#### Requirements

* First, [create an OctoAI API token.](/getting-started/how-to-create-an-octoai-access-token)
* Then, complete [Python SDK Installation & Setup.](/python-sdk/installation-and-setup).
* If you use the `OCTOAI_TOKEN` envvar for your token, you can instantiate the asset\_orch client with `asset_orch = AssetOrchestrator()`
* Then, complete [Python SDK Installation & Setup.](/python-sdk/installation-and-setup).
* If you use the `OCTOAI_TOKEN` envvar for your token, you can instantiate the asset\_orch client with `asset_library = AssetLibrary()`

#### Overview of AssetOrchestrator API
#### Overview of AssetLibrary API

```Python Python
from octoai.clients.asset_orch import AssetOrchestrator
from octoai.client import OctoAI

if __name__ == "__main__":
# If you have an OCTOAI_TOKEN set as an environment variable, you do not need to pass a token.
# If one is not set, you can use:
# asset_orch = AssetOrchestrator(token="your OctoAI API Token goes here")
asset_orch = AssetOrchestrator()
# asset_library = AssetLibrary(token="your OctoAI API Token goes here")
client = OctoAI()
asset_library = client.asset_library
# You can get a list of the public OctoAI assets
print(asset_orch.list(is_public=True, owner="octoai"))
print(asset_library.list(is_public=True, owner="octoai"))
# You can get a specific asset, either one you created or in this example an OctoAI asset
asset = asset_orch.get(is_public=True, owner="octoai", name="product_photography_v1")
asset = asset_library.get("octoai:product_photography_v1")
# And also create, delete, or use those assets to generate images as in the below example.
asset_orch.delete("asset_id_goes_here")
asset_library.delete("asset_id_goes_here")

```

#### Creating a LoRA and Generating an Image

You will need a `safetensors` file in order to use this example, and in our case one is named `origami-paper.safetensors`. I'll be using a lora trained on origami that I can use with the words "origami" and "paper".

In this example, we will be adding a LoRA then using it to generate an image. You can also add [checkpoints](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.asset%5Forch.CheckpointData), [vae](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.asset%5Forch.VAEData), and [textual inversions](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.asset%5Forch.TextualInversionData).

You can check the reference guide for more information about the returned [Asset](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.asset%5Forch.Asset) object or options for the [create](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.asset%5Forch.AssetOrchestrator.create) method.
In this example, we will be adding a LoRA then using it to generate an image. You can also add `checkpoints`, `vae`, and `textual inversions`.

```Python Python
from octoai.clients.asset_orch import LoraData, AssetOrchestrator
from octoai.clients.image_gen import ImageGenerator
from octoai.client import OctoAI
from octoai.asset_library import Data_Lora
from octoai.util import to_file

if __name__ == "__main__":
# OCTOAI_TOKEN set as an environment variable so do not need to pass a token.
asset_orch = AssetOrchestrator()
image_gen = ImageGenerator()

client = OctoAI()
asset_library = client.asset_library
image_gen = client.image_gen

asset_name = "origami-paper-test"
# There is also TextualInversionData, VAEData, and CheckpointData.
lora_data = LoraData(
lora_data = Data_Lora(
data_type="fp16",
engine="image/stable-diffusion-v1-5",
file_format="safetensors",
)

asset = asset_orch.create(
asset = asset_library.create_from_file(
file="origami-paper.safetensors",
data=lora_data,
name=asset_name,
description="origami-paper stable diffusion 1.5",
)

image_gen_resp = image_gen.generate(
engine="sd",
image_gen_resp = image_gen.generate_sd(
prompt="rainbow origami tailong dragon",
num_images=4,
loras={asset: 0.8}
loras={"asset": 0.8}
)

# Some images can be removed for safety.
# Please see the ImageGenerator client docs for more information.
for image in image_gen_resp.images:
image.to_pil().show()
for i, image in enumerate(image_gen_resp.images):
to_file(image, f"result{i}.jpg")

# You can clean up your asset with the following:
asset_orch.delete(asset.id)
asset_library.delete(asset.id)
```

![astropus.png](https://www.datocms-assets.com/45680/1703720706-ai-generated-rainbow-origami-tailong-dragon.png?max-w=2000&auto=compress)
@@ -103,11 +103,12 @@ In this example, there is a directory named `images` that contains files with a

```Python Python
import os
from octoai.clients.asset_orch import AssetOrchestrator, FileData
from octoai.client import OctoAI
from octoai.asset_library import Data_File

if __name__ == "__main__":
# OCTOAI_TOKEN set as an environment variable so do not need to pass a token.
asset_orch = AssetOrchestrator()
client = OctoAI()

dir_path = "./assets/images/" # Set your dir_path here to your file assets.
files = []
@@ -120,26 +121,26 @@ if __name__ == "__main__":
split_file_name = file.split(".")
asset_name = split_file_name[0]
file_format = split_file_name[1]
file_data = FileData(
file_data = Data_File(
file_format=file_format,
)
asset = asset_orch.create(
asset = client.asset_library.create_from_file(
file=dir_path + file,
data=file_data,
name=asset_name,
)
```

You can then use `asset_orch.list()` to see the assets have been created and uploaded and a result that looks something like:
You can then use `octoai.asset_library.list()` to see the assets have been created and uploaded and a result that looks something like:

```
[
id: asset_01234567891011121314151617, name: save_the_paper_poodle, status: ready,
id: asset_01234567891011121314151618, name: save_the_other_paper_poodle, status: ready,
id: asset_01234567891011121314151619, name: result2, status: ready,
id: asset_01234567891011121314151620, name: result3, status: ready,
id: asset_01234567891011121314151621, name: result1, status: ready,
id: asset_01234567891011121314151622, name: result0, status: ready,
id: asset_01234567891011121314151617, name: save_the_paper_poodle, status: ready,
id: asset_01234567891011121314151618, name: save_the_other_paper_poodle, status: ready,
id: asset_01234567891011121314151619, name: result2, status: ready,
id: asset_01234567891011121314151620, name: result3, status: ready,
id: asset_01234567891011121314151621, name: result1, status: ready,
id: asset_01234567891011121314151622, name: result0, status: ready,
id: asset_01234567891011121314151600, name: origami-paper-test, status: uploaded]
```

Original file line number Diff line number Diff line change
@@ -2,35 +2,36 @@
title: "Image Generator Python client"
---

For a quick glance at features for the Image Generation Client, please see the [Image Generator](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#module-octoai.clients.image%5Fgen) reference.

The `ImageGenerator` class specializes in supporting image generation in your application, and guiding what options are available to modify your outputs. It will return a list of all images using the [Image](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.types.Image) type. It allows you to use both Stable Diffusion 1.5 and Stable Diffusion XL for text to image and image to image use cases, and set parameters and prompts either with weighted prompts with the `prompt` field as was common with Stable Diffusion 1.5 or human-readable descriptions using `prompt_2` with Stable Diffusion XL 1.0.
The `ImageGenClient` class specializes in supporting image generation in your application, and guiding what options are available to modify your outputs. It will return a list of all images using the `ImageGeneration` type. It allows you to use both Stable Diffusion 1.5 and Stable Diffusion XL for text to image and image to image use cases, and set parameters and prompts either with weighted prompts with the `prompt` field as was common with Stable Diffusion 1.5 or human-readable descriptions using `prompt_2` with Stable Diffusion XL 1.0.

This guide will walk you through a text to image example, and then we will use the resulting image to demonstrate the image to image use case.

#### Requirements

* First, [create an OctoAI API token.](/getting-started/how-to-create-an-octoai-access-token)
* Then, complete [Python SDK Installation & Setup.](/python-sdk/installation-and-setup).
* If you use the `OCTOAI_TOKEN` envvar for your token, you can instantiate the image\_gen client with `client = ImageGenerator()`
* Then, complete [Python SDK Installation & Setup.](/python-sdk/installation-and-setup).
* If you use the `OCTOAI_TOKEN` envvar for your token, you can instantiate the image\_gen client with `client = OctoAI().image_gen`

#### Simple Text to Image Generation Example

```Python Python
from octoai.clients.image_gen import ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
# If OCTOAI_TOKEN is not set as an envvar, you can also pass a token to the client with:
# If OCTOAI_TOKEN is not set as an envvar, you can also pass a token to the client with:
# ImageGenerator(token="YOUR_TOKEN_HERE")
client = ImageGenerator()
client = OctoAI()
# images is a list of Images from octoai.types
image_resp = client.generate(engine="sdxl", prompt="photorealistic, poodle, intricately detailed")
image_resp = client.image_gen.generate_sdxl(
prompt="photorealistic, poodle, intricately detailed"
)
images = image_resp.images

# images can be filtered for safety, so since we only generated 1 image by default, this verifies
# we actually have an image to show.
if image_resp.removed_for_safety == 0:
images[0].to_pil().show()
if not images[0].removed_for_safety:
to_file(images[0], "output.jpg")
```

After running this simple prompt, you should hopefully have an output somewhat similar to the image below:
@@ -41,17 +42,19 @@ A good start and in our next example, we'll use more features to help guide our

#### Text to Image Generation Example

For a full list of features available, please see the [ImageGenerator.generate() reference](https://octoml.github.io/octoai-python-sdk/octoai.clients.html#octoai.clients.image%5Fgen.ImageGenerator). One of the simplest ways to customize your outputs is a style preset, negative\_prompt, loras, and model selection.
One of the simplest ways to customize your outputs is a style preset, negative\_prompt, loras, and model selection.

```Python Python
from octoai.clients.image_gen import ImageGenerator
from octoai.util import to_file
from octoai.client import OctoAI

if __name__ == "__main__":
client = OctoAI()

prompt = "photorealistic, colorful, poodle, intricately detailed"
file_name = "pretty_poodle_cinematic.jpeg"

client = ImageGenerator()
images_resp = client.generate(engine="sdxl",
images_resp = client.image_gen.generate_sdxl(
prompt=prompt,
negative_prompt="horror, scary, low-quality, extra limbs, cartoon",
checkpoint="crystal-clear",
@@ -60,10 +63,10 @@ if __name__ == "__main__":
steps=50,
)
images = images_resp.images
# It can also be helpful to run another generate method with
# It can also be helpful to run another generate method with
# num_images = image_resp.removed_for_safety to get your desired total images
if images_resp.removed_for_safety == 0:
images[0].to_file(file_name)
if not images[0].removed_for_safety:
to_file(images[0], file_name)

```

@@ -76,21 +79,22 @@ Much more realistic! Now that we have our cinematic poodle, let's go ahead and u
Image to Image Generation lets you use a base image, in our case the above `pretty_poodle.jpeg` to shape the feel of your outputs image. In our case, we'd expect some focal point in the center, and a blurred, bright background, but otherwise our output can look anywhere from completely different or quite similar depending on our prompt. In this case, let's go for a complete different style of outputs and stray from the usual theme of poodles to a corgi in the rain.

```Python Python
from octoai.clients.image_gen import ImageGenerator
from octoai.types import Image
from octoai.util import to_file, from_file
from octoai.client import OctoAI

if __name__ == "__main__":
client = ImageGenerator()
init = Image.from_file("pretty_poodle_cinematic.jpeg")
images_resp = client.generate(
engine="sdxl",
prompt="corgi in the rain",
init_image=init, # Only used for image-to-image
strength=0.8, # Only used for image-to-image
style_preset="anime"
client = OctoAI()

init = from_file("pretty_poodle_cinematic.jpeg")
images_resp = client.image_gen.generate_sdxl(
prompt="corgi in the rain",
init_image=init, # Only used for image-to-image
strength=0.8, # Only used for image-to-image
style_preset="anime"
)
images = images_resp.images
images[0].to_file("rain_corgi.jpeg")

to_file(images[0], "rain_corgi.jpeg")
```

![astropus.png](https://www.datocms-assets.com/45680/1703719904-ai-generated-rain_corgi.jpeg?max-w=2000&auto=compress)
Original file line number Diff line number Diff line change
@@ -7,19 +7,19 @@ sidebarTitle: "Python SDK installation"

The SDK currently supports Python versions 3.8.1 and upwards. It is strongly recommended that you use a virtual environment such as [Conda](https://conda.io/projects/conda/en/latest/index.html) or [venv](https://docs.python.org/3/library/venv.html) to manage Python packages for your development environment. This helps prevent incompatible dependencies with packages installed irrelevant to your current project or that conflict with system dependencies.

You can view our full release history on [PyPi](https://pypi.org/project/octoai-sdk/#history) for the latest version of the Python SDK.
You can view our full release history on [PyPi](https://pypi.org/project/octoai/#history) for the latest version of the Python SDK.

For Mac and Linux, use pip to install the octoai-sdk. Python package. We strongly recommend that you install OctoAI in a virtualenv, to avoid conflicting with your system packages.
For Mac and Linux, use pip to install the `octoai` sdk Python package. We strongly recommend that you install OctoAI in a virtualenv, to avoid conflicting with your system packages.

```shell shell
python3 -m pip install octoai-sdk
python3 -m pip install octoai
```

Please refer to [CLI & SDK Installation](/bring-your-own-model/cli-and-sdk-installation) for how to install the SDK in conjunction with the CLI for the authoring tool.

### Setting API token as an environment variable

In order to access endpoints, [create an OctoAI API token](/getting-started/how-to-create-an-octoai-access-token). Set `OCTOAI_TOKEN` to the token value wherever you set your environment variables, such as your `.bashrc` or `.env` file.
In order to access endpoints, [create an OctoAI API token](/getting-started/how-to-create-an-octoai-access-token). Set `OCTOAI_TOKEN` to the token value wherever you set your environment variables, such as your `.bashrc` or `.env` file.

```bash bash
export OCTOAI_TOKEN=YOUR_TOKEN_HERE
@@ -28,15 +28,15 @@ export OCTOAI_TOKEN=YOUR_TOKEN_HERE
Then when you instantiate the client, it will detect the `OCTOAI_TOKEN` as an envvar and set it for you.

```Python Python
from octoai.client import Client
from octoai.client import OctoAI

client = Client()
client = OctoAI()
```

Alternatively, on creation of the [Client](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client) class, you can set your token, or the Client also accepts a path to where you've stored your API token from the `config_path` variable. Please see the [Client](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client) docs for more information.
Alternatively, on creation of the `OctoAI` class, you can set your token, or the Client also accepts a path to where you've stored your API token from the `config_path` variable. Please see the `OctoAI` docs for more information.

```Python Python
from octoai.client import Client
from octoai.client import OctoAI

client = Client(token=YOUR_OCTOAI_API_TOKEN_HERE)
client = OctoAI(api_key="YOUR_OCTOAI_API_TOKEN_HERE")
```
Original file line number Diff line number Diff line change
@@ -4,169 +4,36 @@ title: "Python SDK inference"

### OctoAI Python SDK at a glance

If you need assistance with any specifics for using the OctoAI Python SDK, please see the [Python SDK Reference](https://octoml.github.io/octoai-python-sdk/octoai.html).

The OctoAI Python SDK is intended to help you use OctoAI endpoints. At its simplest form, it allows you to run inferences against an endpoint by providing a dictionary with the necessary inputs.

```Python Python
from octoai.client import Client
import time
from octoai.client import OctoAI

client = Client()
client = OctoAI()

# It allows you to run inferences
output = client.infer(endpoint_url="your-endpoint-url", inputs={"keyword": "dictionary"})

# It also allows for inference streams for LLMs
for token in client.infer_stream("your-endpoint-url", inputs={"keyword": "dictionary"}):
if token.get("object") == "chat.completion.chunk":
# Do stuff with the token
if token.get("object") == "chat.completion.chunk":
# Do stuff with the token
pass

# And for server-side asynchronous inferences
future = client.infer_async("your-endpoint-url", {"keyword": "dictionary"})
# Typically, you'd collect additional futures then poll for status, but for the sake of example...
while not client.is_future_ready(future):
time.sleep(1)
time.sleep(1)
# Once the results are ready, you can use them in the same way as you
# typically do for demo endpoints
result = client.get_future_result(future)

# And includes healthChecks
if client.health_check("your-healthcheck-url") == 200:
# Run some inferences
pass

```

The [infer](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client.infer) and [infer\_stream](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client.infer%5Fstream) methods are synchronous.

#### Example: Whisper Speech Recognition

Whisper is a natural language processing model that converts audio to text. Like with Stable Diffusion, we’ll use the base64 library for encoding an mp3 or a wav file into a base64 string.

```Python Python
from octoai.client import Client
import base64

whisper_url = "https://whisper-demo-kk0powt97tmb.octoai.run/predict"
whisper_health_check = "https://whisper-demo-kk0powt97tmb.octoai.run/healthcheck"

# First, we need to convert an audio file to base64.
file_path = "she_sells_seashells_by_the_sea_shore.wav"
with open(file_path, "rb") as f:
encoded_audio = base64.b64encode(f.read())
base64_string = encoded_audio.decode("utf-8")

# These are the inputs we will send to the endpoint, including the audio base64 string.
inputs = {
"language": "en",
"task": "transcribe",
"audio": base64_string,
}

OCTOAI_TOKEN = "API Token goes here from guide on creating OctoAI API token"
# The client will also identify if OCTOAI_TOKEN is set as an environment variable
# So if you have it set, you can simply use:
# client = Client()
client = Client(token=OCTOAI_TOKEN)
if client.health_check(whisper_health_check) == 200:
outputs = client.infer(endpoint_url=whisper_url, inputs=inputs)
transcription = outputs["transcription"]
assert "She sells seashells by the seashore" in transcription
assert (
"She sells seashells by the seashore"
in outputs["response"]["segments"][0]["text"]
)
```

With this particular test file, we will have “She sells seashells by the sea shore.” printed in our command line.

#### Whisper Outputs

The above `outputs` variable returns JSON in something like the following format.

```json
{
prediction_time_ms: 626.42526,
response: {
segments: [ [Object] ],
word_segments: [
[Object]
]
},
transcription: ' She sells seashells by the seashore.'
}

```

Each `segment` is an object that looks something like:

```json
{
start: 5.553,
end: 8.66,
text: ' She sells seashells by the seashore.',
words: [
{
word: 'She',
start: 5.553,
end: 5.633,
score: 0.945,
speaker: null
},
{
word: 'sells',
start: 5.653,
end: 5.814,
score: 0.328,
speaker: null
},
// etc...
],
speaker: null
}

```

Each `word_segment` is an object that looks something like:

```
{ word: 'She', start: 0.010, end: 0.093, score: 0.883, speaker: null }
```

### Python SDK asynchronous inference

The asynchronous inference API addresses longer inferences so you can so you can provide responses faster to clients. The inference data is stored for 24 hours and is then deleted. This can be used simply in the Python SDK due to it managing your headers and also authentication, as well as providing helper methods to manage the responses received from the server.

```Python Python
from octoai.client import Client
from octoai.types import Audio
import time

audio_base64 = Audio.from_file("she_sells_seashells.wav").audio_b64 # put your file location here
inputs = {"language": "en", "task": "transcribe", "audio": audio_base64}
OCTOAI_API_TOKEN = "API Token goes here from guide on creating OctoAI API token"
# The client will also identify if OCTOAI_TOKEN is set as an environment variable
client = Client(token=OCTOAI_API_TOKEN)

whisper_url = "https://whisper-demo-kk0powt97tmb.octoai.run/predict"
whisper_health_check = "https://whisper-demo-kk0powt97tmb.octoai.run/healthcheck"

# First, you can verify the endpoint is healthy.
if client.health_check(whisper_health_check) == 200:
future = client.infer_async(whisper_url, inputs)

# Typically, you'd collect additional futures then poll for status,
# but for the sake of example...
while not client.is_future_ready(future):
time.sleep(1)

# Once the results are ready, you can use them in the same way as you
# typically do for demo endpoints
result = client.get_future_result(future)

assert (
"She sells seashells by the seashore"
in result["response"]["segments"][0]["text"]
)
```

The pattern of creating a future with the same URL and inputs is the same regardless of the endpoint you're using. If you merge these steps with [client.infer\_async](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client.infer%5Fasync) and [client.is\_future\_ready](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client.is%5Ffuture%5Fready) as well as [client.get\_future\_result](https://octoml.github.io/octoai-python-sdk/octoai.html#octoai.client.Client.get%5Ffuture%5Fresult), all endpoints can be used asynchronously on the server side, allowing you to collect futures and poll for when one is ready then surface those results.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: "Upgrading from the octoai-sdk"
sidebarTitle: "Upgrading from octoai-sdk"
---

### Uninstall octoai-sdk

```bash
pip uninstall octoai-sdk
pip install octoai
```

### Upgrading your code

The various OctoAI APIs are now accessable from a single client:

```python
from octoai.client import OctoAI

client = OctoAI()

# The various APIs are now accessible from the client
client.text_gen

client.image_gen

client.fine_tuning

client.asset_library
```

### Image Generation API changes

In the `octoai.image_gen` API instead of specifying which
engine to use, use the corresponding `generate_*` method.

### Text Generation API changes

The text generation models with `octoai.text_gen` API are now specified with a `str` model name instead of an `enum`.

Streaming requests are made with the corresponding `octoai.text_gen.*_stream` method.

### Asset Library (Asset Orchestrator) API changes

The Asset Orchestrator has been renamed to Asset Library and can be accessed via `octoai.asset_library`. Assets can be created with the `octoai.asset_lirbary.create_from_file`.

### `octoai.service` API changes

The `octoai.service` API has been removed. You can make inferences to compute service endpoints via `OctoAI` client:

```python
from octoai.client import OctoAI

client = OctoAI()

# octoai.infer()
```

However you will need to continue to use the older `octoai-sdk` for the full `octoai.service` API which includes service authoring.
Loading

0 comments on commit 7fd61d0

Please sign in to comment.