Using vision models on SambaNova Cloud, users can process multimodal inputs consisting of text and images. These models understand and analyze images to then generate text based on the context. Learn how to query SambaNova Cloud’s vision models using OpenAI’s Python client.

​
Make a query with an image

With the Cloud API’s OpenAI Compatibility, the vision model request follows OpenAI’s multimodal input format. The model receives both text and an image in a structured request. Making a call with a vision model is similar to the call for Text Generation, but differs by passing in the encoded the image through its file path, represented by the variable image_path. A helper function converts the path into base64 string by so that it can be passed to the model along with text.
1

Step 1

Make a new Python file and copy the code below.;
This example uses a deprecated model, Llama-3.2-11B-Vision-Instruct.
import openai
import base64

client = openai.OpenAI(
    base_url="https://api.sambanova.ai/v1",
    api_key=<"YOUR API KEY">,
)

# Helper function to encode the image
def encode_image(image_path):
  with open(image_path, "rb") as image_file:
    return base64.b64encode(image_file.read()).decode('utf-8')

# The path to your image
image_path = "sample.JPEG"

# The base64 string of the image
image_base64 = encode_image(image_path)

print(image_base64)

response = client.chat.completions.create(
    model="Llama-3.2-11B-Vision-Instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}
            ]
        }
    ]
)

print(response)
2

Step 2

Use your SambaNova API key to replace <"YOUR API KEY">in the construction of the client.
3

Step 3

Select an image and move it to a suitable path that you can specify in the lines.
# The path to your image
image_path = "sample.JPEG"
4

Step 4

Verify the prompt to pair with the image in the content portion of the user prompt.
5

Step 5

Run the Python file to receive the text output.;
In this example, the "sample.JPEG" had contained an image of hiking, and your response will reflect your selected image.

​
Response example

This example uses a deprecated model, Llama-3.2-90B-Vision-Instruct.
{
  "id": "chatcmpl-456",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "Llama-3.2-90B-Vision-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "This image shows a group of people hiking in the mountains during sunset. The scene is surrounded by lush greenery and a golden sky."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
   "prompt_tokens": 18,
    "completion_tokens": 176,
    "total_tokens": 194,
    "completion_tokens_per_sec": 70.03654436130857,
    "completion_tokens_after_first_per_sec": 88.00019470257922,
    "completion_tokens_after_first_per_sec_first_ten": 92.49063822514935,
    "time_to_first_token": 0.5243418216705322,
    "total_latency": 2.5129737854003906,
    "total_tokens_per_sec": 77.19937276189695
  }
}

​