OpenAI client

This notebook demonstrates how to instantiate the OpenAI object. OpenAI is a subclass of the BaseLLMClient Abstract class.

The OpenAI Client works using an OpenAI compatible server. An OpenAI-compatible server refers to an inference server designed to host and serve large language models (LLMs) while exposing an API endpoint that mimics the structure and functionality of OpenAI’s official API.

An example of such servers are vLLM or Ollama.

For more thorough look at OpenAIClient’s structure, look to the OpenAIClient or BaseLLMClient API reference.

Setting up the Server

Before the Client can be instantiated the server the open source LLM will be hosted on must be setup for the Client to function and be able to communicate with the desired LLM.

There are various ways to instantiate the server depending on which application is being used to create it. The recommended method is using vLLM, though for smaller use cases Ollama can be used.

For a more thorough explanation of server usage look to their respective documentation, linked above, for basic usage however an example will be provided.

Run these command in a Linux terminal to set up vLLM server for OpenAIClient

Installation:

uv pip install vllm

vLLM OpenAI-compatible server setup:

vllm serve "Qwen/Qwen3-8B"

Once the server is set up, the server by default starts at http://localhost:8000

Setting up the Client

To set up the OpenAI Client there are 2 required parameters, the server Base URL, which is set up in the server setup. As well as the desired open source model (this should be the same model as given in the server setup).

from ai_sentinel import OpenAIClient

# Initialize an Open Source LLM Client
client = OpenAIClient(
    base_url="http://localhost:8000/v1",
    model="Qwen/Qwen3-8B"
)

Once the Client is set up, all further usage in the code will be with the ToxicityGuard

The following is the recommended format to follow for key naming convention when working with Google Gemini

GEMINI_API_KEY='YOUR_API_KEY'