← Multiplayer Game System handbook

Companion AI: setup, providers, and extension guide

Implemented scope

Plugins/NiceCompanionAI supplies a Blueprint-callable GameInstance subsystem for server-only text requests. It works independently of CMC/Mover and falls back to a localized fixed guidance line without a gateway. Services/CompanionAI is a separate private Python HTTP service with provider adapters. This is the transport foundation for the future NPC plugin, not a complete companion actor, chat UI, voice system, or faction/command implementation.

The Unreal method currently sends a single user message per request. The gateway contract supports bounded conversation history for future NPC sessions. There is no persistent AI memory yet, no automatic gameplay context retrieval, and no model selection UI. Selecting a provider uses server environment configuration and a gateway restart; Unreal does not need to be rebuilt.

Provider/model support means text request translation and response parsing. It does not imply every provider feature or every model is supported. Streaming, audio, tools/function calling, embeddings, image input, and managed agents are outside this interface. No live paid/free provider requests were made during development. Request-format fixture tests are not an account-level compatibility guarantee.

Free options

Checked against official documentation on 16 October 2026:

Use a small local instruct model for development or a quota-limited hosted option for a prototype. Do not assume any free cloud tier will cover a released multiplayer game's traffic. The default offline provider uses no model, needs no API account, and returns a fixed guidance message.

Provider coverage

Set COMPANION_PROVIDER to a preset below. Always select an exact currently available model in the provider's console; no cloud model is hard-coded.

Preset Protocol Credential environment variable Official reference
offline fixed guidance none local implementation
ollama OpenAI-compatible chat none for local endpoint Ollama
openai Chat Completions OPENAI_API_KEY OpenAI
anthropic Messages ANTHROPIC_API_KEY Anthropic
gemini generateContent GEMINI_API_KEY Gemini
azure Azure OpenAI v1 chat AZURE_OPENAI_API_KEY Microsoft
bedrock Converse via optional boto3 AWS SDK credential chain / IAM role AWS
cohere v2 Chat COHERE_API_KEY Cohere
groq compatible chat GROQ_API_KEY Groq
mistral compatible chat MISTRAL_API_KEY Mistral
deepseek compatible chat DEEPSEEK_API_KEY DeepSeek
xai compatible chat XAI_API_KEY xAI
openrouter compatible chat OPENROUTER_API_KEY OpenRouter
together compatible chat TOGETHER_API_KEY Together
fireworks compatible chat FIREWORKS_API_KEY Fireworks
compatible configurable compatible chat COMPANION_PROVIDER_API_KEY operator-selected compatible service

Azure requires COMPANION_ENDPOINT=https://YOUR-RESOURCE.openai.azure.com/openai/v1/chat/completions and COMPANION_MODEL set to the deployment name. The native preset uses API-key authentication, not Entra token refresh. Bedrock requires installing boto3 in the gateway's own Python environment, AWS region configuration, IAM permissions, and model access; it does not use a static bearer API key. This does not implement Google Vertex AI IAM or other managed variants of the direct Gemini API.

Start a local gateway

Use Python 3.10 or later. All providers except Bedrock use the standard library, so they require no pip install. Keep Python and provider secrets on the server. Run from the project root in PowerShell:

$env:COMPANION_PROVIDER = 'offline'
# Generate once for this development session. Do not commit or share the value.
$env:COMPANION_GATEWAY_TOKEN = [guid]::NewGuid().ToString('N') + [guid]::NewGuid().ToString('N')
$env:COMPANION_GATEWAY_URL = 'http://127.0.0.1:8765/v1/companion'
python Services/CompanionAI/gateway.py

The gateway stays in the foreground. Start Unreal or the game server from another process that inherits the same COMPANION_GATEWAY_URL and COMPANION_GATEWAY_TOKEN. An already-running editor will not see environment changes made in a different shell. For example, use a second shell with those same values, or start the gateway in a separate terminal process and then launch Unreal from the configured parent shell. Do not write the token into .ini files or Blueprint defaults.

For local Ollama, install and start Ollama separately, pull a model suitable for the machine and license, then set:

$env:COMPANION_PROVIDER = 'ollama'
$env:COMPANION_MODEL = 'YOUR_INSTALLED_MODEL_NAME'
python Services/CompanionAI/gateway.py

For Groq, use the same gateway token and URL, set COMPANION_PROVIDER=groq, set COMPANION_MODEL to an available model ID, and inject GROQ_API_KEY into the gateway process through your secret manager or process environment. For Gemini, use gemini, an available Gemini model ID, and GEMINI_API_KEY. Credentials are never read from HTTP request fields. Restart the gateway after changing provider/model; the game transport stays the same.

Configuration reference

Variable Default / meaning
COMPANION_PROVIDER offline
COMPANION_MODEL empty; required for every model provider
COMPANION_ENDPOINT preset endpoint, except Azure/custom must set one; for Gemini this is the API root
COMPANION_KEY_ENV override the name of the environment variable containing the provider key
COMPANION_TIMEOUT 20 seconds; range 1–60
COMPANION_MAX_TOKENS 256; range 1–2048; reasoning models may spend some budget on reasoning
AWS_REGION us-east-1, Bedrock only
COMPANION_BIND 127.0.0.1, gateway listener
COMPANION_PORT 8765
COMPANION_GATEWAY_TOKEN required by gateway, at least 32 ASCII characters with no whitespace; also present on the trusted Unreal server
COMPANION_GATEWAY_URL read by Unreal; unset means local fixed guidance

Cloud endpoint overrides require HTTPS. Loopback HTTP is allowed for development. Redirects are rejected to avoid forwarding credentials to redirected hosts. There is no automatic retry or switch to a different paid provider. Provider failures return fixed offline guidance with fallback=true; detailed upstream bodies and dialogue are not logged.

Unreal integration

  1. Enable/build NiceCompanionAI (enabled by default). A future C++ NPC module should declare NiceCompanionAI as a module and plugin dependency when using its public API.
  2. In the authoritative NPC/companion interaction handler, validate player ownership, companion permissions, distance, and a per-player cooldown/quota first. Client requests must enter through a player-owned server RPC; do not put an unrestricted RPC on a world NPC.
  3. Get UNiceCompanionAISubsystem from the GameInstance and bind OnReply before calling AskCompanion(Message). The returned GUID correlates the reply with the requesting player/companion. Invalid GUID means rejected before dispatch. Local fallback is deferred to the next world tick.
  4. Maintain a server-side map from GUID to the authorized recipient. Revalidate that the player and companion still exist when a reply arrives. Deliver personal dialogue through an owning-client RPC; replicate party narration only to the relevant party.
  5. OnReply(RequestId, Text, bFallback, bSuccess) reports results. bSuccess=false indicates a transport/response failure despite supplying fallback text. A gateway-generated fallback has bSuccess=true, bFallback=true. Missing gateway settings intentionally select local fallback.
  6. Call CancelRequest when the conversation closes or the owning NPC/player goes away. Cancellation suppresses local delivery; a provider request already accepted upstream may still incur usage. World cleanup during travel and subsystem shutdown cancel pending/deferred delivery. No persistence, gameplay rewards, or inventory mutation is performed by this subsystem.

Standalone, listen server, and dedicated server can call the subsystem. A remote client is rejected by a runtime net-mode check as well as Blueprint authority metadata. The subsystem also applies a 0.5-second instance-wide request interval and permits at most four pending/deferred requests; this is overload protection, not a per-player billing policy. NPC ownership routing and UI are not implemented here yet.

Gateway contract

POST /v1/companion, Authorization: Bearer <gateway token>, JSON body:

{"messages":[{"role":"user","content":"How do I equip armor?"}]}

Only user and assistant history roles are accepted. The system prompt is owned by the gateway. Requests cannot choose a model, endpoint, API key, token limit, or system instructions. Limits: 32 KiB body, 16 history entries, 2,000 characters per entry, 12,000 total content characters, final entry must be from the user, four active generations. The gateway returns:

{"text":"Open your inventory and choose the Armor tab.","provider":"configured-provider","fallback":false}

The example text illustrates the contract; it is not a claim that the inventory UI already exists. Responses are bounded to 8,000 characters. Thinking blocks are omitted. No response field executes a command.

Add a provider

For an OpenAI-compatible text endpoint, either use compatible with COMPANION_ENDPOINT, COMPANION_MODEL, and COMPANION_PROVIDER_API_KEY, or add a named entry to PRESETS in providers.py:

"my_provider": (ChatCompletions, "https://api.example.com/v1/chat/completions", "MY_PROVIDER_API_KEY"),

Check the provider's official API documentation, including token parameter names and response differences. The shared adapter uses max_completion_tokens for OpenAI/Azure and max_tokens elsewhere; a provider that differs needs its own adapter.

For a native protocol:

  1. Subclass Adapter in providers.py and implement complete(config, system, messages, send=post_json).
  2. Resolve secrets through self.key(config) or a server SDK credential chain. Never accept credentials or an endpoint from player requests.
  3. Translate the normalized system string and {role, content} messages into that protocol. Apply config.max_tokens and config.timeout.
  4. Call the injected send(url, headers, payload, timeout) for HTTP APIs. SDK adapters must explicitly disable uncontrolled retries and use bounded timeouts.
  5. Parse only visible text and return Reply(text, config.provider). Reject empty/malformed responses. Raise ProviderError with a short safe code on failure; never include request payloads, authorization headers, or raw provider responses in error strings.
  6. Register (YourAdapter, default_endpoint, default_key_environment_name) in PRESETS. Keep all model IDs operator-configurable.
  7. Add tests with an injected fake send. Cover exact headers/body, multi-block text, empty/malformed replies, errors, timeout behavior, and missing credentials. Add a documentation row and setup example.
  8. Run contract tests and then perform an explicitly configured live smoke test with your account and chosen model before deploying it. Verify quota/billing and data terms. Adding an adapter does not require changing character, quest, inventory, or NPC movement code.

Deployment boundary and remaining work

The supplied standard-library HTTP server is a private development service, not a public player endpoint. For production use a managed reverse proxy/service with TLS, restricted inbound access, request-rate and body-size enforcement, connection limits, per-player quotas, and observability without raw dialogue. The four-generation limit does not bound the development HTTP server's total connection threads.

Dedicated servers can authenticate to a private gateway using a server-managed secret. For player-hosted games, do not distribute a publisher master gateway token to listen-server users. Use a future backend that validates EOS identities/session tickets and issues short-lived, scoped credentials, or let hosts use their own local model/provider account. EOS-authenticated AI access is not implemented in this slice. No EOS portal changes have been made.

The NPC plugin now supplies companion conversation UI, an owner-only bounded recent conversation and offline gameplay hints; see VehiclesAndNPCs.md. Rich story context, spoiler policy, moderation and per-player cost limits remain production integration tasks. The fixed fallback can later become a curated localized hint catalog. Persistent character progression and shared party story state remain separate from provider conversation data.

Verification

From the project root:

python -m unittest discover -s Services/CompanionAI -p 'test_*.py' -v

Tests cover each adapter family and compatible preset, credential failure, malformed replies, input constraints, endpoint transport, redirect rejection, HTTP authentication, endpoint injection rejection, route handling, and offline fallback. They use fixtures and loopback networking only. Unreal build status and remaining multiplayer tests are recorded in the architecture document.