# Shared AI providers — SDK 1.8.0

Hackemist supplies a server-side AI gateway. The operator configures provider credentials and assigns a provider/model to each role; apps call `Hackemist.ai` without handling those credentials. Provider settings can be changed centrally without rebuilding integrated apps. Vision, transcription, moderation, specialist text roles and custom roles are supported alongside generation. Apps must implement their AI feature and call the SDK. Merely enabling a role does not add AI logic to existing code.

## Operator setup

1. Open **Admin → AI providers**. A stable server `SETTINGS_KEY` is required for encrypted credentials and backup recovery.
2. Add a provider ID and display name, select a **Provider preset**, and enter its API key. Presets configure the correct adapter and API base URL. **Custom / other compatible service** allows a manually configured adapter and public HTTPS base URL, for example `https://api.example.com/v1`. Native adapters use their fixed official endpoints. Save a separate entry for each account or service; up to 24 entries are supported.
3. Choose a provider from each role's dropdown and enter its exact model ID. **Load available models** reads the saved provider's model catalog and adds autocomplete suggestions; you can always enter a model ID manually. Select a model that supports the role's input/output contract. Unassigned roles are unavailable. Use **Add custom AI role** for other app features; up to 64 role assignments are supported.
4. Set account, app, site daily request limits, concurrent request limits and the maximum text output tokens. Set separate spending limits with the providers. Hackemist request limits are not monetary budgets.
5. Enable the service and save. **Check saved connection** reads the provider's model-discovery endpoint without requesting generation. This checks connectivity/authentication, not assigned-model access, media generation or billing readiness. Saving never requests generation.
6. The creator enables the roles their app uses in **Studio → App details → AI features**, or through MCP. Players must sign in with a verified email. The trusted player asks for data-sharing consent for each role, once per running session and configuration revision.

| Adapter | Roles | API contract |
| --- | --- | --- |
| OpenAI | general, vision, image, video, speech, transcription, embeddings, moderation | Responses, GPT Image generations, Videos, audio speech/transcription, embeddings, moderation |
| OpenAI-compatible | general, vision, image, embeddings, transcription, moderation | Chat Completions (`max_tokens`), image generations (`b64_json`), float embeddings, standard audio transcriptions/moderations where implemented |
| Anthropic | general, vision | Messages API with text/base64 image input |
| Google Gemini | general, vision, image, embeddings | `generateContent` (including vision/native image models), `embedContent` |
| Hackemist AI gateway protocol | all standard contracts and custom JSON | Operator-owned `/generate`, `/jobs/{id}` and `/models` gateway; supports future services |

Compatibility varies by service and model. Provider presets offer the contracts their adapter implements; they do not claim every model supports every modality. The compatible adapter requires the documented endpoint and response shapes. Use a custom gateway for a new protocol, or add an adapter in `app/ai_providers.py`. Credentials and models are operator settings, not app-controlled fields. Azure deployment routes, arbitrary agent tools, streaming, image edits, video reference media and Gemini Veo are not native adapter features. A custom gateway can implement additional services. Native OpenAI video supplies the built-in video contract.

### Major provider presets

Model IDs are deliberately configurable rather than frozen to a changing list of model releases. A provider's ordinary API account/key is required; a chat subscription or coding-plan credential does not automatically provide general application API access. For regional or dedicated endpoints, use **Custom** with the matching adapter and a newly entered key.

| Preset | Default API base URL | References |
| --- | --- | --- |
| OpenAI | `https://api.openai.com/v1` | [OpenAI API](https://developers.openai.com/api/docs/guides/text) |
| Claude (Anthropic) | `https://api.anthropic.com/v1` | [Messages API](https://platform.claude.com/docs/en/api/messages/create) |
| Gemini (Google) | `https://generativelanguage.googleapis.com/v1beta` | [Gemini API](https://ai.google.dev/api) |
| DeepSeek | `https://api.deepseek.com` | [Official quickstart](https://api-docs.deepseek.com/) |
| Kimi (Moonshot) | `https://api.moonshot.ai/v1` | [Official quickstart](https://platform.moonshot.ai/docs/guide/start-using-kimi-api) |
| GLM (Z.ai) | `https://api.z.ai/api/paas/v4` | [Official quickstart](https://docs.z.ai/guides/overview/quick-start) |
| Grok (xAI) | `https://api.x.ai/v1` | [Chat compatibility](https://docs.x.ai/developers/model-capabilities/legacy/chat-completions) |
| Mistral | `https://api.mistral.ai/v1` | [API compatibility](https://docs.mistral.ai/resources/migration-guides) |
| Groq | `https://api.groq.com/openai/v1` | [API compatibility](https://console.groq.com/docs/openai) |
| OpenRouter | `https://openrouter.ai/api/v1` | [Official quickstart](https://openrouter.ai/docs/quickstart) |
| Together AI | `https://api.together.ai/v1` | [API compatibility](https://docs.together.ai/docs/openai-api-compatibility) |

The compatible presets expose text/vision, Groq additionally exposes the standard transcription contract, and Together additionally exposes embeddings. Other specialized APIs from those companies may use different request formats; use the custom gateway for those until a matching native adapter is added. Model discovery checks model catalog access, not modality support or generation readiness.

Existing keys remain when a saved entry's API key is blank. Changing an entry's adapter or API base URL requires re-entering its key. Remove or disable an entry to stop using it. Provider configuration and project role changes take effect on subsequent requests; stale consent and video job configurations are rejected. Previously dispatched work may still incur provider charges.

## Creator funding

Shared AI uses prepaid app wallets funded by each app's creator. In Admin → AI providers, set each provider's default whole credits per request and optional per-role overrides, then configure priced Stripe or Paystack credit packs. Creators buy credits in Studio → App details → AI features → Manage app AI credits. Requests stop before provider dispatch when an app cannot cover its next request, unless an administrator grants a creator or app funding bypass. Read [credit funding and operation](/docs/ai-credits.md), also `hackemist://docs/ai-credits`.

## App integration

The runtime injects the SDK. Test in the signed-in browser player or interactive Studio preview. Standalone/MCP previews, guest players, offline clients and native clients without an AI bridge cannot use this gateway. Do not put provider keys, MCP credentials, account cookies or launch credentials in a bundle.

Local file selection/editing and remote AI are separate operations. The app chooses which content to submit; the suite does not automatically upload the original document. The request flows from the browser app through the trusted player and suite gateway to the configured provider, then returns to the same browser app. Text/JSON results can be displayed directly; file results can be decoded into browser Blobs for preview, local editing and export. Read the [file-processing and data-flow contract](/docs/hosting.md#user-files-browser-processing-and-ai) before describing an app as browser-only.

```js
const capabilities = await Hackemist.ai.capabilities();
if (!capabilities.roles.general?.funded) {
  showStatus('AI is unavailable. Ask the app creator or site administrator.');
} else {
  // Call from your feature's own UI. The player supplies trusted consent UI.
  const result = await Hackemist.ai.generate({
    role: 'general',
    input: {system: 'Summarize the supplied document faithfully.', prompt: documentText}
  });
  if (result.status === 'completed') showSummary(result.text);
  else if (result.status === 'cancelled') showStatus('AI sharing cancelled.');
}
```

`showStatus`, `showSummary` and `documentText` above belong to your app. Handle rejected promises and keep the user's original document. Render generated text as text or sanitize it; never execute generated code or insert unsanitized HTML. Extract document text locally before submitting it. Use the vision/transcription contracts below when a feature needs image or audio input.

| Role | `input` | Completed result |
| --- | --- | --- |
| `general` | `{prompt, system?}` | `{status:'completed', text, request_id}` |
| `reasoning`, `coding`, `translation`, `document_analysis` | `{prompt, system?}` | Same text contract, with independently assigned providers/models |
| `vision` | `{prompt, system?, images:[{media_type,data_base64}]}` | Text describing/answering questions about the images |
| `image` | `{prompt, size?}` | `files` with one generated image |
| `video` | `{prompt, seconds?, size?}` | Initially `{status:'queued', job_id, retry_after_ms}`; poll for an MP4 |
| `speech` | `{input, voice?}` | `files` with an MP3; label playback as AI-generated speech |
| `embeddings` | `{input}` | `{status:'completed', embedding, request_id}` |
| `transcription` | `{audio:{media_type,data_base64}, language?}` | `{status:'completed', text, request_id}` |
| `moderation` | `{input}` | `{status:'completed', flagged, categories, category_scores, request_id}` |
| custom role ID | The assigned contract's input; custom JSON accepts an object | Normalized text/files/embedding or JSON `output`; can return a background job |

Text fields are limited to 64 KiB of UTF-8 each; speech input is limited to 4096 UTF-8 bytes. Vision accepts one to four PNG/JPEG/WebP images, each up to 4 MiB and at most 8 MiB combined. Transcription accepts WAV/MP3/M4A/WebM/OGG/FLAC audio up to 8 MiB, with an optional two/three-letter lowercase language code. Use strict base64 without a data-URL prefix; remote input URLs are rejected. The complete request must fit within 12 MiB. Text extraction from a PDF/DOCX remains app-owned; use the document-analysis text role or render selected pages into vision images.

Maximum returned text is 1 MiB, media is 64 MiB per file (native speech 16 MiB), and embedding vectors contain at most 32768 finite numbers. Provider JSON responses, including base64, must fit within 24 MiB; raw native video downloads have their separate 64 MiB allowance. Image sizes: `1024x1024` (default), `1024x1536`, `1536x1024`. Gemini uses the corresponding aspect ratio; exact dimensions depend on its model. OpenAI image models must support PNG output. Compatible image services must support `response_format:'b64_json'` or return a public HTTPS image URL.

```js
// imageBase64 contains the selected local image bytes encoded by your app.
const result = await Hackemist.ai.generate({
  role: 'vision',
  input: {prompt: 'Describe this diagram and read its labels.',
          images: [{media_type: 'image/png', data_base64: imageBase64}]}
});
if (result.status === 'completed') showDescription(result.text);
```

Video defaults to 4 seconds, `1280x720`. Supported durations: 4, 8, 12 seconds; sizes: `1280x720`, `720x1280`. Speech defaults to `alloy`; supported voices: alloy, ash, ballad, coral, echo, fable, nova, onyx, sage, shimmer. Individual models can impose narrower limits. See your provider's model documentation.

```js
let result = await Hackemist.ai.generate({
  role: 'video', input: {prompt: 'A paper boat on a quiet pond', seconds: 4}
});
while (result.status === 'queued' || result.status === 'in_progress') {
  await new Promise(resolve => setTimeout(resolve, result.retry_after_ms || 5000));
  result = await Hackemist.ai.getJob(result.job_id);
}
if (result.status === 'completed') {
  const file = result.files[0];
  const bytes = Uint8Array.from(atob(file.data_base64), char => char.charCodeAt(0));
  const blob = new Blob([bytes], {type: file.media_type});
  // You can also preview the Blob locally and revoke its object URL afterwards.
  await Hackemist.downloadFile({filename: file.filename, mediaType: file.media_type, data: blob});
}
```

Use polling with a cancel/stop control in real UI. Stop when the app closes and handle `failed`, `cancelled` and rejected requests. Poll no faster than once every five seconds. Video and custom gateway job receipts are bound to the exact launch/account and expire after one day; provider content can expire sooner. Download completed work promptly. Polling an existing receipt never creates a new generation. Repeatedly retrieving a completed job refetches its content and is rate limited.

## Custom roles and future modalities

Create a named role such as `ocr`, `research`, `music_generation`, `video_analysis` or `three_d_generation` in Admin. Role IDs use 1–48 lowercase letters, digits, underscores/hyphens and start with a letter. The role has a label, provider/model and an input/output contract. A specialist text feature can use the general text contract; OCR can use vision. Built-in role contracts remain stable. The creator selects the new role in Studio/MCP and the app calls the same `ai.generate` method with that role ID. No database migration or SDK method change is needed for new role names.

For services that do not implement a native protocol, configure an operator-controlled HTTPS **Hackemist AI gateway protocol** provider. It keeps any service-specific keys on that gateway's backend. The suite authenticates to the configured gateway using its own encrypted bearer API key and applies the normal account, consent, quota, public-network and size limits. It does not execute gateway code uploaded as an app.

Gateway protocol version 1:

- `GET {base}/models` returns `{"data":[{"id":"my-model"}]}` for optional admin model discovery.
- `POST {base}/generate` receives `{protocol_version:1, role, task, model, input, max_output_tokens}`. `task` names a standard contract or `custom`. Custom input is a finite JSON object within 1 MiB; other tasks retain their typed input contracts.
- A synchronous response is `{status:'completed', text? , embedding?, files?, output?}` with at least one output. Standard tasks must return their matching output type; moderation returns `flagged`, `categories` and `category_scores`. `output` is finite JSON within 1 MiB. `files` contains at most four `{media_type,data_base64}` entries. Supported output MIME types include PNG/JPEG/WebP, MP4, MP3/WAV/OGG/M4A/FLAC, PDF, DOCX/XLSX/PPTX, JSON/text/ZIP and GLB/glTF/OBJ/STL. Files use the suite's generated filenames and binary limits.
- Long-running work returns `{status:'queued',job_id:'opaque_receipt'}` (or `in_progress`). Provider receipts use 1–200 letters, digits, underscores/hyphens. The suite retains this receipt privately and returns its own launch-bound job ID to the app.
- `GET {base}/jobs/{provider_receipt}` returns `queued`, `in_progress`, `failed`, or a completed normalized response. Polling never starts another generation. Untrusted gateway error text, arbitrary response fields and raw provider receipts are not forwarded to the app.

For example, a music backend can return an MP3; a 3D backend can return a GLB; a research backend can return JSON `output`. This is an extension contract, not a claim that every upstream provider implements music, 3D, research or other future modalities today. The operator or service developer implements the appropriate gateway mapping and verifies that provider's access and output. Gateway services must independently constrain any URLs/tool actions their custom inputs support.

## MCP and manual integration

MCP authoring tools use the account's existing scopes and owned-project boundary:

1. Call `ai_providers_list` (`projects:read`) for enabled role assignments, models, task contracts, custom role labels, provider presets and adapter support. It never returns API keys or private custom endpoint details.
2. Read `hackemist://docs/ai-providers`.
3. Call `ai_options_get({project_id})` (`projects:read`).
4. Call `ai_options_update({project_id, base_revision, roles:['general','image']})` (`projects:write`). A stale revision returns a conflict; reload before retrying. Roles can be selected before the operator enables their provider.
5. Call `ai_credits_get({project_id})` to read the app's balance and rates. Fund its wallet through Studio, or obtain an admin funding bypass. Authoring credentials cannot buy credits.
6. Implement feature UI using the injected SDK, test with a verified account, and publish the app normally.

The same project options are available through cookie/CSRF-authenticated `GET/PUT /api/projects/{id}/ai` and the Studio form. Admin configuration is `GET/PUT /api/admin/ai`; updates require current administrator authorization and `base_revision`. Blank API keys preserve existing keys; returned provider objects contain `key_configured`, never a secret. Do not send this read-only flag back in an update.

MCP supports discovery, documentation and project role setup. Paid runtime generation is intentionally tied to the interactive browser player's authenticated launch and consent; authoring tokens do not grant paid generation. No provider-domain CSP exception or external-backend permission request is needed for the shared gateway. Apps can still use reviewed external backends separately.

## Failure, privacy and operation

Insufficient app credits, disabled service/provider, missing assignment, unselected project role, expired/revoked sessions, unverified email and read-only previews fail before provider dispatch. The gateway validates the current account, app access, exact issuing session and revisions again. An app cannot choose the suite's endpoint, API key, model, headers or output-token cap. Native adapters expose no agent tools; a custom gateway defines and constrains its own functionality. Sharing consent covers text, images and audio and resets when role/provider configuration changes or the player closes.

Limits count attempted generations, including failures/timeouts, across all server workers. They reset at **00:00 UTC**. Generation is never retried automatically: a timeout or lost response may already have incurred charges, and starting it again can incur another charge. Apps must ask the player before retrying uncertain work. The server serializes admission briefly, releases SQL before network work, and limits active requests. Poll requests have separate cooldown/rate limits.

Keys are Fernet-encrypted with `SETTINGS_KEY` in existing settings storage and encrypted backups; retain the key for recovery. Prompts, generated text and media are not stored in SQL or audit entries. Seven days of request/job metadata support usage limits; video receipts expire after one day. The selected provider receives app-submitted input and applies its own retention rules. Text generation disables OpenAI response storage where supported; this does not override a provider's other retention policies. Apps decide whether and how to save resulting work through the existing save SDK.

Outbound calls require HTTPS on port 443, validate public DNS addresses, pin the chosen address for TLS while verifying the original hostname, and ignore environment proxies. JSON requests never follow redirects. Binary media downloads allow at most two public HTTPS redirects, stripping credentials at each redirect. Provider errors are sanitized. Results are bounded, private and noncacheable; a live paid API check with the operator's actual model access remains necessary after configuration.

The PostgreSQL upgrade uses additive migration **14** for AI options/requests and **15** for prepaid app credits. SQLite initialization and portable backup/transfer metadata include all seven tables. Rebuild/restart the control and runtime images together to deploy SDK 1.8.0. Shared AI and credit purchases start disabled on existing installations; no keys, funds, exemptions, external accounts or paid generations are created during upgrade.

The updated control service runs missing PostgreSQL migrations automatically during schema-guarded startup; SQLite creates the corresponding tables during initialization. A Git commit or push by itself does not run them. Back up the target installation and deploy the updated services before checking the target database's `hackemist.postgres_schema_migrations` table for versions 14 and 15. Development acceptance ran these upgrades only against disposable databases and did not migrate production.

## Provider references

- [OpenAI text generation / Responses](https://developers.openai.com/api/docs/guides/text)
- [OpenAI image generation](https://developers.openai.com/api/reference/resources/images/methods/generate)
- [OpenAI video generation](https://developers.openai.com/api/docs/guides/video-generation)
- [OpenAI vision](https://developers.openai.com/api/docs/guides/images-vision), [transcription](https://developers.openai.com/api/docs/guides/speech-to-text), [moderation](https://developers.openai.com/api/docs/guides/moderation), [speech](https://developers.openai.com/api/reference/resources/audio/subresources/speech/methods/create) and [embeddings](https://developers.openai.com/api/reference/resources/embeddings/methods/create)
- [Anthropic Messages](https://platform.claude.com/docs/en/api/messages/create)
- [Gemini Generate Content](https://ai.google.dev/api/generate-content), [native image generation](https://ai.google.dev/gemini-api/docs/generate-content/image-generation), and [embeddings](https://ai.google.dev/api/embeddings)