Source: https://www.hype-stack.dev/docs/packs-templates/packs/pack-ai-chat

# AI Chat

A complete AI workspace: chat with streaming Markdown, voice in and out, image generation, and a managed document
library that grounds answers in your own files. This is the `pack-ai-chat` pack. It is the largest pack in the catalog,
and the one the [Aether](/docs/packs-templates/templates/aether) template is built on.

## What you need

- An **OpenAI** API key. It covers text, transcription, speech, realtime voice, and embeddings out of the box.
- Optionally **Anthropic** or **Google** keys, to route the text model elsewhere.
- A **billing pack**. Usage is metered against the organization's plan, so this pack depends on the `pack-billing`
  family and refuses to install without one.
- Postgres, Valkey, and RustFS (uploads and generated images land in a private bucket).

## What it depends on

- A **starter pack**, for auth, organizations, and permissions.
- A **billing pack**: [Stripe](/docs/packs-templates/packs/pack-billing-stripe),
  [Lemon Squeezy](/docs/packs-templates/packs/pack-billing-lemonsqueezy), or
  [Polar](/docs/packs-templates/packs/pack-billing-polar).

## What you get

- `/chat`: conversation history, streaming responses, dictation, read-aloud, and image generation, on web and in the
  Expo app.
- `/knowledge`: upload documents, watch them index, and have answers cite them. Versioned, with per-organization limits.
- Durable execution. Every run and tool call is a row, so a run that dies mid-stream is recovered by a job, and
  cancellation is a state change, not a dropped socket.
- Usage accounting per organization: tokens, operations, voice minutes, and storage, reserved before a run and settled
  after, against caps from the plan.
- In the [admin app](/docs/packs-templates/packs/admin): an AI usage dashboard with 30-day charts, a workspace
  leaderboard against plan caps, and an **AI usage** tab on every organization for its allowance and model mix.

## Background work

Document indexing runs on the template's [task queue](/docs/backend#task-queue). Every upload becomes one
`knowledge-indexing` task, written in the same transaction as the upload, so a file is never "queued" with nothing
behind it. Each backend instance indexes `KNOWLEDGE_INDEXING_CONCURRENCY` documents at a time (2 by default), so one
slow PDF does not hold up everyone else. A failed run retries with backoff and resumes from the last embedded batch.
After 5 retries the file shows as failed with a **Try again** button. A file no retry can fix (no text, too many pages)
fails right away with the reason.

Two cron jobs register with the template's scheduler on install:

| Job                | Schedule         | What it does                                                                |
| ------------------ | ---------------- | --------------------------------------------------------------------------- |
| `ai-chat-recovery` | every 30 seconds | Finishes or fails runs whose process died                                   |
| `knowledge-sweep`  | every minute     | Purges deleted documents, closes out dead uploads, re-queues stranded files |

Everything starts with the API, so nothing extra runs in production. If indexing starts competing with requests, lower
`KNOWLEDGE_INDEXING_CONCURRENCY` or add a backend replica: every instance indexes.

Needs template v1.7.0 or newer for the task queue.

## Environment variables

| Variable                                                                           | Purpose                                                     |
| ---------------------------------------------------------------------------------- | ----------------------------------------------------------- |
| `OPENAI_API_KEY`                                                                   | Required                                                    |
| `ANTHROPIC_API_KEY`, `GOOGLE_GENERATIVE_AI_API_KEY`                                | Optional, for other text providers                          |
| `AI_TEXT_MODEL`                                                                    | `provider:model`, default `openai:gpt-5.4-mini`             |
| `AI_TRANSCRIPTION_MODEL`, `AI_SPEECH_MODEL`, `AI_REALTIME_MODEL`                   | Voice models                                                |
| `AI_STORAGE_BUCKET`                                                                | Private bucket for uploads and images, default `ai-private` |
| `AI_MAX_OUTPUT_TOKENS`, `AI_MAX_RUN_SECONDS`, `AI_MAX_VOICE_SECONDS`               | Hard caps per run                                           |
| `AI_USER_STORAGE_BYTES`                                                            | Storage cap per user                                        |
| `KNOWLEDGE_EMBEDDING_MODEL`                                                        | Embedding model for the document library                    |
| `KNOWLEDGE_MAX_FILE_BYTES`, `KNOWLEDGE_MAX_PAGES`, `KNOWLEDGE_MAX_TEXT_CHARACTERS` | Upload limits                                               |
| `KNOWLEDGE_INDEXING_CONCURRENCY`                                                   | Documents indexed at once per backend instance, default `2` |

Every cap has a sensible default. You only need `OPENAI_API_KEY` to start.

## After installing

The pack adds eleven Prisma models (conversations, messages, runs, tool calls, assets, voice sessions, usage accounts
and reservations, knowledge documents, versions, and chunks). Say yes when the CLI offers to run the migration, or run
`pnpm --filter @hype-stack/backend migration:create` afterwards.

## Related

- [Aether](/docs/packs-templates/templates/aether), the template built on this pack
- [Billing (Stripe)](/docs/packs-templates/packs/pack-billing-stripe) for the plan caps usage is metered against
- [Storage](/docs/backend/storage) for the private bucket