Local AI Basics

You do not need machine-learning experience to use Pitchblend. These terms explain why the app sometimes downloads a large file, answers slowly, or needs you to provide context.

Model

A model is the learned data that produces text. A local model is stored and run on this computer; an online model runs on a provider's computers. Different models vary in speed, quality, context size, and tool support.

The model is not a search engine or a guaranteed fact database. It predicts a useful response from your prompt and available context, and it can confidently be wrong. Verify medical, legal, financial, security, and other important claims independently.

Runtime and inference

The runtime is the software that loads the model and performs inference—generating an answer. Pitchblend bundles the platform runtime but downloads model weights separately. The first request after launch can be slower while the model loads into memory.

Prompt, context, and context window

Your prompt is the instruction you send. Context is everything the model sees for that request: parts of the conversation, relevant memories, file passages, and tool results. The context window is the maximum amount that fits.

A longer prompt is not automatically better. State the goal, include only useful details, and ask for the format you want.

Tokens

Models process pieces of text called tokens. A token may be a word, part of a word, punctuation, or whitespace. Max Tokens in Generation settings limits the potential response length; it does not count pages precisely.

Chat history and memory

History is the transcript you can reopen. Memory is a separate set of saved facts that may be recalled across chats, only in one private chat, or across chats in one private folder. Turning memory off does not erase facts already saved.

Files and RAG

RAG means retrieval-augmented generation: finding document text and adding it
to a prompt without retraining the model. Pitchblend has settings labeled RAG,
but automatic default-file retrieval is not connected to Chat in v1.0.

Typing @ in Chat references a processed file directly and adds up to its
first 2,000 extracted characters to that request. An online model can also use
the document-search tool for literal filename or content matching. Neither is
a semantic search over every passage, and a returned passage is evidence to
examine, not proof that the final answer is correct.

Local-first is not the same as offline

Local text inference can run without sending the prompt to a model provider.
Tool support depends on the model and backend: the curated Windows/Linux Qwen
model supports native tool calls, while the current curated macOS MLX model
does not advertise them. A tool keeps its own data boundary—web search and
image generation use network services, while document search reads local text
and supplies matching excerpts to the selected model. Downloads, identity and
license checks, online models, and sync setup also use a network. The shipped
voice control uses an online text agent even when normal Chat is local, and its
local MMS/Sesame speech components can download required assets on first
initialization. See
Local AI and network boundaries
before enabling connected features.