Agent
An application workflow where a model can select from bounded actions, use state, and continue through defined steps. The model should not be the authority for permissions or side effects.
AI engineering glossary
Context window
The maximum amount of input and output a model can consider in one request. A large window does not remove the need to select relevant context.
Embedding
A numeric representation of text used to compare semantic similarity. Embeddings are useful for retrieval but do not guarantee that a retrieved passage is correct or current.
Evaluation set
A curated collection of representative inputs and expected behaviors used to compare system versions. It should include known difficult and high-risk cases.
Grounding
Providing a model with relevant evidence or constraints so its answer is tied to known information rather than unsupported assumptions.
Hallucination
A response that presents unsupported or incorrect information as if it were factual. Strong prompts alone are not a complete defense; evidence, validation, and product boundaries matter.
Idempotency
A property of an operation where repeating the same request does not create an additional side effect. It is important when tool calls can retry.
Input tokens
The units of text sent to a model, including instructions, conversation history, retrieved context, and tool results. They are often the largest controllable cost in retrieval-heavy workflows.
Latency budget
A target amount of time allocated across the steps of a user request. It makes tradeoffs visible before a slow workflow reaches users.
P95 latency
The response time at or below which 95 percent of requests complete. It exposes tail delays that an average can hide.
Model routing
Selecting a model or execution path based on task characteristics such as complexity, risk, latency target, or cost constraint.
Overlap
Repeated text shared by adjacent chunks. It can preserve a thought across a boundary but increases index size and duplicate retrieval risk.
Output tokens
The units of text generated by a model. Output limits affect both cost and response time.
RAG
Retrieval-augmented generation: retrieving external content at request time and giving it to a model as context for an answer.
Reranking
A second-stage process that reorders retrieved candidates using a more precise relevance method before they enter the model context.
Retry rate
The share of requests that make an additional attempt after a failure or low-confidence outcome. Retries change both cost and tail latency.
Schema validation
Checking that a response conforms to an expected structured shape, such as valid JSON with required fields. It verifies format, not factual correctness.
System prompt
Instructions that establish the role, boundaries, and response rules for a model interaction. It should be versioned and evaluated like other product configuration.
Temperature
A sampling parameter that influences output variability. Lower values may improve repeatability for structured tasks but do not guarantee correctness.
Tool call
A structured request from a model to an application capability such as search, retrieval, or a business action. The application must validate it before execution.
Trace
A record of the steps in a request, often including versions, timings, tool activity, and outcomes. Good traces support debugging without logging unnecessary sensitive data.
Vector database
A storage and search system optimized for vector similarity queries. It is one component of retrieval, alongside metadata, chunking, filtering, and evaluation.
Versioning
Recording identifiable versions of prompts, model settings, retrieval configuration, tools, and evaluation data so behavior changes can be investigated.
Human in the loop
A deliberate point where a person reviews, approves, corrects, or takes over a workflow. It is most useful when the decision or side effect is consequential.