AI terminology has a habit of arriving in groups. One week everyone is talking about prompts. The next week it is agents, harnesses, MCP servers, context windows and “agentic workflows”.

Some of these terms describe genuinely useful ideas. Others are ordinary software concepts wearing a futuristic hat.

Here is a practical guide to the language of modern AI, including the terms that are becoming more common as AI systems move from answering questions to taking action.

I'll use this page to list and explain as many of the various AI terms and jargon that I (and my agent du jour) can think of.

If I miss any, feel free to let me know at [email protected] 

Glossary of terms commonly used in AI conversations 

The foundations

Artificial intelligence

Artificial intelligence, or AI, is the broad category. It covers systems designed to perform tasks that normally require human judgement, such as recognising patterns, understanding language, making predictions or choosing between options.

It is a very large umbrella. Underneath it sit machine learning, generative AI, language models and agents.

Machine learning

Machine learning is the process of training a system to recognise patterns from examples rather than programming every rule manually.

A traditional program might be told exactly how to identify an invoice. A machine-learning system might be shown thousands of invoices and learn what tends to make one look like an invoice.

Generative AI

Generative AI creates new content: text, images, audio, video, code or structured data.

The important word is “generative”. It is not merely retrieving a document from a filing cabinet. It is producing an answer, summary, image or piece of code based on patterns learned from data and the instructions it receives.

Model

A model is the trained AI system itself.

Different models have different strengths. One might be fast and inexpensive for classification. Another might be better at complex reasoning. Another might handle images or audio. Choosing a model is a little like choosing a member of staff for a task: you do not send the accountant to repair the roof simply because the accountant is available. I mean ... if they're super cheap ... maybe you do😉

Foundation model

A foundation model is a general-purpose model trained on a broad range of data and adapted for many different tasks.

Large language models are one kind of foundation model. The same underlying model may be used for writing, analysis, coding, extraction, translation and conversation.

LLM

LLM stands for Large Language Model.

An LLM works with language by predicting likely sequences of tokens. Given the words and context it has received, it generates what it considers a useful continuation.

That does not mean it “knows” things in the human sense. It means it is very good at modelling patterns in language and using those patterns to produce an answer. The answer might even be another prompt: "What prompt would be best for producing the image requested by the user".

Multimodal model

A multimodal model can work with more than one type of input or output, such as text, images, audio and video.

This is useful when the task involves a photograph of a damaged building, a scanned document, a voice recording or a spreadsheet rather than plain text alone.
These models train on mixed media like images, in the same way as the basic LLM trains on words and letters. It learns how pixels go together to produce an image that humans will recognise, and then actually be happy with.

The building blocks of AI use

Training data

Training data is the material used to train a model.

It may include text, code, images, audio or other data. The quality, variety, legality and suitability of training data all affect the resulting model.

Training data is not the same as the information you provide during an ordinary conversation. Sending a document to a model for analysis does not automatically mean that document becomes part of the model’s permanent training.

Parameters

Parameters are the internal values a model adjusts during training. They are part of what allows the model to represent patterns. Loosely put, the number of parameters is a good way to judge how capable the model is - how much data it can use to represent patterns that we think are a good result. You might think of it like "years of relevant experience" in a CV.

You do not usually need to manage parameters directly. The term matters mainly because model size, capability and cost are often discussed in relation to the number of parameters.

Tokens

Tokens are the chunks of text that models process.

A token might be a whole short word ("blue"), part of a longer word ("blue-footed booby"), punctuation or a small piece of code. Tokenisation varies between models, so one token is not always one word.

Tokens matter because they influence:

• how much text a model can process at once;

• how much a request costs;

• how quickly a response is generated; and

• whether a long conversation eventually needs to be shortened or summarised.

The model does not see your lovingly crafted 2,000-word briefing as “2,000 words”. It sees a token bill.

Context window

The context window is the amount of information a model can consider in one request.

This may include the current prompt, previous conversation, attached documents, tool results and instructions. A larger context window allows the model to work with more material, but it does not guarantee better judgement.

A model can have a very large context window and still confidently miss the point. Some people manage that without any help from software.

Input and output

Input is what you send to the model: instructions, questions, documents, images, conversation history and tool results.

Output is what the model sends back: text, code, structured data, tool requests or other generated content.

Similar to paying for water that comes out of your tap (input), and how you also pay for the water and waste that goes back out through the plumbing (output - probably best not to think about that one too much).

Input costs and output costs

Many AI services charge separately for input and output tokens.

Input costs cover what the model receives. Output costs cover what it generates. Output is often more expensive because generation requires the model to produce new material one piece at a time.

This distinction matters when building an AI workflow. A system that repeatedly sends a large document, a long conversation and several tool results may spend more on input than expected before it has written a single useful sentence.

Usage limits

Usage limits control how much you can use a model or service within a period.

Limits might apply to:

• the number of requests;

• the number of tokens;

• the amount of uploaded data;

• the number of tool calls;

• the speed of requests; or

• the account’s spending limit.

Free plans, paid subscriptions and APIs may all have different limits. A workflow that works beautifully in a test may stop working when ten people use it at once. This is why “it worked on my machine” remains a respected technical tradition!

Latency

Latency is the time between making a request and receiving a result.

Latency matters when AI is used in live chat, voice systems, customer service or a workflow where people are waiting. A more capable model may produce a better answer but take longer.

Inference

Inference is the process of using a trained model to produce an output.

Training is how the model learns its general capabilities. Inference is what happens every time you ask it to do something.

Prompting and instructions

Prompt

A prompt is the input or instruction given to an AI system.

A prompt can be a simple question, a detailed task description, a template, a set of examples or a request to return structured data.

System instructions

System instructions are higher-level rules that define how a model should behave. They may specify its role, tone, boundaries, available tools and required output format.

For organisations, good system instructions are often more valuable than clever one-off prompts. They turn “please be sensible” into something closer to an agreed operating procedure.

Few-shot prompting

Few-shot prompting gives the model examples of the desired input and output.

For example, you might provide three examples of correctly categorised enquiries before asking it to categorise a fourth. The examples help the model infer the pattern.

The name is an alternative to One-shot prompting where you ask a single question and get a response.

Structured output

Structured output asks the model to return information in a predictable format, such as JSON, a table or a fixed set of fields.

This is useful when the result will be passed into another system. If an AI system is extracting names, dates and contact details from forms, structured output is generally safer than asking it for “a helpful summary” and hoping the next system can interpret the poetry.

Knowledge and retrieval

RAG

RAG stands for Retrieval-Augmented Generation.

Instead of relying only on what the model learned during training, a RAG system retrieves relevant information from your own documents or databases and supplies it to the model before it answers.

This is useful for:

• internal policies;

• grant guidelines;

• product documentation;

• service procedures;

• meeting records; and

• frequently changing information.

• A SharePoint bot that can actually find useful information in the 57 different abandoned sites your old IT guy didn't document well enough ... (this one's definitely personal)

RAG can make answers more grounded, but it is not magic. If the wrong document is retrieved, the model may give a beautifully written answer based on the wrong thing.

Embeddings

Embeddings are numerical representations of meaning.

They allow a system to compare pieces of text based on similarity rather than exact matching. A search for “how do I apply for funding?” might find a document titled “Community grant application process” even though the words are not identical.

Vector database

A vector database stores embeddings and helps retrieve items that are semantically similar.

It is commonly used as part of a RAG system. The name sounds more exotic than the job: it is essentially a fast way to find things that mean something similar.

The words "foot" and "shoe" are related right? There's a short vector between them. But the words "foot" and "blue-footed booby" are not quite the same ... you can see how it is more likely that someone come up with "shoe" when asked about related words and how "blue-footed booby" is probably a lot less likely ... (it's a private joke, don't worry about it! Thanks, Rochelle).

Grounding

Grounding means connecting an AI response to reliable information, such as an approved document, database, source or tool result.

Grounding is especially important when the answer affects money, legal obligations, health, public communications or organisational decisions.

Hallucination

A hallucination is an answer that sounds plausible but is false, invented or unsupported.

Models can hallucinate because they generate likely language, not because they consult a built-in truth meter. The more confident the wording, the more important it is to check the underlying evidence.

Think of a common myth that lots of people tell eachother often, that you know is actually made up, but people seem to enjoy retelling eachother. 

Agentic AI

Agent

An agent is an AI system that can pursue a goal through multiple steps, often using tools, memory and decision-making rather than producing one answer and stopping.

A simple chatbot answers a question. An agent might:

1. understand the request;

2. search a knowledge base;

3. check a calendar;

4. draft an email;

5. ask for approval; and

6. send the approved message.

The crucial difference is not that the agent is “intelligent”. It is that the system can take a sequence of actions within defined boundaries.

Agentic AI

Agentic AI refers to systems designed to plan, decide, use tools and act towards an objective.

The term is increasingly used for systems that go beyond chat. Examples include an agent that triages enquiries, prepares a report, updates a CRM record or checks a set of documents against a policy.

“Agentic” is also becoming a useful marketing adjective. It should not automatically be accepted as a useful technical description.

Workflow

A workflow is a defined sequence of steps.

Some workflows use AI for one step, such as classifying an enquiry. Others use AI throughout the process.

A workflow is usually more predictable than a fully autonomous agent because the steps and decision points are explicit.

Orchestration

Orchestration is the logic that coordinates models, tools, data sources and steps in an AI system.

It decides what happens next, which model or tool should be used, what information should be passed along and when a human should be involved.

Harness 

A harness is the surrounding software that helps an AI model do useful work.

It may provide:

• instructions;

• tools;

• access to files;

• memory;

• permissions;

• feedback;

• monitoring;

• retries; and

• rules about when to stop.

The model is the engine. The harness is the vehicle, dashboard, brakes and occasionally the person shouting, “Perhaps do not email the entire database just yet.” It is a file or set of files that contain specific instructions on what is allowed or disallowed behaviour. It helps you control, or harness the AI, just like the bridle on a horse.

This term is especially common in discussions about coding agents, autonomous systems and agent development.

Tool calling

Tool calling allows a model to request an action from an external function or service.

The model might ask to:

• search a website;

• retrieve a file;

• query a database;

• calculate a figure;

• create a calendar event; or

• send information to another system.

The model does not necessarily perform the action itself. It produces a structured request, and the surrounding software decides whether to carry it out.

Function calling

Function calling is a structured form of tool calling. The developer defines available functions and the information each function requires.

For example, a function might be called create_support_ticket and require a customer name, category and description.

MCP

MCP stands for Model Context Protocol.

It is a standard way for AI applications to connect to external tools and sources of context. An MCP server can expose capabilities such as searching files, querying a service or interacting with an application.

MCP is rising quickly because it provides a common language for connecting models to tools instead of creating a completely different integration for every AI application.

The practical question is not “Does it support MCP?” but “What exactly can this connection access, and what is it allowed to do?”

Handoff 

A handoff occurs when one agent passes work to another agent or specialist.

For example, a general enquiry agent might hand a safeguarding question to a specialist agent, or a triage agent might send a technical issue to a technical-support workflow. Modern agent frameworks increasingly support this kind of delegation.

Memory

Memory is information retained between interactions.

It might include user preferences, previous decisions, organisational context or the history of a task. Memory can be useful, but it also creates questions about accuracy, retention, privacy and deletion.

A system that remembers the wrong thing is not more helpful. It is merely more confidently familiar.

Planning

Planning is the process of breaking a goal into smaller steps (that are then completely ignored by some models - looking at you ChatGPT!)

An agent may create a plan before acting, revise the plan as it receives new information or choose a different route when a tool fails.

Planning can help with complex tasks, but it also creates more opportunities for mistakes. Every extra step is another place where the system can misunderstand the assignment.

Reasoning model

A reasoning model is designed to spend more effort working through complex problems before producing an answer.

Reasoning is useful for analysis, coding, planning and multi-step decisions. It may also increase latency and cost.

A reasoning model can still reason its way to the wrong conclusion. More thinking is not the same as better information.

Human-in-the-loop

Human-in-the-loop means a person reviews, approves or guides an AI system at an important stage.

For example, an agent might prepare a grant application but require a person to approve the final version. It might classify an enquiry but send anything sensitive to a human.

For most organisations, human oversight is not a sign that the AI has failed. It is a sign that someone has remembered who is accountable.

IBM training data used to say that computers will never be held accountable so they should never make decisions - and they were right! Keep the human in the loop.

Guardrails

Guardrails are rules and controls that limit what an AI system can do.

They may cover:

• approved tools;

• permitted data;

• prohibited requests;

• spending limits;

• human approval;

• output checks;

• access controls; and

• logging.

Good guardrails are specific enough to guide behaviour and simple enough for people to understand.

Prompt injection

Prompt injection is an attempt to manipulate an AI system through instructions hidden in user input, documents, webpages or tool results.

For example, a document might contain text telling the agent to ignore its original instructions and reveal confidential information. Systems that read external content need to treat that content as data, not automatically as authority.

Tool permission

Tool permission defines what an agent is allowed to do.

Reading a document is different from editing it. Drafting an email is different from sending it. Looking up a balance is different from transferring money.

Permissions should match the risk of the action. An agent does not need the keys to the whole building to turn on the lights.

Tracing

Tracing records what an AI system did during a task: which instructions it received, which tools it called, what results came back and where the process ended.

Tracing is useful for debugging, evaluation, governance and explaining an outcome after something goes wrong.

Evaluation

Evaluation, or evals, is the process of testing whether an AI system performs well on representative tasks.

A useful evaluation set might contain real but anonymised examples of enquiries, documents or decisions. It should test accuracy, consistency, safety, cost and whether the system knows when to ask for help.

Observability

Observability is the ability to understand what is happening inside a running AI system.

It includes logs, traces, error rates, latency, token usage, tool calls and human corrections. In agentic systems, observability is essential because the final answer may hide a long chain of intermediate actions.

Terms you will increasingly hear

AI-native

An AI-native organisation is designed around AI capabilities from the beginning rather than adding them later to an existing process.

The term is often used loosely. A company with a chatbot on its website is not necessarily AI-native. It may simply have a chatbot on its website...

Copilot

A copilot assists a person with a task while leaving the person in control.

Examples include drafting, summarising, suggesting next steps or preparing analysis for review.

It's used a lot in Microsoft circles as they've made a brand out of it but it's basically just an Agent.

Autopilot

Autopilot implies that the system can perform a task with little or no human intervention.

This can be useful for low-risk, repetitive work. It is a more serious claim than “copilot”, so it deserves more serious testing.

Agent-to-agent communication 

This refers to agents passing tasks or information between one another.

It may be useful in complex systems, but it also introduces more handoffs, permissions and failure points. A committee meeting between six software agents is still a committee meeting.

Computer use 

Computer-use systems can interact with software interfaces by clicking, typing, scrolling and reading screens.

This is useful when no clean API exists, but it can be fragile. A redesigned button, unexpected pop-up or changed login flow may be enough to confuse the system.

Still clunky, but can sometimes solve a problem as a last resort.

Long-running agent 

A long-running agent works on a task over an extended period rather than completing one short interaction.

Examples include monitoring a mailbox, researching a topic over several hours or checking a workflow for changes. These systems require careful limits, logs, scheduling and recovery procedures.

Model routing 

Model routing chooses between different models depending on the task.

You don't get your head architect to update your blog - you get an intern (yeah ok, you get an agent to do it but you know what I mean). Balancing spend with cognitive demand.

A fast inexpensive model may handle routine classification, while a more capable model handles an unusual or high-value request. This can improve cost and speed, provided the routing rules are tested.

Synthetic data

Synthetic data is artificially generated data used for training, testing or evaluation.

It can help create examples where real data is limited or sensitive. It can also reproduce the weaknesses and assumptions of the system that generated it, so it should not be treated as automatically neutral.

How people might use these ideas

A small organisation might start with a simple workflow:

• an LLM drafts a response;

• RAG supplies the organisation’s approved policy;

• structured output extracts the key details;

• a human reviews the draft;

• the system logs the decision; and

• an agent sends the approved response.

A more advanced version might use:

• an agent to classify the request;

• a handoff to a specialist;

• an MCP connection to search internal files;

• a tool call to create a task;

• memory to retain relevant context; and

• guardrails to prevent unauthorised actions.

The sensible order is to begin with the work, not the vocabulary.

Ask:

1. What task are we trying to improve?

2. What information does the system need?

3. What is the cost of a mistake?

4. Which actions require human approval?

5. How will we know whether it is working?

6. What happens when the model is wrong, unavailable or overconfident?

You do not need an agent for every problem. Sometimes a well-written template, a clearer process or a shared spreadsheet remains the correct answer. It is less exciting, but it rarely invents a policy meeting that never happened.

The short version

• A model generates or analyses information.

• An LLM works primarily with language.

• Tokens measure the material models process.

• Context windows limit how much information can be considered at once.

• Input and output tokens affect cost.

• RAG supplies relevant organisational information.

• Tools let models interact with external systems.

• An agent combines models, tools, instructions and decision-making to pursue a goal.

• A harness provides the surrounding controls and capabilities.

• MCP provides a common way to connect AI applications with tools and context.

• Guardrails, permissions and human review keep useful automation from becoming expensive theatre.

• Evals, tracing and observability show whether the system works in the real world.

The important question is not whether your organisation is using the newest term.

It is whether the system helps people do useful work safely, clearly and with fewer avoidable headaches.