AI & Machine Learning Glossary
Plain-language definitions of artificial intelligence and machine learning terms, from the foundations through large language models, generative AI, training, and evaluation.
Terms
Vendor-neutral definitions, grouped by topic. Jump to a section, or browse the whole list. For the rest of the collection, see the glossary hub.
A broad field of computer science focused on building systems that perform tasks normally requiring human intelligence, such as perception, reasoning, language, and decision-making. It spans rule-based methods, machine learning, and modern neural networks.
A subfield of AI in which systems learn patterns from data rather than following explicitly programmed rules. A model is trained on examples and then makes predictions or decisions on new, unseen inputs.
A training approach where a model learns from labeled examples, each pairing an input with its correct output. The model's goal is to generalize the input-to-output mapping so it can predict labels for new data.
A training approach that finds structure in data without labeled outputs, such as grouping similar items or reducing dimensionality. Common tasks include clustering and density estimation.
A method that trains on a small amount of labeled data combined with a large amount of unlabeled data. It aims to improve accuracy when labels are expensive or scarce.
A technique where the training signal is generated automatically from the data itself, for example by predicting a masked or withheld part of the input. It lets models learn useful representations from large unlabeled datasets.
A learning paradigm where an agent takes actions in an environment and learns a policy by maximizing cumulative reward over time. It relies on trial and error guided by feedback signals rather than labeled examples.
A supervised task that assigns inputs to one of a fixed set of discrete categories, such as spam versus not spam. The model outputs a class label or a probability for each class.
A supervised task that predicts a continuous numeric value, such as a price or temperature, rather than a discrete category. Models are evaluated by how close predictions are to the true values.
An unsupervised task that groups data points so that items in the same group are more similar to each other than to those in other groups. Algorithms include k-means and hierarchical clustering.
A failure mode where a model learns noise and specific details of the training data so closely that it performs poorly on new data. It is marked by low training error but high error on held-out data.
A failure mode where a model is too simple to capture the underlying patterns in the data, leading to poor performance on both training and new data. It often indicates insufficient model capacity or training.
The balance between errors from overly simple assumptions (bias) and errors from excessive sensitivity to training data (variance). Reducing one often increases the other, and good models minimize their combined effect.
A set of techniques that discourage overly complex models to reduce overfitting, typically by adding a penalty on large parameter values to the loss function. Common forms include L1 and L2 regularization.
A model's ability to perform well on new, unseen data rather than just the data it was trained on. Strong generalization is the central goal of machine learning.
An optimization algorithm that iteratively adjusts model parameters in the direction that most reduces the loss, using the gradient of the loss function. It is the core method for training most neural networks.
A variant of gradient descent that updates parameters using small random batches of data rather than the whole dataset at once. This makes training faster and able to scale to large datasets.
A model that makes predictions by following a branching series of yes/no questions about the input features until reaching a leaf that holds the output. Trees are interpretable but can overfit without pruning.
An ensemble method that trains many decision trees on random subsets of data and features, then averages or votes across their predictions. It usually improves accuracy and robustness over a single tree.
An ensemble technique that builds models sequentially, where each new model corrects the errors of the combined previous ones. It often achieves strong performance on structured tabular data.
A classification algorithm that finds the boundary that best separates classes with the widest possible margin between them. Kernel functions let it handle nonlinearly separable data.
A simple algorithm that classifies or predicts a point based on the labels or values of its closest examples in the training data. It makes no assumptions about data distribution but can be slow at prediction time.
An approach that combines the predictions of multiple models to produce a result that is usually more accurate and robust than any single model. Techniques include bagging, boosting, and stacking.
Methods that compress data from many features into fewer while preserving important structure. It is used for visualization, noise reduction, and making downstream models more efficient.
A training mode in which the model updates incrementally as new data arrives, rather than being retrained on a fixed dataset. It suits streaming data and environments that change over time.
A model composed of layers of interconnected units (neurons) that transform inputs through weighted connections and nonlinear activation functions. It learns by adjusting the weights to minimize a loss.
A subfield of machine learning that uses neural networks with many layers to learn hierarchical representations from raw data. It powers most modern advances in vision, speech, and language.
The simplest form of artificial neuron, computing a weighted sum of inputs and applying a threshold to produce an output. It is the historical building block that led to modern multilayer networks.
A nonlinear function applied to a neuron's output that lets a network model complex, nonlinear relationships. Common examples include ReLU, sigmoid, and tanh.
An activation function that outputs the input if it is positive and zero otherwise. It is widely used because it is simple, efficient, and helps mitigate the vanishing gradient problem.
An activation function that squashes any input into a value between 0 and 1, producing an S-shaped curve. It is often used to output probabilities but can suffer from vanishing gradients.
A function that converts a vector of scores into a probability distribution that sums to one, used for multi-class classification outputs. The largest score receives the highest probability.
The algorithm that computes how much each weight in a neural network contributed to the error by applying the chain rule backward through the layers. These gradients are then used to update the weights.
A learnable parameter that scales the influence of one input or connection on a neuron's output. Training adjusts weights so the network produces better predictions.
A learnable constant added to a neuron's weighted sum that shifts the activation, allowing the model to fit data that does not pass through the origin. It is distinct from statistical or societal bias.
A value learned during training, such as a weight or bias, that defines a model's behavior. The number of parameters is a common measure of a model's size and capacity.
A neural network that uses convolutional filters to detect local patterns such as edges and textures, making it well suited to images and other grid-like data. Deeper layers combine simple patterns into complex features.
A neural network designed for sequential data that maintains a hidden state carried from one step to the next. It can model order and context but struggles with long-range dependencies.
A type of recurrent network with gating mechanisms that control what information to keep, update, or forget over time. It handles long-range dependencies better than a basic RNN.
A neural network architecture that processes sequences in parallel using self-attention to weigh the relevance of every element to every other. It underpins most modern large language models.
A technique that lets a model focus on the most relevant parts of the input when producing each part of the output by assigning learned weights to different positions. It enables modeling of long-range relationships.
A form of attention in which each element of a sequence attends to all other elements of the same sequence to build context-aware representations. It is the core operation inside transformers.
An extension of attention that runs several attention operations in parallel, each learning to focus on different relationships, then combines their results. It lets a model capture varied kinds of context at once.
A difficulty in training deep networks where gradients shrink as they propagate backward, so early layers learn very slowly. Architectural choices like ReLU, residual connections, and gating help address it.
A shortcut that adds a layer's input directly to its output, letting information and gradients bypass intermediate transformations. It makes very deep networks easier to train.
A technique that normalizes the inputs to each layer across a batch to stabilize and speed up training. It can also have a mild regularizing effect.
A regularization method that randomly disables a fraction of neurons during each training step, forcing the network not to rely on any single unit. It reduces overfitting and improves generalization.
A neural network trained to compress input into a compact code and then reconstruct it, learning efficient representations in the process. Variants are used for denoising, anomaly detection, and generation.
A neural network, usually a transformer, trained on vast amounts of text to predict and generate language. It can perform many tasks such as answering questions, summarizing, and writing code.
A family of transformer-based language models that are pre-trained to predict the next token and then adapted to downstream tasks. The name describes the architecture and training approach rather than a single product.
The basic unit of text that a language model processes, which may be a word, part of a word, or a character. Models read and generate text as sequences of tokens rather than raw characters.
The process of splitting text into tokens that a model can process, often using subword units learned from data. The chosen scheme affects vocabulary size, sequence length, and how rare words are handled.
The maximum amount of text, measured in tokens, that a language model can consider at once when generating a response. Content beyond this limit must be truncated, summarized, or retrieved as needed.
The input text given to a language model to elicit a response, which may include instructions, questions, examples, or context. Its wording strongly influences the quality of the output.
The practice of designing and refining prompts to get more accurate, relevant, or well-formatted responses from a language model. Techniques include giving clear instructions, examples, and step-by-step guidance.
A set of instructions given to a language model that establishes its role, behavior, and constraints before the user's messages. It shapes tone and boundaries throughout a conversation.
Asking a model to perform a task using only instructions, with no worked examples provided in the prompt. It relies on knowledge the model gained during pre-training.
Providing a model with a small number of examples in the prompt to demonstrate the desired task before asking it to respond. The examples guide the format and behavior without changing the model's weights.
A prompting technique that encourages a model to generate intermediate reasoning steps before giving a final answer. It often improves performance on tasks that require multi-step reasoning.
A model's ability to adapt to a task from instructions and examples given in the prompt, without any update to its parameters. The learning lasts only for the duration of that context.
A sampling setting that controls the randomness of a model's output, where lower values make responses more deterministic and higher values more varied. It scales the probabilities before a token is selected.
A decoding method that samples the next token from the smallest set of options whose probabilities add up to a threshold p. It balances diversity and coherence by excluding unlikely tokens.
A decoding method that restricts the next-token choice to the k most probable options before sampling. It limits unlikely outputs while still allowing variety.
When a language model produces content that is fluent and plausible but factually incorrect or not grounded in its sources. It is a key reliability challenge for generative systems.
A technique that retrieves relevant documents from an external knowledge source and supplies them to a language model to ground its response. It reduces hallucination and lets models use current or private information without retraining.
The core training objective of most language models, which learn to predict the following token given the preceding ones. Repeated prediction lets the model generate coherent text.
A model that generates output one element at a time, feeding each produced element back in to help predict the next. Most text-generating language models work this way.
The process of turning a model's predicted token probabilities into actual output text, using strategies such as greedy selection, beam search, or sampling. The chosen strategy affects fluency and diversity.
A decoding strategy that keeps several of the most promising candidate sequences at each step and expands them, rather than committing to a single best token. It can produce higher-quality output than greedy decoding.
Capabilities that appear in large models at sufficient scale but are largely absent in smaller ones, such as multi-step reasoning on certain tasks. Their onset can be hard to predict from smaller models.
A system that uses a language model to plan and take actions toward a goal, often by calling tools, querying data, and iterating on results. It goes beyond single responses to carry out multi-step tasks.
AI systems that create new content such as text, images, audio, or code rather than only classifying or predicting. They learn the patterns of training data and sample new examples from them.
A large model trained on broad data at scale that can be adapted to many downstream tasks. It serves as a general-purpose base rather than being built for a single application.
A generative model that learns to create data by gradually removing noise from a random starting point, reversing a process that adds noise step by step. It is widely used for high-quality image generation.
A generative framework in which a generator creates samples and a discriminator tries to tell real from fake, each improving through competition. The process can produce realistic images and other data.
A generative model that encodes data into a probability distribution in a latent space and decodes samples from it to create new data. It enables smooth interpolation between generated examples.
A compressed internal representation where a model encodes the essential features of data, with similar items placed close together. Generative models sample from this space to create new outputs.
The task of producing an image that matches a natural-language description. Modern systems typically combine a text encoder with a diffusion or transformer-based image generator.
A model that can process or generate more than one type of data, such as text together with images, audio, or video. It learns shared representations that connect the different modalities.
The task of automatically generating a natural-language description of the contents of an image. It combines visual understanding with language generation.
A generative technique that recomposes the content of one input in the visual or stylistic manner of another. It is commonly used to render photos in the style of a particular artwork.
The generation of natural-sounding spoken audio from written text. Neural approaches model prosody and voice characteristics to produce human-like speech.
The use of generative models to produce source code from natural-language descriptions or partial code. It assists with writing, completing, and explaining programs.
Generating output that is guided by a specified input or constraint, such as a class label, text prompt, or reference image. The condition steers what the model produces.
The initial training phase where a model learns general patterns from a large, broad dataset before any task-specific adaptation. It produces reusable representations that later fine-tuning builds on.
Continuing to train a pretrained model on a smaller, task-specific dataset to specialize its behavior. Unlike retrieval, it changes the model's weights to encode new patterns or style.
Reusing knowledge a model gained on one task or dataset to improve learning on a related task. It reduces the data and compute needed compared with training from scratch.
A function that measures how far a model's predictions are from the desired outputs, producing a single number the training process tries to minimize. Its choice shapes what the model optimizes for.
A loss function for classification that penalizes the model more heavily when it assigns low probability to the correct class. It is the standard objective for training language and classification models.
A hyperparameter that controls how large a step the optimizer takes when updating parameters. Too high can cause unstable training, while too low makes training slow.
One complete pass of the training algorithm over the entire training dataset. Models are usually trained for multiple epochs until performance stops improving.
The number of training examples processed together before the model updates its parameters once. It affects training speed, memory use, and the stability of the gradient estimate.
A configuration value set before training, such as learning rate or batch size, that controls the learning process rather than being learned from data. Tuning them can significantly affect performance.
The process of searching for the combination of hyperparameters that yields the best model performance. Methods include grid search, random search, and Bayesian optimization.
The algorithm that updates a model's parameters using gradients to reduce the loss. Common optimizers include SGD, Adam, and their variants.
A widely used optimization algorithm that adapts the learning rate for each parameter using estimates of recent gradients and their variance. It often converges faster and more reliably than plain SGD.
A training method that fine-tunes a model using a reward signal derived from human preferences between model outputs. It is used to make models more helpful and aligned with human expectations.
Fine-tuning a language model on examples framed as instructions and desired responses so it better follows natural-language commands. It improves zero-shot usefulness across many tasks.
A parameter-efficient fine-tuning method that trains a small number of added low-rank matrices while keeping the original weights frozen. It greatly reduces the memory and storage needed to adapt large models.
A family of techniques that adapt large models by updating only a small fraction of parameters or adding lightweight modules. It makes customization feasible without retraining the whole model.
A technique where a smaller student model is trained to mimic the outputs of a larger teacher model, transferring much of its capability at lower cost. It is used to produce faster, cheaper models.
Reducing the numerical precision of a model's weights and activations, for example from 32-bit to 8-bit, to shrink its size and speed up inference. It trades a small amount of accuracy for efficiency.
The phase in which a trained model is used to make predictions or generate outputs on new inputs. It is distinct from training, which is when the model's parameters are learned.
A technique that halts training once performance on held-out validation data stops improving, preventing overfitting. It also saves compute by avoiding unnecessary epochs.
Expanding a training dataset by creating modified copies of existing examples, such as rotated images or paraphrased text. It improves robustness and reduces overfitting.
An individual measurable property or input variable used by a model to make predictions. The set of features describes each example the model sees.
The process of selecting, transforming, and creating input variables to improve a model's performance. Good features can matter as much as the choice of algorithm.
A dense numeric vector that represents an item such as a word, sentence, or image in a way that captures meaning and similarity. Items with related meaning have nearby vectors.
An ordered list of numbers used to represent data in a form that models can process. In machine learning, inputs, features, and embeddings are commonly expressed as vectors.
A database optimized for storing embeddings and finding the items whose vectors are most similar to a query vector. It is a common backbone for semantic search and retrieval-augmented generation.
Transforming numeric features to a common range or distribution, such as normalization or standardization, so that no single feature dominates by magnitude. It helps many algorithms train faster and more reliably.
A way of representing a categorical value as a vector with a single 1 in the position for that category and 0s elsewhere. It lets models process categories without implying an order between them.
The portion of data used to fit a model's parameters during learning. The model directly adjusts its weights based on these examples.
A held-out portion of data used during development to tune hyperparameters and compare models without touching the final test set. It helps detect overfitting while making design choices.
A held-out portion of data used only once, at the end, to estimate how a finished model will perform on unseen inputs. It must not influence training or tuning to give an honest estimate.
The practice of dividing data into separate subsets for training, tuning, and final evaluation. Keeping them separate prevents overly optimistic performance estimates.
An evaluation method that repeatedly splits data into training and validation folds so every example is used for validation once. It gives a more reliable performance estimate than a single split.
The process of annotating raw data with the correct outputs needed for supervised learning, such as tagging images or categorizing text. Label quality strongly affects model accuracy.
When information that would not be available at prediction time leaks into training, producing misleadingly high performance that fails in practice. It often stems from improper splitting or features derived from the target.
A situation where some classes appear far more often than others in the data, which can bias a model toward the majority class. Techniques like resampling and reweighting help address it.
Artificially generated data that mimics the statistical properties of real data, used to augment training or protect privacy. Its usefulness depends on how well it reflects real-world patterns.
A data point that differs markedly from the rest of the dataset and may reflect noise, error, or a rare genuine case. Handling outliers carefully can improve model robustness.
The fraction of predictions a model gets correct out of all predictions made. It can be misleading when classes are imbalanced, since a trivial majority guess may score high.
Among the items a model labeled positive, the fraction that are actually positive. High precision means few false positives, which matters when false alarms are costly.
Among all items that are actually positive, the fraction the model correctly identified. High recall means few missed positives, which matters when overlooking a case is costly.
The harmonic mean of precision and recall, giving a single number that balances the two. It is useful when you need both few false positives and few missed positives.
A table that breaks down a classifier's predictions into true positives, true negatives, false positives, and false negatives. It reveals the types of errors a model makes, not just the overall rate.
A plot of the true positive rate against the false positive rate across all classification thresholds. It shows the tradeoff between catching positives and raising false alarms.
A single number summarizing a ROC curve, equal to the probability that a model ranks a random positive above a random negative. A value of 1.0 is perfect and 0.5 is no better than chance.
A regression metric that averages the squares of the differences between predicted and actual values. Squaring penalizes larger errors more heavily.
A regression metric that averages the absolute differences between predicted and actual values. It is more robust to outliers than mean squared error.
A metric for language models that measures how well the model predicts a sample, with lower values indicating better prediction. It can be read as how many choices the model is effectively uncertain among per token.
A metric for evaluating machine-generated text, especially translation, by comparing overlapping word sequences with one or more reference texts. Higher scores indicate closer matches to the references.
A set of metrics for evaluating summaries and generated text by measuring overlap of words and sequences with reference texts. It is commonly reported for summarization tasks.
A standardized dataset and task used to compare the performance of different models under consistent conditions. Benchmarks enable progress tracking but can be gamed or become saturated.
The correct, reference answer against which a model's predictions are compared during evaluation. Its accuracy sets the ceiling for how meaningful a metric can be.
A simple reference model or method whose performance a new model must beat to show it adds value. Common baselines include predicting the most frequent class or a simple linear model.
An evaluation approach that uses a language model to score or compare outputs according to given criteria. It scales qualitative assessment but can inherit the judge model's biases.
The effort to ensure AI systems pursue goals and behave in ways consistent with human values and intentions. It covers both training methods and ongoing oversight.
Systematic and unfair differences in a model's outputs across groups, often reflecting biases present in training data or design choices. It can lead to discriminatory or inequitable outcomes.
The goal of ensuring a model's decisions do not unjustly favor or harm particular groups. Multiple mathematical definitions exist, and they can conflict with one another.
The degree to which the reasons behind a model's output can be understood by people. It supports trust, debugging, and accountability, especially for high-stakes decisions.
The extent to which a human can understand the internal mechanisms or decision process of a model. Simpler models are often more interpretable than large neural networks.
A set of methods and tools that make model behavior and predictions understandable to humans. Techniques include feature-importance scores and local explanations for individual predictions.
A model whose internal workings are opaque, so decisions cannot be easily inspected or explained from its structure alone. Large neural networks are often described this way.
Deliberately probing an AI system for harmful, unsafe, or unintended behaviors by adversarially testing it. Findings are used to strengthen safeguards before and after deployment.
Rules, filters, or constraints placed around an AI system to keep its behavior within safe and acceptable bounds. They can block disallowed inputs or outputs at run time.
An attack in which crafted input text manipulates a language model into ignoring its instructions or performing unintended actions. It is a leading security concern for applications built on LLMs.
Techniques that trick a model into bypassing its safety restrictions to produce content it would normally refuse. It is a form of adversarial manipulation that safety measures aim to prevent.
An input deliberately modified, often imperceptibly, to cause a model to make a mistake. Studying them reveals model weaknesses and motivates more robust training.
A model's ability to maintain reliable performance when inputs are noisy, shifted, or adversarially altered. Robust systems degrade gracefully rather than failing unpredictably.
The protection of personal or sensitive information used in training or inference from exposure or misuse. Approaches include anonymization, access controls, and privacy-preserving training methods.
A formal framework that adds carefully calibrated noise so that the presence or absence of any single individual's data has a bounded effect on results. It provides mathematical guarantees against re-identification.
A design pattern where people review, correct, or approve an AI system's outputs or decisions. It adds oversight for high-stakes or uncertain cases.
The gradual decline in a deployed model's performance as real-world data diverges from the data it was trained on. Monitoring and periodic retraining help counteract it.
The human tendency to over-trust automated outputs and under-scrutinize them, even when they are wrong. It can amplify the harm of model errors in decision-making.
Keep Reading
Whether you need overflow capacity for one acquisition or a repeatable conversion partner across an active pipeline, we'll help you define the scope, inspect the source data, and build a practical path into Applied Epic.
Built for insurance agencies, brokerages, aggregators, and acquisition teams that need experienced conversion capacity without building another internal department.