What are AI coding assistants?

LLMs are, on their own, able to take in code, context and instructions as input and output new code or code changes in return. But a standalone LLM can’t open files, use external tools and applications, access your terminal or run commands—tasks that are essential to creating and maintaining a production-ready codebase. While an LLM is indeed AI that can assist you with coding, a raw LLM is not what is meant by the term “AI coding assistant” in modern industry parlance.

A true AI coding assistant is, conversely, a more robust, comprehensive software product that uses an LLM (or multiple LLMs) as its engine. It can be understood as an application that pairs an LLM “brain” with a toolkit that acts as its arms and legs, all of which is accessed through a single user interface (UI) and operated through carefully crafted logic and workflows.

Though much of the functionality of a coding assistant can be achieved through manually coded logic and laboriously constructed prompts, AI coding assistants are designed to automate and streamline that work. For instance, built-in retrieval pipelines provide context awareness without requiring the entire codebase to be dumped into each input prompt, consuming the LLM’s entire context window. MCP servers, operating through the Model Context Protocol (MCP) originally introduced by Anthropic, facilitate communication with external services and tools such as APIs, databases and files. Structured workflows allow an AI assistant to use specialized tools to make precise, localized code edits, whereas a standalone LLM must rewrite an entire file or code block to make a single change.

Less achievable through manually orchestrated workflows are the collaboration across development teams and integration with essential coding platforms that AI coding assistants can provide. Most modern coding assistants can be incorporated directly into common integrated development environments (IDEs), such as Visual Studio Code (VS Code) or PyCharm. Some, such as GitHub Copilot, are native plugins or extensions for prominent IDEs. Some, like IBM Bob, can function as both a shell—a command line interface (CLI), such as Bob Shell, that can be either operated independently or integrated with IDEs using a simple wrapper—or as a self-contained IDE with comprehensive, built-in tools for debugging, version control, refactoring and test generation.

The semantics of differentiating between AI coding assistants and AI agents can be confusing. When it comes to AI coding tools, terminology is driven as much by marketing as it is by distinct, well-defined and universally-agreed-upon characteristics and capabilities.

The most important thing to note is that these concepts are not mutually exclusive. In fact, the industry has been increasingly conflating the two terms to the point that they’re sometimes used interchangeably. It’s perhaps most useful to understand each as describing different dimensions of an AI coding tool, rather than as describing separate types of products.

• Assistant describes the product’s relationship to its human user. It’s an informal, UX-oriented term that basically defines the tool’s job description: using AI to assist a human with coding tasks. In theory, this could aptly describe anything from a simple tool for code completion and other straightforward autocomplete-driven functionalities to a complex, comprehensive end-to-end suite for AI-powered software engineering.

• Agent describes the product’s technical architecture. In short, any software that surrounds an LLM with the tools, environments, guidelines, guardrails and reasoning frameworks to autonomously plan and execute coding tasks could fairly be called an “AI coding agent,” inasmuch as it’s an AI agent designed for coding tasks.

Most modern coding assistants constitute agentic AI: they receive natural language instructions, then independently formulate the specific steps necessary to carry out those instructions, run commands, evaluate results and iterate on their final output before presenting it to the user. It’s therefore generally accurate, on a technical level, to use either term to describe them.

But in practice, one would more appropriately use the term “AI coding agent” (or simply “coding agent”) to describe an individual agent, programmed to carry out a specific task or responsibility. For instance, a software engineer might use an AI coding assistant to build an autonomous agent whose purpose is to ingest new Jira tickets and proactively address minor issues. You might build another agent to monitor changes to the codebase and update documentation accordingly. In essence, these coding agents would be created by and operate within your coding assistant.

While each coding assistant on the market offers its own unique workflows, logic, features and focus areas, a coding assistant will generally comprise the following essential components.

A coding assistant must have an interface through which the user may interact with it. That interface may be either a simple text-based command-line interface (CLI) or, in the case of IDE-based assistants, a graphical user interface (GUI). The latter could entail either a dedicated GUI for the coding assistant or an extension of or plugin within the GUI of their primary IDE.

The appropriate choice will usually depend on some combination of the user’s skill level, use case, operating environment and token budget.

CLI-based assistants such as Bob Shell, Aider or Pi enable developers to direct their coding assistant from within their machine’s native terminal (or their third-party terminal of choice). For software engineers who primarily work through their terminal, this offers the most seamless, speedy, token-efficient and customizable experience. For coding assistants that must operate within “headless” environments wherein GUIs (and monitor screens, for that matter) aren’t available—such as continuous integration/continuous delivery (CI/CD) services—CLI-based tools are often the only option.

CLI coding assistants provide more precise control over tools and processes, allowing developers to explicitly script workflows instead of being beholden to the abstractions and built-in workflow logic of an IDE. The output of a system command, test or workflow can be directly piped into the next input prompt to the AI assistant, enabling developers to seamlessly chain commands together.

CLI-based assistants do require significantly greater skill and development knowledge to operate, making them a poor choice for beginners and vibe coders. Users must be comfortable navigating terminal environments, file paths and other architectural elements that are typically hidden beneath UI abstractions in everyday computing. Their barebones UI also precludes some of the functionality offered by IDE-based assistants, such as inline buttons, chat sidebars, real-time code review or click-to-accept code completion suggestions.

IDE-based AI coding assistants offer a more robust and user-friendly experience through a classic GUI. For developers who primarily work through an IDE or beginners who are unfamiliar with terminal commands, IDE-based tools will provide a more streamlined and low-friction experience.

IDE-based tools allow for a more feature-rich environment, as there are more ways and locations in which options and information can be presented to and ingested by the user within a GUI with a moveable, clickable cursor. For instance, an IDE-based coding assistant’s GUI can provide side-by-side file comparisons and color-coded inline diffs to clearly convey proposed changes. Sidebars and context menus provide opportunities to display refactoring suggestions. The assistant can respond to where your text cursor is positioned in real-time, and context-aware edit suggestions can be accepted or rejected with a single click.

That functionality comes at the expense of control and, in some cases, cost-efficiency. IDE-based assistants are inherently more token-hungry than their CLI-based counterparts, as an IDE must constantly bundle a wide range of contextual information into each raw prompt it sends to the LLM under the hood. Working through the IDE’s set of abstractions instead of explicit commands is more accessible and intuitive for most users, but that comes with a sacrifice in customization capabilities.

LLMs are central to every AI coding assistant: the “assistant” can perhaps best be understood as a software structure that allows the user to extract maximum performance and utility from an LLM. The choice of which specific LLM to use is therefore an essential architectural decision for any AI coding tool.

Some coding assistants are model agnostic, but many constrain the user to specific LLMs. Claude Code, for instance, operates exclusively through Anthropic’s Claude models. Cursor uses its proprietary “Composer” model for code generation.

In most cases, a coding assistant built upon one-model approach is inefficient from both a cost and latency perspectives: some tasks require the precision and reasoning of a large frontier model, but many others are better served by a smaller, faster and less token-hungry LLM. IBM Bob, for instance, uses a multi-model orchestration that draws upon a mix of frontier proprietary models (including Claude), open-source Mistral models and IBM Granite models alongside specialized fine-tune models for security and next-edit prediction. Bob routes each task to the most suitable model: simple tasks go to lighter models, while central planning and complex tasks go to larger models.

Agentic reasoning logic is how the coding assistant’s LLM “brain” digests a macro-objective (such as “figure out why login keeps crashing on mobile”) and decomposes it into practical execution steps. Different reasoning strategies lend themselves to different types of tasks: a coding assistant will often be programmed to deploy a variety of strategies to meet the needs of a user’s request. 

Tools are the “arms and legs” of a coding assistant, allowing it to interact with its environment and do more than edit or generate isolated code snippets in a vacuum. Built-in tooling commonly offered within modern coding assistants enables tasks such as:

• Extract relevant information from and take actions within external services, such as task management software, internal documentation, databases, calendars or other applications  

• Make precise, localized code changes to specific parts of a file rather than a full rewrite

• Run terminal and shell commands

• Perform security checks

• Execute rule-based actions in predefined scenarios

• Validate outputs against (and enforce) system guidelines and guardrails

For instance, the coding assistant might call a built-in tool that peruses your organization’s Slack channels for context relevant to the task at hand. Informed by that context, it might call another tool to make a surgical change to a specific area of code and a third tool to generate and run a unit test.

Actions involving file systems, databases and external services are generally mediated by MCP servers. The emergence and widespread adoption of Model Context Protocol (MCP), which provides a universal standard for communication between LLMs and external APIs, has significantly increased the ease of interoperability between code assistants, the LLMs that power them and the many services they must interact with.

The function of those tools is often to identify, retrieve and act upon context that’s essential to the task a coding assistant is being asked to perform, but wouldn’t have been included in an LLM’s training data.

• Vector databases: A coding assistant must repeatedly discern which handful of files, amongst thousands of documents or millions of lines of code, contain the context needed to inform each step of a task. Storing individual files in a database of vector embeddings—mathematical representations of each document as an array of numbers—allows for relevant documents to be identified through semantic search. Context from retrieved files can then be injected into the LLM’s workflow through retrieval augmented generation (RAG).

• Rules files: Rather than requiring the user to constantly, repeatedly provide a list of detailed instructions for each task, code assistants usually rely on rules files, such asClaude.md files in Claude Code or custom rules and custom modes in IBM Bob. These are standardized markdown files that explicitly dictate coding standards, desired behaviors and architectural considerations to the LLM.

• Cross-session memory: LLMs are, by default, stateless. Beyond the context window of the current session, an LLM has no access to information from previous sessions. Coding assistants therefore store session logs and persistent metadata caches that allow the assistant to “remember” important takeaways, patterns, obstacles (such as code defects and the debugging strategies used to remedy them) and project-specific build configurations. This is what allows the coding assistant to “learn” over time.

The powerful autonomy of coding assistants (and the coding agents they create to implement a task) can be a double-edged sword. Left unmonitored and unchecked, a coding assistant could potentially enact far-ranging updates that break functionality, expose credentials and other secrets or enable malicious code into your organization’s systems.

High-quality coding assistants therefore facilitate guardrails and automated checkpoint schemas to ensure that any consequential actions have been reviewed and approved by a human before they’re executed. IBM Bob, for instance, requires human permission for most actions by default—any exceptions must be explicitly set to Auto-Approve by the user.

Many coding assistants’ configuration settings enable a user to sandbox a given agent or project, keeping it contained within specifically approved environments for reasons of security or relevance. In IBM Bob, specific files and directories can simply be added to a .bobignore file to prevent Bob from interacting with them. When using the CLI-based Bob Shell in folders that haven’t been added to “trusted folders,” Bob runs in a restricted safe mode to minimize vulnerabilities.

Many modern coding assistants provide automated checkpointing, to facilitate experimentation with code changes and easy rollback of updates with undesirable consequences.

When multiple sub-agents—whether created within the coding assistant or built elsewhere—have been deployed for a given task, most coding assistants use the Agent2Agent (A2A) protocol to coordinate communication between them.

For coding assistants like Bob that leverage multiple LLMs, a task-aware routing system dynamically delegates each sub-task to the appropriate model based on complexity, computational demands and token budgets.

AI coding assistants enable a wide range of usage patterns, from vibe coding—driven entirely by natural language prompts and model outputs, with minimal engagement with the code itself—to more deliberate and strategic agentic coding (or agentic engineering).

Vibe coding is generally the province of users without training or experience in software development, many of whom lack basic knowledge of even common programming languages such as Python or JavaScript, or (at the hands of more experienced developers) experimentation and prototyping. The user essentially delegates the entire process to the coding assistant, eschewing most manual input, review and testing beyond whatever instructions and fixes are conveyed by the user’s natural language instructions. If vibe coded projects are to be used in real-world scenarios, great care must be taken to avoid code quality issues or security risks.

Agentic engineering, conversely, is more akin to pair programming, in which the coding assistant is truly an assistant working alongside the developer—who actively steers the project and code—as an extra set of hands and eyes. This more sophisticated practice of agentic coding unlocks the full capabilities and potential of AI coding assistants, enabling more sustainable and productive use of AI coding tools in production environments.