What is OpenSearch?

Built on Apache Lucene and originally derived from Elasticsearch—another search and analytics engine—OpenSearch provides a scalable and distributed architecture for real-time search, observability, log analytics and security analytics use cases.

OpenSearch includes OpenSearch Dashboards for data visualization and application monitoring. It also features a broad ecosystem of plugins, application programming interfaces (APIs) and clients that support analytics workflows across modern data environments.

Because it is developed as an open source project with a community-driven roadmap, organizations can use OpenSearch without licensing restrictions or vendor lock-in. Its compatibility with earlier versions of Elasticsearch—along with its extensible plugin framework—allows teams to adopt OpenSearch as a flexible analytics engine for operational workloads, machine learning pipelines and search applications.

Today’s organizations generate significant volumes of data that can be invaluable, but only if the data is indexed, searchable and available in real time. OpenSearch delivers this functionality through an open source search architecture designed for scale, cost efficiency and interoperability.

In practice, OpenSearch offers:

OpenSearch started as a community response to licensing changes for Elasticsearch and Kibana, a popular visualization layer. Earlier versions of Elasticsearch were released under the Apache 2.0 license, but subsequent releases adopted the Server Side Public License (SSPL) and Elastic License. These licenses restricted open source reuse, creating challenges for organizations that relied on freely deployable and redistributable search software.

To preserve an open search ecosystem, Amazon Web Services (AWS) forked (that is, created an independent copy of) the last Apache 2.0 versions of Elasticsearch and Kibana, creating the OpenSearch Project. The project introduced new features and enhancements under an open governance model, and expanded compatibility with Elasticsearch APIs and client libraries to simplify migration.

Since then, the OpenSearch Project has evolved independently. It features a community-driven roadmap, contributions from multiple providers and a growing ecosystem of plugins hosted on GitHub. While it remains compatible with many Elasticsearch patterns, OpenSearch has expanded its feature set with plugins for vector search, anomaly detection and advanced observability tools.

While both projects share a common origin, their paths have diverged. Elasticsearch continues under SSPL and Elastic License with a proprietary feature development strategy. OpenSearch, by contrast, remains Apache 2.0–licensed, prioritizing openness, extensibility and operational visibility. As a result, organizations choosing between the two now evaluate not just features, but also governance models, licensing terms and long-term ecosystem direction.

Compatibility continues to be an important bridge between the projects: OpenSearch still supports many Elasticsearch APIs, query patterns and client libraries from earlier versions, helping teams migrate with minimal refactoring. It also preserves similar repository structures and index formats, maintaining familiarity for users transitioning from Elasticsearch.

OpenSearch is built on a distributed architecture designed for scale and real-time performance. Its core components include clusters, nodes, indices, shards and documents—all working together to store and retrieve data efficiently.

Nodes are servers or containerized instances that perform indexing, querying and storage operations. Common node types include:

• Master nodes: Manage cluster state, coordinate shard placement and maintain metadata.
  
  
• Data nodes: Store documents and shards, as well as execute indexing and search operations.
  
  
• Client (coordinating) nodes: Route search queries, aggregate results and support load balancing without storing data.

A cluster is a collection of one or more nodes that work together to manage data and execute queries. Clusters provide redundancy and load balancing so that node failures do not affect overall performance. Each cluster maintains metadata about indices, shards and routing information.

An index is a logical namespace similar to a relational database table. It contains mappings that define the structure of JSON documents and references to the shards that store those documents. The term “index” is also used as a verb to describe the act of populating an index with data.

Documents are JSON objects that represent individual records. Put simply, it’s the data being stored and searched for. When indexed, fields within each document are analyzed, tokenized and stored in inverted indices.

Shards are the fundamental storage units in OpenSearch where documents live. Each index consists of primary shards and optional replica shards.

• Primary shards store the initial copy of the data.
  
  
• Replica shards provide redundancy and increase read throughput.

Because each shard is a standalone Lucene instance (a self-contained search engine library), OpenSearch distributes shards across nodes to parallelize search operations and scale performance.

So, how does this all come together? When a document is indexed, OpenSearch analyzes the content and applies text analyzers and tokenizers. After processing, it writes the terms into the appropriate shard.

Indexing is handled by data nodes and can be distributed across the cluster for speed and reliability. Queries are then submitted to a coordinating node, which identifies the shards containing relevant data, forwards the query to those shards and aggregates the results.

Think of it as a restaurant kitchen with different stations. Indexing is like prepping ingredients and sending them to the right station so it’s ready when the order comes in. When a query arrives, the coordinating node acts like the expediter—calling out what’s needed, gathering each station’s contribution and delivering one finished plate.

OpenSearch includes built-in features for search, analytics and observability. Plugins and extensions expand functionality, allowing teams to tailor the platform for specialized workloads.

• Full-text search and relevance scoring: Supports phrase queries, relevance tuning and filters using Apache Lucene.
  
  
• Distributed indexing and retrieval: Stores data across primary and replica shards, enabling parallel indexing and low-latency queries.
  
  
• Aggregations and analytical queries: Summarize and analyze data in real time for trend detection and operational monitoring.
  
  
• SQL query syntax: Queries indexed data using familiar Structured Query Language (SQL) constructs and returns results in JSON or tabular formats.
  
  
• Piped Processing Language (PPL): A pipeline-style syntax for exploring logs, metrics and other operational datasets.
  
  
• Index State Management (ISM): Automates index lifecycle operations such as rollover and retention.
  
  
• Data Prepper (ingest pipelines): Filters, enriches and transforms data before indexing for observability and security.
  
  
• Dashboards and visualization: Creates visualizations, operational panels and reports from logs, metrics and traces.
  
  
• Authentication and access control: Provides granular access controls over indices, documents and fields with support for Lightweight Directory Access Protocol (LDAP), Security Assertion Markup Language (SAML) and Active Directory.
  
  
• Observability primitives: Provides built-in support for key observability data used to monitor distributed systems.

While not exhaustive, these popular extensions enable advanced analytics, machine learning (ML) and observability scenarios:

• Anomaly Detection: Detects unusual patterns in logs and metrics using the Random Cut Forest algorithm.
  
  
• k-NN and vector search: Supports semantic search and similarity search, along with recommendation workloads using approximate nearest neighbor (k-NN) techniques.
  
  
• ML Commons: Runs machine learning models directly within OpenSearch, supporting training and inference.
  
  
• Performance Analyzer: Provides detailed resource and performance metrics across clusters, helping teams optimize CPU and query throughput.
  
  
• Cross-cluster replication: Replicates indices across clusters to support disaster recovery, redundancy and workload isolation.
  
  
• Trace Analytics: Visualizes traces from distributed systems and helps teams understand service dependencies and latency paths.

Organizations that prefer a managed experience can also use Amazon OpenSearch Service, which automates scaling, backups, node replacement and maintenance for OpenSearch clusters on AWS.

OpenSearch Dashboards is the visualization and analytics interface for OpenSearch. It provides an interactive environment for exploring indexed data, building visualizations and creating operational dashboards used across observability, security analytics and application monitoring workflows. For instance, teams can leverage Dashboards to visualize trends in metrics and investigate anomalies in near real time.

OpenSearch Dashboards supports the creation of charts, tables, maps, notebooks and custom panels. It also includes features designed to streamline analysis. Notebooks allow users to combine visualizations and text into a single narrative, while operational panels organize observability visualizations created with Piped Processing Language into a unified display.

Because OpenSearch Dashboards shares a user interface (UI) heritage with Kibana, many data teams find the workflow familiar. However, it is developed under its own roadmap and includes capabilities that reflect the broader OpenSearch feature set.

OpenSearch supports a wide range of use cases across industries, including:

• Log analytics and operational intelligence
• Observability workflows
• Security analytics and threat detection
• Search engine applications
• Data visualization and reporting
• Machine learning–enhanced analytics

Teams index logs from applications, infrastructure and cloud services to analyze performance issues and troubleshoot outages. OpenSearch supports high-volume ingest and real-time analytics, which makes it suitable for distributed production systems, such as a multinational e-commerce site.

With support for metrics, logs and traces, OpenSearch provides an integrated observability platform. Trace Analytics visualizes service interactions, while application analytics correlates telemetry to understand system behavior and pinpoint latency or failures. Dashboards and PPL queries allow teams to investigate issues quickly and create reusable operational views.

OpenSearch’s anomaly detection and ML Commons algorithms enable organizations to apply search and analytics techniques across security operations. Teams use it to detect unusual patterns in authentication logs or application behavior, as well as trigger notifications when conditions or thresholds are met.

Organizations use OpenSearch as the search engine behind websites, product catalogs and enterprise content systems. Full-text search, autocomplete, phrase matching and vector search support a range of user experience and recommendation use cases.

OpenSearch Dashboards provides interactive visualizations, reporting and notebooks that help teams explore data, monitor trends, track KPIs and share insights with stakeholders.

With ML Commons, teams can run model-driven operations inside OpenSearch, such as clustering, classification and forecasting. These capabilities support use cases like fraud detection, demand prediction, customer segmentation and enrichment of downstream data pipelines.