Elasticsearch component

The elasticsearch component indexes, updates, gets, deletes and searches documents in Elasticsearch over its REST API.

Scheme: elasticsearch: · Also: opensearch: (same component, see opensearch) · Crate: cameleer-datastores · Cargo feature: datastores · Consumer: no · Producer: yes · Apache Camel: Elasticsearch

When to use it

• Index documents, logs or events as they flow through a route.
• Retrieval for AI agents: run a full-text search and pass the top documents to an LLM as context (retrieval-augmented generation).
• Bulk-load or re-index data, and scroll through large result sets.

URI format

elasticsearch://clusterName?hostAddresses=host:9200&operation=<op>[&indexName=<index>][&options]

clusterName is a label. hostAddresses is a comma-separated list tried in order (default localhost:9200). Requests use HTTP, or HTTPS with enableSSL=true.

Example

POST /articles indexes the JSON body into the articles index and returns the document id.

- route:
    id: index-article
    from:
      uri: "platform-http:/articles?httpMethodRestrict=POST"
      steps:
        - unmarshal:
            json: {}
        - to: "elasticsearch://search?hostAddresses={{env:ES_HOSTS:localhost:9200}}&user={{env:ES_USER:elastic}}&password={{env:ES_PASSWORD:changeme}}&operation=Index&indexName=articles"

GET /search?q=rust returns the three best-matching articles. Put the result in a prompt and an LLM can answer from your own documents. Queries with characters other than letters, digits, spaces and simple punctuation are rejected, because the query text is placed inside JSON.

- route:
    id: search-articles
    from:
      uri: "platform-http:/search?httpMethodRestrict=GET"
      steps:
        - filter:
            simple: "${header.q} !regex '^[A-Za-z0-9 ,.?_-]+$'"
            steps:
              - setHeader:
                  name: CamelHttpResponseCode
                  constant: 400
              - setBody:
                  constant: "invalid query"
              - stop: {}
        - setBody:
            simple: '{"query": {"match": {"text": "${header.q}"}}}'
        - to: "elasticsearch://search?hostAddresses={{env:ES_HOSTS:localhost:9200}}&user={{env:ES_USER:elastic}}&password={{env:ES_PASSWORD:changeme}}&operation=Search&indexName=articles&size=3&enableDocumentOnlyMode=true"
        - marshal:
            json: {}

A body that is JSON text, like the one built by setBody above, is sent as the full search request, so it must contain query (and can add knn, aggs or sort). A body that is a map is treated as the query and wrapped into {"query": ...}, unless it already has a query key.

Rust DSL

ctx.add_routes(|b| {
    b.from("direct:index")
        .to("elasticsearch://search?hostAddresses=localhost:9200&operation=Index&indexName=articles");
    b.from("direct:search")
        .to("elasticsearch://search?hostAddresses=localhost:9200&operation=Search&indexName=articles&enableDocumentOnlyMode=true");
})?;

Operations

Operation names are case-insensitive; BulkIndex is accepted for Bulk.

Options

maxRetryTimeout, certificatePath (use sslContextParameters), waitForActiveShards and connectionManagerMaxConnections are accepted and ignored.

Message headers

With useScroll=true the exchange property CamelElasticsearchScrollQueryCount holds the number of scroll requests.

Run it with Docker

docker run --rm -p 8080:8080 -e ES_HOSTS=es1.example.com:9200 -e ES_USER=app -e ES_PASSWORD \
  -v "$PWD:/routes:ro" pmsudhi/cameleer run /routes/articles.camel.yaml

Common questions

How do I connect to Elastic Cloud?

Use hostAddresses=<deployment>.es.<region>.cloud.es.io:443&enableSSL=true&apiKey=<encoded key>.

Can I run vector (kNN) search?

Send a full search request with a knn section as the body; it is passed to Elasticsearch as is. Computing the query vector is up to you, for example with the openai embeddings endpoint. The embedded test server does not implement kNN, so this path is not covered by tests.

How do I get all results, not just the first page?

Set useScroll=true. Every page is fetched with _search/scroll and the scroll context is cleared at the end.

Related

• opensearch: the same component for OpenSearch.
• AI components guide
• Apache Camel Elasticsearch component

Found a problem?

If something on this page doesn't match what Cameleer does, or you hit a bug while building with it, please help fix it, whether you are a person or a coding agent: open an issue that describes what you saw, then fork the repository, fix it, add or update a test, run cargo test --workspace, and open a pull request. CONTRIBUTING.md explains the ground rules.