Cameleer

Azure Storage Blob component

The azure-storage-blob component uploads, downloads, lists, copies and deletes blobs in Azure Blob Storage (block, append and page blobs), and polls a container for new blobs.

Scheme: azure-storage-blob: · Crate: cameleer-cloud · Cargo feature: cloud · Consumer: yes · Producer: yes · Apache Camel: Azure Storage Blob Service

When to use it

  • Archive files, reports and message payloads in Azure.
  • Read files that partners drop into a container, then delete them.
  • Append log lines to an append blob from many routes or devices.
  • React to every blob change in an account with the change feed consumer.

URI format

text
azure-storage-blob://accountName[/containerName][?options]

Example

Upload a daily report. The blob name comes from the CamelAzureStorageBlobBlobName header (or the blobName option):

yaml example: azure-storage-blob-upload-report
- route:
    id: upload-report
    from:
      uri: "timer:daily-report?repeatCount=1"
      steps:
        - setBody:
            constant: "region,total;eu,42;us,17"
        - setHeader:
            name: CamelAzureStorageBlobBlobName
            simple: "reports/${date:now:yyyy-MM-dd}.csv"
        - setHeader:
            name: CamelAzureStorageBlobContentType
            constant: text/csv
        - to: "azure-storage-blob://{{env:AZURE_STORAGE_ACCOUNT:mystorageaccount}}/reports?operation=uploadBlockBlob&accessKey=RAW({{env:AZURE_STORAGE_KEY:}})"
        - log: "uploaded ${header.CamelAzureStorageBlobBlobName} (etag ${header.CamelAzureStorageBlobETag})"

Poll a container and delete each blob after it was processed:

yaml example: azure-storage-blob-consume-incoming
- route:
    id: incoming-blobs
    from:
      uri: "azure-storage-blob://{{env:AZURE_STORAGE_ACCOUNT:mystorageaccount}}/incoming?accessKey=RAW({{env:AZURE_STORAGE_KEY:}})&deleteAfterRead=true"
      steps:
        - log: "received ${header.CamelAzureStorageBlobBlobName} (${header.CamelAzureStorageBlobBlobSize} bytes)"
        - to: "log:incoming"

Instead of the account key you can use a SAS token (sasToken=RAW(...)) or Microsoft Entra ID (credentialType=AZURE_IDENTITY, for managed identities); see Cloud credentials.

Rust DSL

rust
ctx.add_routes(|b| {
    b.from("azure-storage-blob://{{env:AZURE_STORAGE_ACCOUNT:mystorageaccount}}/incoming?accessKey=RAW({{env:AZURE_STORAGE_KEY:}})&deleteAfterRead=true")
        .log("received ${header.CamelAzureStorageBlobBlobName} (${header.CamelAzureStorageBlobBlobSize} bytes)")
        .to("log:incoming");
})?;

Options

Common Azure options (accessKey, sasToken, credentialType, Entra ID options, uriEndpointOverride, overrideEndpoint, proxyHost, proxyPort, trustAllCertificates) are described in Cloud credentials.

optiondefaultdescription
operationlistBlobContainersContainers: listBlobContainers, createBlobContainer, deleteBlobContainer. Blobs: listBlobs, uploadBlockBlob, getBlob, deleteBlob, downloadBlobToFile, downloadLink, copyBlob, stageBlockBlobList, commitBlobBlockList, getBlobBlockList. Append blobs: createAppendBlob, commitAppendBlob. Page blobs: createPageBlob, uploadPageBlob, resizePageBlob, clearPageBlob, getPageBlobRanges. Leases (Cameleer): acquireBlobLease, renewBlobLease, releaseBlobLease, breakBlobLease. Change feed: getChangeFeed.
blobNameBlob (the CamelAzureStorageBlobBlobName header wins).
prefixlistBlobs and consumer: only blobs with this prefix.
maxResultsPerPagePage size of listings (all pages are read).
deleteAfterReadfalseCameleer: consumer deletes blobs after successful processing (otherwise it reads the container again on every poll, as in Camel).
createAppendBlobtruecommitAppendBlob creates the blob when missing.
createPageBlobtrueuploadPageBlob creates the blob when missing.
pageBlobSize512Size of createPageBlob.
leaseBlobfalseHold a lease while writing or deleting.
leaseDurationInSeconds60Lease duration (-1 = infinite).
commitBlockListLatertruestageBlockBlobList stages without committing.
blockListTypeCOMMITTEDgetBlobBlockList: COMMITTED, UNCOMMITTED or ALL.
fileDirTarget directory of downloadBlobToFile.
downloadLinkExpiration1 hourValidity (ms) of downloadLink.
sourceBlobAccessKeyKey of the source account of a cross-account copyBlob.
changeFeedfalseCameleer: consume the account's change feed instead of a container.
changeFeedStartTime, changeFeedEndTimeChange feed window (epoch millis or ISO-8601).

The consumer also accepts the scheduled poll options (delay, initialDelay, scheduler.cron, ...).

Message headers

headerdirectiondescription
CamelAzureStorageBlobOperationinOverrides operation.
CamelAzureStorageBlobBlobName, CamelAzureStorageBlobContainerNamein/outBlob and container.
CamelAzureStorageBlobContentType, CamelAzureStorageBlobMetadatain/outContent type and metadata (map, x-ms-meta-*).
CamelAzureStorageBlobETag, CamelAzureStorageBlobLastModified, CamelAzureStorageBlobBlobSize, CamelAzureStorageBlobContentMD5, CamelAzureStorageBlobBlobTypeoutBlob properties.
CamelAzureStorageBlobPrefixinListing prefix.
CamelAzureStorageBlobPageBlobRange, CamelAzureStorageBlobPageBlobSize, CamelAzureStorageBlobCreateAppendBlob, CamelAzureStorageBlobCreatePageBlobinAppend and page blobs.
CamelAzureStorageBlobCommittedBlockCountoutBlocks of an append blob.
CamelAzureStorageBlobLeaseId, CamelAzureStorageBlobLeaseDurationin/outLeases.
CamelAzureStorageBlobBlockIds, CamelAzureStorageBlobCommitBlobBlockListLater, CamelAzureStorageBlobBlockListTypein/outBlock lists.
CamelAzureStorageBlobDownloadLink, CamelAzureStorageBlobDownloadLinkExpirationin/outdownloadLink.
CamelAzureStorageBlobFileDir, CamelAzureStorageBlobFileNameindownloadBlobToFile.
CamelAzureStorageBlobSourceBlobAccountName, CamelAzureStorageBlobSourceBlobContainerName, CamelAzureStorageBlobSourceBlobName, CamelAzureStorageBlobCopyId, CamelAzureStorageBlobCopyStatusin/outcopyBlob.
CamelAzureStorageBlobChangeFeedStartTime, CamelAzureStorageBlobChangeFeedEndTime, CamelAzureStorageBlobChangeFeedEventType, CamelAzureStorageBlobChangeFeedEventTime, CamelAzureStorageBlobChangeFeedEventIdin/outChange feed.

Run it with Docker

sh
docker run --rm -e AZURE_STORAGE_ACCOUNT -e AZURE_STORAGE_KEY \
  -v "$PWD:/routes:ro" pmsudhi/cameleer run /routes/blobs.camel.yaml

Common questions

Why do I get ContainerNotFound?

Uploads do not create the container. Create it once with operation=createBlobContainer (or in the portal or your infrastructure code).

How do I test against Azurite?

Use the account devstoreaccount1 with Azurite's well-known key and add uriEndpointOverride=http://127.0.0.1:10000&overrideEndpoint=true; requests then use Azurite's path style (http://127.0.0.1:10000/devstoreaccount1/...).

How do I share a blob with someone for an hour?

Use operation=downloadLink: the body becomes a read-only SAS URL valid for downloadLinkExpiration milliseconds (default one hour).

How do I use a managed identity?

Set credentialType=AZURE_IDENTITY and drop the key: tokens come from the environment, workload identity or the managed identity endpoint.

Found a problem?

If something on this page doesn't match what Cameleer does, or you hit a bug while building with it, please help fix it, whether you are a person or a coding agent: open an issue that describes what you saw, then fork the repository, fix it, add or update a test, run cargo test --workspace, and open a pull request. CONTRIBUTING.md explains the ground rules.

Edit this page on GitHub