File component

Q: How do I avoid reading a file that is still being written?

Use readLock=changed (waits until the size and time stop changing), or have the writer use a done file (doneFileName) or a temporary name that the include pattern does not match.

Q: How do I keep the original file where it is?

noop=true. Cameleer remembers the files it has read (in memory, or in an idempotentRepository).

Q: How do I process files in order?

sortBy=file:modified (oldest first) or sortBy=reverse:file:name.

Q: Where do processed files go by default?

Into a .camel subfolder, as in Apache Camel. Set move, delete=true or noop=true to change that.

The file component reads files from a directory (polling for new ones) and writes message bodies to files.

Scheme: file: · Crate: cameleer-core · Cargo feature: none, always included (build features) · Consumer: yes · Producer: yes · Apache Camel: File

When to use it

Watch a drop folder on an edge gateway: machines, scanners or other programs write files, a route picks each one up, processes it and moves it aside.
Write exports, reports or received messages to disk.
Feed documents from a folder into an AI pipeline (summarize, classify, extract) one file at a time.

URI format

text

file:directoryName[?options]

The directory is relative to the working directory unless it is absolute. On Windows, use forward slashes (file:C:/data/inbox).

Example

A drop folder for CSV files. A file is picked up only after its size has stopped changing (readLock=changed), copied to the outbox under a new name, and moved into a dated archive folder; on failure it goes to .failed.

yaml example: file-drop-folder

- route:
    id: drop-folder
    from:
      uri: "file:{{env:INBOX:inbox}}?include=.*\\.csv&readLock=changed&move=.done/${date:now:yyyyMMdd}/${file:name}&moveFailed=.failed"
      steps:
        - log: "picked up ${file:name} (${file:size} bytes)"
        - to: "file:{{env:OUTBOX:outbox}}?fileName=${file:name.noext}-processed.csv&tempPrefix=.part-"

tempPrefix writes to a temporary name first and renames when complete, so a program watching the outbox never sees a half-written file.

Rust DSL

rust

fn drop_folder_route(b: &mut RouteBuilder, inbox: &str, outbox: &str) {
    b.from(&format!("file:{}?include=.*\\.csv&readLock=changed&move=.done/${{date:now:yyyyMMdd}}/${{file:name}}&moveFailed=.failed", inbox))
        .route_id("drop-folder")
        .log("picked up ${file:name} (${file:size} bytes)")
        .to(&format!("file:{}?fileName=${{file:name.noext}}-processed.csv&tempPrefix=.part-", outbox));
}

Options

Consumer:

option	default	description
`delay`	`500`	Milliseconds between polls (with `initialDelay` 1000, `useFixedDelay`, `greedy`, `timeUnit`, `startScheduler`, `sendEmptyMessageWhenIdle`, `scheduler`).
`noop`	`false`	Leave files in place (implies `idempotent=true`).
`delete`	`false`	Delete files after processing.
`move`	`.camel`	Where to move processed files (file language expression).
`moveFailed`		Where to move files whose processing failed.
`preMove`		Where to move files before processing.
`include`, `exclude`		Regular expressions on the file name.
`includeExt`, `excludeExt`		Comma-separated extensions, e.g. `csv,txt`.
`antInclude`, `antExclude`		Ant patterns on the relative path, e.g. `*/.xml`. `antFilterCaseSensitive` (default `true`).
`fileName`		Consume only this file (expression).
`filter`		`#bean` with a custom file filter.
`recursive`	`false`	Also read subdirectories (`minDepth`, `maxDepth`).
`maxMessagesPerPoll`	`0`	Limit per poll (`0` = no limit).
`sortBy`		`file:name`, `file:modified` or `file:size`, with `reverse:` / `ignoreCase:` prefixes; groups separated by `;`.
`shuffle`	`false`	Random order.
`readLock`	`none`	`markerFile`, `changed`, `fileLock`, `rename`, or `none`.
`readLockCheckInterval`, `readLockTimeout`, `readLockMinLength`, `readLockMinAge`	1000 ms, 10000 ms, 1, 0	Read lock tuning.
`readLockMarkerFile`, `readLockDeleteOrphanLockFiles`	`true`	Marker file handling.
`idempotent`, `idempotentKey`, `idempotentRepository`		Skip files already processed.
`doneFileName`		Only consume a file when this done file exists.
`charset`		Read the body as text in this charset.
`includeHiddenFiles`, `includeHiddenDirs`	`false`	Also read hidden entries.
`startingDirectoryMustExist`, `directoryMustExist`	`false`	Fail when the directory is missing.
`autoCreate`	`true`	Create the directory.

Producer:

option	default	description
`fileName`	message id	File name expression; header `CamelFileName` also sets it.
`fileExist`	`Override`	`Override`, `Append`, `Fail`, `Ignore`, `Move` (with `moveExisting`), `TryRename`.
`tempFileName`, `tempPrefix`		Write to a temporary name, then rename.
`doneFileName`		Write a done file after the file.
`appendChars`		Characters appended after each write (e.g. `\n`).
`charset`		Encoding of string bodies.
`flatten`	`false`	Drop directories from the file name.
`keepLastModified`	`false`	Keep the `CamelFileLastModified` time.
`allowNullBody`	`false`	Write an empty file for a null body.
`eagerDeleteTargetFile`	`true`	Delete an existing target before renaming a temp file.
`jailStartingDirectory`	`true`	Refuse to write outside the directory.
`autoCreate`	`true`	Create missing directories.

Expressions support ${file:name}, ${file:name.noext}, ${file:ext}, ${file:onlyname}, ${file:parent}, ${file:path}, ${file:absolute.path}, ${file:size}, ${file:modified}, ${date:now:FORMAT}, ${date:file:FORMAT}, ${exchangeId} and any other Simple expression.

Message headers

header	direction	description
`CamelFileName`	out / in	Relative file name; producer: target name.
`CamelFileNameOnly`	out	Name without path.
`CamelFileAbsolutePath`, `CamelFileAbsolute`	out	Absolute path / whether the path is absolute.
`CamelFileRelativePath`, `CamelFilePath`, `CamelFileParent`	out	Path details.
`CamelFileLength`	out	Size in bytes.
`CamelFileLastModified`	out	Modification time.
`CamelFileNameConsumed`	out	Name of the consumed file.
`CamelFileNameProduced`	out	Producer: the absolute path written.
`CamelOverruleFileName`	in	Producer: overrides the name once.

The body of a consumed file is its bytes. File errors raise GenericFileOperationFailedException.

Run it with Docker

mkdir -p inbox outbox
docker run --rm --user "$(id -u):$(id -g)" -e INBOX=/data/inbox -e OUTBOX=/data/outbox \
  -v "$PWD:/data" -v "$PWD/drop-folder.camel.yaml:/routes/drop-folder.camel.yaml:ro" \
  pmsudhi/cameleer run /routes/drop-folder.camel.yaml

The data folder is mounted writable; the route file stays read-only.

Common questions

How do I avoid reading a file that is still being written?

Use readLock=changed (waits until the size and time stop changing), or have the writer use a done file (doneFileName) or a temporary name that the include pattern does not match.

How do I keep the original file where it is?

noop=true. Cameleer remembers the files it has read (in memory, or in an idempotentRepository).

How do I process files in order?

sortBy=file:modified (oldest first) or sortBy=reverse:file:name.

Where do processed files go by default?

Into a .camel subfolder, as in Apache Camel. Set move, delete=true or noop=true to change that.

ftp, ftps, sftp: the same options for remote servers.
Apache Camel: File component.

Found a problem?

If something on this page doesn't match what Cameleer does, or you hit a bug while building with it, please help fix it, whether you are a person or a coding agent: open an issue that describes what you saw, then fork the repository, fix it, add or update a test, run cargo test --workspace, and open a pull request. CONTRIBUTING.md explains the ground rules.