Cameleer

SFTP component

The sftp component downloads files from and uploads files to an SSH server over SFTP, polling a remote directory for new files.

Scheme: sftp: · Crate: cameleer-ssh · Cargo feature: ssh · Consumer: yes · Producer: yes · Apache Camel: SFTP

Cameleer has its own SSH-2 and SFTP v3 implementation (interop-tested against OpenSSH), with password, public key, agent and certificate authentication. The consumer and producer options and headers are those of ftp.

When to use it

  • Pick up files at remote or edge sites (stores, plants, field devices) that expose SFTP.
  • Exchange files with partners and banks over SFTP.
  • Push configuration or firmware files to devices.

URI format

text
sftp://[username@]host[:port]/directory[?options]

The directory is relative to the login (home) directory; sftp://host//abs/dir is absolute. The default port is 22.

Example

A central server collects JSON files that remote sites leave in outbox/, saves them locally and deletes them on the site. The host key must match known_hosts (strictHostKeyChecking=yes).

yaml example: sftp-site-pickup
- route:
    id: site-pickup
    from:
      uri: "sftp://{{env:SFTP_USER:edge}}@{{env:SFTP_HOST:localhost:22}}/outbox?password={{env:SFTP_PASSWORD:changeme}}&knownHostsFile={{env:KNOWN_HOSTS:known_hosts}}&strictHostKeyChecking=yes&include=.*\\.json&delete=true&delay=30000"
      steps:
        - log: "picked up ${file:name} from ${header.CamelFileHost}"
        - to: "file:{{env:LOCAL_DIR:incoming}}"

For key authentication, replace password=... with privateKeyFile=/routes/id_ed25519 (and privateKeyPassphrase for an encrypted key).

Rust DSL

rust
fn site_pickup_route(b: &mut RouteBuilder, sftp: &str, password: &str, known_hosts: &str, local_dir: &str) {
    b.from(&format!(
        "sftp://{}/outbox?password={}&knownHostsFile={}&strictHostKeyChecking=yes&include=.*\\.json&delete=true&delay=30000",
        sftp, password, known_hosts
    ))
    .route_id("site-pickup")
    .log("picked up ${file:name} from ${header.CamelFileHost}")
    .to(&format!("file:{}", local_dir));
}

Options

Connection and authentication:

optiondefaultdescription
username, password$USERLogin (also user[:password]@ in the URI).
privateKeyFile, privateKeyUri, privateKeyPrivate key: path, file: URI, or key text / #bean. OpenSSH or PEM.
privateKeyPassphrasePassphrase of an encrypted key.
useAgent, agentSocketfalseAuthenticate with an SSH agent (SSH_AUTH_SOCK or the given socket).
certificateFileOpenSSH user certificate for the key.
knownHostsFile, knownHostsUri, knownHostsknown_hosts path, URI or text.
useUserKnownHostsFiletrueAlso read ~/.ssh/known_hosts.
strictHostKeyCheckingnoyes: the host key must be known. no (Apache Camel's default) accepts unknown hosts; changed or revoked keys are always rejected.
preferredAuthenticationse.g. publickey,keyboard-interactive,password.
ciphers, keyExchangeProtocols, serverHostKeys, macs, compressionAlgorithmsAlgorithm lists.
compression0A level above 0 enables zlib compression.
connectTimeout10sConnect timeout.
soTimeout5mRead timeout.
disconnectfalseClose the connection after each poll or upload.
sendNooptrueCheck pooled connections before use.
maximumReconnectAttempts, reconnectDelay3, 1sReconnects.
throwExceptionOnConnectFailedfalseFail the poll on connect errors.
autoCreatetrueCreate the remote directory.
bulkRequests16Pipelined SFTP read / write requests.
chmod, chmodDirectoryOctal permissions of uploaded files / created directories.
charsetText encoding of bodies.

Consumer: the scheduler options (delay 500 ms, initialDelay 1000 ms, ...), noop, delete, move, moveFailed, preMove, include, exclude, includeExt, excludeExt, antInclude, antExclude, antFilterCaseSensitive, fileName, recursive, minDepth, maxDepth, maxMessagesPerPoll, sortBy, idempotent, idempotentKey, idempotentRepository, download, ignoreFileNotFoundOrPermissionError.

Producer: fileName, fileExist, moveExisting, tempFileName, tempPrefix, eagerDeleteTargetFile, allowNullBody, flatten, doneFileName.

Not supported: readLock (accepted with a warning and ignored) and proxies.

Message headers

The same as ftp: CamelFileName, CamelFileNameOnly, CamelFileAbsolutePath, CamelFileRelativePath, CamelFilePath, CamelFileParent, CamelFileLength, CamelFileLastModified, CamelFileNameConsumed, CamelFileHost, and CamelFileNameProduced on upload.

Run it with Docker

sh
ssh-keyscan -p 22 site7.example.com > known_hosts
docker run --rm --user "$(id -u):$(id -g)" -e SFTP_HOST=site7.example.com:22 -e SFTP_USER -e SFTP_PASSWORD \
  -e KNOWN_HOSTS=/routes/known_hosts -e LOCAL_DIR=/data/incoming -v "$PWD:/data" \
  -v "$PWD/site-pickup.camel.yaml:/routes/site-pickup.camel.yaml:ro" -v "$PWD/known_hosts:/routes/known_hosts:ro" \
  pmsudhi/cameleer run /routes/site-pickup.camel.yaml

Common questions

How do I log in with a private key?

privateKeyFile=/path/id_ed25519; add privateKeyPassphrase=... if it is encrypted. Ed25519, ECDSA and RSA keys in OpenSSH or PEM format work.

Why should I set strictHostKeyChecking=yes?

With the default no, the first host key seen is accepted, so a man in the middle on the first connection is not detected. Provide knownHostsFile and strictHostKeyChecking=yes in production.

Can I upload with SFTP?

Yes: to: sftp://user@host/inbox?password=...&tempPrefix=.part-.

How do I wait for a file to be fully written on the remote side?

readLock is not supported for SFTP. Have the sender write under a temporary name and rename the file when it is complete, and match only the final names with include.

Found a problem?

If something on this page doesn't match what Cameleer does, or you hit a bug while building with it, please help fix it, whether you are a person or a coding agent: open an issue that describes what you saw, then fork the repository, fix it, add or update a test, run cargo test --workspace, and open a pull request. CONTRIBUTING.md explains the ground rules.

Edit this page on GitHub