Deploy a Fluentd data collector

Fluentd is an open source data collector that unifies data from multiple sources and routes it to storage, analytics or monitoring services. This guide deploys a minimal HTTP collector that writes received events to the application logs. You can then adapt its inputs, filters and outputs to your needs.

Prerequisites

• A Clever Cloud account
• Clever Tools
• Git
• Ruby 3.2 or later with Bundler

Create the Fluentd application

Create a project directory, initialize Git and create a Linux application:

mkdir myFluentd
cd myFluentd
git init
clever create -t linux -a myFluentd

Clever Tools targets your personal organisation by default. To use another organisation, add --org ORGANISATION or -o ORGANISATION to the clever create command.

Create a Gemfile to install the tested Fluentd version:

source "https://rubygems.org"

gem "fluentd", "1.19.3"

Generate and commit the dependency lock file:

bundle lock

Create a .gitignore file to exclude local Bundler files:

.bundle/
vendor/

Configure Fluentd

Create a fluent.conf file:

<source>
  @type http
  bind 0.0.0.0
  port "#{ENV.fetch('PORT', 8080)}"
</source>

<match **>
  @type stdout
</match>

<label @FLUENT_LOG>
  <match **>
    @type stdout
  </match>
</label>

This configuration receives JSON events over HTTP and sends them to standard output. The @FLUENT_LOG label handles Fluentd’s own logs separately.

Create a mise.toml file with the build and run tasks used by the Linux runtime:

[tasks.build]
description = "Install Fluentd and its dependencies"
run = "bundle config set --local path vendor/bundle && bundle install"

[tasks.run]
description = "Start Fluentd"
run = "bundle exec fluentd --no-supervisor -c fluent.conf"

Mise is available on Clever Cloud. During deployment, the Linux runtime runs the task named build in the build phase and the task named run to start the application. See Mise tasks and the Linux runtime documentation for details.

Deploy Fluentd

Commit and deploy the project:

git add .
git commit -m "Deploy Fluentd"

clever deploy

Display the application URL or add a custom domain. A custom domain also requires DNS configuration:

clever domain
clever domain add your.website.tld

Send a test event to the application, replacing the example hostname with its URL:

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"message":"Hello from Fluentd"}' \
  https://your-fluentd.example.com/app.log

Display the application logs to confirm that Fluentd received the event:

clever logs

The minimal HTTP input in this guide is publicly reachable. Before using it in production, add access controls suitable for your clients and replace the standard-output match with the Fluentd output plugins required by your architecture.

Learn more