Fluent is an event collector system. It’s said that Fluent is generalized version of syslogd, which can deal with JSON object for the log message.

Fluent collects events from various data sources and write them to files, database or other storages:

Web apps  ---+                 +--> file
             |                 |
             +-->           ---+
/var/log  ------>  fluentd  ------> mail
             +-->           ---+
             |                 |
apache    ----                 +--> fluentd

Fluent also supports log transfer:

Web server
+---------+
| fluentd -------
+---------+|     |
 +---------+     |
                 |
Proxy server     |    Log server, Amazon S3, HDFS, ...
+---------+      +--> +---------+
| fluentd ----------> | fluentd ||
+---------+|     +--> +---------+|
 +---------+     |     +---------+
                 |
Database server  |
+---------+      |
| fluentd ---------> mail
+---------+|
 +---------+

An event consists of tag, time and record. Tag is a string separated with ‘.’ (e.g. myapp.access). It is used to categorize events. Time is a UNIX time when the event occurs. Record is a JSON object.

$ gem install fluentd
$ # install sample configuration file to the directory
$ sudo fluentd -s
$ fluentd &
$ echo '{"json":"message"}' | fluent-cat debug.test

Web site

fluentd.org/

Documents

fluentd.org/doc/

Source repository

github.com/fluent

Author

Sadayuki Furuhashi

Copyright

© 2011 FURUHASHI Sadayuki

License

Apache License, Version 2.0

Patches contributed by:

• Abhishek Parolkar

• Chad Skidmore

• Eiichiro Iwata

• Francesc Esplugas

• hirochachacha

• Masahiro Nakagawa

• Sakuro Ozawa

• Satoshi Tagomori

• Takashi Nagayasu

• Tsuyoshi Ozawa

• uu59

• Waldemar Quevedo Salinas

• Yuichi Tateno