Skip to content

janheuninck/logstash-gelf

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

55 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

logstash-gelf Build Status

Provides logging to logstash using the Graylog Extended Logging Format (GELF). This implementation comes with support for three areas:

See also http://logging.paluch.biz/ for further documentation.

Settings

Following settings can be used:

  • host (since version 1.2.0, Mandatory): Hostname/IP-Address of the Logstash Host
    • tcp:(the host) for TCP, e.g. tcp:127.0.0.1 or tcp:some.host.com
    • udp:(the host) for UDP, e.g. udp:127.0.0.1 or udp:some.host.com
    • (the host) for UDP, e.g. 127.0.0.1 or some.host.com
  • port (since version 1.2.0, Optional): Port, default 12201
  • graylogHost (until version 1.1.0, Mandatory): Hostname/IP-Address of the Logstash Host
  • graylogPort (until version 1.1.0, Optional): Port, default 12201
  • originHost (Optional): Originating Hostname, default FQDN Hostname
  • extractStackTrace (Optional): Post Stack-Trace to StackTrace field, default false
  • filterStackTrace (Optional): Perform Stack-Trace filtering (true/false), default false
  • mdcProfiling (Optional): Perform Profiling (Call-Duration) based on MDC Data. See MDC Profiling, default false
  • facility (Optional): Name of the Facility, default logstash-gelf
  • threshold/level (Optional): Log-Level, default INFO
  • filter (Optional): Class-Name of a Log-Filter, default none
  • additionalFields (Optional): Post additional fields. Eg. .GelfLogHandler.additionalFields=fieldName=Value
  • mdcFields (Optional): Post additional fields, pull Values from MDC. Name of the Fields are comma-separated mdcFields=Application,Version,SomeOtherFieldName

MDC Profiling

MDC Profiling allows to calculate the runtime from request start up to the time until the log message was generated. Only available in log4j environments. You must set one value in the MDC:

profiling.requestStart.millis: Time Millis of the Request-Start (Long or String) Two values are set by the Log Appender:

profiling.requestEnd: End-Time of the Request-End in Date.toString-representation profiling.requestDuration: Duration of the request (e.g. 205ms, 16sec)

Including it in your project

Maven:

<dependency>
    <groupId>biz.paluch.logging</groupId>
    <artifactId>logstash-gelf</artifactId>
    <version>1.2.2</version>
</dependency>

JBoss Module Download:

<dependency>
    <groupId>biz.paluch.logging</groupId>
    <artifactId>logstash-gelf</artifactId>
    <version>1.2.2</version>
    <classifier>logging-module</classifier>
</dependency>

or http://search.maven.org/remotecontent?filepath=biz/paluch/logging/logstash-gelf/1.2.2/logstash-gelf-1.2.2-logging-module.zip

Properties:

handlers = biz.paluch.logging.gelf.jul.GelfLogHandler, java.util.logging.ConsoleHandler

.handlers = biz.paluch.logging.gelf.jul.GelfLogHandler, java.util.logging.ConsoleHandler
.level = INFO

biz.paluch.logging.gelf.jul.GelfLogHandler.host=udp:localhost
biz.paluch.logging.gelf.jul.GelfLogHandler.port=12201
biz.paluch.logging.gelf.jul.GelfLogHandler.facility=java-test
biz.paluch.logging.gelf.jul.GelfLogHandler.extractStackTrace=true
biz.paluch.logging.gelf.jul.GelfLogHandler.filterStackTrace=true
biz.paluch.logging.gelf.jul.GelfLogHandler.timestampPattern=yyyy-MM-dd HH:mm:ss,SSSS
biz.paluch.logging.gelf.jul.GelfLogHandler.maximumMessageSize=8192
biz.paluch.logging.gelf.jul.GelfLogHandler.additionalFields=fieldName1=fieldValue1,fieldName2=fieldValue2
biz.paluch.logging.gelf.jul.GelfLogHandler.level=INFO
log4j configuration -------------- Properties:
log4j.appender.gelf=biz.paluch.logging.gelf.log4j.GelfLogAppender
log4j.appender.gelf.Threshold=INFO
log4j.appender.gelf.Host=udp:localhost
log4j.appender.gelf.Port=12201
log4j.appender.gelf.Facility=java-test
log4j.appender.gelf.ExtractStackTrace=true
log4j.appender.gelf.FilterStackTrace=true
log4j.appender.gelf.MdcProfiling=true
log4j.appender.gelf.TimestampPattern=yyyy-MM-dd HH:mm:ss,SSSS
log4j.appender.gelf.MaximumMessageSize=8192
log4j.appender.gelf.AdditionalFields=fieldName1=fieldValue1,fieldName2=fieldValue2
log4j.appender.gelf.MdcFields=mdcField1,mdcField2

XML:

<appender name="gelf" class="biz.paluch.logging.gelf.log4j.GelfLogAppender">
    <param name="Threshold" value="INFO" />
    <param name="Host" value="udp:localhost" />
    <param name="Port" value="12201" />
    <param name="Facility" value="java-test" />
    <param name="ExtractStackTrace" value="true" />
    <param name="FilterStackTrace" value="true" />
    <param name="MdcProfiling" value="true" />
    <param name="TimestampPattern" value="yyyy-MM-dd HH:mm:ss,SSSS" />
    <param name="MaximumMessageSize" value="8192" />
    <param name="AdditionalFields" value="fieldName1=fieldValue1,fieldName2=fieldValue2" />
    <param name="MdcFields" value="mdcField1,mdcField2" />
</appender>
log4j2 configuration --------------

Log4j v2 supports an extensive and flexible configuration in contrast to other log frameworks (JUL, log4j v1). This allows you to specify your needed fields you want to use in the GELF message. An empty field configuration results in a message containing only

  • timestamp
  • level (syslog level)
  • host
  • facility
  • message
  • short_message

You can add different fields:

  • Static Literals
  • MDC Fields
  • Log-Event fields (using Pattern Layout)

In order to do so, use nested Field elements below the Appender element.

Static Literals

<Field name="fieldName1" literal="your literal value" />

MDC Fields

<Field name="fieldName1" mdc="name of the MDC entry" />

Log-Event fields

See also: Pattern Layout

Set the desired pattern and the field will be sent using the specified pattern value.

Additionally, you can add the host-Field, which can supply you either the FQDN hostname, the simple hostname or the local address.

Option Description
host{["fqdn"
"simple"
"address"]}
Outputs either the FQDN hostname, the simple hostname or the local address. You can follow the throwable conversion word with an option in the form %host{option}.
%host{fqdn} default setting, outputs the FQDN hostname, e.g. www.you.host.name.com.
%host{simple} outputs simple hostname, e.g. www.
%host{address} outputs the local IP address of the found hostname, e.g. 1.2.3.4 or affe:affe:affe::1.

XML:

<Configuration>
    <Appenders>
        <Gelf name="gelf" graylogHost="udp:localhost" graylogPort="12201" extractStackTrace="true"
              filterStackTrace="true" mdcProfiling="true" maximumMessageSize="8192">
            <Field name="timestamp" pattern="%d{dd MMM yyyy HH:mm:ss,SSS}" />
            <Field name="level" pattern="%level" />
            <Field name="simpleClassName" pattern="%C{1}" />
            <Field name="className" pattern="%C" />
            <Field name="server" pattern="%host" />
            <Field name="server.fqdn" pattern="%host{fqdn}" />
            <Field name="fieldName2" literal="fieldValue2" /> <!-- This is a static field -->
            <Field name="mdcField2" mdc="mdcField2" /> <!-- This is a field using MDC -->
        </Gelf>
    </Appenders>
    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="gelf" />
        </Root>
    </Loggers>
</Configuration>    
JBoss 7 configuration -------------- You need to include the library as module (see download above), then add following lines to your configuration:

standalone.xml

<custom-handler name="GelfLogger" class="biz.paluch.logging.gelf.jboss7.JBoss7GelfLogHandler" module="biz.paluch.logging">
    <level name="INFO" />
    <properties>
        <property name="threshold" value="INFO" />
        <property name="host" value="udp:localhost" />
        <property name="port" value="12201" />
        <property name="facility" value="java-test" />
        <property name="extractStackTrace" value="true" />
        <property name="filterStackTrace" value="true" />
        <property name="mdcProfiling" value="true" />
        <property name="timestampPattern" value="yyyy-MM-dd HH:mm:ss,SSSS" />
        <property name="maximumMessageSize" value="8192" />
        <property name="additionalFields" value="fieldName1=fieldValue1,fieldName2=fieldValue2" />
        <property name="mdcFields" value="mdcField1,mdcField2" />
    </properties>
</custom-handler>
Logback configuration -------------- logback.xml Example:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>

<configuration>
    <contextName>test</contextName>
    <jmxConfigurator/>

    <appender name="gelf" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:localhost</host>
        <port>12201</port>
        <facility>java-test</facility>
        <extractStackTrace>true</extractStackTrace>
        <filterStackTrace>true</filterStackTrace>
        <mdcProfiling>true</mdcProfiling>
        <timestampPattern>yyyy-MM-dd HH:mm:ss,SSSS</timestampPattern>
        <maximumMessageSize>8192</maximumMessageSize>
        <additionalFields>fieldName1=fieldValue1,fieldName2=fieldValue2</additionalFields>
        <mdcFields>mdcField1,mdcField2</mdcFields>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
    </appender>

    <root level="DEBUG">
        <appender-ref ref="gelf" />
    </root>
</configuration>

MDC Profiling allows to calculate the runtime from request start up to the time until the log message was generated. You must set one value in the MDC:

profiling.requestStart.millis: Time Millis of the Request-Start (Long or String)

Two values are set by the Log Appender:

  • profiling.requestEnd: End-Time of the Request-End in Date.toString-representation
  • profiling.requestDuration: Duration of the request (e.g. 205ms, 16sec)

License

About

Graylog Extended Log Format (GELF) implementation in Java and log4j appender for logstash

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 100.0%