Tracer Example

http://camel.apache.org/tracer-example.html

http://camel.apache.org/tracer.html

Tracer Interceptor

Camel supports a tracer interceptor that is used for logging the route executions at INFO level.

The Tracer is an InterceptStrategy which can be applied to a DefaultCamelContext or SpringCamelContext to ensure that there is a TracerInterceptor created for every node in the DSL.

You can enable or disable the Tracer's logging dynamically, by calling the tracer's setEnabled method.

Options

Option	Default	Description
formatter		Sets the Trace Formatter to use. Will default use org.apache.camel.processor.interceptor.DefaultTraceFormatter.
enabled	true	Flag to enable or disable this tracer
logLevel	INFO	The logging level to use: FATAL, ERROR, WARN, INFO, DEBUG, TRACE, OFF
logName		The log name to use. Will default use org.apache.camel.processor.interceptor.TraceInterceptor.
traceFilter	null	An exchange Predicate to filter the tracing.
traceInterceptors	false	Flag to enable or disable tracing of interceptors
traceExceptions	true	Flag to enable or disable tracing of thrown exception during processing of the exchange
traceOutExchanges	false	Flag to enable fine grained tracing with a callback for both IN (before) and OUT (after). Is disabled by default which means there is only one trace callback executed.
logStackTrace	false	When tracing exception you can control whether the stack trace should be logged also. If not then only the exception class and message is logged.
useJpa	false	To use a JpaTraceEventMessage from camel-jpa component as the TraceEventMessage. This requires that camel-jpa.jar is on the classpath.
destinationUri	null	Optional destination uri to route TraceEventExchange containing TraceEventMessage with details about the trace. Can be used for custom processing to store traces in database using JPA.
jpaTraceEventMessageClassName	null	Camel 2.3: Fully class name for a custom org.apache.camel.processor.interceptor.TraceEventMessage class which contains the traced information. For example you can use your custom JPA @Entity class to store traced information in a database according to your schema layout.
traceHandler	null	Camel 2.3: To use a custom org.apache.camel.processor.interceptor.TraceEventHandler where you can control what happens when a trace event occurs.
traceInterceptorFactory	null	Camel 2.3: To use a custom org.apache.camel.processor.interceptor.TraceInterceptorFactory where you can create the runtime trace instance which does the actual tracing. It should be a Processor instance. The default tracer is implemented in the class org.apache.camel.processor.interceptor.TraceInterceptor.

Formatting

The tracer formats the execution of exchanges to log lines. They are logged at INFO level in the log category: org.apache.camel.processor.interceptor.TraceInterceptor.
The tracer uses by default the org.apache.camel.processor.interceptor.DefaultTraceFormatter to format the log line.

DefaultTraceFormatter has the following options:

Option	Default	Description
breadCrumbLength	0	Fixed length of the bread crumb. 0 = no fixed length. Setting a value to e.g. 80 allows the tracer logs to be aligned for easier reading.
nodeLength	0	Fixed length of the node. 0 = no fixed length. Setting a value to e.g. 40 allows the tracer logs to be aligned for easier reading.
showBreadCrumb	true	Outputs the unique unit of work for the exchange. To be used for correlation so you can identify the same exchange.
showNode	true	Previous and destination node, so you can see from -> to.
showExchangeId	false	To output the unique exchange id. Currently the breadcrumb is sufficient.
showShortExchangeId	false	To output the unique exchange id in short form, without the hostname.
showProperties	false	Output the exchange properties
showHeaders	true	Output the in message headers
showBodyType	true	Output the in body Java type
showBody	true	Output the in body
showOutHeaders	false	Output the out (if any) message headers
showOutBodyType	false	Output the out (if any) body Java type
showOutBody	false	Output the out (if any) body
showExchangePattern	true	Output the exchange pattern
showException	true	Output the exception if the exchange has failed
showRouteId	true	Camel 2.8: Output the id of the route
maxChars		Is used to limit the number of chars logged per line. The default value is 10000 from Camel 2.9 onwards.

Logging stream bodies Camel Tracer will by default not log stream or files bodies from Camel 2.8 onwards. You can force Camel to log those by setting the property on the CamelContext properties camelContext.getProperties().put(Exchange.LOG_DEBUG_BODY_STREAMS, true);

Example:

ID-claus-acer/4412-1222625653890/2-0 -> to(mock:a)                , Pattern:InOnly , Headers:{to=James} , BodyType:String , Body:Hello London

ID-claus-acer/3690-1214458315718/2-0 is the breadcrumb with the unique correlation id.
node3 is the id of the node in the route path. Is always shown.
To[mock:a] is the destination node.
InOnly is the exchange pattern. Is always shown.
Then the rest is properties, headers and the body.

Showing from and to

The trace log will output both the from and to so you can see where the Exchange came from, such as:

>>> direct:start --> process(MyProcessor)
>>> process(MyProcessor) --> to(mock:a)
>>> to(mock:a) --> to(mock:b)

Enabling

To enable tracer from the main run

java org.apache.camel.spring.Main -t

or

java org.apache.camel.spring.Main -trace

and the tracer will be active.

Enabling from Java DSL

context.setTracing(true);

You can configure tracing at a higher granularity as you can configure it on camel context and then override and set it per route as well. For instance you could just enable tracer for one particular route.

INFO  TraceInterceptor     - ID-davsclaus-local-54403-1246038742624-0-0 >>> from(direct:start) --> MyProcessor     , Pattern:InOnly, Headers:{to=James}, BodyType:String, Body:Hello London
INFO  TraceInterceptor     - ID-davsclaus-local-54403-1246038742624-0-0 >>> MyProcessor --> mock:a                 , Pattern:InOnly, Headers:{to=James}, BodyType:String, Body:Hello London
INFO  TraceInterceptor     - ID-davsclaus-local-54403-1246038742624-0-0 >>> mock:a --> mock:b                      , Pattern:InOnly, Headers:{to=James}, BodyType:String, Body:Hello London
...
INFO  TraceInterceptor     - ID-davsclaus-local-54403-1246038742624-0-1 >>> from(direct:start) --> MyProcessor     , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
INFO  TraceInterceptor     - ID-davsclaus-local-54403-1246038742624-0-1 >>> MyProcessor --> mock:a                 , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling
INFO  TraceInterceptor     - ID-davsclaus-local-54403-1246038742624-0-1 >>> mock:a --> mock:b                      , Pattern:InOnly, Headers:{from=Claus}, BodyType:String, Body:This is Copenhagen calling

Configuring from Java DSL

The tracer options can be configured from the Java DSL like this:

    public void configure() throws Exception {
        // add tracer as an interceptor so it will log the exchange executions at runtime
        // this can aid us to understand/see how the exchanges is routed etc.
        Tracer tracer = new Tracer();
        formatter.getDefaultTraceFormatter().setShowBreadCrumb(false);
        formatter.getDefaultTraceFormatter().setShowNode(false);
        ...
        getContext().addInterceptStrategy(tracer);

Using predicates to filter exchanges

In the code below we want the tracer only to trace if the body contains the text London. As this is just an example can of course set any Predicate that matches your criteria:

    Tracer tracer = new Tracer();
    // set the level to FATAL so we can easily spot it
    tracer.setLogLevel(LoggingLevel.FATAL);
    // and only trace if the body contains London as text
    tracer.setTraceFilter(body().contains(constant("London")));

Enabling from Spring XML

There is now a trace attribute you can specify on the *<camelContext/> for example

  <camelContext trace="true" xmlns="http://activemq.apache.org/camel/schema/spring">
    ...
  </camelContext>

You can see this in action with the SpringTraceTest and its spring.xml file

Another option is to just include a spring XML which defines the Tracer bean such as the one that is automatically included if you run the Main with -t above.

Configuration from Spring

You can configure the tracer as a Spring bean. Just add a bean with the bean class org.apache.camel.processor.interceptor.Tracer and Camel will use it as the Tracer.

<!-- we can configure the tracer by defining the tracer bean and set the properties as we like -->
<!-- the id name can be anything its the class that must be org.apache.camel.processor.interceptor.Tracer -->
<bean id="camelTracer" class="org.apache.camel.processor.interceptor.Tracer">
    <property name="traceExceptions" value="false"/>
    <property name="traceInterceptors" value="true"/>
    <property name="logLevel" value="ERROR"/>
    <property name="logName" value="com.mycompany.messages"/>
</bean>

<camelContext trace="true" xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="direct:start"/>
        <to uri="mock:result"/>
    </route>
</camelContext>

You can configure the formatting of tracer as a Spring bean. Just add a bean with the id traceFormatter and Camel will lookup this id and use the formatter, as the example below illustrates:

<bean id="traceFormatter" class="org.apache.camel.processor.interceptor.DefaultTraceFormatter">
    <property name="showBody" value="true"/>
    <property name="showBodyType" value="false"/>
    <property name="showBreadCrumb" value="false"/>
    <property name="maxChars" value="100"/>
</bean>

<camelContext trace="true" xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="direct:start"/>
        <to uri="mock:result"/>
    </route>
</camelContext>

Enable tracing of out messages

You can trace messages coming out of processing steps. To enable this, configure the tracer as follows

Java DSL

Tracer tracer = new Tracer();
tracer.setTraceOutExchanges(true);

// we configure the default trace formatter where we can
// specify which fields we want in the output
DefaultTraceFormatter formatter = new DefaultTraceFormatter();
formatter.setShowOutBody(true);
formatter.setShowOutBodyType(true);

// set to use our formatter
tracer.setFormatter(formatter);

getContext().addInterceptStrategy(tracer);

or

Spring DSL

<bean id="camelTracer" class="org.apache.camel.processor.interceptor.Tracer">
  <property name="traceOutExchanges" value="true" />
</bean>

<bean id="traceFormatter" class="org.apache.camel.processor.interceptor.DefaultTraceFormatter">
  <property name="showOutBody" value="true" />
  <property name="showOutBodyType" value="true" />
</bean>

Running with these options, you'll get output similar to:

INFO  TraceInterceptor - ID-mojo/59899-1225474989226/2-0 -> transform(body) , Pattern:InOnly , Headers:{to=James} , BodyType:String , Body:Hello London
INFO  TraceInterceptor - transform(body) -> ID-mojo/59899-1225474989226/2-0 , Pattern:InOnly , Headers:{to=James} , BodyType:String , Body:Hello London , OutBodyType:String , OutBody:Hello London

Using Custom Formatter

You can now implement your own org.apache.camel.processor.interceptor.TraceFormatter to be used for logging trace messages to the log.

The sample below shows how to configure a Tracer from Java DSL using custom formatter:

// we create a tracer where we want to use our own formatter instead of the default one
Tracer tracer = new Tracer();

// use our own formatter instead of the default one
MyTraceFormatter formatter = new MyTraceFormatter();
tracer.setFormatter(formatter);

// and we must remeber to add the tracer to Camel
getContext().addInterceptStrategy(tracer);

And here we have our custom logger that implements the TraceFormatter interface where we can construct the log message how we like:

// here we have out own formatter where we can create the output we want for trace logs
// as this is a test we just create a simple string with * around the body
class MyTraceFormatter implements TraceFormatter {

    public Object format(TraceInterceptor interceptor, ProcessorDefinition<?> node, Exchange exchange) {
        return "***" + exchange.getIn().getBody(String.class) + "***";
    }
}

Using Destination for custom processing and routing

Tracer supports custom processing of trace events. This can be used to route a trace event to a JPA endpoint for persistence in a database.

This works by Camel creates a new TraceEventExchange containing:

• snapshot of the original traced Exchange as a immutable TraceEventMessage containing String values of the fields, when the interception occurred. This ensures the fields contains the exact data at the given time of interception.
• the original Exchange can be accessed using getTracedExchange()

Beware to access the original Exchange to avoid causing any side effects or alter its state. Prefer to access the information from TraceEventMessage

Camel routes the TraceEventExchange synchronously from the point of interception. When its completed Camel will continue routing the original Exchange.

The sample below demonstrates this feature, where we route traced Exchanges to the direct:traced route:

// we create a tracer where we want to send TraveEvents to an endpoint
// "direct:traced" where we can do some custom processing such as storing
// it in a file or a database
Tracer tracer = new Tracer();
tracer.setDestinationUri("direct:traced");
// we disable regular trace logging in the log file. You can omit this and
// have both.
tracer.setLogLevel(LoggingLevel.OFF);
// and we must remember to add the tracer to Camel
getContext().addInterceptStrategy(tracer);

Then we can configure a route for the traced messages:

   from("direct:traced").process(new MyTraceMessageProcessor()).to("file://myapp/logs/trace);

And our processor where we can process the TraceEventMessage. Here we want to create a CSV format of the trace event to be stored as a file. We do this by constructing the CSV String and the replace the IN body with our String instead of the TraceEventMessage.

class MyTraceMessageProcessor implements Processor {

    public void process(Exchange exchange) throws Exception {
        // here we can transform the message how we like want it
        TraceEventMessage msg = exchange.getIn().getBody(DefaultTraceEventMessage.class);

        // we want to store it as a CSV with from;to;exchangeId;body
        String s = msg.getFromEndpointUri() + ";" + msg.getToNode() + ";" + msg.getExchangeId() + ";" + msg.getBody();

        // so we replace the IN body with our CSV string
        exchange.getIn().setBody(s);
    }
}

Using JPA as datastore for trace messages

See Tracer Example for complete documentation and how to use this feature.

Traced route path during runtime

Tracer also traces the actual route path taken during runtime. Camel will store the route path taken on the UnitOfWork when Tracer is enabled.

The example below demonstrates how we can use that for error handling where we can determine at which node in the route graph the error triggered.
First we define our route:

protected RouteBuilder createRouteBuilder() throws Exception {
    return new RouteBuilder() {
        public void configure() throws Exception {
            errorHandler(deadLetterChannel("mock:error").redeliveryDelay(0).maximumRedeliveries(3));

            // must enable tracer to trace the route path taken during runtime
            context.setTracing(true);

            // let our my error processor handle all exceptions
            onException(Exception.class).handled(true).process(new MyErrorProcessor());

            // our route where an exception can be thrown from either foo or bar bean
            // so we have enable tracing so we can check it at runtime to get the actual
            // node path taken
            from("direct:start").to("bean:foo").to("bean:bar");
        }
    };
}

And then our custom error processor where we can handle the exception and figure out at which node the exception occurred.

private static class MyErrorProcessor implements Processor {
    public void process(Exchange exchange) throws Exception {
        TracedRouteNodes traced = exchange.getUnitOfWork().getTracedRouteNodes();

        // get the list of intercepted nodes
        List<RouteNode> list = traced.getNodes();
        // get the 3rd last as its the bean
        Processor last = list.get(list.size() - 3).getProcessor();

        // wrapped by JMX
        if (last instanceof InstrumentationProcessor) {
            InstrumentationProcessor ip = (InstrumentationProcessor) last;
            last = ip.getProcessor();
        }

        // set error message
        exchange.getOut().setFault(true);
        exchange.getOut().setBody("Failed at: " + last.toString());
    }

    public String toString() {
        return "MyErrorProcessor";
    }
}

 

=============

 

Tracer Example

Introduction

This example demonstrates the Tracer. Tracer is a tracing feature build in camel core to log snapshots of Exchanges while they are routed. This allows you to see:

• how a given Exchange was routed
• a snapshot of the content of the Exchange at any given node it passed in the route

When used Camel will by default log the snapshot at INFO level. This example demonstrates how to persist trace snapshots usingJPA into a database. This allows you to store this information and query them from a SQL prompt, giving you full power to analyze the data.

Requirements

This requires Camel 2.0, the camel-jpa component and configuration of the target database.

Data Model

Camel uses the org.apache.camel.processor.interceptor.JpaTraceEventMessage JPA @Entity as data model.
This class has been enhanced with JPA annotations.

The class has the following properties in the JPA model:

Property	Type	Description
Id	Long	Primary key that is generated by the database
Timestamp	Date	Timestamp when the snapshot was generated. Is the system time of the JMV in which Camel is running
FromEndpoint	String	the URI of the starting consumer the Exchange was created (usually a from in the route).
PreviousNode	String	id of the previous step in the route. Is null if there wasn't a previous node such as the start.
ToNode	String	id of the next step in the route
ExchangeId	String	unique id of the Exchange
ShortExchangeId	String	id of the Exchange without the machine name
ExchangePattern	String	The Exchange Pattern such as InOnly or InOut
Properties	String	The Exchange properties dumped as a String
Headers	String	The Exchange IN headers dumped as a String
Body	String	The Exchange IN body dumped as a String
BodyType	String	The Exchange IN body java type such as String, org.w3c.Document, com.mycompany.MyOrder etc.
OutHeaders	String	The Exchange OUT (if any) headers dumped as a String
OutBody	String	The Exchange OUT body (if any) dumped as a String
OutBodyType	String	The Exchange OUT body (if any) java type such as String, org.w3c.Document, com.mycompany.MyOrder etc.
CausedByException	String	The Exchange exception (if any) dumped as a String including stacktrace

The table name for persisting trace events is: CAMEL_MESSAGETRACED

Configuration of the database

The Tracer uses standard JPA configuration for setting the database. In the META-INF/persistence.xml file we setup the service unit and the database configuration as:

<persistence xmlns="http://java.sun.com/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             version="1.0">

  <persistence-unit name="tracer" transaction-type="RESOURCE_LOCAL">
    <class>org.apache.camel.processor.interceptor.jpa.JpaTraceEventMessage</class>

    <properties>
      <property name="hibernate.dialect" value="org.apache.camel.example.tracer.FixedDerbyDialect"/>
      <property name="hibernate.connection.driver_class" value="org.apache.derby.jdbc.EmbeddedDriver"/>
      <property name="hibernate.connection.url" value="jdbc:derby:target/derby;create=true"/>
      <property name="hibernate.hbm2ddl.auto" value="create"/>

      <!-- debugging flags -->
      <!-- <property name="hibernate.show_sql" value="true"/>
      <property name="hibernate.format_sql" value="true"/> -->
    </properties>

  </persistence-unit>
</persistence>

What is important is to add the JpaTraceEventMessage as a class in the persistence.xml file to register our data model:

<class>org.apache.camel.processor.interceptor.JpaTraceEventMessage</class>

In this example we use Hibernate JPA and a HSQLDB as database.

Running the example

The README.txt states how to run the example from either ANT or Maven.

Here we show running with Maven:

mvn camel:run

When the application starts it start:

• in the console
• a GUI for browsing the SQL database

Select the console where the application should prompt you to enter some words. Try entering:Camel
The application should respond with a text quote.

You can also enter multiple quotes separate with space, and the response should be the best quote based on the list of words given.
See the file src/main/resources/META-INF/spring/camel-context.xml to give you an idea how it works.

You can enter: Camel Beer and it should be smart enough to find a quote for the beer

Seeing the trace events

When the program was started a GUI application was started as well. Its a SQL prompt for the database. So try entering:

select * from camel_messagetraced

And it should return the list of trace events in the SQL.

We enter this sql: select id, shortExchangeId, previousNode, toNode, body from camel_messagetraced and get the output as the picture below:

Routing

The diagram below illustrates the route diagram generated using Visualisation.

We receive an Exchange from the in stream, then its split using the splitWords method. Then the quote method is invoked before its aggregated and finally sent to the stream out to be printed in the console.

Trace the routing

If we look at the 6 rows from the traced SQL (the first picture) and with the route diagram in mind we can get a better understand how theExchange was routed.

1. The Exchange does not have a previousNode so its the first step where its consumed from the input stream and that its going to the splitter.
2. The exchange id has changed and this is the output of the splitter as it creates a new Exchange. We can also see this one has one word in the body. ThisExchange is being routed to the quote bean next.
3. This is the 2nd output from the splitter containing the 2nd word. This Exchange is being routed to the quote bean next.
4. This is the Beer Exchange where we can see the output from the quote server and that its being routed to the aggregator.
5. This is the Camel Exchange where we can see the output from the quote server and that its being routed to the aggregator.
6. This is the result of the aggregator where the Exchange ending with id 0-2 "was the winner" and is being routed as the aggregated result to the stream out.

Configuration ofJPA tracing in Camel

In Camel you need to configure it to use JPA for tracing. We do this as by adding a tracer in theMETA-INF/camel-context.mxl file:

<!-- use camel jpa trace so we can see all the traced exchanges in a database -->
<bean id="camelTracer" class="org.apache.camel.processor.interceptor.Tracer">
  <!-- turn on jpa tracing, otherwise the TraceEventMessage is a non JPA Entity class -->
  <property name="useJpa" value="true"/>
  <!-- the destination to send the traced messages -->
  <property name="destination" ref="traced"/>
  <!-- we turn ordinary trace logging off so we only see the traced messages in the database,
       but you can have both if you like -->
  <property name="logLevel" value="OFF"/>
</bean>

To properly configure JPA for tracing we must complete these two steps:
1. enable the JPA tracing by setting the property useJpa=true
2. set the destination or destinationUri to a JPA producer endpoint

In this example we set the desintation that refers to a endpoint we define in the camel context:

<endpoint id="traced" uri="jpa://org.apache.camel.processor.interceptor.JpaTraceEventMessage?persistenceUnit=tracer"/>

Here its important that the endpoint is configure with the {org.apache.camel.processor.interceptor.JpaTraceEventMessage}} as entity name and the persistenceUnit as an option. In out example we use tracer.

Then the following is standard Spring JPA configuration:

<!-- this is ordinary JPA spring configuration -->
<bean id="transactionTemplate" class="org.springframework.transaction.support.TransactionTemplate">
  <property name="transactionManager">
    <bean class="org.springframework.orm.jpa.JpaTransactionManager">
      <property name="entityManagerFactory" ref="entityManagerFactory"/>
    </bean>
  </property>
</bean>

<!-- this is ordinary JPA spring configuration -->
<bean id="jpaTemplate" class="org.springframework.orm.jpa.JpaTemplate">
  <property name="entityManagerFactory" ref="entityManagerFactory"/>
</bean>

<!-- this is ordinary JPA spring configuration -->
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalEntityManagerFactoryBean">
  <!-- we have named our presistence unit tracer in the META-INF/persistence.xml file -->
  <property name="persistenceUnitName" value="tracer"/>
</bean>

However we must set the persistenceUnitName to the same unit name we defined inpersistence.xml, such as tracer as we are using in this example.

And if you are wondering how the Camel route is defined then its here:

<route>
  <!-- in stream to allow you to enter some text in the console -->
  <from uri="stream:in?initialPromptDelay=4000&amp;promptDelay=2000&amp;promptMessage=Enter some words:"/>

  <!-- split the text using parallel execution -->
  <split parallelProcessing="true">
    <!-- use methodCall expression to split the words, using a java bean to do it -->
    <method bean="quoteService" method="splitWords"/>

    <!-- for each split message invoke the quote server to get a quote of the word -->
    <to uri="bean:quoteService?method=quote"/>

    <!-- now we need to find the best quote, so we aggregate all the splitted words
         we use our own strategy how to aggregate -->
    <aggregate strategyRef="myAggregateStrategy">
      <!-- correlate everything using constant true, as they are all from the same source -->
      <correlationExpression>
        <constant>true</constant>
      </correlationExpression>
      <!-- complete after 1 sec on inactivity -->
      <completionTimeout>
        <constant>1000</constant>
      </completionTimeout>
      <!-- send the result to stream out so we can see the response in the console -->
      <to uri="stream:out"/>
    </aggregate>
  </split>
</route>