Logging in OSGI Enterprise Applications, Part 4

Part 4 will demonstrate how to start the logging - framework and all the bridges inside an OSGI Enterprise Application.

To remember: At the end of the last (third) part of my blog - series "Logging in OSGI Enterprise Applications" we were able to log all events from „classic“ log-frameworks, OSGI LogServices, BundleEvents and FrameworkEvents using a SLF4J Log - implementation (LOGBack):

If we want to use it all inside a complex OSGI Enterprise Application its important to start the bundles using the right start-level. We also have to decide which bundles can be auto-started from the framework and which not. 

As an example I'll use again my OSGI - Equinox - Riena - EasyBeans - Hibernate - Server. I'll talk in detail about this server later in one of the follow-ups.

From OSGI Launch Configuration:

Default Start - Level is set to 3. 

Our osgi-over-slf4j - Bundle must be started before (Start-Level = 2) and the EasyBeans - Agent - Bundle later (Start-Level = 4)

We also have to watch that some bundles cannot be started with Auto-Start = true ! These bundles will be started under control from EasyBeans Agent bundle.

What happens now starting the OSGI server application ?

1. The OSGI - Framework Equinox will be started
   
2. Our org.ekkehard.osgi.over.slf4j bundle will be started. The Activator calls in the start() method SLF4JBridgeHandler.install() to catch all log - events from java.util.logging - also the Activator contains Logger - Statements. All this causes starting and activating SLF4J - API and SLF4J - implementation (LOGBack) - bundles: our Logging Framework is alive !
   
3. All bundles with Auto-Start = true will be started then from the Framework. This includes all Eclipse Riena bundles. Riena uses LOG4J - so the LOG4J - bridge will be started. Riena also uses Equinox - Extended - OSGI - Log - Services.
   
4. After starting all Default - Start - Level - bundles, the easybeans.agent will be started. This agent starts all EasyBeans components and Hibernate. EasyBeans uses Commons - Logging and java.util.logging - Hibernate uses SLF4J. Then all our logging -  framework - bridges are started and active. EasyBeans then starts all Entity - bundles and EJB3 - bundles from our server and registers the EJB3 - Business - Interfaces (Local oder Remote) als OSGI - Services.
   
5. The org.ekkehard.server bundle tracks OSGI Managed Services and recognizes, when all required services are configured from EasyBeans and Hibernate and ready-to-go. Then org.ekkehard.server registers Services as Remote Services for Eclipse Riena.
   

Remark: This is only a small part of all the activities and tasks happening while starting the server. More about 4. and 5. later in another blog series. Lets first finishing the logging.

No black holes ?

If we did it all well, then our SLF4J - Implementation LOGBack should start so early that we get all log - events and nothing is lost in space. 

To test this we configure our logback-test.xml as follows: set Root Level to DEBUG, output to Console, swich DEBUG - Mode of the configuration itself on and start the server.

You should get a log like this:

• Line 1: equinox - osgi - Console
  
• Line 2 - 7: Debug-output from LOGBack configuration
  
• Line 8 - 9: Logging-output from osgi-over-slf4j - bundle
  

There's no other logging-output on the console before starting SLF4J / LOGBack :-)

The next blog will look at the configuration of LOGBack and some special Fragment - Bundles for configuration and also extending the LOGBack - implementation.

• Part 1: An overview
  
• Part 2: How to catch all log events from "classic" logging - frameworks
  
• Part 3: How to catch all log events from OSGI Log Services 
  
• Part 4: How to start logging bundles the right way inside an OSGI Enterprise application
  
• Part 5: ... follows (Stay tuned...)
  

Perhaps some of my older blogs about Logging und OSGI are also interesting:

• Logging (Riena - EasyBeans - Equinox) reloaded
  
• Catch-22 Logging in OSGI Environments
  

 the blog in german.

Logging in OSGI Enterprise Applications, Part 3

Part 3 explains how to catch all log events logged by OSGI Bundles using  OSGI Log Services. 

In part 2 you've seen that catching log entries from "classic" Log - Frameworks was very easy:

• ✓ Choose the right bundles
  
• ✓ Create a config - file 
  

-thats all :-)

Remark: java.util.logging needs also this one-liner: SLF4JBridgeHandler.install();

Running an OSGI Enterprise Application you'll also find bundles logging their log - entries using OSGI Services:

• org.osgi.service.log.LogService 
  
• org.osgi.service.log.LogReaderService 
  

You have also to watch for Services extending the "normal" OSGI Log Services: Eclipse Riena per ex. uses from Equinox Incubator:

• org.eclipse.equinox.log.ExtendedLogService 
  
• org.eclipse.equinox.log.ExtendedLogReaderService
  

You can read Log entries using the LogReaderService. You only have to register your LogListener. 

The LogListener gets all LogEntry (ExtendedLogEntry), then you can log them using SLF4J-API to your LOGBack Implementation. 

You have to use the same config file (logback.xml or logback-test.xml) as described in part 2 (logging the entries from „classic“ Log - frameworks). Details about configuration will follow in another Blog of this series.

Short overview:

We already have following bundles used as bridges between „classic“ Log-Frameworks and SLF4J / LOGBack:

• slf4j.api
  
• jul.to.slf4j
  
• log4j.over.slf4j
  
• jcl.over.slf4j
  

There are also the bundles of the SLF4J - implementation LOGBack:

• logback.core
  
• logback.classic
  

You'll find more details in part 2 of this Blog series.

Bundle osgi.over.slf4j (a bridge between OSGI Service and LOG4J)

We need a bundle as bridge between OSGI Log Services and SLF4J / LOGBack. Please create a Plug-In project using PDE:

• my.namespace.osgi.over.slf4j
  

Your MANIFEST.MF should look like:

This osgi.over.slf4j - bundle has its own org.slf4j.Logger:

We'll use this Logger to log events from this bundle itself and also to log all catched LogEntry from other bundles.

To track LogReader Services we can use a ServiceTracker:

We create our LogServiceTracker with this filter to get only services from type LogReaderService or ExtendedLogReaderService:

(|(objectClass=org.osgi.service.log.LogReaderService) (objectClass=org.eclipse.equinox.log.ExtendedLogReaderService))

A LogListener for „normal“ OSGI LogServices:

LogEntry will be logged using the Logger of our osgi.over.slf4j - bundle.

A LogListener for „extended“ Equinox OSGI LogServices:

ExtendedLogEntry already contains an own Logger. 

We'll use an org.slf4j.Logger with same name - but to distinguish we use a prefix („X.“). Details will follow in another Blog.

Then we delegate LogEntry and ExtendedLogEntry to the org.slf4j.Logger:

We have to map the OSGI LogService Levels. We can get the origin message and (optional) exception from LogEntry and delegate to our Logger.

Different from Log4J or Commons-Logging is using a Marker. Markers are part of the SLF4J API, but at the moment only the LOGBack - implementation uses them. We are using Marker to print the bundle-name and perhaps also the service-name. Then we know which bundle / service originally creates the LogEntry or ExtendedLogEntry. 

Remark: More about Marker you'll find in one of the next Blogs of this series.

Logging BundleEvent and FrameworkEvent 

Its a good idea also to log org.osgi.framework.BundleEvent and org.osgi.framework.FrameworkEvent. 

To do this we need these Listener:

Dependent from the Event - Type we create the message and level.

Remark: More about SLF4J - Marker in one of the follow-ups.

--------------------------------------------------------------------------------

Now we are ready to catch and log all events from „classic“ Log-Frameworks, OSGI LogServices, BundleEvents or FrameworkEvents using SLF4J / LOGBack:

This was the third part of my blog series "Logging in OSGI Enterprise Applications":

• Part 1: An overview
  
• Part 2: How to catch all log events from "classic" logging - frameworks
  
• Part 3: How to catch all log events from OSGI Log Services 
  
• Part 4: ... follows (Stay tuned...)
  

Perhaps some of my older blogs about Logging und OSGI are also interesting:

• Logging (Riena - EasyBeans - Equinox) reloaded
  
• Catch-22 Logging in OSGI Environments

the blog in german.

Logging in OSGI Enterprise Applications, Part 1

Logging is an important part of ERP Applications and should be easy using a modern logging framework ;-)

My actual work-in-progress (german ERP solution) contains: Equinox as OSGI Framework, Eclipse Riena for Remote Services + UI, EasyBeans as EJB3 Container + JPA (Hibernate) etc.

This means I have to integrate different logging - frameworks:

• org.osgi.service.log
  
• org.eclipse.equinox.log (Extended Log Services from 1.1.0.HEAD)
  
• org.apache.commons.logging
  
• org.apache.log4j
  
• org.slf4j
  
• java.util.logging (JSR47)
  

If you already followed some of my blogs, then you probably know that its not easy to design a clean OSGI architecture for a complex business application.

Actually I was confronted with the newest snapshot of EasyBeans containing a new Hibernate release (3.4) now using SLF4 as logging framework. This causes me to rethink my logging architecture.

The logging solution should make my OSGI client / server - application independent from logging frameworks. I'll explain all in detail later.  At first I'll give you an overview to see the differences between "classic" Java logging frameworks and OSGI Logging Services.

"Classic" Java Logging Frameworks

•  Apache Commons - Logging (JCL)
  
•  Log4J - Logging
  
•  SLF4J - Logging
  
•  JDK - Logging (java.util.logging)
  

Usually logging works this way:

Logger definition:

Commons-Logging:

private static Log logger = LogFactory.getLog(MyClass.class);

Log4J:

private static Logger logger = Logger.getLogger(MyClass.class.getName());

SLF4J:

private static Logger logger = LoggerFactory.getLogger(MyClass.class);

JDK:

private static Logger logger = Logger.getLogger(MyClass.class.getName());

Logging a message:

logger.info(„my message“)

Because concatenating Strings is slowly we also find: 

if (logger.isDebugEnabled())

   logger.debug("message part 1" + aVar

   + "message part 2");

Remark: If you're using SLF4J (FAQ) in many cases you can simple use a parameterized message string:

logger.debug("message part 1 {} message part 2", aVar);

Configuring logging - frameworks:

Configuration can be made using Java code - but mostly you'll find a configuration file - wether in properties or XML format. 

Whats different if you're using these logging frameworks under OSGI ?

• you need a bundle for each JAR. You can find bundles for Commons-Logging, LOG4J and SLF4J partly in Eclipse or get them from a bundle repository
  
• logging itself is the same as in normal Java applications
  
• you have to place the configuration file into a Fragment-Bundle
  

OSGI Logging Services

Under OSGI you can use Logging Services:

• Standard OSGI : org.osgi.log (OSGI Logging, implemented by org.equinox.log)
  
• Extended: org.equinox.log (1.1.0.HEAD Version from CVS or Riena download), extended Logging)
  

OSGI Logging uses Services and Listeners. Equinox implements OSGI Log Services as ManagementServices. 

• LogService or ExtendedLogService - writes all log messages
  
• LogReaderService or ExtendedLogReaderService - where you can read log messages from
  
• LogListener - gets the messages from ReaderService and you can handle them
  

OSGI LogService can be compared with the Logger from "classic" Java logging. Its a good idea to place LogService Logger objects into the Activator of a bundle.

LogService logger = ...

Logging a message is nearly the same as logging in non - OSGI environments - of course we have to test if the service is available:

if (logger != null) {

     logger.log(LogService.LOG_INFO, „my message“);

}

The LogEvent can then be read using the LogReaderService. You can "track" this ReaderService  and add your own LogListener. This LogListener gets all messages and you can persist the messages or print to console or use one of the "classic" logging frameworks.

Equinox allows you to configure how many messages should be cached from LogReader Service and at which Log Level. But important to know: The LogReaderService sends always all messages to all registered LogListener independent from your configuration !

"Classic" Logging or OSGI Log Services ?

Lets look at the differences between "classic" Java - logging und OSGI - Log Services:

a) Which messages will be logged - and when ?

"Classic" Logging only creates log - messages if the Logger answers "yes" to „should I really log this message at those level for 'the.logger.name'“ ? It depends from the content of your configuration file, what you want to log at which level. If the Logger answers "no" - nothing will be logged.

OSGI Log Services always log ALL messages. The Log Reader Service gets them ALL. The Log Reader Service will inform all Log Listeners about all logged messages. Also the Log Reader Service will cache a configurable amount of messages.

The main task of a Log Listener is to filter the messages and to persist or print them.

b) What will be logged ? And whats the content of the Log Messages

Here you'll see the most important content of a Log Entry:

The different logging frameworks are also using different Log Levels, so they have to be mapped.

The logged Message is a String or will be created from myObject.toString().

MDC (Mapped Diagnostic Context) - a Thread - local Map where you can store context informations, per ex. user info of client sessions.

Marker - a tree of Strings you can mark messages with.

In an OSGI environment its very useful to log informations about the Bundle (and perhaps the Service Reference) where the logged message comes from.

Using OSGI Log Services you'll get this for free. If you're using SLF4J / LOGBack you can also store Bundle- and Service Names. (Using the Marker - more about this later)

c) Whats the best to use ? "Classic" Logging Frameworks or OSGI Log Services ?

Don't ask this question if you're developing an OSGI Enterprise - Application ;-)

Why not ? 

• You'll never know in advance, which (foreign) projects you'll have to integrate while the whole life-cycle of your business application. 
  
• You have to live with 3rdParty Bundles using all kind of logging - frameworks and -strategies. (Sometimes they change it as with Hibernate happened)
  

But you have to find the answer, HOW to combine and integrate them all without loosing flexibility. If you've solved this issue, then it also doesn't matter what you're using in your own bundles, because your logging architecture consumes it all :-)

Perhaps specific use cases can force you to use 'classic' logging and OSGI Log Services.

Integration of all Logging Frameworks and Log Services

The heart of my solution is SLF4J with LOGBack as native implementation. 

SLF4J (Logback) has already bridges for all 'classic' logging frameworks: this enables you to use bundles logging to Commons-Logging (JCL), Log4J, JDK (java.util.logging) or SLF4J. All logged messages will be transparently routed to LOGBack.

Also all OSGI LogServices - wether Standard or Extended - will be catched from Log Listeners and delegated to SLF4J (LOGBack) delegiert.

So you have ONE place to handle all your logging :-) It depends only from your configuration, if LOGBack prints messages to Console or sends them to a socket (local or remote), to your filesystem, RDBMS or to a LogServer or ... 

(Logging) - Integration of Eclipse Riena

If you're using Eclipse Riena M4 there are currently two small problems to solve:

1. Log4J-over-SLF4J

The bridge Log4J-over-SLF4J only  provides the package org.apache.log4j, but  org.eclipse.riena.core has a dependency to org.apache.log4j.xml. Please change this to "optional" in the MANIFEST.MF:

Import-Package: org.apache.log4j,

org.apache.log4j.xml;resolution:=optional,

org.osgi.framework;version="1.3.0",

org.osgi.service.cm;version="1.2.0",

org.osgi.service.log;version="1.3.0"

2. multiple log entries

Starting with M5 Riena's logging can be configured and its promised that you can switch it all OFF. While using M4 you'll notice two extra messages printed from Rienas LogListener to Console. (Bugzilla 247680)

(Logging) - Integration of EasyBeans OSGI

EasyBeans logging is now transparently routed to LOGBack without any problems. Since some days the Snapshots of EasyBeans 1.1.0 includes Hibernate 3.4, which itself now is using SLF4J. This also causes no problems because LOGBack is native Implementation of SLF4J.

additional informations

This blog is only the first one of a serie about Logging and OSGI. Perhaps it makes sense that you also take a look at my (older) blogs to know the hard way I went ;-)

• Logging (Riena - EasyBeans - Equinox) reloaded
  
• Catch-22 Logging in OSGI Environments
  

Links:

• Simple Logging Facade for Java (SLF4J)
  
• LOGBack - Loggingframework, native SLF4J implementation
  
• Eclipse Riena (Remote Services, UI enhancements)
  
• Eclipse Equinox (OSGI Framework)
  
• EasyBeans OSGI (EJB3 Container as OSGI bundles)
  
• Hibernate (JPA - integrated into EasyBeans)
  

How goes it on ?

This blog is only part 1 of the series „Logging in OSGI Enterprise Applications“ giving only a short overview - the following themes will be later discussed in detail with source codee examples - so stay tuned.

• HowTo registrier OSGI Logging Services
  
• Tracking (standard and extended) OSGI Logging Reader Services
  
• Adding Log Listeners to LogReader Services
  
• HowTo configure LOGBack for OSGI
  
• Mapping of LogLevels
  
• Details of used Bundles, Manifest - files, dependencies,...
  
• Fragment-Bundles
  
• OSGI Launch configurations
  
• Testing the whole stuff
  
• Use of EasyBeans JDBC Pooling to persist messages from LOGBack in RDBMS
  
• Performance issues (frameworks, "classic" logging vs. OSGI LogServices)
  
• HowTo migrate existing code
  
• Useful Eclipse PlugIns (Log4E, LogBack Console)
  
• Client-/Server Logging Strategy, Remote Services, MDC - Userinfo...
  
• The „black hole“: how to log before the services are ready ?
  
• Use of SLF4J Marker to put Bundle- und Service - informations into
  
• What happens if other bundles are also using Log Listeners and log messages ?
  
• ...
  

Remark: In my article I'm not covering PAX OSGI Logging. PAX uses Log4J as implementation and also integrates some logging frameworks (JCL, JDK, SLF4J, Avalon). Perhaps you should also take a look at PAX before deciding how to use logging in your OSGI application.

A more detailed complete article is already in work about „Logging in OSGI Enterprise Applications“ - and I'll continue this blog series.

Examples with sourcecode will also follow - I'm jst preparing a sample of my  Equinox -based Riena-Easybeans-Server, where the logging of course is an important aspect.

Hopefully you can get some ideas from my experiences and ideas how to solve the logging problem. I promise: if you're developing OSGI enterprise applications - one day you'll run into logging problems, so its better you're prepared.

 in german.

Eclipse Projects + OSX - some Tips

More and more developers are using eclipse under OSX. Here are two tips from my current experiences:

1. Never name an Eclipse Project *.service

A project name like my.namespace.something.service causes OSX to think its an OSX service and the Finder will show it as an OSX Service and not as folder:

Using right-mouse-click you can look into the "package":

Last months it happens sometimes, that my *.service projects suddenly contain only 0 Bytes and the projects were empty. So I renamed all those projects.

2..DS_Store can confuse P2

Using the Finder of OSX to view folders and files from Eclipse installations causes the Finder to create .DS_Store files.

Suddenly I got on error while starting Eclipse (Bugzilla 247177). Reason was an unexpected .DS_Store file.

Using this command you can remove all .DS_Store files from a whole folder hierarchy:

    find . -name *.DS_Store -type f -exec rm {} \;

(Its always a good idea on OSX to make „.“ - Systemfiles visible, because many important Eclipse files start with „.“: .project, .classpath, ...)

blog in german

PDE and MDSD [problem generated code and nested src]

Its a good practice in a model-driven project to separate generated code from manually written code. (Generated code normaly will not be in your cvs / svn)

Eclipse Modeling and openArchitectureWare makes this easy: you can define different outlets in your workflow and XPand templates for generated code. manual code, resources etc.

As result you get a Plug-In Project with some nested src - folders. This is no problem for the Eclipse Java Compiler.

If you try to Export PDE - as deployable Plug-In you run into an error: 

Errordialog:

Logfile:

PDE cannot export Plug-In projects with nested src folders. (s.a. Bugzilla with a small example project from me 241830)

Workarounds: 

1. Before PDE - Exporting copy the Plug-In Project into a temporary second Plug-In and merge all src - folders into the one and only /src root folder. (Now the workflow while developing and testing isn't as easy as before)
   
2. Instead of PDE Export you can use a normal Java - Export as JAR (Loosing the comfort from PDE to auto-detect the Manifest file, versioning ....)
   
3. Create some ANT Tasks...
   

But I'm still looking for the pure PDE solution. Perhaps anyone has an idea how to make PDE happy with nested src folders ?

blog in german