How to log data from multiple threads?

There are huge numbers of threads running in parallel continuously (let's assume this continuous part)). All the threads want to log some application data, basically a set of values.

What would be the best approach to log this data? single/multiple file?
What would be the best approach to make backup of this log?
What would be the approach to read data from backup file and convert it into something useful?

Several threads like this and this suggest log4net and log4j but I want to know the actual process? Also how multiple threads write to same log file? Is file level lock required for each thread? How does all this work?

Any pointer towards understanding all the details would be appreciated.

Thanks.

java multithreading logging

Edit Report

Regarding point 1, I usually log everything (feature-related) to the same file but the log line always includes some context information that allows me to track (via grep or something else) the flow of the context/request.

Example (a scenario with calls):

DEBUG|CallID#12: Establishing new AUDIO call from AA to BB
DEBUG|CallID#34: Call accepted by ZZ at ...
DEBUG|CallID#99: Call terminated by callee (SS)

This way it's if someone asks "what happened to call from AA to BB at 12:34 today?" I just grep either AA to BB (or the time it happened) and then, once I get the call id, getting the full details of the call is just a matter of grepping again with the id.

Other stuff like chat, presence, etc would go in its own file (wouldn't make much sense to mix this info all in a single monolithic file).

If you want per-thread (instead of per action/request) just log the name of the thread that's performing the action.

Regarding point 2, daily rotation with log4j.

Not sure I understood point 3... Maybe you mean parse a log file to retrieve some patterns? Any tool that supports regex will do the trick (grep being the most handy).

answered 2010-10-05T12:29:43.583

How to log data from multiple threads?

1 Answer

Your Answer