Friday, January 4, 2008

Sorting Log files/como files

In my earlier blog, I had given a como viewer, which works on 'n' different files like a tail command works in a single file. This was necessary because the application runs in 'n' different threads and each thread produces its own log file, making monitoring of the processes on single window difficult.

This also gives another challenge, once the application has finished its process, we would like to view the events that occurred in different threads in chronological order of time in a single file, so that it is easy to troubleshoot/diagnose errors that occurred during its execution.

Here is another tool, comosort.py, which takes 'n' number of files as input and gives a single file as output. (You can find the text version here)

The python script does the following,

a. Takes the directory, where the como/log files are there. (Please note that the directory should have only set of files which you want to be combined together - DO NOT have old log files in that directory)
b. read all the files which starts with tSA
c. sorts the contents by thread number, time, job name and creates a single file called sortcomolist. The log information is separated by "_".


You can import it to excel and delimit it by "_" to get the final output.


The script uses mainly python's LIST, FUNCTIONS (map) and sorting by key.