Lennart Koopmann

{ :blog => true }

Tag Results

5 posts tagged log

Introducing the Graylog2 Analytics Shell

I spent my last evenings hacking on a new Graylog2 feature which will be very useful for running powerful analytics on your log messages: The Graylog2 Analytics Shell. The detailled description is in the wiki.

Use this shell to extract information from your log messages. I always like the use case of API monitoring. You could easily find out if that one call is still used by consumers - Or even which consumers are still using it with a simple distinct call:

all.distinct({_oauth_consumer_key}, processed_controller = /^PrivateMessages#.+/)

Watch the shell in action in this short screencast: (Try the full screen mode and HD video)

This is the first working implementation but there is still stuff missing. For example:

  • Shell history available with up and down arrow keys
  • Improved design and layout
  • A history of count and distinct results
  • Tab completion (ZOMG!!)
  • Saving and sharing of commands
  • Short names for streams, to avoid typing the raw stream ID

The shell will be included in the next version: 0.9.6

Post your comments here or discuss in the mailing list.

mongo_analyzer

I wrote a little Sinatra application that can be very helpful at optimizing your MongoDB applications. It was very handy while optimizing the Graylog2 MongoDB parts so I made it public and configurable.

It allows you to enable/disable the MongoDB profiler and see it’s results in an overview page. Just start the profiler, run the MongoDB parts of you application you want to test and reload the mongo_analyzer overview page. It will list all queries that took longer than 100ms when in slow queries mode or all queries that were executed when in all queries mode.

You can also view and manage the indexes of your collections to try out effects on the query execution times.

I consider this software as beta. Please report bugs or feature requests in the GitHub issue tracker.

Download and installation instructions: https://github.com/lennartkoopmann/mongo_analyzer

Easily log all your Rails Exceptions to Graylog2

I wrote and published a gem that you can load as Rack middleware in your Rails application. It transparently fetches all your Exceptions (and throws them back to the application so it does not affect your own Exception handling) and sends them as a GELF message to your Graylog2 installation.

Installation

Install the gem. It has my GELF gem as dependency and will install it automatically:

lennart ~$ sudo gem install graylog2_exceptions
Successfully installed gelf-0.9.1
Successfully installed graylog2_exceptions-0.5.3

Now open your config/environment.rb and add the gem as Rack middleware: (Make sure to require 'graylog2_exceptions' before.)

config.middleware.use "Graylog2Exceptions", { :host => '127.0.0.1', :port => '12201', :local_app_name => 'mama' }

Parameter explanation:

  • host: The hostname or IP of your Graylog2 server
  • port: The GELF port your Graylog2 server is listening on (Standard: 12201)
  • local_app_name: This name will be used as “host” field (Optional)
  • level: The syslog level for all Exceptions (Optional)

That’s it: Restart your app and all your Exceptions will be logged to Graylog2!

(Source code of the gem at GitHub)

Git: Show all commits since a tag

This git command shows you all commits after tag ‘0.9’: ~$ git log 0.9..

lennart@sundaysister:~/workspace/graylog2-web-interface$ git log 0.9..
commit 699ea1984279b5c5db2f419ac740b488bd874ed9
Author: Lennart Koopmann <lennart@scopeport.org>
Date:   Sun Aug 1 20:13:47 2010 +0200

    build script now deleting dashboard logo in release

commit cf033a2e7425ac790fc4c08217111fd0d1c46bd9
Author: Lennart Koopmann <lennart@scopeport.org>
Date:   Sun Aug 1 20:12:45 2010 +0200

    First version of dashboard! Includes new setting: Maximum messages

commit ac100bf05687d14307a5ed055f67ca3ef4081e8e
Author: Lennart Koopmann <lennart@scopeport.org>
Date:   Sun Aug 1 19:10:46 2010 +0200

    deleted old files, reset change that was not meant to be committed