Re: High performance exception/traceback reporting system

[Daniël W. Crompton <daniel.crompton () gmail com>]

Hi Cal,

I've been thinking of this issue over the weekend and imagined one of the
solutions to have a messaging system which accepted inputs from different
languages and transforms these into a report which can be put into a bug
reporting tool which could use hashes to fingerprint the bugs so you can see
which are the errors which are causing the most problems.

Sounds like a good plugin for a SIEM too.

D.


On 13 February 2011 17:50, Cal Leeming [Simplicity Media Ltd] <
cal.leeming () simplicitymedialtd co uk> wrote:

> Hi,
>
> I haven't started development on this yet, I will post the location of the
> project once I've begun (hopefully next week!)
>
> Cal
>
>
> On Sun, Feb 13, 2011 at 4:36 PM, Daniël W. Crompton <
> daniel.crompton () gmail com> wrote:
>
> > Is there any place I can retrieve the code?
> >
> > D.
> >
> >
> > On 11 February 2011 18:17, Cal Leeming [Simplicity Media Ltd] <
> > cal.leeming () simplicitymedialtd co uk> wrote:
> >
> > > Hey all,
> > >
> > > For the last two years, I've been meaning to write a reporting server
> > > which allows webapps to post their exception tracebacks, which are then
> > > viewable from a centralized location. After having Thunderbird corrupt my
> > > mailbox due to over 250 thousand debug emails, this project has now been
> > > given a bit more priority ;)
> > >
> > > The current prototype stores basic exception information (the file path,
> > > line number, exception type, exception value, originating webapp, node
> > > hostname etc) in the database, and the traceback details are then
> > > serialized, dumped into a file, and the path to that file stored against the
> > > row. A web interface then allows you to browse through these exceptions
> > > (currently via Django admin), and view them using the same prettified
> > > exception page which it shows for actual exceptions. This prettified page
> > > also shows the variables within each frame in the stack, which is very
> > > handy!
> > >
> > > From a developers point of view, this makes life extremely easy, because
> > > all your webapps report to a single place, you can do sphinx searches,
> > > alerts, custom reports etc, and it looks pretty lol.
> > >
> > > The entire thing is going to be open source, and will eventually be a
> > > one-click install with a set up page etc.
> > >
> > > Here are some of the features I am planning on adding, but if anyone has
> > > any suggestions as to what they would like to see in this, please feel free
> > > to mention them!
> > >
> > >    - Tracebacks can be sent to the server primarily via POST request,
> > >    but custom plugins will allow it to pull in via other means (such as mail
> > >    attachments)
> > >    - Alerts can be given different classifications (for example, you
> > >    could configure specific nodes, webapps, or exception types to alert you via
> > >    BulkSMS)
> > >    - Prettified traceback page should initially support Python/PHP,
> > >    other languages can be added as and when.
> > >    - Basic authentication / IP restrictions for the admin login
> > >    - Authentication support for when the tracebacks are POST'd to the
> > >    server
> > >    - Tar source should pre-package a lightweight nginx/uwsgi/python
> > >    environment, so it is self sufficient (this will need to be security
> > >    maintained obviously).
> > >    - A nice, pretty, easy to use interface, because this just makes
> > >    people feel all nice and warm inside ^_^
> > >
> > > I don't want to go as far as to say that it should be used to collect
> > > error_log outputs, I think that would be going a bit too far, the main
> > > reason for having a system like this is simply due to the sheer amount of
> > > information usually contained within a traceback dump, and the Django
> > > prettifier makes it so much easier to debug with!
> > >
> > > Thoughts/criticisms welcome!
> > >
> > > Cal
> > >
> > > _______________________________________________
> > > Full-Disclosure - We believe in it.
> > > Charter: http://lists.grok.org.uk/full-disclosure-charter.html
> > > Hosted and sponsored by Secunia - http://secunia.com/
> >
> >
> >
> > --
> > blaze your trail
> >
> > --
> > Daniël W. Crompton <daniel.crompton () gmail com>
> >
> > <http://specialbrands.net/>
> >
> > <http://specialbrands.net/>
> > http://specialbrands.net/
> > <http://twitter.com/webhat> 
> > <http://www.facebook.com/webhat><http://plancast.com/webhat><http://www.linkedin.com/in/redhat>


-- 
blaze your trail

--
Daniël W. Crompton <daniel.crompton () gmail com>

<http://specialbrands.net/>

<http://specialbrands.net/>
http://specialbrands.net/
<http://twitter.com/webhat>
<http://www.facebook.com/webhat><http://plancast.com/webhat><http://www.linkedin.com/in/redhat>

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/