[yapc] Messaging, interoperability and log aggregation - a new framework
admin at yapcna.org
Thu Apr 19 01:00:03 PDT 2012
Tomas Doran will give a talk at YAPC::NA_2012 described as:
In this talk, I will talk about why log files are horrible, logging
log lines, and more structured performance metrics from large scale
production applications as well as building reliable, scaleable and
flexible large scale software systems in multiple languages.
Why (almost) all log formats are horrible will be explained, and why
JSON is a good solution for logging will be discussed, along with a
number of message queuing, middleware and network transport
technologies, including STOMP, AMQP and ZeroMQ.
The Log::Stash framework will be introduced, along with the
logstash.net project which the perl code is interoperable with. These
are pluggable frameworks in ruby/java/jruby and perl with pre-written
sets of inputs, filters and outputs for many many different systems,
message formats and transports.
They were initially designed to be aggregators and filters of data
for logging. However they are flexible enough to be used as part of
your messaging middleware, or even as a replacement for centralised
message queuing systems.
You can have your cake and eat it too - an architecture which is
flexible, extensible, scaleable and distributed. Build discrete,
loosely coupled components which just pass messages to each other
Integrate and interoperate with your existing code and code bases
easily, consume from or publish to any existing message queue,
logging or performance metrics system you have installed. Â
Simple examples using common input and output classes will be
demonstrated using the framework, as will easily adding your own
custom filters. A number of common messaging middleware patterns will
be shown to be trivial to implement.
Some higher level use-cases will also be explored, demonstrating log
indexing in ElasticSearch and how to build a responsive platform API
Interoperability is also an important goal for messaging middleware.
The logstash.net project will be highlighted and well discuss
crossing the single language barrier, allowing us to have full
integration between java, ruby and perl components, and to easily
write bindings into libraries we want to reuse in any of those
[From the YAPC::NA_Blog.]
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the yapc