Pierre-Yves is CTO and co-founder at Exoscale where he is responsible for product, architecture, and strategic technology choices. Pierre-Yves relies on experience in the architecture of very large corporate systems as well as technical product design in several startups. Pierre-Yves is a member of the Apache Foundation, a Datastax MVP for Apache Cassandra, an OpenBSD developer as well as a core maintainer of Riemann and Collectd.
As the utility of processing heterogeneous data reaches an increasing number of sectors and businesses, the need to get faster insights and avoid large batch cycles is growing as well.
This talk will first outline what the key concepts of big-data and how stream processing fits into this picture. We will then show-case a reference architecture for building stream processing pipelines and walk through how to implement real-time billing from resource events in a distributed system.