Applying serverless architecture pattern to distributed data processing

Serverless architectures refer to applications that significantly depend on "cloud" services (knows as Backend as a Service) or on custom code that's run in ephemeral runtime (Function as a Service or "FaaS").

To application developers "serverless" mean app where some certain logic of it is still written by the developer but unlike traditional architectures or microservices is run in stateless compute runtime that are event-triggered, may only last for one invocation, and fully managed by a cloud. Serverless helps developers to transfer responsibility of keeping their apps up and running as well as scaling out their workload capacity without involving devops/ops as we got used to.

In this talk we will go through whole "serverless" thing: from decomposing app and its logic to microservices and further to smaller bits, i.e. functions to defining data flow through functions and building their fault-tolerant pipeline.

Moreover, we will go through demo that highlight key takeaways:

• what is functions, what it could and could not be
• how design scalable architecture without getting into troubles by hitting concrete bottlenecks
• how combine multiple programming languages into a single app
• why Python is one of the best programming languages designed exactly for serverless