Spark ML pipelines under the hood I Dynamic Talks [RUS]

Vitalii Monastyrev, BigData Engineer, Grid Dynamics “Modern IT companies actively develop the Data Science stack in their projects to predict profits for subsequent quarters, configure targeted advertising, build a recommendation system, and much more. Quite often, the data used to build Machine Learning models weighs hundreds of gigabytes or more. In this case, many questions often arise: - How to work with so much data - How to generate features - How to train models - How to integrate work between Data Engineer and Data Science teams