Cloud computing is becoming a key solution to provide the underlying infrastructure to run big data analysis in a pay-per-use basis without up-front investment, especially beneficial for SMEs. These business opportunities for SMEs are strategic for Europe & Brazil.
Data-driven decision processes guide many sectors of our economy. In complex systems that do not lend themselves to intuitive models (e.g, natural sciences, social and engineered systems), data-driven modelling hypothesis generation and high performance computing scenario-based simulations have a key role to understanding system behavior and interactions.
Running effectively big data analysis systems is challenging.The execution time of a complex job is generally unknown in advance. Thus, determining the optimal number of nodes in a cluster, shared among multiple users performing heterogeneous tasks, is an important problem.The incoming workload might change, runtime management policies need react to workload fluctuations and application mutual runtime interferences.
EUBra-BIGSE has developed a framework for ensuring the Quality of Service (e.g., guaranteeing that jobs are completed before a given deadline) of data analytics services on top of cloud computing infrastructures, providing:
All the components are platform-agnostic and work on different Cloud Management Platforms (CMPs) avoiding changes to the application code thanks to a vendor-agnostic implementation of the monitoring platform and configuration and adaptation mechanisms. The EUBra-BIGSEA solution includes configuration and contextualisation services and a repository for deployment recipes.