EUROPE - BRAZIL COLLABORATION OF BIG DATA SCIENTIFIC RESEARCH THROUGH CLOUD-CENTRIC APPLICATIONS

EUROPE - BRAZIL COLLABORATION OF BIG DATA SCIENTIFIC RESEARCH THROUGH CLOUD-CENTRIC APPLICATIONS

Cloud & Quality of Service

Cloud computing is becoming a key solution to provide the underlying infrastructure to run big data analysis in a pay-per-use basis without up-front investment, especially beneficial for SMEs. These business opportunities for SMEs are strategic for Europe & Brazil.
Data-driven decision processes guide many sectors of our economy. In complex systems that do not lend themselves to intuitive models (e.g, natural sciences, social and engineered systems), data-driven modelling hypothesis generation and high performance computing scenario-based simulations have a key role to understanding system behavior and interactions.

Running effectively big data analysis systems is challenging.The execution time of a complex job is generally unknown in advance. Thus, determining the optimal number of nodes in a cluster, shared among multiple users performing heterogeneous tasks, is an important problem.The incoming workload might change, runtime management policies need react to workload fluctuations and application mutual runtime interferences.

EUBra-BIGSE has developed a framework for ensuring the Quality of Service (e.g., guaranteeing that jobs are completed before a given deadline) of data analytics services on top of cloud computing infrastructures, providing: 

  1. Smart policies for vertical and horizontal elastic adjustment of resources allocated to meet deadlines and dynamic adjustment of workloads at the level of Virtual Machines, containers and physical servers.
  2. A rich and comprehensive API to enable Virtual Appliances to monitor performance indicators ranging from the low-level to the application-specific, spanning different resources.
  3. The support of advanced business models (price-based, dynamic re-scheduling of data analyses to exploit the best use of infrastructural resources, minimising costs) for on-premise or public infrastructures.
  4. The use of standards for the definition of the software appliances.

All the components are platform-agnostic and  work on different Cloud Management Platforms (CMPs) avoiding changes to the application code thanks to a vendor-agnostic implementation of the monitoring platform and configuration and adaptation mechanisms. The EUBra-BIGSEA solution includes configuration and contextualisation services and a repository for deployment recipes. 

 

Find out more about the EUBra-BIGSEA Quality of Service solutions: 

EC3 - Elastic cloud computing

Performance guarantee for Big Data applications

dagSIM