Resource Management (Resource

Allocation, Provisioning, Scheduling

and Load Balancing)
Cloud as a Marketplace of
“Computing Utilities”
Compute Cloud Cluster (VM Pool)
User Pool node
User
VM VM VM
Cloud Manager
Coordinator

Cloud Broker 1 Cloud Broker N Publish Offers Pool node Pool node
...... VM VM VM

VM VM SaaS
Negotiate/Bid
Request Directory
Capacity PaaS
Bank Cloud
Coordinator IaaS
Auctioneer
Storage Cloud
Cloud
Coordinator

Enterprise Cloud Exchange

Resource (CEx)
Server
(Proxy) Storage Cloud Compute Cloud

Enterprise IT Consumer
Market-Oriented Cloud Architecture:
QoS negotiation and SLA-based Resource Allocation
Users/
Brokers

Service Request Examiner and

Admission Control
- Customer-driven Service Management
- Computational Risk Management
SLA - Autonomic Resource Management
Resource
Allocator Pricing Accounting

VM Service Request
Monitor Dispatcher Monitor

Virtual
Machines
(VMs)

Physical
Machines
Market-Oriented Cloud Architecture:
Cloud to expand (lease more resources) with increase in demand
Users/
Brokers

Service Request Examiner and

Admission Control
- Customer-driven Service Management
- Computational Risk Management
SLA - Autonomic Resource Management
Resource
Allocator Pricing Accounting

VM Service Request
Monitor Dispatcher Monitor

Virtual
Machines
(VMs)

Multi-Clouds
Physical
Machines
Market-Oriented Cloud Architecture:
Cloud to shrink (unleash resources) with decrease in demand
Users/
Brokers

Service Request Examiner and

Admission Control
- Customer-driven Service Management
- Computational Risk Management
SLA - Autonomic Resource Management
Resource
Allocator Pricing Accounting

VM Service Request
Monitor Dispatcher Monitor

Virtual
Machines
(VMs)

Physical
Machines
Introduction to Resource Allocation
Resource Allocation Process
Parameters for resource allocating

• Response time: Minimum time to respond to a service request to perform the task.

• Reliability: The ability to successfully complete the runtime.

• Performance: The number of tasks performed on the request of cloud users.

• Execution Time: It is also defined as completion time, which is the time taking to satisfy the demands of cloud users.

• Workload: The amount of processing to be done out for a particular amount of time. It’s the ability to process cloud
computing jobs.

• Utilization: The overall amount of resources currently used in data centers. Cloud computing involves maximizing the
use of resources to optimize the revenue and income of cloud providers to the satisfaction
Parameters for resource allocating

• Throughput: In cloud computing, the total number of tasks fully performed is within a given period.

• SLA: This is an agreement that describes the QoS offered by cloud providers to cloud users. The Cloud provider is
committed to delivering its best service can serve the need of a cloud Customer and avoid violating the SLA.

• Power: The VM placement & migration strategies used in the cloud data center must reduce their consumption.

• Fault-tolerant: The system should continue to provide service in spite of the failure of resources.

• Cost: The amount to be billed for the use of CC facilities. This is an expense to cloud customers and a benefit and
income to cloud providers

• Bandwidth/speed: Maximum data transmission rate of the network links

• Availability: In cloud computing, it represents a collection of services that allow accessibility, maintenance, reliability,
durability, and serviceability of the resources that depend on a request of cloud consumers to perform the specified
or necessary activity
Parameters for resource allocating
VM Provisioning

VM Allocation Policy C1 C2

Alloc

● Management of VMs (creation...): OpenStack

● Defined @ Data Center level
VM1 VM2
• Different Data Centers in the same simulation may use
different policies VMM
● Which host will receive the VM? PEs
• Load balance
• Consolidation (green IT)
● Migration Hosts

Data Center
VM Provisioning

VM Scheduler C1 C2

Alloc

● Defined in Host level

• Different hosts in the same Data Center may VM1 VM2
have different policies
● How to share PEs among VMs in the host?
VMM
• Xen: RR, Credit
PEs
• Time-shared, space-shared, proportional
● Provisioning policies for resources (memory and

bandwidth) also customizable

Hosts

Data Center
Task scheduling

Cloudlet Scheduler
C1 C2

Alloc
• Defined in VM level
• Different VMs in the same host may have different
VM1 VM2
policies
• How to share processing power allocated to a VM among
VMM
Cloudlets?
• OS scheduling PEs

• Time-shared and space-shared

Hosts

Data Center
Broker
C1 C2
VMScheduler
Datacenter Broker
VM1 VM2

VMM
“Cloud scheduling”
PEs
• Selects the Data Center to host VMs
Hosts
• Selects VMs to run Cloudlets
• Application model: PS? BoT? Workflow? Data Center

• Economic decisions
Network

VM1 VM2

C1
C2
User/Broker
Resource management in CC

Resource Management Strategies can be grouped into 2 – Resource Provisioning and Resource
scheduling. Then each of those paradigms can be split into various categories.
Resource Provisioning

Resource provisioning is defined as the act of allocating virtualized resources to users.

Resource Provisioning

• To enable the virtualized resources for allocation to users.

• Create VMs (once, user made request ) and allocate them for the user on
demand.
• Responsible to meet users need based on QoS, SLA and matching the
resources based on upcoming workloads.
• NEED: to detect and select the best/optimal resources based on the
requirements with minimal maintenances.
• Map upcoming requests to the running VMs considering QoS and SLA.
Types of Resource Provisioning

• On-demand provisioning: an intermediate level plan pay per hour basis,

if demand exceeds the reserved value, it arranges resources higher cost
than advanced reservation resources.

• Advanced reservation: long-term plan to reserve the resources in

advance for a specific time period, useful in federated cloud as well as
EC2. prediction of future demand and prices, Overprovisioning and
under-provisioning are key issues

• Spot instances: short-term plan to bid on unused resources, Amazon's

third plan to offer unused resources at lower cost in comparison with on-
demand and advanced reservation (AWS, Google, and Azure), price vary
frequently based on supply and demand is major issue.
Advantages of cloud resource
provisioning

• Efficient resource provisioning techniques reduce makespan time and

response time for submitted workloads.

• The issues of overprovisioning and under provisioning can be reduced

through the optimized utilization of resources.

• Better resource provisioning can be brought to cloud environments

through reducing VMs’ startup delay.

• Both the robustness and fault tolerance capabilities can be brought

using effective cloud resource provisioning algorithms.

• Power consumption can be also reduced using a resource provisioning

algorithm without violating the SLA.
Resource/task Scheduling

Scheduling is the art of analyzing the required QoS parameters with the aim to determine which
activity should be performed. In clouds, scheduling is responsible for:

i. selecting the most optimal VM to execute a task using a heuristic/meta-heuristic algorithm,

and
ii. ensuring the fulfillment of QoS constraints.
Scheduling

• To generate an order of tasks assignments to the allocated VMs considering

quality of service (QoS) parameters and SLA.
• To select optimal virtual machines for execution of tasks using either heuristic
or meta-heuristic algorithm.
• On Demand: SP may assign to random VM, unequal distribution, may occur
over provisioning,
• Long Term Reservation: under provisioning,
• Over provisioning and under provisioning type of problem increase the cost of
services due to unnecessary wastage of resources and time.
Scheduling

• Objective of resource provisioning with scheduling (RPS) is to provision VMs

to users without violation of SLA and fulfill users demand.
• Threshold can be used understand the under/over provisioning.
• Fitness function (FFQoS) is calculated based upon the required QoS
parameters for each workload and compared with the value calculated
without considering QoS parameters (FFnonQoS) for provisioning.
• If value of FFQoS is less than the value of FFnonQoS then it will provision;
otherwise it analyses the workload again after resubmission of SLA by the
cloud consumer through re-negotiation. RP is success then scheduling.
• It may be used to satisfy budget and deadline.
Scheduling

• Before allocation of workload at VMs, cloud running resources are monitored

and load is calculated to verify overloading constraint.
• If VMs are overloaded, VMs are increased using the horizontal scalability
concept.
• Scheduling algorithm can be categorized as: static and dynamic, online and
offline mode scheduling (online scheduling immediately user made request,
in offline batch processing ), preemptive and non-preemptive scheduling.
Resource Scheduling
Provisioning and Scheduling of cloud resources
Resource Scheduling
Different Strategies of scheduling in clouds
Resource Scheduling

Static and dynamic scheduling: Static scheduling algorithms re- quire advance information about incoming tasks
(i.e., task count, task length, task deadline, etc.) and available resources (memory, processing power, node
processing capacity, etc.).
Resource Scheduling

User level and system level scheduling: In user level scheduling, the problem of service provisioning between
service providers and consumers is tackled by the scheduler, which shows a high efficiency, especially when market-
oriented virtualized resources are delivered as a service to users.

Online (immediate) and offline (batch) scheduling: With regard to online scheduling, the scheduler maps the
customer request to running VMs such that the scheduling process is kept stable over time by performing a single
scheduling for each task at a time.

Preemptive and non-preemptive scheduling: In centralized scheduling, all tasks are gathered by a master processor
unit and then are sent to slave processing units where every processor takes over a single dispatch queue
Resource Scheduling

Centralized and distributed scheduling: In centralized scheduling, all tasks are gathered by a master processor unit
and then are sent to slave processing units where every processor takes over a single dispatch queue. On the other
hand, there is no central control unit in distributed scheduling.

Cooperative and non-cooperative scheduling: In cooperative scheduling, all processors achieve their common goal
through collaboration when making a scheduling decision. In non-cooperative scheduling, every individual
processor makes its decision independently while other processors are not affected or ever alerted
Need for Scheduling

• Scheduling mainly aims to handle end-users’ incoming requests by finding out the best cloud
resources that should improve both the resource utilization rate and key performance parameters
(QoS parameters).

• Cloud computing has diverse performance indicators, like makespan, monetary cost, execution
cost, response time, energy consumption, reliability, etc.

• An efficient task scheduling algorithm must be used to analyze and improve these parameters in
order to fulfill the requirements of both end-users and service providers without violating the
SLA.
Categorization of cloud task
scheduling schemes

Traditional Scheduling -

Heuristic Scheduling - Heuristic algorithms depend on the nature of the problem and perform very
well with certain problems while present low performance with others.

Meta-Heuristic Scheduling - meta-heuristic algorithms have gained huge circulation because of

their effectiveness in solving complex and large computational problems.
Meta-heuristic = Heuristic + Randomization.
Categorization of cloud task
scheduling schemes
Mainstream Algorithms under Heuristic algorithms

• Min-Min - Shortest Task will be picked and mapped to a VM

• Max-Min - The longest task is assigned to the VM for execution
• FCFS – Assigns the incoming task based on arrival time. First come first serve.
• HEFT - Heterogeneous Earliest Finish Time is a list-based scheduling heuristic in which a task priority list is firstly
built so that optimal allocation decisions are then made locally for each task based on the task’s estimated
completion time.
• SJF – It assigns the task with the shortest CPU burst time. If 2 tasks have same burst time, FCFS is implemented.
• RR – Round Robin assigns tasks immediately and allocates the available resources to the incoming task
• MCT – Minimum Completion Time schedules the tasks based on their expected minimum execution time
• Sufferage – is a heuristic technique in which a resource is mapped immediately with a task which would likely
suffer the most according to a “sufferage ”threshold value which is associated with its expected completion time.
Categorization of cloud task
scheduling schemes
Mainstream Algorithms under Heuristic algorithms
Meta-heuristics based approaches
in cloud task scheduling
Meta-heuristics based approaches
in cloud task scheduling

Nature of Scheduling Problem

• Single-Objective
• Multi-Objective

Primary Objective of Scheduling

• Makespan
• Monetary cost
• Computational cost
• Reliability and Availability
• Elasticity or Scalability
• Energy consumption
• Security
• Resource utilization
• Throughput
Meta-heuristics based approaches
in cloud task scheduling

Task Resource Mapping Scheme

• Static Scheduling - Static scheduling requires prior information regarding the tasks
for making a schedule decision before a task starts to execute.

• Dynamic Scheduling - Dynamic scheduling can occur during task execution and does
not require knowledge of all task properties. This is useful for handling the
fluctuating demands of cloud users, especially when maximizing the resource
utilization is a higher priority than reducing the execution time.

• AI-Based Scheduling - highly-technical and specialized methodology that supports

the creation of an intelligent technique working and reacting, like humans, to
schedule and assign resources with different aspects.

• Prediction-Based Scheduling - It is associated with the behavior of methods and

various measures while allocating resources.
Meta-heuristics based approaches
in cloud task scheduling

Scheduling Constraints

• Deadline

• Priority

• Budget

• Fault Tolerance
Cloud Task Scheduling
Cloud Task Scheduling
Open Issues and Challenges

• Resource Scheduling
• Quality of Service (QoS)
• Service level agreements (SLAs)
• Self-management service
• Energy management
• Dynamic Scalability
• Reliability
• Security
• Scheduling based on emerging meta-heuristic approaches
Future Trends

• Priority of users: Cloud is ultimately a business model; thus, during the execution of submitted applications, the
prioritization of cloud consumers should be taken into consideration.
• Green computing: Energy-aware task scheduling needs extensive research so that computing resources can be used
more user- and environment-friendly by reducing the use of contaminated materials.
• Resource controlling: Key mechanisms, such as monitoring task migration, VM migration, memory or CPU utilization,
etc., should be handled in a more controlled manner.
• Workload prediction: There is a need for more effective workload estimation techniques to predict the scale of
upcoming workload, thereby increasing both the throughput and resource utilization.
• Network Bandwidth: Generally, network bandwidth has not received enough attention in the majority of current
techniques, although dis- regarding it might cause communication delay, data loss, general network failure, etc.
Future Trends

• Fog computing: Traditional elastic cloud suffers from issues regarding security and delays which can be solved based on
the new trend of fog computing which provides a higher level of heterogeneity and decentralization.
• Failure prediction: Resource failures including resource missing, storage failure, network failure, hardware failure,
software failure, computing failure, database failure, overflow, underflow, and timeout can be predicted using diverse
ML techniques.
• Failure management: The features regarding the management of task migration and failure have been tackled by a few
scheduling algorithms; therefore, future research should address those features for maintaining the availability and
constancy of the system.
• IoT: Managing the IoT devices and multimedia contents are critical recent trends on task scheduling in cloud.
• Next generation computing: Nano-computing-based/Quantum, non- traditional architecture is an attractive
environment that should be involved in the next generation cloud.
Classification of Scheduling in CC
Classification of Scheduling in CC