SSRN 5071987
SSRN 5071987
net/publication/387712417
CITATIONS READS
0 21
1 author:
Saurabh Deochake
SentinelOne
20 PUBLICATIONS 65 CITATIONS
SEE PROFILE
All content following this page was uploaded by Saurabh Deochake on 04 January 2025.
Saurabh Deochake[0000−0002−3757−6463]
SentinelOne Inc.
444 Castro St, Mountain View, CA 94041 USA
saurabh.deochake@sentinelone.com
1 Introduction
In today’s fast-changing digital landscape, cloud computing has become an es-
sential tool for businesses looking to promote innovation, improve agility, and
achieve scalability. Organizations across industries are progressively shifting to
the cloud, drawn by the promise of increased performance and lower infrastruc-
ture expenses. Therefore, with the increasing growth of public cloud infrastruc-
ture hosting, conventional on-premise data center-based businesses are shifting
their workloads to the public cloud. Compared to traditional data center in-
frastructure, public clouds offer greater elasticity, efficiency, and scalability for
Infrastructure as a Service (IaaS). However, the spike in cloud use frequently
results in unexpected obstacles, particularly in monitoring and controlling cloud
expenses. As cloud services become more prevalent, the complexities of invoic-
ing and resource allocation also become commonplace. This makes it increasingly
difficult for businesses to maintain a tight grip on their financial management.
Cost optimization, therefore, has become an increasingly important concern
for organizations of all sizes as cloud computing adoption continues to grow
year by year. According to a Flexera report, in recent years, the main cloud
2 Saurabh Deochake
initiative for businesses has been cost optimization [1]. Furthermore, Gartner
predicts that "organizations that do not have a cost optimization program in
place will overspend by up to 70%" through 2024 [2]. The impacts of failing to
manage cloud costs are obvious: wasted resources, increased expenses, and lower
overall profitability. Therefore, understanding and implementing effective cloud
cost optimization strategies is critical for businesses to remain competitive in
the modern cloud computing landscape [3].
To manage and optimize these operating expenses (OpEx) costs, the con-
cept of Financial Operations (FinOps) emerges as a game changer. FinOps is a
collaborative method that bridges the gap between finance, engineering, and op-
erations teams, allowing organizations to efficiently control cloud expenditures.
FinOps enables teams to make informed decisions about cloud resource utiliza-
tion by cultivating an accountability and transparency culture, ensuring that
expenditure is in sync with business objectives.
This paper proposes ABACUS - Automated Budget Analysis and Cloud Us-
age Surveillance, an automated FinOps solution developed specifically to address
the challenge of cloud cost optimization. ABACUS leverages automation and
data analytics to give enterprises access to cloud cost attribution, budgeting,
alerts, and infrastructure spend. Furthermore, the solution helps teams discover
unproductive cloud resources and recommend smart cost-cutting methods, such
as temporarily blocking new cloud workloads when budgets are exceeded. ABA-
CUS easily connects with financial planning data, including OpEx costs and bud-
gets. By integrating with existing cloud providers and utilizing cost-attribution
techniques such as resource tagging, it also translates and assigns budget data
to cloud resources belonging to a specific cost center or department.
The structure of this paper is as follows. Section 2 delves into the critical
necessity for cloud cost optimization. Section 3 explores in depth the guiding
principles of FinOps. Furthermore, Section 4 introduces ABACUS as a system
that applies the key tenets of FinOps to the challenge of cost minimization.
Section 5 examines the future work recommended to enhance the solution with
novel features. Finally, Section 6 concludes the paper.
but it may also lead to unexpected cost spikes if not closely monitored. Fur-
thermore, the complexity of cloud billing, which often includes both fixed and
variable costs, makes it difficult for businesses to accurately estimate and budget
for cloud expenses [3]. This lack of transparency can result in unclear spending
and misalignment with business goals.
A lack of visibility into cloud spending can result in inefficient resource allo-
cation and significant loss, with data from a recent Acceldata report indicating
that businesses can lose up to 32% of their cloud budget on idle or underutilized
resources [7]. This issue frequently occurs as organizations seek to identify and
eliminate inefficiencies, especially in environments where several teams provision
and maintain cloud services separately. Such a decentralized strategy can result
in information silos, making it difficult to acquire a full knowledge of cloud ex-
penditures and resource utilization across the firm. Without this information,
engineers and IT management may struggle to successfully optimize resources,
match spending with business goals, or even determine where the most sub-
stantial cost-saving potential exist. As a result of this, the potential for waste
escalates, and the business risks overspending on cloud services that do not de-
liver proportional value.
Furthermore, the rapid expansion of cloud services usually results in shadow
platforms where different departments and teams adopt cloud solutions without
the knowledge or approval of the central platform and finance teams. This lack
of coordination might result in a fragmented approach to cloud cost manage-
ment, with expenses that are not managed or optimized efficiently. Failure to
adequately manage cloud expenses can result in major consequences, ranging
from financial strains and budget overruns to missed chances for research and
development. According to a recent white paper by Civo, 37% of organizations
faced unanticipated cloud charges, highlighting the crucial need for enhanced
cost management strategies [8]. When departments work in silos, the lack of
visibility and control over cloud spending can quickly lead to a situation where
expenses are unexpected and resources are allocated inefficiently. Therefore, this
not only complicates budget planning but also hampers the ability to support
strategic business initiatives, as funds that could drive innovation are instead
absorbed by unmonitored and unnecessary expenses.
Therefore, as businesses rely more on cloud services to power their digi-
tal transformation, efficient cloud cost optimization measures become critical.
Implementing strong cost management techniques and leveraging modern tools
can provide organizations with critical control over their cloud spending. This
includes establishing governance frameworks to ensure accountability, utilizing
tagging and resource management strategies to improve visibility, and imple-
menting automated tools to discover and resolve resource inefficiencies in real
time. To summarize, cloud cost optimization is more than simply a financial im-
perative; it is a strategic requirement for enterprises seeking to properly embrace
cloud computing. The capacity to manage costs while maximizing the value of
cloud investments is important for preserving a competitive advantage in an in-
creasingly digital world. In the following sections, we will look at the ideas of
4 Saurabh Deochake
FinOps and introduce ABACUS, a comprehensive solution that may assist enter-
prises negotiate the intricacies of cloud cost optimization and achieve long-term
financial sustainability.
3 Understanding FinOps
3.1 What is FinOps
FinOps, or Financial Operations, is an emerging discipline at the intersection
of finance and engineering that tackles the complicated task of controlling and
optimizing operating expenses (OpEx) in today’s quickly changing IT ecosystem.
As organizations rely more on cloud services, the need for a structured approach
to cloud financial management has become critical.
At its core, FinOps is about bringing financial accountability to the vari-
able spend and cost model of cloud computing. It is a collaborative effort that
brings together technology, finance, and business teams to make informed de-
cisions about cloud usage and expenditure. The FinOps Foundation which is
a program of the Linux Foundation defines it as "an evolving cloud financial
management discipline and cultural practice that enables organizations to get
maximum business value by helping engineering, finance, technology and busi-
ness teams to collaborate on data-driven spending decisions" [9]. Therefore, as
per the FinOps Foundation, the primary goals of FinOps include the following.
– Improving visibility into cloud costs across multi-cloud environments
– Optimizing resource utilization and eliminating idle resource waste
– Aligning cloud spending with business financial objectives
– Creating a culture of financial accountability among all cloud stakeholders
FinOps involves setting up the right processes, tools, and organizational
structures to help teams make informed decisions about cloud resource allo-
cation and usage. This includes managing costs, budgeting effectively, detecting
anomalies, and employing predictive analytics to keep spending in check. How-
ever, FinOps is not solely focused on cutting costs. Instead, it emphasizes making
strategic trade-offs between speed, cost, and quality. For instance, there are times
when investing more in cloud resources can lead to significant business benefits,
such as faster time-to-market or enhanced performance. Conversely, there are
situations where cost optimization and efficiency become critical to maintaining
financial health.
Adopting FinOps principles means aiming for a balanced approach that aligns
innovation with operational efficiency and cost management. It involves setting
up robust governance frameworks and leveraging tools that provide visibility
into cloud spending, enabling teams to make data-driven decisions. This strategic
approach not only helps in controlling costs but also ensures that cloud resources
are utilized in ways that drive business value. As a result, organizations can
enhance their operational effectiveness, adapt to market changes more rapidly,
and secure a competitive advantage in an increasingly cloud-centric landscape
[10].
ABACUS: A FinOps Service for Cloud Cost Optimization 5
This FinOps approach not only helps in controlling cloud costs but also
enables teams to leverage cloud resources more strategically, ultimately driving
better business outcomes.
6 Saurabh Deochake
– Inform: In the Inform phase, the focus is on establishing visibility and ac-
countability by identifying data sources for cloud cost and usage metrics,
accurately allocating expenses based on tags or business rules, and develop-
ing budgeting and forecasting capabilities. This phase empowers teams with
accurate and timely data, enabling them to make informed decisions and
align cloud spending with business objectives.
– Optimize: The Optimize phase is dedicated to identifying and implementing
opportunities to improve cloud efficiency and cost-effectiveness. Teams work
collaboratively to rightsize underutilized resources, leverage cloud provider
optimization options, and select the best optimization opportunities based
on organizational goals. By optimizing resource utilization and exploring
cost-saving opportunities, organizations can achieve more value from their
cloud investments.
– Operate: The Operate phase focuses on establishing organizational changes
and a culture of cost accountability. Key activities include defining cloud gov-
ernance policies, empowering individuals through training and automation,
and continuously evaluating the alignment between business objectives and
cloud spend. This phase ensures that cloud spending remains under control
and adaptable to changing needs, fostering a culture of financial account-
ability across teams.
4 ABACUS
This section showcases ABACUS, Automated Budget Analysis and Cloud Usage
Surveillance, a service that follows FinOps principles and automates the analysis
of budget for the cloud infrastructure and adjusts the cloud costs accordingly.
8 Saurabh Deochake
4.1 Prerequisites
Since ABACUS service depends on accurate budget and usage analysis of the
cloud resources, it is imperative that certain prerequisites are established to
ensure its effectiveness in managing cloud financial operations and costs.
Analyzing Budgets The initial step in the budget analysis process involves
gathering data on historical cloud spending (HC) and projecting future costs
ABACUS: A FinOps Service for Cloud Cost Optimization 9
where,
– CRB : Cloud Resource Budget
– HCi : Historical Cloud Spend for each instance or service, indexed by i from
1 to n
– G : Projected Growth Factor (expressed as a decimal, e.g., 20% as 0.20)
– C : Cost Control Factor (expressed as a decimal, e.g., 10% as 0.10)
– V : Variability Factor, which accounts for fluctuations in spending or growth
projections (expressed as a decimal, e.g., 5% as 0.05)
– AB : Available Budget
– min : The minimum function, ensuring that the calculated budget does not
exceed the available budget
The variability factor V is calculated based on the historical spending data
using the following steps:
Where:
– n is the number of data points.
– xi are the individual spending values for services or teams.
2. Calculate the sample variance (s2 ) of the historical spending data:
n
1 X
s2 = (xi − x̄)2
n − 1 i=1
Table 1. Cloud Resource Budget Calculations with and without Variability Factor
This calculation indicates that the budget for cloud resources for that par-
ticular team or service for the upcoming period is $113,400, reflecting the orga-
nization’s historical spending adjusted for growth, cost control measures, and a
ABACUS: A FinOps Service for Cloud Cost Optimization 11
·105
1.3
With Variability Factor (V = 0.1)
Without Variability Factor
1.2
Cloud Resource Budget (CRB)
1.1
0.9
0.8
0 10 20 30
Cost Control Factor (C)
Fig. 1. Relationship between Cost Control Factor and Cloud Resource Budget with
and without Variability Factor
The graph 1 demonstrates the inverse relationship between the Cost Control
Factor (C) and the Cloud Resource Budget (CRB). As the Cost Control Fac-
tor increases, the required budget to set aside for upcoming period decreases,
highlighting the impact of cost management strategies on financial planning.
Additionally, with the inclusion of V, which accounts for potential fluctuations
in spending, the budget allocation reflects a more flexible approach to financial
planning ensuring that organizations are better prepared for variability in costs
and historical spending.
the heart of this architecture, serving as the primary interface for both the
Finance and FinOps teams to interact with the system. These teams are in
charge of creating organizational budgets, which are then maintained in the
Organization Budget and Organization Chargeback databases. The Organization
Budget Database contains overall financial allocations, whereas the Organization
Chargeback Database maintains cost data divided by team or project, allowing
for cost attribution and accountability.
Once budget allocations have been made, the Budget Allocation Module dis-
tributes them among multiple cloud accounts maintained by the Cloud Platform
Billing Account. The Cloud Platform Billing Account is a centralized component
that monitors all cloud-related charges in real time. This module guarantees that
each account stays within its budget, so setting a financial boundary.
The Historical Spend Monitor continuously compares expenditure data to
budget restrictions. It uses a threshold-based system in which certain expendi-
ture triggers are configured. As actual spending increases, this module contin-
uously monitors for potential threshold breaches, allowing for proactive budget
management. If the spend reaches a predetermined level, the Cost Evaluation
Engine is activated to conduct a thorough examination of the expenditure pat-
terns. This component investigates aspects such as the services that contribute
to high costs, the rate of spending, and any abnormalities that may suggest
inefficient resource consumption or unplanned costs.
When the Cost Evaluation Engine detects a significant threshold breach, it
transmits essential data to a Message Queue, including the account ID, service
ABACUS: A FinOps Service for Cloud Cost Optimization 13
type, budget, and spending information. This queue acts as a buffer, ensur-
ing that enforcement actions are processed sequentially, avoiding conflicts and
delays. This data is then collected by the Budget Enforcer, who serves as an
enforcement agent. Based on the information obtained, the Budget Enforcer
may halt particular cloud services or impose limits on the affected accounts.
For example, in the diagram, Account C is depicted as having ceased services,
indicating a budget infraction.
The Budget Enforcer consults the Cost Breach Record Database, which
keeps a complete record of all spending breaches and any measures taken. This
database is essential for not just rapid enforcement, but also long-term finan-
cial analysis and compliance assessments. The system provides useful data by
keeping a historical record of budget breaches and the resulting enforcement
measures, which may be used to identify patterns, evaluate policy efficacy, and
enhance future budgeting procedures.
Simultaneously, the Alerting Engine sends real-time messages and alerts
whenever a budget breach is discovered and punished. These warnings are de-
livered to both the Finance and FinOps teams via multiple channels, including
email, Slack, and dashboard notifications. The Alerting Engine sends real-time
alerts each time spending crosses the 50%, 75%, 90%, 100%, and the threshold
of the budget limit. The alerting mechanism guarantees that stakeholders are
swiftly notified, allowing them to take corrective action or explore the underlying
causes of excessive spending.
Furthermore, the architecture enables scalability and adaptation. The ABA-
CUS GUI can expand to accept more accounts or budget categories as the or-
ganization grows. In more specific technical terms, this expansion can be easily
performed via Google Service Account IAM roles in Google Cloud Platform
(GCP) or assuming IAM roles in new accounts in Amazon’s AWS. The system
is also intended to handle larger amounts of data, with the Message Queue guar-
anteeing that enforcement operations are managed efficiently even when under
heavy pressure. The Cost Evaluation Engine can also be upgraded, as mentioned
in the Future Work section of this paper, with machine learning algorithms to
improve spending projections and provide advanced insights, such as detecting
unused resources or advising budget adjustments based on consumption trends.
Finally, ABACUS can be extended by integrating advanced policy-as-code
frameworks like Open Policy Agent (OPA) and HashiCorp Sentinel into the
Infrastructure-as-Code (IaC) workflow to greatly improve cost management dur-
ing cloud deployments. Engineering teams can reduce the risk of exceeding the
budget threshold by developing policies that analyze proposed infrastructure
changes in relation to financial limitations. For example, an OPA policy can
be developed to assess Terraform plans and reject changes that would result in
expenses beyond a predetermined level [11]. Similarly, a Sentinel policy can com-
bine cost estimates produced by Infracost, which offers comprehensive pricing de-
pending on Terraform configurations, to verify that total expected expenses are
under budget [12]. Integrating these technologies into a CI/CD pipeline enables
14 Saurabh Deochake
5 Future Work
While the existing design efficiently tackles the main difficulties of cloud cost
management through real-time monitoring, budget enforcement, and alerting,
there are various paths for future upgrades to enhance the system’s resilience,
scalability, and intelligence. This section discusses a few major future work items
for ABACUS.
Incorporating machine learning (ML) models into the Cost Evaluation Engine
could improve the present framework and allow for proactive budget manage-
ment. Currently, budget enforcement is reactive, responding only after breaches
have occurred. By including predictive algorithms for forecasting, such as time-
series models (e.g., ARIMA, LSTM), the system could predict when budgets
are likely to be depleted based on previous and current data [13]. Anomaly
detection algorithms (such as Isolation Forest and Autoencoders) could spot
anomalous expenditure patterns early, allowing for prompt responses. Further-
more, regression models can evaluate the impact of various usage parameters on
costs, whereas clustering models can disclose specific usage trends. These ML-
driven additions would allow for budget modifications and optimizations before
breaches occurred, improving the system’s responsiveness and accuracy.
Future iterations of the Budget Enforcer will include adaptive enforcement poli-
cies. Instead of using static enforcement actions such as stopping services, the
system could tailor enforcement actions to business-critical workloads or Ser-
vice Level Agreements (SLAs) [14]. Instead of immediately suspending a crucial
production service, the system could send warnings, slow non-essential services,
or impose interim spending constraints to allow critical applications to continue
functioning. This dynamic method would increase flexibility while ensuring that
enforcement measures do not disrupt critical activities.
ABACUS: A FinOps Service for Cloud Cost Optimization 15
6 Conclusion
In conclusion, this paper has emphasized the crucial necessity for cloud cost
optimization in today’s changing digital landscape, as well as the importance
of implementing FinOps concepts to enhance collaboration between finance and
IT teams for successful resource management. The ABACUS algorithm and ar-
chitecture were established, which are intended to improve budget enforcement
and resource allocation through real-time monitoring and automated interven-
tions. Future developments, such as the integration of multi-agent systems and
machine learning models, were also proposed as a way to enable proactive bud-
get management and compliance monitoring. These developments will not only
improve the system’s adaptability and scalability but will also allow enterprises
to negotiate complicated regulatory settings while reaping the financial benefits
of cloud computing.
16 Saurabh Deochake
References
1. Flexera: State of the cloud report (2024), https://info.flexera.com/cm-report-state-
of-the-cloud
2. Gartner: Realize cost savings after migrating to the cloud (2021),
https://www.gartner.com/en/documents/4001072
3. Deochake, S.: Cloud cost optimization: A comprehensive review of strategies and
case studies (Jul 2023), https://arxiv.org/abs/2307.12479
4. Markets, markets: Cloud computing market by service model and de-
ployment model - 2023 (2023), https://www.marketsandmarkets.com/Market-
Reports/cloud-computing-market-234.html
5. Charan, K.V.S., Vardhan, K.H., Reddy, V.J.R.: Cloud computing: A review of
features, benefits, and challenges. International Journal of Advanced Research in
Computer Science and Software Engineering 9(3) (2019)
6. Jayashree, P., Hemalatha, M.: Cloud computing: A comprehensive survey. Inter-
national Journal of Pure and Applied Mathematics 133(1) (2021)
7. Shaikh, R.H.: Guideon cloud costs for data-intensive workloads (Nov
2024), https://www.acceldata.io/blog/cloud-cost-optimization-for-data-intensive-
workloads-a-strategic-approach
8. Civo: The cost of cloud report 2024 - civo.com (2024), https://www.civo.com/cost-
of-cloud-report-2024
9. FinOps-Foundation: What is finops? (2023), https://www.finops.org/introduction/
what-is-finops/
10. Storment, J., Fuller, M.: Cloud FinOps. O’Reilly Media, Inc. (2023)
11. Open Policy Agent: Open policy agent documentation (2024),
https://www.openpolicyagent.org/docs/latest/, accessed: 2024-12-22
12. Infracost: Open policy agent (2024), https://www.infracost.io/docs/integrations/o
pen_policy_agent/
13. Siami-Namini, S., Tavakoli, N., Siami Namin, A.: A comparison of arima and lstm
in forecasting time series. In: 2018 17th IEEE International Conference on Machine
Learning and Applications (ICMLA). pp. 1394–1401 (2018)
14. Beyer, B., Jones, C., Petoff, J., Murphy, N.R.: Site reliability engineering: How
Google runs production systems. O’Reilly Media, Inc. (2016)
15. Deochake, S.R.: Cloud resource management (Jan 18 2024), US Patent App.
17/863,116
16. Wang, Q., Li, W., Mohajer, A.: Load-aware continuous-time optimization for multi-
agent systems: toward dynamic resource allocation and real-time adaptability.
Computer Networks 250, 110526 (Jun 2024)
17. Deochake, S.: Belief-desire-intention (bdi) multi-agent system for cloud market-
place negotiation. Distributed Computing and Artificial Intelligence, 19th Interna-
tional Conference p. 144–153 (Dec 2022)
18. Deochake, S., Mukhopadhyay, D.: An agent-based cloud service negotiation in hy-
brid cloud computing. Advances in Intelligent Systems and Computing p. 563–572
(Dec 2020)
19. Deochake, S., Channapattan, V., Steelman, G.: Bigbird: Big data storage
and analytics at scale in hybrid cloud. arXiv:2203.11472 [cs] (Mar 2022),
https://arxiv.org/abs/2203.11472
20. Williams, B.R., Adamson, J.S.: PCI Compliance. CRC Press (Nov 2022)