What is Platform Engineering?
Platform Engineering involves designing, building, and maintaining the infrastructure and tools that enable developers to create and run applications efficiently. It focuses on creating a stable, scalable, and secure platform that supports the entire software development lifecycle.
Platform Engineering aims to create an environment where developers can focus more on writing code and less on managing infrastructure This accelerates innovation and improves overall software quality.
The need for Platform Engineering
Several common problems emerge when DevOps organizations scale:
- Developers face increasing workloads
- Demand from other teams increases
- Developers need to learn new skills, technologies, and techniques
- Adoption of tools differs between teams, increasing cost and complexity
This can cause a high burden on developers, and the increased complexity can lead to cognitive overload.
The situation can be even worse when organizations have not fully adopted DevOps. In this scenario, scaling up development efforts results in larger work batches and slower releases. Adding more developers fails to increase the flow of valuable features. However, organizations that have not fully adopted DevOps might find it difficult to benefit from Platform Engineering, as we discuss later in the article.
If you recognize the DevOps scaling problem, consider a Platform Engineering approach. You can use our Platform Engineering decision flow to help.
What is an internal developer platform (IDP)?
An IDP is a set of integrated tools and services designed to streamline and automate the development, deployment, and operation of software applications within an organization. It acts as a bridge between developers and the underlying infrastructure, providing a user-friendly interface for managing various aspects of the software development lifecycle.
An IDP typically provides or accommodates one or more of the following capabilities:
- Builds
- Test automation
- Deployments
- Infrastructure
- Monitoring
- Maintenance
- Security
- Cloud and cost management
- Tool cost containment and supplier management
The IDP should be managed as a product, with the platform team continuously gathering feedback from developers and iterating on features to improve usability and effectiveness. This user-centric approach ensures that the platform evolves to meet the changing needs of the development teams and supports the overall goal of accelerating software delivery while maintaining high quality and reliability.
Key elements of Platform Engineering
Platform Engineering involves creating and maintaining an internal developer platform (IDP), working closely with development teams to understand their needs and ensure the platform provides the necessary tools and services.
The IDP typically includes components such as CI/CD pipelines, container orchestration, monitoring, logging, and security services. By automating these elements, the platform team enables developers to focus on writing code and delivering features without worrying about the underlying infrastructure.
Key aspects of how Platform Engineering works include:
- Automation: The platform team automates repetitive tasks and processes, reducing manual intervention and minimizing errors. This includes infrastructure provisioning, software deployments, and scaling operations.
- Standardization: By providing a consistent set of tools and practices, the platform encourages teams to follow the same standards, leading to more predictable and reliable outcomes.
- Self-service: Developers can use the platform’s self-service capabilities to deploy and manage their applications independently. This reduces bottlenecks and allows for faster iteration and delivery.
- Observability: The platform incorporates monitoring and logging tools to provide visibility into system performance and application behavior. This helps diagnose issues quickly and maintains system health.
- Security and compliance: The platform enforces security policies and compliance requirements across all environments, ensuring that applications adhere to organizational and regulatory standards.
Platform Engineering should be part of DevOps
Platform Engineering doesn’t replace DevOps. Instead, it tackles common problems organizations experience as they grow. Platform Engineering helps you to get the benefits of DevOps at scale. An internal developer platform (IDP) helps stream-aligned teams focus on delivering valuable features alongside running their software in production.
There are many ways to address developer burnout with an IDP. Selecting a small set of big-impact areas is essential, rather than trying to do everything. One of the anti-patterns of Platform Engineering is moving all the pain into the platform team.
Platform teams benefit from the DevOps capabilities for culture, leadership, lean product management, and technical practices. You often find DevOps and Platform Engineering together. The Puppet State of DevOps Report found that high-performing DevOps organizations are more likely to use Platform Engineering.
| Category | % with platform engineering |
|---|---|
| Low | 8% |
| Mid | 25% |
| High | 48% |
DevOps applies to all organizations that depend on software delivery and operational performance. Platform Engineering solves particular problems that can constrain performance, especially at scale.
The role of Platform Engineering in 3 stages of DevOps adoption
Generally speaking, the 3 stages of DevOps adoption are:
- Divided: Pre-DevOps, there are separate silos for development and operations with conflicting goals
- Aligned: Aligned Goals and objectives to remove conflict between teams
- Combined: Removing metaphorical walls allows cross-functional teams to build and run software
Platform Engineering exists to solve problems that emerge in the combined phase of DevOps adoption.
Divided
You may have worked in an organization where a developer’s job was ‘done’ after software passed all its tests. Someone else packaged, deployed, and operated the software. This divide was deeper than role definitions. Management would reward developers for delivering more features in less time while operations teams were judged on stability.
The misalignment of goals kept development and operations in constant conflict. As a developer, you felt blocked by the need to wait for maintenance windows. Some technical decisions (like a choice of database technology) blocked progress if operations teams refused to support it. Similarly, management held operations teams responsible for any instability developers introduced.
The first stage of DevOps is to solve the alignment problem. Giving both teams shared goals removes the barriers preventing team collaboration.
Aligned
Development and operations teams must share the goal of regular and stable feature releases. This removes conflict and encourages collaboration.
Collaboration can start small. A developer and system admin reviewing error logs together after a deployment can be an excellent first step.
When developers see the error logs, they realize there’s value in improving the information in error messages. The feedback loop is complete, benefiting their software’s observability.
Operations monitoring dashboards will expand to include application and business metrics alongside infrastructure metrics.
A positive side-effect is everyone involved increases their knowledge beyond their previous silos. As understanding and empathy improve, everyone makes different decisions that are mutually beneficial. This leads to better outcomes for the software and the organization.
Combined
With DevOps well-established, the concept of a single team building and running software emerges. In some organizations, this meant moving operations experts into the development teams. In others, it involved developers taking on full responsibility, often called ‘you build it, you run it.’
There are 2 ways to be successful at ‘you build it, you run it’:
- Constrain design to ensure it stays simple, even as it scales
- Keep teams staffed with specialists who can manage more complex setups
Many teams had a complex setup but lacked the breadth of knowledge across team members to share the workload.
Platform Engineering is the solution for organizations at odds with the success criteria. Often, that happens as teams and organizations scale up. Complexity increases as the number of teams grows. Workflows and toolchains become fragmented, and operations knowledge drops as more developers join the organization.
If you want to know if Platform Engineering would work for your organization, see our decision guide.
Related content: Read our guide to DORA metrics
Best practices for implementing Platform Engineering as part of a DevOps program
The key attributes of successful Platform Engineering efforts are:
- Having a platform as a product approach
- Treating developers as the customer
- Avoiding mandated adoption (the IDP should win market share)
- Making the platform self-service, with support
- Solving real developer problems, not inventing new tech stacks
As Platform Engineering teams form with highly skilled team members, a danger is they might think they know better than developers. When they approach the task with this mindset, they create something developers avoid using. Managers can compound this problem if they mandate the use of the platform. That signals managers also believe Platform Engineers know better than the developers.
Your IDP should compete for a share of the internal developer market. You can use the MONK metrics to guide your Platform Engineering efforts and DevEx metrics to track developer productivity.
Here are a few additional best practices that can help you make the most of platform engineering.
Understand problems faced by developers
You must start your Platform Engineering journey by understanding problems experienced by your developers. To create a successful IDP, you must choose the tech stacks and workflows the platform should support based on the positive impact it could have.
You build a successful IDP with a thoughtful selection of hand-picked tools, not by buying 1 general-purpose platform. Platform Engineers design ‘golden pathway’ workflows and simplify deployments and operations.
A golden pathway is the choices the platform team decides to support. The IDP simplifies each pathway to reduce cognitive burden on developers. It also encourages others to adopt the IDP and the good decisions it represents. For example, rather than development teams maintaining hundreds of infrastructure configuration lines in different file formats (JSON, YAML, XML), you create 1 internal format with a few key fields. This change makes it easier for developers to manage and aligns teams with sharing decisions about items not part of the internal format.
Don’t treat platforms as a shared service
Traditional operations teams are a shared service. Developers raise tickets and wait for someone to do the work. For example, you’d submit a ticket to refresh the staging database. This request would sit in a queue until an operations team member picked it up and ran the steps to copy and anonymize data from a production backup.
This is different from how a Platform Engineering team works. Instead of doing the work for you, the internal platform enables you to do the work. Refreshing the staging database would be available as an automated push-button action. These self-service operations would be made available with an API call, allowing you to add tasks to automated workflows.
The only time you’d raise a ticket would be to get support using the platform. The platform team can improve documentation and simplify the product to reduce support requests. They can also network teams together so they can help each other use the platform.
Treat internal developer platforms as a first-class product
The Puppet State of Platform Engineering Report highlights that Platform Engineering fails when the team doesn’t treat the platform like a product. Without a product focus, platforms don’t solve real problems. Having direct lines of communication between Platform Engineers and the users of the internal developer platform is a crucial sign the organization is taking the right approach.
Merely shifting cognitive load onto a Platform Engineering team doesn’t resolve a problem and may worsen things. High churn in your platform team can negatively impact stream-aligned teams using the IDP.
The Platform Engineering team should focus support on fewer scenarios than many teams have. One of Platform Engineering’s goals is encouraging new development work to align with tighter choices around technology and workflows. Less variety, more focus.
Platform Engineering shouldn’t run as a ticketed queue of work. The platform should allow developers to self-service their typical needs. The platform team can provide documentation, training, and support to help teams deploy and run software themselves.
Platform Engineering brings some design to a critical area of your organization. You don’t have to stop with one well-designed team. Translating your organization’s purpose into clear team structures can improve the flow of value to customers. The concept of a ‘platform team’ in Team Topologies doesn’t mean a Platform Engineering team creating an IDP. Your organization will have other chances to create platforms that improve performance in other areas.
You can design your organization using the Team Topologies team and interaction types. You’ll find them in the Team Topologies book in our DevOps reading list.
Open-source or inner-source your internal developer platform
One of the benefits of creating a product aimed at developers is that they can contribute to it. You can make your platform internally open-source or inner-sourced, so developers can submit improvements. It’s a great way to ensure the platform solves real customer problems.
To get the most out of inner sourcing, make sure you:
- Publish clear contribution guidelines
- Keep documentation up to date
- Guide contributors to solve problems with appropriate abstraction
- Help contributors to use existing features to solve their problems
You need to decide on the right stage to introduce inner sourcing, usually after reaching broad adoption across several teams. InnerSource Commons has more information about inner-sourcing governance and patterns for successful adoption.
Balance developer needs with customer needs
Developer experience (DX) is crucial for solid software delivery performance, but you must balance it with the needs of others. There has been a rise in dysfunctional hyperfocus on developer experience to the detriment of the software they produce.
You can create great developer experiences that are bad for customers and damage the organization. Platform Engineering provides a clear way to improve developer experience while maintaining good customer and business outcomes. Golden pathways allow the right choice to be the easy choice for developers.
Happy and engaged developers create better software if you treat their experience as a part of a broader system.
The platform team can also perform valuable bridging between technical teams and other business areas, like governance, risk, and compliance.
Understanding business needs and helping development teams meet them without red tape adds value beyond technical concerns. The platform team must be exceptional at communication to keep DevOps collaboration alive.
Modern organizations strive to create the right environment for people to do their best work. This is a triple-win, benefiting employees, customers, and the business. Learn more about the benefits of employee engagement in the Gallup Q12 summary.
Related content: Read our guide to the SPACE framework
Summary
Platform Engineering is an example of Team Topologies in action. The team’s IDP should reduce complexity and cognitive overload by focusing on solving developer pain points.
A successful IDP is more likely to combine best-in-class tools rather than all-in-one DevOps platforms. Best-in-class tools include:
Platform Engineering aims to achieve high developer satisfaction without harming the experience for customers or the organization.
Next, you can learn when to adopt Platform Engineering and the patterns and anti-patterns of Platform Engineering.
Help us continuously improve
Please let us know if you have any feedback about this page.