What is an internal developer portal?

Internal developer portals are sometimes abbreviated as IDPs, though that same abbreviation can apply to the related, but not identical, internal developer platform. Essentially, the internal developer platform is the environment that hosts those tools, while the portal is how the developers find, learn about and use them.

IDC data indicates that knowledge workers might spend as much as 2.5 hours per day—about 30% of the workday—searching for information. For developers and engineers, this information is vital in helping to ensure compliance, functionality and non-redundancy. 

Let’s say that a newly hired developer needs to build a new microservice to modify the way a cart works in an e-commerce platform. Without a developer portal, that developer might have to navigate a maze of documentation and internal tools, tools that might be out of date or in conflict with other documentation. The developer might not have a list of existing services and APIs that would help save time in building this new cart. And they might encounter all kinds of dead ends and red herrings as they attempt to learn about the development environment and figure out how things are done.

An internal developer portal solves this issue by gathering a services catalog, software templates, docs, automated workflow information, onboarding guidance and more, all in one place. A crucial component within a DevOps ecosystem, an internal developer portal can reduce cognitive loads and streamline development processes by making institutional knowledge centralized, standardized and transparent.

Because the names are similar and their acronyms identical, internal developer portals and internal developer platforms are often confused. Think of the portal as not merely a dashboard for viewing, but a control panel through which developers discover and use platform components and features.

The platform is the engine. An internal developer platform includes the available infrastructure, tools, APIs and automations.

In practice, some teams and organizations might refer to the platform as including the portal. Some teams might even maintain a platform without a dedicated portal as such; instead, the platform contains documentation and services but might require institutional knowledge and a greater effort to find them.

Developers can use an internal developer platform without a portal at all by using the command line. CLI-only teams do this. For larger and more complex organizations with faster rates of turnover or more varied development teams, an internal developer portal can help ensure a more efficient developer experience.

In general, platform engineering teams are responsible for the building and provisioning of internal developer platforms and configuring the portals that provide access to the platform. Platform engineering leaders are tasked with creating what are called “golden paths,” pre-approved step-by-step methods for completing development work.

Portal shape and scope vary by organization, and there isn’t a single set of features that all portals must include. However, an internal developer portal generally contains several key features and elements:

The service catalog or software catalog is the central inventory of all available services and components. This directory contains information about a service that a developer might need to know for a variety of use cases. 

That information might include:

• Service ownership
  
  
• Other metadata (language, deployment status, dependencies)
  
  
• Access and permissions
  
  
• Resources required by the service
  
  
• Compliance and governance info for standardization

Developer portals support pre-built, automated actions that have been fully vetted, tested and validated by engineering teams. The goal is that instead of having to go through the process of creating, configuring and testing similar actions repeatedly, a user can select a commonly used action from a list. Productivity and workload optimization improve when standardized deployment workflows, CI/CD pipelines and other operational, security and compliance components are ready to use and in line with organizational imperatives. 

In Backstage, a popular open-source IDP originally created by Spotify, these workflows are called software templates; other systems might call them “blueprints.” These templates, according to Backstage’s documentation, “generally consist of a set of steps, and each step can have its own set of required or optional inputs.” Engineers can select from a list of existing software templates, answer a few questions to fill in required fields, and then automatically publish the new repository to GitHub. In some cases, these templates are subject to role-based access control (RBAC). For example, there might be certain components that only senior developers can access.

Templates are often YAML files and include key parameters, the backend actions that will be executed once the end user fills out those input fields and the output presented to the user after execution (like a “completed” message or a link to the newly created repository). 

These templates can be complex under the surface; they could be calling all sorts of APIs and management layers and security measures. Within the portal, however, the user sees only a simple entry for the template. This abstraction generally simplifies and streamlines the development process. It promotes consistency in development and helps maintain control over governance and security.

As companies manage dozens, hundreds or even thousands of individual microservices, it is increasingly difficult, and important, to ensure that documentation is transparent, easy to find and up to date.

Documentation can be published and hosted on a wide variety of platforms, including Git repositories, Jira and Confluence, and even as locally hosted text documents. One of the main functions of an internal developer portal is to consolidate and surface documentation.

Types of documentation commonly found in a portal include:

• Service documentation: What each service does, how it works, what it integrates with.
  
  
• API documentation: Endpoints, authentication, request/response formats and other guidelines for APIs.
  
  
• Troubleshooting and runbooks: FAQs, guides and other documentation laying out common issues and how to resolve them.
  
  
• Onboarding documentation: Information for new hires on best practices, house style and other conventions.

Many internal developer portals include some version of relationship or dependency mapping. The goal with these inclusions is to understand the connections between services, infrastructure, APIs, libraries and more. 

Common types of relationships include:

• Service to API: A software service and the APIs it exposes or relies on for data exchange
  
  
• Service to service: One software service uses another to complete an action, such as an ecommerce ordering system querying an inventory system
  
  
• Service to infrastructure: Many services rely on databases to perform their duties
  
  
• Team and ownership: Which team owns, and which teams use, various services across the organization

This sort of dependency mapping is often seen as challenging and frustrating. Manual mapping is essentially impossible in large organizations since these relationships are regularly evolving, and manual documentation captures a single moment in time. 

To address this issue, modern portals might offer an automated dependency mapping feature. Portals input data and accounts from a variety of sources, including internal databases, external APIs and various service providers and attempt to create a living, real-time dependency graph.  Some services use artificial intelligence (AI) to automate dependency discovery; this tech is promising but quite new.

Software scorecards are real-time performance tracking systems that convey the health and metrics for a given service. Scorecards provide specific feedback by defining standards of success.

Features of software scorecards might include:

• Production readiness: Is a service compliant with existing standards and ready to go? If not, which elements—documentation, ownership, security, for example—are missing or incorrect? 
  
  
• Code quality: Are there any errors or non-compliant conventions in the code? Is the code clean, reproducible and readable? 
  
  
• Overall health: What is the overall status of the service? Status is sometimes represented by using traffic light colors, or Olympic medal grades (gold, silver, bronze) to show whether a service is operating without errors and as expected.
  
  
• Security: Security scorecards can indicate whether dependencies are up to date, when a security scan was most recently deployed or whether there are any exposed endpoints, files or data.
  
  
• Initiatives: Initiative scorecards are temporary and usually designed to grade an organization-wide improvement or upgrade. These scorecards are more like progress trackers for ongoing assignments.

Internal developer portals provide a transparent way to find and use a wide variety of integrations, which can both pull data into the portal and push it out to other services. 

These might include:

• Repositories: Sources like GitHub are used to store and manage code, and integrations with an internal developer portal enable developers to view vital data within the portal dashboard. That data can include deployment history, code ownership, build status and tracked changes.
  
  
• Kubernetes: Also known as k8s or kube, Kubernetes is an open source container orchestration platform for scheduling and automating the deployment, management and scaling of containerized applications. IDPs can help abstract the specifics of container management, providing a simplified way to deploy applications, view logs, check statuses and more.
  
  
• CI/CD systems: IDPs can integrate with continuous integration/continuous delivery (CI/CD) pipelines, automated DevOps workflows that streamline software development, testing and delivery cycles. This integration enables developers to trigger pipelines, provision infrastructure, check deployment statuses and more from a single interface.  
  
  
• Incident management: By integrating with service management tools such as Jira, IDPs provide a one-stop location to track incidents and projects.
  
  
• Observability tools: Portals provide developers with real-time insight into service health and behavior. Observability tools designed to track latency, error rates and types, uptime data and more are often integrated with IDPs.
  
  
• Plugins: Portals might provide plugins that power the integrations, or a marketplace where developers can browse and install plugins themselves.

Internal developer portals help reduce cognitive load for engineering teams by streamlining processes, enhancing security and governance, and promoting development and infrastructure transparency.

Internal developer portals are all about reducing and removing obstacles that can, in other circumstances, waste time and effort. Enterprises use portals to consolidate a wide array of information from disparate sources and organize it into easier-to-navigate interfaces. 

Software catalogs show all available software tools, who owns them and where they can be found. Dependency and relationship graphs show how services interact with each other, with APIs and with internal infrastructure. Software templates provide self-guided pathways for common tasks and alleviate infrastructure provisioning and pipeline configuration work. Documentation is centralized in one place, reducing the hunt across internal boards, cloud storage or chat apps for information. 

All of these benefits can remove bottlenecks, streamline processes and reduce cognitive overhead for developers and engineering teams, promoting developer productivity. Case studies indicate that deployment and onboarding times can be dramatically reduced when using internal developer portals and platforms. 

The attrition rate in tech is unusually high, reaching as high as 21% annually according to BucketList. This means that teams are often onboarding new employees, who are faced with many time-wasting dilemmas. (Who owns various services? Where is the documentation for APIs? And so on.) 

Without a portal to guide new employees to these answers, onboarding can be a frustrating and time-consuming effort. Spotify, which created the popular open-source portal Backstage, noted in 2021 that onboarding engineers took over 60 days, on average, to merge their 10^th pull request. After using Backstage, that dropped to 20 days.

High turnover in tech has consequences beyond frequent onboarding. When employees depart, they often take institutional knowledge with them. At 10 of the largest American tech companies, the average employee tenure is between 1.2 and 2 years. While job tenure has been decreasing in the U.S. overall over the past few years, the tech industry average is less than half the national average.

A well-maintained portal helps ensure that information remains when employees depart, even if those employees are the ones who set up a given service or established a best practice.

Internal developer portals, service catalogs and configuration management databases (CMDBs) fulfill overlapping duties for developers but are not interchangeable terms.