Setting up Consul API GW to get JWT token with Azure AD as IdP

Introduction
In modern micro-services architectures, secure communication and authentication are critical aspects to ensure that only authorized users and services can interact with each other.
When dealing with authentication, JWT (JSON Web Tokens) are commonly used to facilitate stateless and scalable authentication, enabling systems to verify the identity of a user or service by validating a token rather than making frequent calls to a centralized authentication service. Azure Active Directory (Azure AD) is a widely used identity provider (IdP) that supports OAuth2, OpenID Connect (OIDC), and other authentication protocols to issue JWTs.
API Gateways (e.g., Kong, Traefik, NGINX) sit between clients and micro-services, acting as a reverse proxy to route requests while enforcing security policies like authentication, rate limiting, and routing logic. In this setup, Consul API Gateway is used to integrate with identity providers like Azure AD to authenticate and authorize incoming requests via JWTProvider CRD.
Expected Outcome
Azure AD is integrated as the JWT provider for Consul and an API Gateway, allowing Consul to validate incoming JWT tokens issued by Azure AD and enable secure communication within the cluster.
The architecture involves:
Azure AD issuing JWT tokens for authenticated users or services.
The API Gateway validating incoming JWT tokens from clients before forwarding requests to backend services.
Consul validating and using these JWT tokens for secure service-to-service communication, ensuring that only authorized services can interact.
To achieve this, we need to set up a JWT Provider Custom Resource Definition (CRD) in Kubernetes, configure Consul to trust Azure AD as an identity provider, and configure the API Gateway to authenticate requests using JWT tokens from Azure AD.
PrerequisitesBasic Knowledge and Familiarity
JWT Authentication: Basic understanding of JWT tokens, how they are structured, and how they are used for authentication and authorization. Ref. https://developer.hashicorp.com/consul/docs/security/acl/auth-methods/jwt
API Gateway: Familiarity with setting up an API Gateway and how it handles JWT authentication (ref. https://developer.hashicorp.com/consul/docs/connect/gateways/api-gateway & https://developer.hashicorp.com/consul/docs/connect/config-entries/jwt-provider )
Azure Active Directory (Azure AD): Basic knowledge of how Azure AD works, how to register applications, and how to configure OAuth2/OpenID Connect (ref. https://learn.microsoft.com/en-us/power-pages/security/authentication/openid-settings )
Ensure that Consul is installed and running in your environment with following minimum version constraint . You can either run it on-premises, in a cloud VM, or within a Kubernetes cluster.
The helm chart version: 1.5.3
server image version 1.19.2
Dataplane image version: 1.5.3
k8s-control-plane image version: 1.5.3
Follow this guide to setup Consul API GW on K8s cluster and to control access to services inside the mesh.
Use Case
Some common and impactful use cases for setting up this integration:
Centralized Authentication and Authorization for Microservice Architecture - In a microservice architecture, different services need to authenticate and authorize each other to ensure that only legitimate requests are processed. Using Azure AD as the identity provider allows you to centralize authentication, reducing the overhead of managing credentials within individual services.
Federated Authentication Across Multiple Systems - When multiple systems or organizations need to collaborate, federated authentication allows users to authenticate using their existing identity provider (e.g., Azure AD) without requiring them to create new credentials for each system.
Secure API Gateway Authentication with External Identity Providers - Many organizations prefer to integrate their API Gateway with an external identity provider (IdP) like Azure AD for authentication. This removes the need for the API Gateway to manage authentication and offloads security concerns to a trusted IdP.
Multi-Tenant Architecture with Role-Based Access Control (RBAC) - In multi-tenant applications, each tenant requires access to its own set of resources, but with the same underlying microservices. Role-based access control (RBAC) ensures that users can only access resources that belong to their tenant or role.
Procedure
Setting up a Consul Cluster JWT provider with an API Gateway on Azure AD involves configuring both Consul and Azure AD to trust and communicate via JWT tokens for authentication and authorization. Here’s a step-by-step guide for this integration:
1. Configure Azure Active Directory (Azure AD)
First set up Azure AD as the Identity Provider (IdP) to issue JWT tokens that Consul can validate.
Step 1.1: Register an Application in Azure AD
Go to Azure Portal and navigate to Azure Active Directory.
Under App registrations, click on New registration.
Provide a name for your app (e.g., JWT DEMO).
Set the Supported account types based on your needs (e.g., "Accounts in this organizational directory only").
Set the Redirect URI (optional at this stage) if needed, typically used for OAuth flow.
After registration, note the Application (client) ID and Directory (tenant) ID.
 


Step 1.2: Configure API Permissions
Navigate to your registered app in Azure AD > API permissions.
Click Add a permission, then select Microsoft Graph or Other API depending on your use case.
Add the necessary permissions (e.g., openid, profile, email for OIDC).
Grant Admin Consent for the added permissions.
 


Step 1.3: Obtain the OpenID Connect (OIDC) metadata
In Azure AD, you can find the OIDC metadata endpoint by navigating to the App Registration > Endpoints.


Copy the OpenID Connect metadata document URL (e.g., https://login.microsoftonline.com/{tenant-id}/v2.0/.well-known/openid-configuration).


This URL will be used by Consul to verify the JWT tokens issued by Azure AD.
Step 1.4: Create a Client Secret (if using OAuth2 flow)
If you're using OAuth2 flow to authenticate the API Gateway (i.e., for token issuance), you’ll need a client secret:
In the Certificates & secrets section of your app in Azure AD, click New client secret.
Note the value of the client secret, which you’ll need to configure in your API Gateway or Consul service.


 
Postman Configuration
In Postman, we configure an OAuth 2.0 as the Authorization so we can get the Access Token.
Go to the Authorization tab
In the Type column select “OAuth 2.0”
Set “Add authorization data” to “Request Headers”
Set “Grant Type” to “Client Credentials”
Set the Header Prefix value to “Bearer”
Set the Access Token URL value to the “OAuth 2.0 token endpoint (v2)” URL value from Azure AD Application Endpoints.
Set the Client ID value to the “Application (client) ID” value from the Azure AD App.
Set the Client Secret value to the Client secret value of the Azure AD App.
Set the Scope to the Application ID URI you copied from Azure but append /.default at the end (Example: “api://<READCT>/.default”
Set Client Authentication to “Send client credentials in body”


11) Click “Get New Access Token” and then “Proceed”
12) Copy the access token value and click “Use token”


Decoding the Token: The user will need to decode the JWT token to get the Identity Provider URL. It will be done using Consul API GW. This guide demonstrates token decoding using JWT.IO
Go to JWT.IO in your web browser and paste your token into the “Encoded” text box there.
On the Decode side, look for "iss", and copy the URL to a notepad.


2. Configure Consul for JWT Authentication
Now that Azure AD is set up to issue JWT tokens, the user needs to configure Consul to use these tokens for authenticating incoming requests.
Step 2.1: Enable the JWT Authentication Method
In the Consul cluster, the user needs to enable JWT authentication and configure it to trust Azure AD as the Identity Provider.
The JWTProvider CRD will configure Consul to trust JWT tokens issued by Azure AD. This is how Consul will validate incoming JWT tokens for authentication.
Then the user needs to define access control policies in Consul that will govern which users or services can access particular services based on the JWT claims (e.g., roles, user ID). The GatewayPolicy CRD allows you to specify rules for API Gateway behavior, such as which JWT claims should be used for authorization and what roles are required for access.
Ref.JWT provider configuration reference | Consul | HashiCorp Developer
GatewayPolicy configuration reference | Consul | HashiCorp Developer
---
apiVersion: consul.hashicorp.com/v1alpha1
kind: JWTProvider
metadata:
  name: api-jwt-provider
  namespace: consul
spec:
  issuer: "https://sts.windows.net/{tenant-id}/"
  jsonWebKeySet:
    remote:
      uri: "https://login.microsoftonline.com/{tenant-id}/discovery/v2.0/keys"
      jwksCluster:
        discoveryType: STRICT_DNS
      cacheDuration: 5m
  forwarding:
    headerName: user-token
---
apiVersion: consul.hashicorp.com/v1alpha1
kind: GatewayPolicy
metadata:
  name: api-gateway-policy
  namespace: consul
spec:
  targetRef:
    name: api-gateway
    sectionName: http
    group: gateway.networking.k8s.io/v1beta1
    namespace: consul
    kind: Gateway
  override:
    jwt:
      providers:
      - name: "api-jwt-provider"
  default:
    jwt:
      providers:
      - name: "api-jwt-provider"
        verifyClaims:
        - path:
            - appid
          value: <APP_ID>
---
% kubectl get jwtprovider -A
NAMESPACE   NAME               AGE
consul      api-jwt-provider   8s
% kubectl get gatewaypolicy -A
NAMESPACE   NAME                 SYNCED   LAST SYNCED   AGE
consul      api-gateway-policy                          37s
% kubectl get po -n consul
NAME                                           READY   STATUS    RESTARTS      AGE
api-gateway-64fb9f5d5b-248k8                   1/1     Running   0             1d
consul-connect-injector-d47445d8c-tp4t6        1/1     Running   0             1d
consul-server-0                                1/1     Running   2 (55d ago)   1d
consul-terminating-gateway-74db79698-5d79c     1/1     Running   0             1d
consul-webhook-cert-manager-687f69b9c5-49b2q   1/1     Running   0             1d
% kubectl get gateways.gateway.networking.k8s.io -A
NAMESPACE   NAME          CLASS    ADDRESS                                                                  PROGRAMMED   AGE
consul      api-gateway   consul   a64fef46f6f9c4e1cb4dcf81d2bfea1b-846729335.eu-west-1.elb.amazonaws.com   True         1d
 
With above setup, when the user tries to check consul-api-gateway pod logs it will show following messages with different endpoint IPs for the Azure AD.
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.http(23) [Tags: "ConnectionId":"1896444","StreamId":"10323561514510472677"] encoding headers via codec (end_stream=false):
':status', '200'
'content-type', 'text/plain; charset=UTF-8'
'cache-control', 'no-cache, max-age=0'
'x-content-type-options', 'nosniff'
'date', 'Mon, 16 Dec 2024 07:55:53 GMT'
'server', 'envoy'
'x-envoy-upstream-service-time', '1'
'connection', 'close'
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.client(23) [Tags: "ConnectionId":"527138"] response complete
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.http(23) [Tags: "ConnectionId":"1896444","StreamId":"10323561514510472677"] Codec completed encoding stream.
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] closing data_to_write=277 type=0
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] setting delayed close timer with timeout 1000 ms
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.pool(23) [Tags: "ConnectionId":"527138"] response complete
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.pool(23) [Tags: "ConnectionId":"527138"] destroying stream: 0 remaining
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] write flush complete
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896444"] closing socket: 1
2024-12-16T07:55:53.332Z+00:00 [debug] envoy.conn_handler(23) [Tags: "ConnectionId":"1896444"] adding to cleanup list
2024-12-16T07:55:55.715Z+00:00 [debug] envoy.dns(14) dns resolution for login.microsoftonline.com started
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.dns(14) dns resolution for login.microsoftonline.com completed with status 0
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.11:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.17:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.18:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.16:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 20.190.181.6:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.19:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 20.190.181.2:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) transport socket match, socket default selected for host with address 40.126.53.7:443
2024-12-16T07:55:55.717Z+00:00 [debug] envoy.upstream(14) DNS refresh rate reset for login.microsoftonline.com, refresh rate 5000 ms
2024-12-16T07:55:56.282Z+00:00 [debug] envoy.conn_handler(23) [Tags: "ConnectionId":"1896445"] new connection from 192.168.63.75:53898
2024-12-16T07:55:56.282Z+00:00 [debug] envoy.connection(23) [Tags: "ConnectionId":"1896445"] remote close
 Additional Informationhttps://www.ibm.com/support/pages/configuring-azure-active-directory-aad-consul-using-openid-connect-oidc 
https://www.ibm.com/support/pages/node/7263675 
https://github.com/hashicorp/consul/blob/main/CHANGELOG.md#1192-august-26-2024 
https://github.com/hashicorp/consul/pull/21604
https://github.com/hashicorp/consul-k8s/blob/v1.5.3/CHANGELOG.md#153-august-30-2024
 
Tips

Setting up Consul API GW to get JWT token with Azure AD as IdP

How To

Summary

Steps

Introduction

Expected Outcome

Prerequisites

Use Case

Procedure

1. Configure Azure Active Directory (Azure AD)

Step 1.1: Register an Application in Azure AD

Step 1.2: Configure API Permissions

Step 1.3: Obtain the OpenID Connect (OIDC) metadata

Step 1.4: Create a Client Secret (if using OAuth2 flow)

2. Configure Consul for JWT Authentication

Step 2.1: Enable the JWT Authentication Method

Additional Information

Document Location

Historical Number

Was this topic helpful?

Document Information

UID

Share your feedback

Need support?
