could not create a role assignment for acr

Infra as Code
Microsoft 365

Attach ACR Error Are you an owner on this subscription

while running terraform with default contributor permission from Azure DevOps you end up with this error when you try to create a role assignment.

In our case, we are using the service principal for Azure DevOps.

Solution #1

let see how to create a custom role with minimum permission to avoid this error.

Choose subscription “Access control (IAM) – Add – Add custom role –

could not create a role assignment for acr

Clone a role – Contributor

Friendly name for the custom role – aks custom role

Delete this “Microsoft.Authorization/*/Write from NoAction.

Choose next

choose next finish. In Assignable scopes, if you want the same custom role on different subscriptions you need to include them.

Now let’s use the custom role created and add service principal to the custom role

add role assignment – role – aks custom role – select the service principal you want to use.

Choose to add.

Solution #1 –

The quick solution is to change the service principal to Owner permission on the subscription and give legacy graph permission. This is one way of doing it. but certainly not a recommended way. Custom role to give minimum permission to achieve the same. without owner permission on the subscription.

Choose Application Permission – Application.ReadWrite.OwnedBy

Add Permission and click on “Grant Admin Consent”

Intune web-based enrollment for ios devices: how to get started, how to create a sharepoint site, troubleshoot “invalid latest package version” error in microsoft endpoint admin center, leave a reply cancel reply.

Save my name, email, and website in this browser for the next time I comment.

EDITOR PICKS

Deploying to azure vm using terraform infrastructure as code (iac), step-by-step azure sql backup & restore guide, popular posts, how to set google as the default search engine in microsoft edge using intune: a step-by-step guide, device preparation policy – (autopilot v2), popular category.

x-All Posts-x 665
Exchange 2013 183
Microsoft 365 141
Exchange 2010 122
Exchange 2016 105
Azure Active Directory 102
Windows Server 55

I don’t think that is supported in the playground.

In Azure playground service chat, both ACR and AKS service supporting. That why i have i tried. If it is not supporting. What is purpose of ACR services is enabled in kodekloud pro.

please help.

You can’t do anything with Roles in the playground - that is why it is not working. I will ask the team.

Ok @al1 , Please check with team and update me

Regards krishna

Hub23 Deployment Guide

Create an azure container registry, create an azure container registry #, create an acr #.

The ACR name must be globally unique and consist of only lowercase alphanumeric characters, between 5 and 50 characters long. This can be checked using: az acr check-name --name <ACR-NAME> .

Login to the ACR #

Save the login server to a variable #, save the registry id to a variable #, assign acrpush role to the service principal #.

The Service Principal needs an AcrPush role so that it is permitted to both push and pull images to/from the registry. Without this, BinderHub won’t be able to store the images it generates.

You will only have permission to perform this step if you are an owner on the turingmybinder Azure subscription. Otherwise, you should ask IT to assign this role to the Service Principal.

See the Service Principal section for instructions on how to download the Service Principal app ID to a file from the Key Vault.

This browser is no longer supported.

Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.

Fail to pull images from Azure Container Registry to Azure Kubernetes Service cluster

2 contributors

Was this article helpful? Your input is important to us. Please use the Feedback button on this page to let us know how well this article worked for you or how we can improve it.

When you're using Microsoft Azure Container Registry together with Azure Kubernetes Service (AKS), an authentication mechanism must be established. You can set up the AKS to Container Registry integration by using a few simple Azure CLI or Azure PowerShell commands. This integration assigns the AcrPull role for the kubelet identity that's associated with the AKS cluster to pull images from a container registry.

In some cases, trying to pull images from a container registry to an AKS cluster fails. This article provides guidance for troubleshooting the most common errors that you encounter when you pull images from a container registry to an AKS cluster.

Before you begin

This article assumes that you have an existing AKS cluster and an existing container registry. See the following quick starts:

If you need an AKS cluster, deploy one by using the Azure CLI or the Azure portal .

If you need an Azure Container Registry (ACR), create one by using the Azure CLI or the Azure portal .

You also need Azure CLI version 2.0.59 or a later version to be installed and configured. Run az version to determine the version. If you have to install or upgrade, see Install Azure CLI .

Symptoms and initial troubleshooting

The Kubernetes pod's STATUS is ImagePullBackOff or ErrImagePull . To get detailed error information, run the following command and check Events from the output.

We recommend that you start troubleshooting by checking the container registry's health and checking whether the container registry is accessible from the AKS cluster.

To check the container registry's health, run the following command:

If a problem is detected, it provides an error code and description. For more information about the errors and possible solutions, see Health check error reference .

If you get Helm-related or Notary-related errors, it doesn't mean that you an issue is affecting Container Registry or AKS. It indicates only that Helm or Notary isn't installed, or that Azure CLI isn't compatible with the current installed version of Helm or Notary, and so on.

To validate whether the container registry is accessible from the AKS cluster, run the following az aks check-acr command:

The following sections help you troubleshoot the most common errors that are displayed in Events in the output of the kubectl describe pod command.

Cause 1: 401 Unauthorized error

An AKS cluster requires an identity. This identity can be either a managed identity or a service principal. If the AKS cluster uses a managed identity, the kubelet identity is used for authenticating with ACR. If the AKS cluster is using as an identity a service principal, the service principal itself is used for authenticating with ACR. No matter what the identity is, the proper authorization that's used to pull an image from a container registry is necessary. Otherwise, you may get the following "401 Unauthorized" error:

Failed to pull image "<acrname>.azurecr.io/<repository:tag>": [rpc error: code = Unknown desc = failed to pull and unpack image "<acrname>.azurecr.io/<repository:tag>": failed to resolve reference "<acrname>.azurecr.io/<repository:tag>": failed to authorize: failed to fetch oauth token: unexpected status: 401 Unauthorized

Several solutions can help you resolve this error, subject to the following constraints:

Solutions 2 , 3 , and 5 are applicable only to AKS clusters that use a service principal .

Solutions 1 , 2 , 3 , and 4 are applicable for the Azure method of creating the role assignment at Container Registry level for AKS's identity .

Solutions 5 and 6 are applicable for the Kubernetes method of pulling a Kubernetes secret .

Solution 1: Make sure AcrPull role assignment is created for identity

The integration between AKS and Container Registry creates an AcrPull role assignment at container registry level for the AKS cluster's kubelet identity. Make sure that the role assignment is created.

To check whether the AcrPull role assignment is created, use one of the following methods:

Run the following command:

Check in the Azure portal by selecting Azure Container Registry > Access control (IAM) > Role assignments . For more information, see List Azure role assignments using the Azure portal .

Besides the AcrPull role, some built-in roles and custom roles can also contain the " Microsoft.ContainerRegistry /registries/pull/read" action. Check those roles if you've got any of them.

If the AcrPull role assignment isn't created, create it by configuring Container Registry integration for the AKS cluster with the following command:

Solution 2: Make sure service principal isn't expired

Make sure that the secret of the service principal that's associated with the AKS cluster isn't expired. To check the expiration date of your service principal, run the following commands:

For more information, see Check the expiration date of your service principal .

If the secret is expired, update the credentials for the AKS cluster .

Solution 3: Make sure AcrPull role is assigned to correct service principal

In some cases, the container registry role assignment still refers to the old service principal. For example, when the service principal of the AKS cluster is replaced with a new one. To make sure that the container registry role assignment refers to the correct service principal, follow these steps:

To check the service principal that's used by the AKS cluster, run the following command:

To check the service principal that's referenced by the container registry role assignment, run the following command:

Compare the two service principals. If they don't match, integrate the AKS cluster with the container registry again.

Solution 4: Make sure the kubelet identity is referenced in the AKS VMSS

When a managed identity is used for authentication with the ACR, the managed identity is known as the kubelet identity. By default, the kubelet identity is assigned at the AKS VMSS level. If the kubelet identity is removed from the AKS VMSS, the AKS nodes can't pull images from the ACR.

To find the kubelet identity of your AKS cluster, run the following command:

Then, you can list the identities of the AKS VMSS by opening the VMSS from the node resource group and selecting Identity > User assigned in the Azure portal or by running the following command:

If the kubelet identity of your AKS cluster isn't assigned to the AKS VMSS, assign it back.

Modifying the AKS VMSS using the IaaS APIs or from the Azure portal isn't supported , and no AKS operation can remove the kubelet identity from the AKS VMSS. This means that something unexpected removed it, for example, a manual removal performed by a team member. To prevent such removal or modification, you can consider using the NRGLockdown feature .

Because modifications to the AKS VMSS aren't supported, they don't propagate at the AKS level. To reassign the kubelet identity to the AKS VMSS, a reconciliation operation is needed. To do this, run the following command:

Solution 5: Make sure the service principal is correct and the secret is valid

If you pull an image by using an image pull secret , and that Kubernetes secret was created by using the values of a service principal, make sure that the associated service principal is correct and the secret is still valid. Follow these steps:

Run the following kubectl get and base64 command to see the values of the Kubernetes secret:

Check the expiration date by running the following az ad sp credential list command. The username is the service principal value.

If necessary, reset the secret of that service principal by running the following az ad sp credential reset command:

Update or re-create the Kubernetes secret accordingly.

Solution 6: Make sure the Kubernetes secret has the correct values of the container registry admin account

If you pull an image by using an image pull secret , and that Kubernetes secret was created by using values of container registry admin account , make sure that the values in the Kubernetes secret are the same as the values of the container registry admin account. Follow these steps:

In the Azure portal , search for and select Container registries .

In the list of container registries, select your container registry.

In the navigation pane for the container registry, select Access keys .

In the Access keys page for the container registry, compare the container registry values with the values in the Kubernetes secret.

If the values don't match, update or re-create the Kubernetes secret accordingly.

If a Regenerate password operation occurred, an operation that's named "Regenerate Container Registry Login Credentials" will be displayed in the Activity log page of the container registry. The Activity log has a 90-day retention period .

Cause 2: Image not found error

Failed to pull image "<acrname>.azurecr.io/<repository:tag>": [rpc error: code = NotFound desc = failed to pull and unpack image "<acrname>.azurecr.io/<repository:tag>": failed to resolve reference "<acrname>.azurecr.io/<repository:tag>": <acrname>.azurecr.io/<repository:tag>: not found

Solution: Make sure image name is correct

If you see this error, make nsure that the image name is fully correct. You should check the registry name, registry login server, the repository name, and the tag. A common mistake is that the login server is specified as "azureacr.io" instead of "azurecr.io".

If the image name isn't fully correct, the 401 Unauthorized error may also occur because AKS always tries anonymous pull regardless of whether the container registry has enabled anonymous pull access.

Cause 3: 403 Forbidden error

Failed to pull image "<acrname>.azurecr.io/<repository:tag>": rpc error: code = Unknown desc = failed to pull and unpack image "<acrname>.azurecr.io/<repository:tag>": failed to resolve reference "<acrname>.azurecr.io/<repository:tag>": failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden

Solution 1: Make sure AKS virtual network link is set in the container registry's Private DNS zone

If the network interface of the container registry's private endpoint and the AKS cluster are in different virtual networks, make sure that the virtual network link for the AKS cluster's virtual network is set in the Private DNS zone of the container registry. (That link is named "privatelink.azurecr.io" by default.) If the virtual network link isn't in the Private DNS zone of the container registry, add it by using one of the following ways:

In the Azure portal, select the private DNS zone "privatelink.azurecr.io", select Virtual network links > Add under the Settings panel, and then select a name and the virtual network of the AKS cluster. Select OK .

It's optional to select the "Enable auto registration" feature.

Create a virtual network link to the specified Private DNS zone by using Azure CLI .

Solution 2: Add AKS Load Balancer's public IP address to allowed IP address range of the container registry

If the AKS cluster connects publicly to the container registry (NOT through a private link or an endpoint) and the public network access of the container registry is limited to selected networks, add AKS Load Balancer's public IP address to the allowed IP address range of the container registry:

Verify that the public network access is limited to selected networks.

In the Azure portal, navigate to the container registry. Under Settings , select Networking . On the Public access tab, Public network access is set to Selected networks or Disabled .

Obtain the AKS Load Balancer's public IP address by using one of the following ways:

In the Azure portal, navigate to the AKS cluster. Under Settings , select Properties , select one of the virtual machine scale sets in the infrastructure resource group, and check the public IP address of the AKS Load Balancer.

Allow access from the AKS Load Balancer's public IP address by using one of the following ways:

Run az acr network-rule add command as follows:

For more information, see Add network rule to registry .

In the Azure portal, navigate to the container registry. Under Settings , select Networking . On the Public access tab, under Firewall , add the AKS Load Balancer's public IP address to Address range and then select Save . For more information, see Access from selected public network - portal .

If Public network access is set to Disabled , switch it to Selected networks first.

Screenshot about how to add AKS Load Balancer's public IP address to Address range

Cause 4: 443 timeout error

Failed to pull image "<acrname>.azurecr.io/<repository:tag>": rpc error: code = Unknown desc = failed to pull and unpack image "<acrname>.azurecr.io/<repository:tag>": failed to resolve reference "<acrname>.azurecr.io/<repository:tag>": failed to do request: Head "https://<acrname>.azurecr.io/v2/<repository>/manifests/v1": dial tcp <acrprivateipaddress>: 443: i/o timeout

The "443 timeout" error occurs only when you connect privately to a container registry by using Azure Private Link .

Solution 1: Make sure virtual network peering is used

If the network interface of the container registry's private endpoint and the AKS cluster are in different virtual networks, make sure that virtual network peering is used for both virtual networks. You can check virtual network peering by running the Azure CLI command az network vnet peering list --resource-group <MyResourceGroup> --vnet-name <MyVirtualNetwork> --output table or in the Azure portal by selecting the VNETs > Peerings under the Settings panel. For more information about listing all peerings of a specified virtual network, see az network vnet peering list .

If the virtual network peering is used for both virtual networks, make sure that the status is "Connected". If the status is Disconnected , delete the peering from both virtual networks, and then re-create it. If the status is "Connected", see the troubleshooting guide: The peering status is "Connected" .

For further troubleshooting, connect to one of the AKS nodes or pods , and then test the connectivity with the container registry at TCP level by using the Telnet or Netcat utility. Check the IP address with the nslookup <acrname>.azurecr.io command, and then run the telnet <ip-address-of-the-container-registry> 443 command.

For more information about connecting to AKS nodes, see Connect with SSH to Azure Kubernetes Service (AKS) cluster nodes for maintenance or troubleshooting .

Solution 2: Use Azure Firewall Service

If the network interface of the container registry's private endpoint and the AKS cluster are in different virtual networks, in addition to virtual network peering, you may use Azure Firewall Service to set up a Hub-spoke network topology in Azure . When you set up the firewall rule, you need to use network rules to explicitly allow the outbound connection to the container registry private endpoint IP addresses.

Cause 5: No match for platform in manifest

The host operating system (node OS) is incompatible with the image that's used for the pod or container. For example, if you schedule a pod to run a Linux container on a Windows node, or a Windows container on a Linux node, the following error occurs:

Failed to pull image "<acrname>.azurecr.io/<repository:tag>": [ rpc error: code = NotFound desc = failed to pull and unpack image "<acrname>.azurecr.io/<repository:tag>": no match for platform in manifest : not found, ]

This error can occur for an image that's pulled from any source, as long as the image is incompatible with the host OS. The error isn't limited to images that are pulled from the container registry.

Solution: Configure the nodeSelector field correctly in your pod or deployment

Specify the correct nodeSelector field in the configuration settings of your pod or deployment. The correct value for this field's kubernetes.io/os setting ensures that the pod will be scheduled on the correct type of node. The following table shows how to set the kubernetes.io/os setting in YAML:

Container type	YAML setting
Linux container
Windows container

For example, the following YAML code describes a pod that needs to be scheduled on a Linux node:

More information

If the troubleshooting guidance in this article doesn't help you resolve the issue, here are some other things to consider:

Check the network security groups and route tables associated with subnets, if you've got any of those items.

If a virtual appliance like a firewall controls the traffic between subnets, check the firewall and Firewall access rules .

Third-party information disclaimer

The third-party products that this article discusses are manufactured by companies that are independent of Microsoft. Microsoft makes no warranty, implied or otherwise, about the performance or reliability of these products.

Contact us for help

If you have questions or need help, create a support request , or ask Azure community support . You can also submit product feedback to Azure feedback community .

Was this page helpful?

Additional resources

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

Notifications You must be signed in to change notification settings

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

az aks create gives Could not create a role assignment for subnet. Are you an Owner on this subscription? Operation failed with status: 'Bad Request'. Details: Internal server error #10191

x10an14 commented Aug 8, 2019 • edited Loading

I'm unable to create a kubernetes cluster with the below cli command, with the settings/inputs given.

I've written some shellscripts to automate the creation and set-up of the K8s cluster and associated resources ( / /etc.).
Below are some relevant files and the bash output of said terminal execution, where you can see me create from scratch all the resources, with actual valid inputs/outputs (I'll run the delete script before I post this though).

/usr/bin/env bash export PS4='# [${BASH_SOURCE}:${LINENO}]: ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' set -o errexit # Stop script on first error (non-zero returncode) set -o pipefail # Stop script on first error in a piped command (default only checks last pipe-command) set -o verbose # Verbose debugprinting of executing scripts set -o xtrace # Show commands being executed through debugprint during execution

/usr/bin/env bash FILE_DIR=$(dirname $0) source "${FILE_DIR}"/_bash_support/debug.sh if [[ -v $SERVICE_PRINCIPAL_NAME ]] || [[ -z "${SERVICE_PRINCIPAL_NAME}" ]]; then source "${FILE_DIR}/inputs.sh" fi set -o nounset # Throw error on first variable-reference to an unset variable "${FILE_DIR}/network/network.azcli" "${FILE_DIR}/keyvault/keyvault.azcli" "${FILE_DIR}/aks/providers.azcli" "${FILE_DIR}/aks/k8s.azcli" "${FILE_DIR}/acr/acr.azcli" "${FILE_DIR}/helm/helm.sh" "${FILE_DIR}/rbac/set-access.azcli" "${FILE_DIR}/ingress/ingress.azcli"

: The below terminal copy/paste contains the output of my scripts having all sourced the above contents as the first thing they do.

/Documents/team-k8s-project $ setup/init_env.sh set -o xtrace # Show commands being executed through debugprint during execution if [[ -v $SERVICE_PRINCIPAL_NAME ]] || [[ -z "${SERVICE_PRINCIPAL_NAME}" ]]; then source "${FILE_DIR}/inputs.sh" fi # [setup/init_env.sh:6]: main(): [[ -v '' ]] # [setup/init_env.sh:6]: main(): [[ -z '' ]] # [setup/init_env.sh:7]: main(): source setup/inputs.sh #!/usr/bin/env bash # Throw error on first variable-reference to an unset variable set -o nounset ## [setup/inputs.sh:4]: source(): set -o nounset # Commonly used variables ## _MUST_ fill-in variables! export location="westeurope" ## [setup/inputs.sh:9]: source(): export location=westeurope ## [setup/inputs.sh:9]: source(): location=westeurope export subscription_name="{REDACTED}" ## [setup/inputs.sh:10]: source(): export subscription_name={REDACTED} ## [setup/inputs.sh:10]: source(): subscription_name={REDACTED} export team_name="${subscription_name}-${location}-team" ## [setup/inputs.sh:11]: source(): export team_name={REDACTED}-westeurope-team ## [setup/inputs.sh:11]: source(): team_name={REDACTED}-westeurope-team export product_name="${team_name}-team" ## [setup/inputs.sh:12]: source(): export product_name={REDACTED}-westeurope-team-team ## [setup/inputs.sh:12]: source(): product_name={REDACTED}-westeurope-team-team export environment='dev' ## [setup/inputs.sh:13]: source(): export environment=dev ## [setup/inputs.sh:13]: source(): environment=dev ## Derivatives export subscription_name_capitalized="$(tr '[:lower:]' '[:upper:]' <<< ${subscription_name:0:1})${subscription_name:1}" ### [setup/inputs.sh:16]: source(): tr '[:lower:]' '[:upper:]' ## [setup/inputs.sh:16]: source(): export subscription_name_capitalized={REDACTED} ## [setup/inputs.sh:16]: source(): subscription_name_capitalized={REDACTED} export resource_group="${product_name}-rg" ## [setup/inputs.sh:17]: source(): export resource_group={REDACTED}-westeurope-team-team-rg ## [setup/inputs.sh:17]: source(): resource_group={REDACTED}-westeurope-team-team-rg export subscription_id=$(az account list \ --query "[?name=='${subscription_name_capitalized}'].[id]" \ -o tsv \ ) ### [setup/inputs.sh:21]: source(): az account list --query '[?name=='\''{REDACTED}'\''].[id]' -o tsv ## [setup/inputs.sh:21]: source(): export subscription_id=15ea10b3-c25a-4569-b4c2-46054c6f5227 ## [setup/inputs.sh:21]: source(): subscription_id=15ea10b3-c25a-4569-b4c2-46054c6f5227 # Resource specific variables ## Azure Container Registry (ACR) export acr_name="$(echo "${team_name}-acr" | sed -r 's/-//g')" ### [setup/inputs.sh:25]: source(): echo {REDACTED}-westeurope-team-acr ### [setup/inputs.sh:25]: source(): sed -r s/-//g ## [setup/inputs.sh:25]: source(): export acr_name={REDACTED}westeuropeteamacr ## [setup/inputs.sh:25]: source(): acr_name={REDACTED}westeuropeteamacr export acr_rg="${team_name}-rg" ## [setup/inputs.sh:26]: source(): export acr_rg={REDACTED}-westeurope-team-rg ## [setup/inputs.sh:26]: source(): acr_rg={REDACTED}-westeurope-team-rg export acr_email="[email protected]" ## [setup/inputs.sh:27]: source(): export [email protected] ## [setup/inputs.sh:27]: source(): [email protected] export secret_name="acr-auth" ## [setup/inputs.sh:28]: source(): export secret_name=acr-auth ## [setup/inputs.sh:28]: source(): secret_name=acr-auth ## Azure Keyvault (KV) export kv_name="${team_name}" ## [setup/inputs.sh:31]: source(): export kv_name={REDACTED}-westeurope-team ## [setup/inputs.sh:31]: source(): kv_name={REDACTED}-westeurope-team export kv_rg="${team_name}-rg" ## [setup/inputs.sh:32]: source(): export kv_rg={REDACTED}-westeurope-team-rg ## [setup/inputs.sh:32]: source(): kv_rg={REDACTED}-westeurope-team-rg export k8s_rbac_keyname="k8s-rbac-client" ## [setup/inputs.sh:33]: source(): export k8s_rbac_keyname=k8s-rbac-client ## [setup/inputs.sh:33]: source(): k8s_rbac_keyname=k8s-rbac-client ## Azure Kubernetes Service (AKS) export aks_name="${team_name}-aks" ## [setup/inputs.sh:36]: source(): export aks_name={REDACTED}-westeurope-team-aks ## [setup/inputs.sh:36]: source(): aks_name={REDACTED}-westeurope-team-aks export aks_rg="${team_name}-rg" ## [setup/inputs.sh:37]: source(): export aks_rg={REDACTED}-westeurope-team-rg ## [setup/inputs.sh:37]: source(): aks_rg={REDACTED}-westeurope-team-rg export aks_subnet="${aks_name}-subnet" ## [setup/inputs.sh:38]: source(): export aks_subnet={REDACTED}-westeurope-team-aks-subnet ## [setup/inputs.sh:38]: source(): aks_subnet={REDACTED}-westeurope-team-aks-subnet export k8s_namespace="${product_name}-${environment}" ## [setup/inputs.sh:39]: source(): export k8s_namespace={REDACTED}-westeurope-team-team-dev ## [setup/inputs.sh:39]: source(): k8s_namespace={REDACTED}-westeurope-team-team-dev ## Networking export vnet="${team_name}" ## [setup/inputs.sh:42]: source(): export vnet={REDACTED}-westeurope-team ## [setup/inputs.sh:42]: source(): vnet={REDACTED}-westeurope-team export vnet_rg="${team_name}-rg" ## [setup/inputs.sh:43]: source(): export vnet_rg={REDACTED}-westeurope-team-rg ## [setup/inputs.sh:43]: source(): vnet_rg={REDACTED}-westeurope-team-rg export subnet="${aks_name}-subnet" ## [setup/inputs.sh:44]: source(): export subnet={REDACTED}-westeurope-team-aks-subnet ## [setup/inputs.sh:44]: source(): subnet={REDACTED}-westeurope-team-aks-subnet # Misc. ## Show my service principals ## az ad sp list --subscription "${subscription_name_capitalized}" --show-mine -o table export SERVICE_PRINCIPAL_NAME={REDACTED} ## [setup/inputs.sh:49]: source(): export SERVICE_PRINCIPAL_NAME={REDACTED} ## [setup/inputs.sh:49]: source(): SERVICE_PRINCIPAL_NAME={REDACTED} export SERVICE_PRINCIPAL_ID="http://${SERVICE_PRINCIPAL_NAME}" ## [setup/inputs.sh:50]: source(): export SERVICE_PRINCIPAL_ID=http://{REDACTED} ## [setup/inputs.sh:50]: source(): SERVICE_PRINCIPAL_ID=http://{REDACTED} set -o nounset # Throw error on first variable-reference to an unset variable # [setup/init_env.sh:9]: main(): set -o nounset "${FILE_DIR}/network/network.azcli" # [setup/init_env.sh:11]: main(): setup/network/network.azcli set -o xtrace # Show commands being executed through debugprint during execution echo -e "\n--- Network setup start" # [setup/network/network.azcli:6]: main(): echo -e '\n--- Network setup start' --- Network setup start if [[ -v $SERVICE_PRINCIPAL_NAME ]] || [[ -z "${SERVICE_PRINCIPAL_NAME}" ]]; then source "${FILE_DIR}/../inputs.sh" fi # [setup/network/network.azcli:8]: main(): [[ -v {REDACTED} ]] # [setup/network/network.azcli:8]: main(): [[ -z {REDACTED} ]] set -o nounset # Throw error on first variable-reference to an unset variable # [setup/network/network.azcli:11]: main(): set -o nounset # Create core network az group create \ --location $location \ --name $vnet_rg \ --subscription $subscription_id # [setup/network/network.azcli:14]: main(): az group create --location westeurope --name {REDACTED}-westeurope-team-rg --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 { "id": "/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg", "location": "westeurope", "managedBy": null, "name": "{REDACTED}-westeurope-team-rg", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": null } az network vnet create \ --name $vnet \ --resource-group $vnet_rg \ --location $location \ --address-prefix 10.40.0.0/16 # [setup/network/network.azcli:19]: main(): az network vnet create --name {REDACTED}-westeurope-team --resource-group {REDACTED}-westeurope-team-rg --location westeurope --address-prefix 10.40.0.0/16 { "newVNet": { "addressSpace": { "addressPrefixes": [ "10.40.0.0/16" ] }, "ddosProtectionPlan": null, "dhcpOptions": { "dnsServers": [] }, "enableDdosProtection": false, "enableVmProtection": false, "etag": "W/\"8c77e8d4-a1be-498c-9a06-29d3c3cea19a\"", "id": "/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg/providers/Microsoft.Network/virtualNetworks/{REDACTED}-westeurope-team", "location": "westeurope", "name": "{REDACTED}-westeurope-team", "provisioningState": "Succeeded", "resourceGroup": "{REDACTED}-westeurope-team-rg", "resourceGuid": "c15424c5-e428-4432-b693-29d50905e55e", "subnets": [], "tags": {}, "type": "Microsoft.Network/virtualNetworks", "virtualNetworkPeerings": [] } } # The name GatewaySubnet does not meet the naming convention because Azure is looking at it! az network vnet subnet create \ --resource-group $vnet_rg \ --vnet-name $vnet \ --name "GatewaySubnet" \ --address-prefixes 10.40.0.0/24 # [setup/network/network.azcli:26]: main(): az network vnet subnet create --resource-group {REDACTED}-westeurope-team-rg --vnet-name {REDACTED}-westeurope-team --name GatewaySubnet --address-prefixes 10.40.0.0/24 { "addressPrefix": "10.40.0.0/24", "addressPrefixes": null, "delegations": [], "etag": "W/\"e54ef133-b8b5-4eeb-8d16-b267b325fcfc\"", "id": "/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg/providers/Microsoft.Network/virtualNetworks/{REDACTED}-westeurope-team/subnets/GatewaySubnet", "ipConfigurationProfiles": null, "ipConfigurations": null, "name": "GatewaySubnet", "natGateway": null, "networkSecurityGroup": null, "privateEndpoints": null, "provisioningState": "Succeeded", "purpose": null, "resourceGroup": "{REDACTED}-westeurope-team-rg", "resourceNavigationLinks": null, "routeTable": null, "serviceAssociationLinks": null, "serviceEndpointPolicies": null, "serviceEndpoints": null, "type": "Microsoft.Network/virtualNetworks/subnets" } az network vnet subnet create \ --resource-group $vnet_rg \ --vnet-name $vnet \ --name $subnet \ --address-prefixes 10.40.1.0/24 # [setup/network/network.azcli:32]: main(): az network vnet subnet create --resource-group {REDACTED}-westeurope-team-rg --vnet-name {REDACTED}-westeurope-team --name {REDACTED}-westeurope-team-aks-subnet --address-prefixes 10.40.1.0/24 { "addressPrefix": "10.40.1.0/24", "addressPrefixes": null, "delegations": [], "etag": "W/\"602169fe-651e-443a-94fd-eae8ca93c4e2\"", "id": "/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg/providers/Microsoft.Network/virtualNetworks/{REDACTED}-westeurope-team/subnets/{REDACTED}-westeurope-team-aks-subnet", "ipConfigurationProfiles": null, "ipConfigurations": null, "name": "{REDACTED}-westeurope-team-aks-subnet", "natGateway": null, "networkSecurityGroup": null, "privateEndpoints": null, "provisioningState": "Succeeded", "purpose": null, "resourceGroup": "{REDACTED}-westeurope-team-rg", "resourceNavigationLinks": null, "routeTable": null, "serviceAssociationLinks": null, "serviceEndpointPolicies": null, "serviceEndpoints": null, "type": "Microsoft.Network/virtualNetworks/subnets" } echo "--- Network setup stop" # [setup/network/network.azcli:38]: main(): echo '--- Network setup stop' --- Network setup stop "${FILE_DIR}/keyvault/keyvault.azcli" # [setup/init_env.sh:12]: main(): setup/keyvault/keyvault.azcli set -o xtrace # Show commands being executed through debugprint during execution echo -e "\n--- Keyvault setup start" # [setup/keyvault/keyvault.azcli:6]: main(): echo -e '\n--- Keyvault setup start' --- Keyvault setup start if [[ -v $SERVICE_PRINCIPAL_NAME ]] || [[ -z "${SERVICE_PRINCIPAL_NAME}" ]]; then source "${FILE_DIR}/../inputs.sh" fi # [setup/keyvault/keyvault.azcli:8]: main(): [[ -v {REDACTED} ]] # [setup/keyvault/keyvault.azcli:8]: main(): [[ -z {REDACTED} ]] set -o nounset # Throw error on first variable-reference to an unset variable # [setup/keyvault/keyvault.azcli:11]: main(): set -o nounset if ! az group exists --name "${kv_rg}" --subscription "${subscription_id}"; then az group create \ --location $location \ --name $kv_rg \ --subscription $subscription_id \ --verbose fi # [setup/keyvault/keyvault.azcli:13]: main(): az group exists --name {REDACTED}-westeurope-team-rg --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 true if ! az keyvault show --name "${kv_name}"; then # Create development key vault az keyvault create \ --name "${kv_name}" \ --location "${location}" \ --resource-group "${kv_rg}" \ --subscription "${subscription_id}" \ --verbose fi # [setup/keyvault/keyvault.azcli:21]: main(): az keyvault show --name {REDACTED}-westeurope-team { "id": "/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg/providers/Microsoft.KeyVault/vaults/{REDACTED}-westeurope-team", "location": "westeurope", "name": "{REDACTED}-westeurope-team", "properties": { "accessPolicies": [ { "applicationId": null, "objectId": "694534a6-1c88-4bce-9c7e-5b8a77005356", "permissions": { "certificates": [ "get", "list", "delete", "create", "import", "update", "managecontacts", "getissuers", "listissuers", "setissuers", "deleteissuers", "manageissuers", "recover" ], "keys": [ "get", "create", "delete", "list", "update", "import", "backup", "restore", "recover" ], "secrets": [ "get", "list", "set", "delete", "backup", "restore", "recover" ], "storage": [ "get", "list", "delete", "set", "update", "regeneratekey", "setsas", "listsas", "getsas", "deletesas" ] }, "tenantId": "0abe8783-2c3e-4c42-9848-54e419bcdeb0" } ], "createMode": null, "enablePurgeProtection": null, "enableSoftDelete": null, "enabledForDeployment": false, "enabledForDiskEncryption": null, "enabledForTemplateDeployment": null, "networkAcls": null, "provisioningState": "Succeeded", "sku": { "name": "standard" }, "tenantId": "0abe8783-2c3e-4c42-9848-54e419bcdeb0", "vaultUri": "https://{REDACTED}-westeurope-team.vault.azure.net/" }, "resourceGroup": "{REDACTED}-westeurope-team-rg", "tags": {}, "type": "Microsoft.KeyVault/vaults" } echo "--- Keyvault setup stop" # [setup/keyvault/keyvault.azcli:31]: main(): echo '--- Keyvault setup stop' --- Keyvault setup stop "${FILE_DIR}/aks/providers.azcli" # [setup/init_env.sh:13]: main(): setup/aks/providers.azcli set -o xtrace # Show commands being executed through debugprint during execution echo -e "\n--- Providers setup start" # [setup/aks/providers.azcli:6]: main(): echo -e '\n--- Providers setup start' --- Providers setup start if [[ -v $SERVICE_PRINCIPAL_NAME ]] || [[ -z "${SERVICE_PRINCIPAL_NAME}" ]]; then source "${FILE_DIR}/../inputs.sh" fi # [setup/aks/providers.azcli:8]: main(): [[ -v {REDACTED} ]] # [setup/aks/providers.azcli:8]: main(): [[ -z {REDACTED} ]] set -o nounset # Throw error on first variable-reference to an unset variable # [setup/aks/providers.azcli:11]: main(): set -o nounset az provider register \ --namespace Microsoft.ContainerService \ --subscription $subscription_id # [setup/aks/providers.azcli:13]: main(): az provider register --namespace Microsoft.ContainerService --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 echo "--- Providers setup stop" # [setup/aks/providers.azcli:17]: main(): echo '--- Providers setup stop' --- Providers setup stop "${FILE_DIR}/aks/k8s.azcli" # [setup/init_env.sh:14]: main(): setup/aks/k8s.azcli set -o xtrace # Show commands being executed through debugprint during execution echo -e "\n--- Kubernetes setup start" # [setup/aks/k8s.azcli:6]: main(): echo -e '\n--- Kubernetes setup start' --- Kubernetes setup start if [[ -v $SERVICE_PRINCIPAL_NAME ]] || [[ -z "${SERVICE_PRINCIPAL_NAME}" ]]; then source "${FILE_DIR}/../inputs.sh" fi # [setup/aks/k8s.azcli:8]: main(): [[ -v {REDACTED} ]] # [setup/aks/k8s.azcli:8]: main(): [[ -z {REDACTED} ]] set -o nounset # Throw error on first variable-reference to an unset variable # [setup/aks/k8s.azcli:11]: main(): set -o nounset # Get k8s rbac credentials if ! az keyvault secret list --vault-name "${kv_name}" | \ jq -r '.[] | .id' | \ awk -F '/' '{print $NF}' | \ grep -q "${k8s_rbac_keyname}-id"; then # If secret does not exist in keyvault, create it az keyvault secret set \ --vault-name "${kv_name}" \ --name "${k8s_rbac_keyname}-id" \ --value "$(pwgen --ambiguous --secure 16 1)" fi # [setup/aks/k8s.azcli:14]: main(): az keyvault secret list --vault-name {REDACTED}-westeurope-team # [setup/aks/k8s.azcli:15]: main(): jq -r '.[] | .id' # [setup/aks/k8s.azcli:16]: main(): awk -F / '{print $NF}' # [setup/aks/k8s.azcli:17]: main(): grep -q k8s-rbac-client-id echo -e "\tGet client ID from keyvault" # [setup/aks/k8s.azcli:24]: main(): echo -e '\tGet client ID from keyvault' Get client ID from keyvault client_id=$(az keyvault secret show \ --vault-name $kv_name \ --name "${k8s_rbac_keyname}-id" \ --query "value" \ --output tsv) ## [setup/aks/k8s.azcli:29]: main(): az keyvault secret show --vault-name {REDACTED}-westeurope-team --name k8s-rbac-client-id --query value --output tsv # [setup/aks/k8s.azcli:29]: main(): client_id=kRuebd7khTr39JUq [ -z "${client_id}" ] && exit 1 # [setup/aks/k8s.azcli:30]: main(): '[' -z kRuebd7khTr39JUq ']' if ! az keyvault secret list --vault-name "${kv_name}" | \ jq -r '.[] | .id' | \ awk -F '/' '{print $NF}' | \ grep -q "${k8s_rbac_keyname}-secret"; then # If secret does not exist in keyvault, create it client_id= az keyvault secret set \ --vault-name "${kv_name}" \ --name "${k8s_rbac_keyname}-secret" \ --value "$(pwgen --ambiguous --secure 64 1)" fi # [setup/aks/k8s.azcli:32]: main(): az keyvault secret list --vault-name {REDACTED}-westeurope-team # [setup/aks/k8s.azcli:33]: main(): jq -r '.[] | .id' # [setup/aks/k8s.azcli:34]: main(): awk -F / '{print $NF}' # [setup/aks/k8s.azcli:35]: main(): grep -q k8s-rbac-client-secret echo -e "\tGet client secret from keyvault" # [setup/aks/k8s.azcli:43]: main(): echo -e '\tGet client secret from keyvault' Get client secret from keyvault client_secret=$(az keyvault secret show \ --vault-name $kv_name \ --name "${k8s_rbac_keyname}-secret" \ --query "value" \ --output tsv) ## [setup/aks/k8s.azcli:48]: main(): az keyvault secret show --vault-name {REDACTED}-westeurope-team --name k8s-rbac-client-secret --query value --output tsv # [setup/aks/k8s.azcli:48]: main(): client_secret=EsuWzXCWKiuthT3ubrsfP3pxLMfn4feqrWm3hJyyYvNrsTFka9iMywYyHPtmUNdY [ -z "${client_secret}" ] && exit 1 # [setup/aks/k8s.azcli:49]: main(): '[' -z EsuWzXCWKiuthT3ubrsfP3pxLMfn4feqrWm3hJyyYvNrsTFka9iMywYyHPtmUNdY ']' # Get k8s eu west 1 subnet id echo -e "\tGet subnet ID" # [setup/aks/k8s.azcli:52]: main(): echo -e '\tGet subnet ID' Get subnet ID k8s_subnet_id=$(az network vnet subnet show \ --resource-group $vnet_rg \ --vnet-name $vnet \ --name $aks_subnet \ --subscription $subscription_id \ --query id \ -o tsv) ## [setup/aks/k8s.azcli:59]: main(): az network vnet subnet show --resource-group {REDACTED}-westeurope-team-rg --vnet-name {REDACTED}-westeurope-team --name {REDACTED}-westeurope-team-aks-subnet --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 --query id -o tsv # [setup/aks/k8s.azcli:59]: main(): k8s_subnet_id=/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg/providers/Microsoft.Network/virtualNetworks/{REDACTED}-westeurope-team/subnets/{REDACTED}-westeurope-team-aks-subnet # Get tenant id echo -e "\tGet tenant ID" # [setup/aks/k8s.azcli:62]: main(): echo -e '\tGet tenant ID' Get tenant ID tenant_id=$(az account show \ --query tenantId \ -o tsv) ## [setup/aks/k8s.azcli:65]: main(): az account show --query tenantId -o tsv # [setup/aks/k8s.azcli:65]: main(): tenant_id=0abe8783-2c3e-4c42-9848-54e419bcdeb0 # Set the current subscription because the explicit argument does NOT working properly echo -e "\tGet current subscription ID" # [setup/aks/k8s.azcli:68]: main(): echo -e '\tGet current subscription ID' Get current subscription ID current_subscription=$(az account show \ --query id \ -o tsv) ## [setup/aks/k8s.azcli:71]: main(): az account show --query id -o tsv # [setup/aks/k8s.azcli:71]: main(): current_subscription=15ea10b3-c25a-4569-b4c2-46054c6f5227 echo -e "\tSet subscription to current" # [setup/aks/k8s.azcli:72]: main(): echo -e '\tSet subscription to current' Set subscription to current az account set \ --subscription $subscription_id # [setup/aks/k8s.azcli:73]: main(): az account set --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 # Create the k8s echo -e "\tCreate resource group $aks_rg" # [setup/aks/k8s.azcli:77]: main(): echo -e '\tCreate resource group {REDACTED}-westeurope-team-rg' Create resource group {REDACTED}-westeurope-team-rg az group create \ --location $location \ --name $aks_rg \ --subscription $subscription_id # [setup/aks/k8s.azcli:78]: main(): az group create --location westeurope --name {REDACTED}-westeurope-team-rg --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 { "id": "/subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg", "location": "westeurope", "managedBy": null, "name": "{REDACTED}-westeurope-team-rg", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": null } # NOTE: Make sure the subscription limits are greater than 10 cores # NOTE: The nodepool-name does not meet the naming convention because the limit of characters echo -e "\tCreate Kubernetes cluster (this will take a while)" # [setup/aks/k8s.azcli:85]: main(): echo -e '\tCreate Kubernetes cluster (this will take a while)' Create Kubernetes cluster (this will take a while) az aks create \ --resource-group $aks_rg \ --name $aks_name \ --node-count 3 \ --generate-ssh-keys \ --network-plugin kubenet \ --service-principal $client_id \ --client-secret $client_secret \ --service-cidr 192.168.0.0/17 \ --dns-service-ip 192.168.0.2 \ --pod-cidr 192.168.128.0/17 \ --docker-bridge-address 172.17.0.1/16 \ --vnet-subnet-id $k8s_subnet_id \ --kubernetes-version 1.13.5 \ --node-osdisk-size 200 \ --node-vm-size Standard_DS3_v2 \ --admin-username kadmin \ --nodepool-name euwest \ --subscription $subscription_id \ --enable-addons monitoring \ --verbose # [setup/aks/k8s.azcli:86]: main(): az aks create --resource-group {REDACTED}-westeurope-team-rg --name {REDACTED}-westeurope-team-aks --node-count 3 --generate-ssh-keys --network-plugin kubenet --service-principal kRuebd7khTr39JUq --client-secret EsuWzXCWKiuthT3ubrsfP3pxLMfn4feqrWm3hJyyYvNrsTFka9iMywYyHPtmUNdY --service-cidr 192.168.0.0/17 --dns-service-ip 192.168.0.2 --pod-cidr 192.168.128.0/17 --docker-bridge-address 172.17.0.1/16 --vnet-subnet-id /subscriptions/15ea10b3-c25a-4569-b4c2-46054c6f5227/resourceGroups/{REDACTED}-westeurope-team-rg/providers/Microsoft.Network/virtualNetworks/{REDACTED}-westeurope-team/subnets/{REDACTED}-westeurope-team-aks-subnet --kubernetes-version 1.13.5 --node-osdisk-size 200 --node-vm-size Standard_DS3_v2 --admin-username kadmin --nodepool-name euwest --subscription 15ea10b3-c25a-4569-b4c2-46054c6f5227 --enable-addons monitoring --verbose Use existing SSH public key file: /home/user/.ssh/id_rsa.pub Waiting for AAD role to propagate Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[### ] 10.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[####### ] 20.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[########## ] 30.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[############## ] 40.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[################## ] 50.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[##################### ] 60.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[######################### ] 70.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[############################ ] 80.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Waiting for AAD role to propagate[################################ ] 90.0000%Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://graph.windows.net/0abe8783-2c3e-4c42-9848-54e419bcdeb0/getObjectsByObjectIds?api-version=1.6 Could not create a role assignment for subnet. Are you an Owner on this subscription? Operation failed with status: 'Bad Request'. Details: Internal server error command ran in 124.497 seconds. [2019-08-08 13:16:20] 1 user@xmachine:~/Documents/team-k8s-project $

For the command not to fail, so that the k8s cluster could be created and the script(s) could continue.

👍 1 reaction

ghost commented Aug 13, 2019

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc

Sorry, something went wrong.

griffinator76 commented Oct 29, 2019 • edited Loading

I am getting the same error trying to update an existing cluster to attach an ACR:

generates the error

I have the "global admin" role.

TitaniumDave commented Dec 2, 2019

Hi ,

Global Admin is an AAD role not a subscription level role. Owner is a subscription role and assigned via the subscription blade:

screen.

Alternatively you can provide Global Admins with the ability to assign subscription level permissions, how to do that is detailed on Microsoft page (Access Management for Azure resources is what you want).

Hope that helps.

GersonDias commented Dec 13, 2019

I'm facing the same problem with the Owner role

palma21 commented Dec 16, 2019

do you have a commands ran list and output?

Herr-Sepp commented Jan 14, 2020

I've had the same problem come up recently.
i tried to create a AKS-cluster with this command:

and i got

I am Service Administrator for this development subcription so this should be working. Even after i gave me the owner role is got this error message.

In the past i have successfully created aks cluster with "-attach-acr" on this account.

I'm not sure what's wrong.

i tried before to create this aks-cluster with the installed aks-preview extensions where at the moment --attach-acr is broken ( ), so i decided to remove the preview extension but now i got this error.

Herr-Sepp commented Jan 14, 2020 • edited Loading

i found the problem and the solution.
az aks was using an old serviceprincipal that i had delete some time ago.

to fix this you must delete the file "aksServicePrincipal.json" under

after that the az aks create command run through without an error.
But the Error message make it difficult to find it.

Maybe you can check if the Service principal is valid before you try to assign a role to it?

👍 2 reactions

palma21 commented Jan 17, 2020

Thanks for chasing this we'll take a look at that error message to see if we can clarify it.

krowlandson commented May 28, 2020 • edited Loading

I've also come across this and found that many of the commands use the which can cause this issue. In my case, the Azure CLI commands appear to be performing a lookup of the "Object ID" for the Service Principal being used to run the command before validating a Role Assignment against the dependent resource.

I found multiple situations where I got a warning or error similar to the following:

This doesn't always cause the job to fail. For example, when creating a new AKS Cluster attached to an existing Subnet which has the appropriate Role Assignments, this will be treated as a warning and the command will continue to run. By contrast, trying to connect the AKS Cluster to an Azure Container Registry using will fail.

Looking at the debug output, I was getting an authentication error (HTTP 403) when trying to connect to:

To resolve this, I needed to to the Service Principal for the permission:

This is in addition to assigning the Owner role on the Subscription, or ensuring the Service Principal has suitable Role Assignments on each dependent resource.

Unfortunately granting permissions to these APIs isn't always possible due to internal security policies, so it would be good to find an alternative way to handle this in the Azure CLI but hopefully this piece of information will help anyone else who comes across this.

👍 3 reactions

Laasya-07 commented Jul 5, 2021

I am getting same error. Also, I dont have aksServicePrincipal.json and Service Principal has owner permissions on the subscription.

Waiting for AAD role to propagate[### ] 10.0000%
Waiting for AAD role to propagate[####### ] 20.0000%
Waiting for AAD role to propagate[########## ] 30.0000%
Waiting for AAD role to propagate[############## ] 40.0000%
Waiting for AAD role to propagate[################## ] 50.0000%
Waiting for AAD role to propagate[##################### ] 60.0000%
Waiting for AAD role to propagate[######################### ] 70.0000%
Waiting for AAD role to propagate[############################ ] 80.0000%
Waiting for AAD role to propagate[################################ ] 90.0000%WARNING: Could not create a role assignment for subnet. Are you an Owner on this subscription?
Waiting for AAD role to propagate[### ] 10.0000%
Waiting for AAD role to propagate[####### ] 20.0000%
Waiting for AAD role to propagate[########## ] 30.0000%
Waiting for AAD role to propagate[############## ] 40.0000%
Waiting for AAD role to propagate[################## ] 50.0000%
Waiting for AAD role to propagate[##################### ] 60.0000%
Waiting for AAD role to propagate[######################### ] 70.0000%
Waiting for AAD role to propagate[############################ ] 80.0000%
Waiting for AAD role to propagate[################################ ] 90.0000%ERROR: Could not create a role assignment for ACR. Are you an Owner on this subscription?

Laasya-07 commented Jul 8, 2021

Hello all, the reason of could not create role assignment for subnet is because of the maxPods. The count has to be greater than 30.

I'm unable to create a kubernetes cluster with the below cli command, with the settings/inputs given.

: The below terminal copy/paste contains the output of my scripts having all sourced the above contents as the first thing they do.

For the command not to fail, so that the k8s cluster could be created and the script(s) could continue.

The reason behind subnet role assignment is because of the maxpods. Try giving a value of maxPods greater than 30 and the CIDR value to /21 or so.

navba-MSFT commented Feb 2, 2022

Apologies for the late reply. This github issue has been open for quite sometime. Could you please let us know if you still need assistance on this? Awaiting your reply.

x10an14 commented Feb 2, 2022

I am no longer working on that project/issue, so I don't need support anymore now no.

Closing issue, others can reference this if they feel they've ended up in the same situation.

No branches or pull requests

Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers
Advertising & Talent Reach devs & technologists worldwide about your product, service or employer brand
OverflowAI GenAI features for Teams
OverflowAPI Train & fine-tune LLMs
Labs The future of collective knowledge sharing
About the company Visit the blog

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Get early access and see previews of new features.

unable to attach acr with aks

I have use following command to attach AKS with ACR

But the error still persist while fetching image. Then I did a little more investigation to find what all service principal ids are attached using following

and I get [] empty array list. Any clue what I might be missing?

az role assignment list --assignee --scope

azure-devops

What is the value of $MYACR ? If the registry is not in the same subscription as the cluster you will have to pass the full resource id (not just the name) – danielorn Commented Feb 12, 2021 at 11:51
name of container registry. aks and acr both are in same subscription – user269867 Commented Feb 16, 2021 at 17:11
What's about the issue now? Do you solve it? – Charles Xu Commented Feb 17, 2021 at 8:29

2 Answers 2

You can first set the subscription using the following command and then can try with further ones to map ACR to AKS

there are two ways to get this sorted

Map the ACR to AKS

Cannot find user or service principal in graph database for "xxx",If the assignee is an appId, make sure the corresponding service principal is created with 'az ad sp create --id xxx'. – user269867 Commented Feb 16, 2021 at 15:58

First of all check if your cluster is already attached to ACR.

If already attached you will get a message like "Your cluster can now pull images from ACR". If you get an error code like 403 you can attach ACR to existing cluster with this command

You can also attach ACR to AKS from different subscription.

Your Answer

Reminder: Answers generated by artificial intelligence tools are not allowed on Stack Overflow. Learn more

Sign up or log in

Post as a guest.

Required, but never shown

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy .

Not the answer you're looking for? Browse other questions tagged kubernetes azure-devops azure-aks or ask your own question .

The Overflow Blog
The hidden cost of speed
The creator of Jenkins discusses CI/CD and balancing business with open source
Featured on Meta
Announcing a change to the data-dump process
Bringing clarity to status tag usage on meta sites
What does a new user need in a homepage experience on Stack Overflow?
Feedback requested: How do you use tag hover descriptions for curating and do...
Staging Ground Reviewer Motivation

Hot Network Questions

Does the average income in the US drop by $9,500 if you exclude the ten richest Americans?
How to change time on ubuntu 22.04?
Transform a list of rules into a list of function definitions
I'm rewriting a 2019 oneshot and am up to 37,000 words already. Should I make them chapters or keep it as a long oneshot?
What does an isolated dash mean in figured bass?
DateTime.ParseExact returns today if date string and format are set to "General"
A checkered cross - Colombian Sudoku
"It never works" vs "It better work"
Solve cannot find solutions if integer parameters are assumed
A seven letter *
Why does this theta function value yield such a good Riemann sum approximation?
Inductive and projective limit of circles
Is a stable quantifier-free language really possible?
Are all citizens of Saudi Arabia "considered Muslims by the state"?
Can I Use A Server In International Waters To Provide Illegal Content Without Getting Arrested?
Is "in spirit and truth" a hendiadys describing the worshipper who is born again?
Issue with the roots of the Equation of Time
I'm a little embarrassed by the research of one of my recommenders
What does "Two rolls" quote really mean?
Are others allowed to use my copyrighted figures in theses, without asking?
Matrix Multiplication & Addition
Escape from the magic prison
How rich is the richest person in a society satisfying the Pareto principle?
Stained passport am I screwed

IMAGES

Could not create a role assignment for ACR. Are you an Owner on this
Cannot create the role assignment for ACR · Issue #318
Could not create a role assignment for ACR. Are you an Owner on this
How-to setup Continuous Deployment using ACR Tasks with Windows
How-to setup Continuous Deployment using ACR Tasks with Windows
Could not create a role assignment for monitoring addon. · Issue #1249

VIDEO

Goodbye to Restlessness and Fall Asleep Instantly with Heavy Rain & Powerful Thunder at a Farm
Let's Play Starfield Episode 42
Pokemon Battle Pedia: Bea Vs Gary
Secrets of Mechanical Contracting from a Pro
William and the BAFTAs: A Misstep Among the Stars #shorts 🌟💬
22 Indirect Role Assignment

COMMENTS

Could not create a role assignment for ACR. Are you an Owner on this
This issue can be resolved by following either of the two methods. Method-1. Provide Permissions on Azure AD graph API to Service Principal so that it can read information about other objects from the directory and perform Role Assignment. Method-2. Provide Directory Readers role to Service Principal.
Could not create a role assignment for ACR. Are you an Owner on this
Got the message "Could not create a role assignment for ACR. Are you an Owner on this subscription?" when trying to attach the ACR to AKS. Managed to resolve it by giving the Service Principal the following API Permission: "Application.ReadWrite.OwnedBy". The nasty thing here is, is that the Azure CLI doesn't use the new Microsoft Graph API but ...
Cannot attach Azure container registries to Azure kubernetes service
Could not create a role assignment for ACR. Are you an Owner on this subscription? Here are the steps I followed: Login: az login; To move servicePrincipal.json: az ad sp create-for-rbac --skip-assignment; To get the service principal object id: az ad sp list --display-name acrrgdevfahim and got the id from result:
Registry roles and permissions
Assign roles. See Steps to add a role assignment for high-level steps to add a role assignment to an existing user, group, service principal, or managed identity. You can use the Azure portal, Azure CLI, Azure PowerShell, or other Azure tools. When creating a service principal, you also configure its access and permissions to Azure resources such as a container registry.
[BUG] Could not create a role assignment for ACR. Are you an ...
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
ACR not attaching to AKS despite being the owner of ...
Also tried granting the AKS managed identity, Owner permission to both the subscriptions (to see whether that works), that didn't help as well. However, I can see ACR Pull access is assigned to the managed identity of AKS. Your CLI is up-to-date. Linking a similar issue #1517: (Could not create a role assignment for ACR.
How to allow AKS Cluster access to Azure Container Registry: issues
Waiting for AAD role to propagate[##### ] 90.0000%Could not create a role assignment for ACR. Are you an Owner on this subscription? I made sure to not only enable "Admin User" for my ACR, but also assigned a contributor role. I double checked all of the resources I have created, and all have the Owner role enabled; again, all within my ...
Could not create a role assignment for ACR within the subscription
The text was updated successfully, but these errors were encountered: triage-new-issues bot added the triage label Jul 7, 2020. ghost added needs-triage This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved.
Authenticate with managed identity
First use the az acr show command to get the resource ID of the registry: resourceID=$(az acr show --resource-group myResourceGroup --name myContainerRegistry --query id --output tsv) Use the az role assignment create command to assign the AcrPull role to the identity. This role provides pull permissions to the registry. To provide both pull ...
az aks create|update --attach-acr makes undue graph api calls that
Results in Could not create a role assignment for ACR.Are you an Owner on this subscription?.Though this is documented as an expected behavior of ACR, it is not. It is a bug of Azure CLI that makes a useless call to Graph API to get an information it already has.
Attach ACR Error Are you an owner on this subscription
Solution #1 -. The quick solution is to change the service principal to Owner permission on the subscription and give legacy graph permission. This is one way of doing it. but certainly not a recommended way. Custom role to give minimum permission to achieve the same. without owner permission on the subscription.
Managed identity in ACR task
For a complete list of registry roles, see Azure Container Registry roles and permissions. Assign the managed identity a role to read secrets in an Azure key vault. Use the Azure CLI or other Azure tools to manage role-based access to resources. For example, run the az role assignment create command to assign the identity a role to the resource.
Unable to attach ACR to AKS
It is showing could not create a role assignment for ACR. Are you an owner on the subscription. I have created registry and aks cluster with same kodekloud subscri… Hi Team, I cannot able to add the ACR to AKS cluster. It is showing could not create a role assignment for ACR. Are you an owner on the subscription.
Cannot integrat ACR with AKS with --attach-acr #1555
az role assignment create --assignee --scope --role acrpull where the scope is the resourceid of the ACR, and the assignee is the clientid of the kubelet's user assigned managed identity that was automatically created (referred to by in the portal by name of "-agentpool" .
Create an Azure Container Registry
Create an ACR. Note. The ACR name must be globally unique and consist of only lowercase alphanumeric characters, between 5 and 50 characters long. This can be checked using: az acr check-name --name <ACR-NAME>. az acr create --name hub23registry --resource-group hub23 --sku Standard.
Possible to do Azure RBAC Role Assignment for ACR Repository?
0. No, as of November 2021 it is still impossible but there is a task for this on Azure ACR team backlog. Please find the task link. So only way that you can follow right now is repository scoped permissions with token. answered Oct 31, 2021 at 18:05. Kağan Mersin.
Fail to pull images from Azure Container Registry to Azure Kubernetes
The integration between AKS and Container Registry creates an AcrPull role assignment at container registry level for the AKS cluster's kubelet identity. Make sure that the role assignment is created. To check whether the AcrPull role assignment is created, use one of the following methods: Run the following command:
`az aks create` gives `Could not create a role assignment for subnet
Waiting for AAD role to propagate[##### ] 90.0000%Could not create a role assignment for ACR. Are you an Owner on this subscription? I am Service Administrator for this development subcription so this should be working.
kubernetes
Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers; Advertising & Talent Reach devs & technologists worldwide about your product, service or employer brand; OverflowAI GenAI features for Teams; OverflowAPI Train & fine-tune LLMs; Labs The future of collective knowledge sharing; About the company Visit the blog