These logs might be helpful
controlplane $ kubectl describe pod/istio-egressgateway-6f484f9f-kmxbt -n istio-system
Name: istio-egressgateway-6f484f9f-kmxbt
Namespace: istio-system
Priority: 0
Node: node01/172.17.0.39
Start Time: Sat, 08 Jan 2022 04:13:52 +0000
Labels: app=istio-egressgateway
chart=gateways
heritage=Tiller
install.operator.istio.io/owning-resource=unknown
istio=egressgateway
istio.io/rev=default
operator.istio.io/component=EgressGateways
pod-template-hash=6f484f9f
release=istio
service.istio.io/canonical-name=istio-egressgateway
service.istio.io/canonical-revision=latest
sidecar.istio.io/inject=false
Annotations: prometheus.io/path: /stats/prometheus
prometheus.io/port: 15020
prometheus.io/scrape: true
sidecar.istio.io/inject: false
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/istio-egressgateway-6f484f9f
Containers:
istio-proxy:
Container ID:
Image: docker.io/istio/proxyv2:1.10.6
Image ID:
Ports: 8080/TCP, 8443/TCP, 15090/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Args:
proxy
router
--domain
$(POD_NAMESPACE).svc.cluster.local
--proxyLogLevel=warning
--proxyComponentLogLevel=misc:error
--log_output_level=default:info
--serviceCluster
istio-egressgateway
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 100m
memory: 128Mi
Readiness: http-get http://:15021/healthz/ready delay=1s timeout=1s period=2s #success=1 #failure=30
Environment:
JWT_POLICY: third-party-jwt
PILOT_CERT_PROVIDER: istiod
CA_ADDR: istiod.istio-system.svc:15012
NODE_NAME: (v1:spec.nodeName)
POD_NAME: istio-egressgateway-6f484f9f-kmxbt (v1:metadata.name)
POD_NAMESPACE: istio-system (v1:metadata.namespace)
INSTANCE_IP: (v1:status.podIP)
HOST_IP: (v1:status.hostIP)
SERVICE_ACCOUNT: (v1:spec.serviceAccountName)
CANONICAL_SERVICE: (v1:metadata.labels['service.istio.io/canonical-name'])
CANONICAL_REVISION: (v1:metadata.labels['service.istio.io/canonical-revision'])
ISTIO_META_WORKLOAD_NAME: istio-egressgateway
ISTIO_META_OWNER: kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-egressgateway
ISTIO_META_MESH_ID: cluster.local
TRUST_DOMAIN: cluster.local
ISTIO_META_UNPRIVILEGED_POD: true
ISTIO_META_ROUTER_MODE: standard
ISTIO_META_CLUSTER_ID: Kubernetes
Mounts:
/etc/istio/config from config-volume (rw)
/etc/istio/egressgateway-ca-certs from egressgateway-ca-certs (ro)
/etc/istio/egressgateway-certs from egressgateway-certs (ro)
/etc/istio/pod from podinfo (rw)
/etc/istio/proxy from istio-envoy (rw)
/var/lib/istio/data from istio-data (rw)
/var/run/secrets/istio from istiod-ca-cert (rw)
/var/run/secrets/kubernetes.io/serviceaccount from istio-egressgateway-service-account-token-mks47 (ro)
/var/run/secrets/tokens from istio-token (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
istiod-ca-cert:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio-ca-root-cert
Optional: false
podinfo:
Type: DownwardAPI (a volume populated by information about the pod)
Items:
metadata.labels -> labels
metadata.annotations -> annotations
limits.cpu -> cpu-limit
requests.cpu -> cpu-request
istio-envoy:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 43200
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio
Optional: true
egressgateway-certs:
Type: Secret (a volume populated by a Secret)
SecretName: istio-egressgateway-certs
Optional: true
egressgateway-ca-certs:
Type: Secret (a volume populated by a Secret)
SecretName: istio-egressgateway-ca-certs
Optional: true
istio-egressgateway-service-account-token-mks47:
Type: Secret (a volume populated by a Secret)
SecretName: istio-egressgateway-service-account-token-mks47
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m7s default-scheduler Successfully assigned istio-system/istio-egressgateway-6f484f9f-kmxbt to node01
Warning FailedMount 4m4s kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istiod-ca-cert istio-token], unattached volumes=[config-volume istiod-ca-cert podinfo egressgateway-ca-certs istio-token egressgateway-certs istio-egressgateway-service-account-token-mks47 istio-envoy istio-data]: timed out waiting for the condition
Warning FailedMount 117s (x10 over 6m7s) kubelet, node01 MountVolume.SetUp failed for volume "istiod-ca-cert" : configmap "istio-ca-root-cert" not found
Warning FailedMount 117s (x10 over 6m7s) kubelet, node01 MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: the API server does not have TokenRequest endpoints enabled
Warning FailedMount 107s kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istiod-ca-cert istio-token], unattached volumes=[istio-data istiod-ca-cert podinfo egressgateway-ca-certs istio-envoy istio-token egressgateway-certs istio-egressgateway-service-account-token-mks47 config-volume]: timed out waiting for the condition
controlplane $ kubectl describe pod/istio-ingressgateway-67df755d66-7tdkd -n istio-system
Name: istio-ingressgateway-67df755d66-7tdkd
Namespace: istio-system
Priority: 0
Node: node01/172.17.0.39
Start Time: Sat, 08 Jan 2022 04:13:51 +0000
Labels: app=istio-ingressgateway
chart=gateways
heritage=Tiller
install.operator.istio.io/owning-resource=unknown
istio=ingressgateway
istio.io/rev=default
operator.istio.io/component=IngressGateways
pod-template-hash=67df755d66
release=istio
service.istio.io/canonical-name=istio-ingressgateway
service.istio.io/canonical-revision=latest
sidecar.istio.io/inject=false
Annotations: prometheus.io/path: /stats/prometheus
prometheus.io/port: 15020
prometheus.io/scrape: true
sidecar.istio.io/inject: false
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/istio-ingressgateway-67df755d66
Containers:
istio-proxy:
Container ID:
Image: docker.io/istio/proxyv2:1.10.6
Image ID:
Ports: 15021/TCP, 8080/TCP, 8443/TCP, 15090/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Args:
proxy
router
--domain
$(POD_NAMESPACE).svc.cluster.local
--proxyLogLevel=warning
--proxyComponentLogLevel=misc:error
--log_output_level=default:info
--serviceCluster
istio-ingressgateway
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Limits:
cpu: 2
memory: 1Gi
Requests:
cpu: 100m
memory: 128Mi
Readiness: http-get http://:15021/healthz/ready delay=1s timeout=1s period=2s #success=1 #failure=30
Environment:
JWT_POLICY: third-party-jwt
PILOT_CERT_PROVIDER: istiod
CA_ADDR: istiod.istio-system.svc:15012
NODE_NAME: (v1:spec.nodeName)
POD_NAME: istio-ingressgateway-67df755d66-7tdkd (v1:metadata.name)
POD_NAMESPACE: istio-system (v1:metadata.namespace)
INSTANCE_IP: (v1:status.podIP)
HOST_IP: (v1:status.hostIP)
SERVICE_ACCOUNT: (v1:spec.serviceAccountName)
CANONICAL_SERVICE: (v1:metadata.labels['service.istio.io/canonical-name'])
CANONICAL_REVISION: (v1:metadata.labels['service.istio.io/canonical-revision'])
ISTIO_META_WORKLOAD_NAME: istio-ingressgateway
ISTIO_META_OWNER: kubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-ingressgateway
ISTIO_META_MESH_ID: cluster.local
TRUST_DOMAIN: cluster.local
ISTIO_META_UNPRIVILEGED_POD: true
ISTIO_META_ROUTER_MODE: standard
ISTIO_META_CLUSTER_ID: Kubernetes
Mounts:
/etc/istio/config from config-volume (rw)
/etc/istio/ingressgateway-ca-certs from ingressgateway-ca-certs (ro)
/etc/istio/ingressgateway-certs from ingressgateway-certs (ro)
/etc/istio/pod from podinfo (rw)
/etc/istio/proxy from istio-envoy (rw)
/var/lib/istio/data from istio-data (rw)
/var/run/secrets/istio from istiod-ca-cert (rw)
/var/run/secrets/kubernetes.io/serviceaccount from istio-ingressgateway-service-account-token-6422d (ro)
/var/run/secrets/tokens from istio-token (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
istiod-ca-cert:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio-ca-root-cert
Optional: false
podinfo:
Type: DownwardAPI (a volume populated by information about the pod)
Items:
metadata.labels -> labels
metadata.annotations -> annotations
limits.cpu -> cpu-limit
requests.cpu -> cpu-request
istio-envoy:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
istio-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 43200
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: istio
Optional: true
ingressgateway-certs:
Type: Secret (a volume populated by a Secret)
SecretName: istio-ingressgateway-certs
Optional: true
ingressgateway-ca-certs:
Type: Secret (a volume populated by a Secret)
SecretName: istio-ingressgateway-ca-certs
Optional: true
istio-ingressgateway-service-account-token-6422d:
Type: Secret (a volume populated by a Secret)
SecretName: istio-ingressgateway-service-account-token-6422d
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 15m default-scheduler Successfully assigned istio-system/istio-ingressgateway-67df755d66-7tdkd to node01
Warning FailedMount 13m kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istio-token istiod-ca-cert], unattached volumes=[ingressgateway-certs ingressgateway-ca-certs istio-envoy istio-token podinfo istio-ingressgateway-service-account-token-6422d config-volume istiod-ca-cert istio-data]: timed out waiting for the condition
Warning FailedMount 10m kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istiod-ca-cert istio-token], unattached volumes=[istio-ingressgateway-service-account-token-6422d istio-envoy config-volume istiod-ca-cert ingressgateway-certs istio-data istio-token podinfo ingressgateway-ca-certs]: timed out waiting for the condition
Warning FailedMount 8m52s (x11 over 15m) kubelet, node01 MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: the API server does not have TokenRequest endpoints enabled
Warning FailedMount 8m31s kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istiod-ca-cert istio-token], unattached volumes=[istio-envoy istio-data istiod-ca-cert istio-token podinfo ingressgateway-ca-certs istio-ingressgateway-service-account-token-6422d config-volume ingressgateway-certs]: timed out waiting for the condition
Warning FailedMount 4m48s (x13 over 15m) kubelet, node01 MountVolume.SetUp failed for volume "istiod-ca-cert" : configmap "istio-ca-root-cert" not found
controlplane $ kubectl describe pod/istiod-6fbf5d76d6-bmh8b -n istio-system
Name: istiod-6fbf5d76d6-bmh8b
Namespace: istio-system
Priority: 0
Node: node01/172.17.0.39
Start Time: Sat, 08 Jan 2022 04:13:48 +0000
Labels: app=istiod
install.operator.istio.io/owning-resource=unknown
istio=pilot
istio.io/rev=default
operator.istio.io/component=Pilot
pod-template-hash=6fbf5d76d6
sidecar.istio.io/inject=false
Annotations: prometheus.io/port: 15014
prometheus.io/scrape: true
sidecar.istio.io/inject: false
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/istiod-6fbf5d76d6
Containers:
discovery:
Container ID:
Image: docker.io/istio/pilot:1.10.6
Image ID:
Ports: 8080/TCP, 15010/TCP, 15017/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Args:
discovery
--monitoringAddr=:15014
--log_output_level=default:info
--domain
cluster.local
--keepaliveMaxServerConnectionAge
30m
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 500m
memory: 2Gi
Readiness: http-get http://:8080/ready delay=1s timeout=5s period=3s #success=1 #failure=3
Environment:
REVISION: default
JWT_POLICY: third-party-jwt
PILOT_CERT_PROVIDER: istiod
POD_NAME: istiod-6fbf5d76d6-bmh8b (v1:metadata.name)
POD_NAMESPACE: istio-system (v1:metadata.namespace)
SERVICE_ACCOUNT: (v1:spec.serviceAccountName)
KUBECONFIG: /var/run/secrets/remote/config
PILOT_TRACE_SAMPLING: 1
PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_OUTBOUND: true
PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_INBOUND: true
ISTIOD_ADDR: istiod.istio-system.svc:15012
PILOT_ENABLE_ANALYSIS: false
CLUSTER_ID: Kubernetes
Mounts:
/etc/cacerts from cacerts (ro)
/var/run/secrets/istio-dns from local-certs (rw)
/var/run/secrets/kubernetes.io/serviceaccount from istiod-service-account-token-hpx8x (ro)
/var/run/secrets/remote from istio-kubeconfig (ro)
/var/run/secrets/tokens from istio-token (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
local-certs:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium: Memory
SizeLimit: <unset>
istio-token:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 43200
cacerts:
Type: Secret (a volume populated by a Secret)
SecretName: cacerts
Optional: true
istio-kubeconfig:
Type: Secret (a volume populated by a Secret)
SecretName: istio-kubeconfig
Optional: true
istiod-service-account-token-hpx8x:
Type: Secret (a volume populated by a Secret)
SecretName: istiod-service-account-token-hpx8x
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 18m default-scheduler Successfully assigned istio-system/istiod-6fbf5d76d6-bmh8b to node01
Warning FailedMount 16m kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istio-token], unattached volumes=[istio-token local-certs cacerts istio-kubeconfig istiod-service-account-token-hpx8x]: timed out waiting for the condition
Warning FailedMount 11m (x2 over 13m) kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istio-token], unattached volumes=[local-certs cacerts istio-kubeconfig istiod-service-account-token-hpx8x istio-token]: timed out waiting for the condition
Warning FailedMount 9m18s kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istio-token], unattached volumes=[cacerts istio-kubeconfig istiod-service-account-token-hpx8x istio-token local-certs]: timed out waiting for the condition
Warning FailedMount 7m4s kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istio-token], unattached volumes=[istiod-service-account-token-hpx8x istio-token local-certs cacerts istio-kubeconfig]: timed out waiting for the condition
Warning FailedMount 106s (x16 over 18m) kubelet, node01 MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: the API server does not have TokenRequest endpoints enabled
Warning FailedMount 20s (x3 over 4m49s) kubelet, node01 Unable to attach or mount volumes: unmounted volumes=[istio-token], unattached volumes=[istio-kubeconfig istiod-service-account-token-hpx8x istio-token local-certs cacerts]: timed out waiting for the condition