SaFi Bank Space : Security in Istio

Each and every microservices have certain security requirements

“Encryption” - When one microservice communicates with another microservice, it’s possible for an attacker to intercept the traffic and modify it before the request reaches the destination. This is known as a “man-in-the-middle-attack”, to prevent this from happening the traffic between services needs to be encrypted.
“Mutual TLS ” - Certain microservices need to implement access control restrictions. For example - only Productpage microservice is allowed to access the Review microservice, not the Details microservice (or) any other microservices in the cluster - for this, we need “access control” - istio allows this using mutual TLS and fine-grained access policies.
“Audit logs” - istio also provides support for audit logs.

Istio Security Architecture https://istio.io/latest/docs/concepts/security/

how istio implements security to achieve the microservices requirements.

So within “istiod” there is a “Certification Authority” that manages “keys” and “certificates” in Istio. This is where the certificates are validated and “certificate signing requests” are approved.
Every time a workload is started, the Envoy proxies request the certificate and key from the istio-agent and that helps secure communication between microservices.
The “Configuration API Server” component distributes all the authentication, authorization, and secure naming policies to the proxies.
Sidecar and ingress/egress proxies work as Policy Enforcement Points. So this means the certificate, keys, authentication, authorization, and secure naming policies are sent to proxies at all times. Every point has security checks(not just the entry point to the network) and its called “Security at depth”

Authentication

Peer Authentication(microservice to microservice)

In a service-oriented architecture, we need to make sure that communication between two services are authenticated - this is done by hardening traffic using verification options like

Mutual TLS
JSON Web Token(JWT) validation

For example - when the “product” service tries to reach the “review” service - the “review” service needs to know that the request is actually coming from the “product” service, not from the external source who pretends to be the fake “product” service.

So the traffic between one service to the other needs to be verified.

with mTLS each microservice gets its own identity which is enforced using “certificate key pairs” - the certificate generation and distribution are all managed automatically by istiod.

Workload-specific peer authentication policy

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "example-peer-policy"
  namespace: "book-info"
spec:
  selector:
    matchLabels:
      app: reviews
  mtls:
    mode: STRICT

Namespace-wide peer authentication policy

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "example-peer-policy"
  namespace: "book-info"
spec:
  mtls:
    mode: STRICT

if we specify the namespace as “istio-system” then it is applicable for entire mesh.

Mesh-wide peer authentication policy

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "example-peer-policy"
  namespace: "istio-system"
spec:
  mtls:
    mode: STRICT

Request Authentication(End User’s access to microservices)

For this istio supports using JSON Web Token(JWT) validation (or) OpenID Connect Providers(ORY Hydra, Keycloak, Firebase, Google)

All the examples used here can be found here: https://github.com/istio/istio/tree/master/samples/bookinfo/platform/kube

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f samples/bookinfo/platform/kube/bookinfo.yaml
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$



*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get all -n default
NAME                                 READY   STATUS    RESTARTS   AGE
pod/details-v1-698b5d8c98-thmpp      2/2     Running   0          17m
pod/productpage-v1-bf4b489d8-4q5bf   2/2     Running   0          17m
pod/ratings-v1-5967f59c58-9frg6      2/2     Running   0          17m
pod/reviews-v1-9c6bb6658-5n6bw       2/2     Running   0          17m
pod/reviews-v2-8454bb78d8-kz7lx      2/2     Running   0          17m
pod/reviews-v3-6dc9897554-vlj5t      2/2     Running   0          17m

NAME                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/details       ClusterIP   10.44.0.228    <none>        9080/TCP   18m
service/kubernetes    ClusterIP   10.44.0.1      <none>        443/TCP    8d
service/productpage   ClusterIP   10.44.3.105    <none>        9080/TCP   17m
service/ratings       ClusterIP   10.44.11.22    <none>        9080/TCP   17m
service/reviews       ClusterIP   10.44.15.238   <none>        9080/TCP   17m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/details-v1       1/1     1            1           17m
deployment.apps/productpage-v1   1/1     1            1           17m
deployment.apps/ratings-v1       1/1     1            1           17m
deployment.apps/reviews-v1       1/1     1            1           17m
deployment.apps/reviews-v2       1/1     1            1           17m
deployment.apps/reviews-v3       1/1     1            1           17m

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/details-v1-698b5d8c98      1         1         1       17m
replicaset.apps/productpage-v1-bf4b489d8   1         1         1       17m
replicaset.apps/ratings-v1-5967f59c58      1         1         1       17m
replicaset.apps/reviews-v1-9c6bb6658       1         1         1       17m
replicaset.apps/reviews-v2-8454bb78d8      1         1         1       17m
replicaset.apps/reviews-v3-6dc9897554      1         1         1       17m
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k create ns bar
namespace/bar created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Let's create a sample application called HTTPbin. HTTPbin helps us to curl the other services. https://github.com/istio/istio/tree/master/samples/httpbin

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f <(istioctl kube-inject -f samples/httpbin/httpbin.yaml) -n bar
serviceaccount/httpbin created
service/httpbin created
deployment.apps/httpbin created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Let’s curl from the HTTPbin to a product page.

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get pod -l app=httpbin -n bar
NAME                      READY   STATUS    RESTARTS   AGE
httpbin-dddb978d5-tc5nn   2/2     Running   0          4m37s
*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get pod -l app=httpbin -n bar -o jsonpath={.items..metadata.name}
httpbin-dddb978d5-tc5nn
*[main][~/git/learn-istio-hardway/istio-1.16.1]$
*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k exec -it httpbin-dddb978d5-tc5nn -c istio-proxy -n bar  -- curl "http://productpage.default:9080" -s -o /dev/null -w "%{http_code}\n"
200
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

So now let’s configure the peer-authentication in default namespace

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -n default -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "default"
spec:
  mtls:
    mode: STRICT
heredoc> EOF
peerauthentication.security.istio.io/default created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Now lets try the curl command to hit the productpage from httpbin. as expected it failed cause we added the “peer authentication policy” - so workloads that do not apply the “mTLS mode strict” cannot reach our application.

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k exec -it "$(k get pod -l app=httpbin -n bar -o jsonpath={.items..metadata.name})" -c istio-proxy -n bar -- curl "http://productpage.default:9080" -s -o /dev/null -w "%{http_code}\n"
000
command terminated with exit code 56
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Authorization

Authorization in istio provides a flexible approach to access control for inbound traffic.

We can control which microservice can reach which microservice which is referred to as east-west traffic using authorization configuration.

With the above authentication policies, we now have the end-user traffic as well as service-to-service traffic secured, what about access controls?

For example:

We would like to only allow the productpage microservice to access the review microservice and not the details microservice or any other services.

Also the productpage microservice is only supposed to run GET calls to the review microservice and not POST or UPDATE calls.

We do not want anyone gaining access to the productpage microservice to make POST calls to the review microservice in an attempt to update or change reviews.

For this purpose, istio provides authorization mechanisms that allow us to define authorization policies to allow or deny requests based on certain criteria.

The microservices don’t require any changes to implement authorization. This is implemented by the envoy proxies authorization engine in run time.

When a request comes through the proxy, the authorization engine evaluates the request context against the current authorization policies and returns the authorization result either allow or deny.

Authorization Policies support ALLOW, DENY and CUSTOM actions.

Authorization policies also be configured to audit the requests.

Sample AuthorizationPolicy - this policy denies all POST requests from the bar namespace to the bookinfo namespace.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: authdenypolicy
  namespace: bookinfo
spec:
  action: DENY
  rules:
  - from:
    - source:
        namespaces: ["bar"]
     to:
     - operation:
          methods: ["POST"]

Test the productpage microservice status

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k run --image=ubuntu test123 -- sleep 100000
pod/test123 created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

*[main][~/git/learn-istio-hardway/istio-1.16.1]$  k exec -it test123 -- curl "http://productpage.default:9080" -s -o /dev/null -w "%{http_code}\n"
200
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Let’s create the AuthorizationPolicy which will work on the whole default namespace. Since this has no selectors, it will affect whole default namespace. And spec section has an empty value so it will not permit any traffic and all requests will be denied.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: allow-nothing
  namespace: default
spec:
  {}

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: allow-nothing
  namespace: default
spec:
  {}
heredoc> EOF
authorizationpolicy.security.istio.io/allow-nothing created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

After applying this AuthorizationPolicy we should get RBAC: Access Denied Error.

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k exec -it test123 -- curl "http://productpage.default:9080"
RBAC: access denied
*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k exec -it test123 -- curl "http://productpage.default:9080"
RBAC: access denied
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Now allow only GET requests to the productpage with the below productpage-viewer authorization policy.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "productpage-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: productpage
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: ["GET"]

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "productpage-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: productpage
  action: ALLOW
  rules:
  - to:
    - operation:
        methods: ["GET"]

heredoc> EOF
authorizationpolicy.security.istio.io/productpage-viewer created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k exec -it test123 -- curl "http://productpage.default:9080" -s -o /dev/null -w "%{http_code}\n"
200
*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k exec -it test123 -- curl "http://productpage.default:9080" -s -o /dev/null -w "%{http_code}\n"
200
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

2. Now create the “details-viewer” authorization policy- this policy will allow traffic coming from service account of bookinfo-productpage

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "details-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: details
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "details-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: details
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]
heredoc> EOF
authorizationpolicy.security.istio.io/details-viewer created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

3. Now create the “reviews-viewer” authorization policy- this policy will allow traffic coming from the service account of bookinfo-productpage

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "reviews-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: reviews
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "reviews-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: reviews
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]
heredoc> EOF
authorizationpolicy.security.istio.io/reviews-viewer created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

4. Now create the “ratings-viewer” authorization policy- this policy will allow traffic coming from the service account of bookinfo-productpage

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "ratings-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: ratings
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: "ratings-viewer"
  namespace: default
spec:
  selector:
    matchLabels:
      app: ratings
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/bookinfo-productpage"]
    to:
    - operation:
        methods: ["GET"]
heredoc> EOF
authorizationpolicy.security.istio.io/ratings-viewer created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get authorizationpolicies -A
NAMESPACE   NAME                 AGE
default     allow-nothing        57m
default     details-viewer       16m
default     productpage-viewer   39m
default     ratings-viewer       5m24s
default     reviews-viewer       11m
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Certificate Managementhttps://istio.io/latest/docs/concepts/security/#pki

Below is the flow of key and certification flow on istio.

So when a microservice is started, it needs to identify itself to the service mesh control plane and retrieve a certificate in order to serve the traffic.

Steps to get the valid cert and private key

istiod has a built-in certificate authority.
The istio-agent creates a private key and a CSR(Certificate signing request) and then sends this CSR with its credentials to Istiod for signing.
The CA in istiod validates the credentials carried out in the CSR. Upon successful validation istiod signs the CSR to generate the certificate.
The istio-agent sends the certificates received from istiod and private key to the Envoy Proxy.
The istio-agent also monitors the expiration of the workload certificate.

The above process repeats periodically for certificates and key rotation.

for production-grade clusters, we should use production-ready CA such as HashiCorp Vault and put your certificates there

Configure `istio` with a root certificate of our own when setting up `istio` cluster

All the tools to generate certificates available here: https://github.com/istio/istio/tree/master/tools/certs

Generate root certificate with below command make -f ../istio-1.16.1/tools/certs/Makefile.selfsigned.mk root-ca

root-ca.conf is the configuration for openSSL to generate root certificate.

root-cert.csr is a generated CSR for the root certificate.

root-cert.pem is a generated root certificate.

root-key.pem is a generated root key.

*[main][~/git/learn-istio-hardway/ca-certs]$ make -f ../istio-1.16.1/tools/certs/Makefile.selfsigned.mk root-ca
generating root-key.pem
Generating RSA private key, 4096 bit long modulus
...........++
.................................................................................................................++
e is 65537 (0x10001)
generating root-cert.csr
generating root-cert.pem
Signature ok
subject=/O=Istio/CN=Root CA
Getting Private key
*[main][~/git/learn-istio-hardway/ca-certs]$ ls -rtlh
total 32
-rw-r--r--  1 bahrathkumaraju  staff   3.2K Feb  7 13:42 root-key.pem
-rw-r--r--  1 bahrathkumaraju  staff   362B Feb  7 13:42 root-ca.conf
-rw-r--r--  1 bahrathkumaraju  staff   1.7K Feb  7 13:42 root-cert.csr
-rw-r--r--  1 bahrathkumaraju  staff   1.8K Feb  7 13:42 root-cert.pem
*[main][~/git/learn-istio-hardway/ca-certs]$

*[main][~/git/learn-istio-hardway/ca-certs]$ cat root-ca.conf
[ req ]
encrypt_key = no
prompt = no
utf8 = yes
default_md = sha256
default_bits = 4096
req_extensions = req_ext
x509_extensions = req_ext
distinguished_name = req_dn
[ req_ext ]
subjectKeyIdentifier = hash
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, nonRepudiation, keyEncipherment, keyCertSign
[ req_dn ]
O = Istio
CN = Root CA
*[main][~/git/learn-istio-hardway/ca-certs]$

2. Now we need to create our intermediate certificates localcluster-cacerts because Root certificates should not be used directly, because it is very dangerous and also not very practical to use them.

*[main][~/git/learn-istio-hardway/ca-certs]$ make -f ../istio-1.16.1/tools/certs/Makefile.selfsigned.mk localcluster-cacerts
generating localcluster/ca-key.pem
Generating RSA private key, 4096 bit long modulus
...................++
.............................................................++
e is 65537 (0x10001)
generating localcluster/cluster-ca.csr
generating localcluster/ca-cert.pem
Signature ok
subject=/O=Istio/CN=Intermediate CA/L=localcluster
Getting CA Private Key
generating localcluster/cert-chain.pem
Intermediate inputs stored in localcluster/
done
rm localcluster/cluster-ca.csr localcluster/intermediate.conf
*[main][~/git/learn-istio-hardway/ca-certs]$ ls -lrth
total 40
-rw-r--r--  1 bahrathkumaraju  staff   3.2K Feb  7 13:42 root-key.pem
-rw-r--r--  1 bahrathkumaraju  staff   362B Feb  7 13:42 root-ca.conf
-rw-r--r--  1 bahrathkumaraju  staff   1.7K Feb  7 13:42 root-cert.csr
-rw-r--r--  1 bahrathkumaraju  staff   1.8K Feb  7 13:42 root-cert.pem
-rw-r--r--  1 bahrathkumaraju  staff    17B Feb  7 13:48 root-cert.srl
drwxr-xr-x  6 bahrathkumaraju  staff   192B Feb  7 13:48 localcluster
*[main][~/git/learn-istio-hardway/ca-certs]$ ls -lrth localcluster
total 32
-rw-r--r--  1 bahrathkumaraju  staff   3.2K Feb  7 13:48 ca-key.pem
-rw-r--r--  1 bahrathkumaraju  staff   1.9K Feb  7 13:48 ca-cert.pem
-rw-r--r--  1 bahrathkumaraju  staff   3.6K Feb  7 13:48 cert-chain.pem
-rw-r--r--  1 bahrathkumaraju  staff   1.8K Feb  7 13:48 root-cert.pem
*[main][~/git/learn-istio-hardway/ca-certs]$

3. Configure istio to use our certificate.

let's create a istio-system namespace again and create secret including all the above created certs.

*[main][~/git/learn-istio-hardway/ca-certs]$ k delete namespace istio-system
namespace "istio-system" deleted
*[main][~/git/learn-istio-hardway/ca-certs]$

*[main][~/git/learn-istio-hardway/ca-certs]$ k create namespace istio-system
namespace/istio-system created
*[main][~/git/learn-istio-hardway/ca-certs]

*[main][~/git/learn-istio-hardway/ca-certs/localcluster]$ k create secret generic cacerts -n istio-system \
>  --from-file=ca-cert.pem \
>  --from-file=ca-key.pem \
>  --from-file=root-cert.pem \
>  --from-file=cert-chain.pem
secret/cacerts created
*[main][~/git/learn-istio-hardway/ca-certs/localcluster]$

4. Now install the istio back again using istioctl utility so that the istio certificate authority will read these certificates and keys from the secret mount files.

*[main][~/git/learn-istio-hardway/ca-certs/localcluster]$ istioctl install --set profile=demo
This will install the Istio 1.16.0 demo profile with ["Istio core" "Istiod" "Ingress gateways" "Egress gateways"] components into the cluster. Proceed? (y/N) y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete                                                                                                                                                                                          Making this installation the default for injection and validation.

Thank you for installing Istio 1.16.  Please take a few minutes to tell us about your install/upgrade experience!  https://forms.gle/99uiMML96AmsXY5d6
*[main][~/git/learn-istio-hardway/ca-certs/localcluster]$




*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get all -n istio-system
NAME                                        READY   STATUS    RESTARTS   AGE
pod/grafana-b854c6c8-vrplb                  1/1     Running   0          7m30s
pod/istio-egressgateway-d84b5f89f-t4jr4     1/1     Running   0          13m
pod/istio-ingressgateway-869ccf7495-hdw87   1/1     Running   0          13m
pod/istiod-689fd979b-brkv5                  1/1     Running   0          13m
pod/jaeger-54b7b57547-cqvgf                 1/1     Running   0          7m30s
pod/kiali-5ff88f8595-xrgwd                  1/1     Running   0          7m29s
pod/prometheus-7b8b9dd44c-nzctg             2/2     Running   0          7m28s

NAME                           TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                                                                      AGE
service/grafana                ClusterIP      10.44.1.39     <none>          3000/TCP                                                                     7m30s
service/istio-egressgateway    ClusterIP      10.44.7.73     <none>          80/TCP,443/TCP                                                               13m
service/istio-ingressgateway   LoadBalancer   10.44.3.147    35.247.162.62   15021:30482/TCP,80:32273/TCP,443:32424/TCP,31400:31630/TCP,15443:30539/TCP   13m
service/istiod                 ClusterIP      10.44.6.183    <none>          15010/TCP,15012/TCP,443/TCP,15014/TCP                                        13m
service/jaeger-collector       ClusterIP      10.44.5.239    <none>          14268/TCP,14250/TCP,9411/TCP                                                 7m30s
service/kiali                  ClusterIP      10.44.12.183   <none>          20001/TCP,9090/TCP                                                           7m29s
service/prometheus             ClusterIP      10.44.0.119    <none>          9090/TCP                                                                     7m29s
service/tracing                ClusterIP      10.44.1.127    <none>          80/TCP,16685/TCP                                                             7m30s
service/zipkin                 ClusterIP      10.44.11.58    <none>          9411/TCP                                                                     7m30s

NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/grafana                1/1     1            1           7m30s
deployment.apps/istio-egressgateway    1/1     1            1           13m
deployment.apps/istio-ingressgateway   1/1     1            1           13m
deployment.apps/istiod                 1/1     1            1           13m
deployment.apps/jaeger                 1/1     1            1           7m30s
deployment.apps/kiali                  1/1     1            1           7m29s
deployment.apps/prometheus             1/1     1            1           7m29s

NAME                                              DESIRED   CURRENT   READY   AGE
replicaset.apps/grafana-b854c6c8                  1         1         1       7m30s
replicaset.apps/istio-egressgateway-d84b5f89f     1         1         1       13m
replicaset.apps/istio-ingressgateway-869ccf7495   1         1         1       13m
replicaset.apps/istiod-689fd979b                  1         1         1       13m
replicaset.apps/jaeger-54b7b57547                 1         1         1       7m30s
replicaset.apps/kiali-5ff88f8595                  1         1         1       7m29s
replicaset.apps/prometheus-7b8b9dd44c             1         1         1       7m29s
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

5. We can also create addons to get our grafana, kiali, prometheus https://github.com/istio/istio/tree/master/samples/addons

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f samples/addons
serviceaccount/grafana created
configmap/grafana created
service/grafana created
deployment.apps/grafana created
configmap/istio-grafana-dashboards created
configmap/istio-services-grafana-dashboards created
deployment.apps/jaeger created
service/tracing created
service/zipkin created
service/jaeger-collector created
serviceaccount/kiali created
configmap/kiali created
clusterrole.rbac.authorization.k8s.io/kiali-viewer unchanged
clusterrole.rbac.authorization.k8s.io/kiali unchanged
clusterrolebinding.rbac.authorization.k8s.io/kiali unchanged
role.rbac.authorization.k8s.io/kiali-controlplane created
rolebinding.rbac.authorization.k8s.io/kiali-controlplane created
service/kiali created
deployment.apps/kiali created
serviceaccount/prometheus created
configmap/prometheus created
clusterrole.rbac.authorization.k8s.io/prometheus unchanged
clusterrolebinding.rbac.authorization.k8s.io/prometheus unchanged
service/prometheus created
deployment.apps/prometheus created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$



*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k rollout status deploy/kiali -n istio-system
deployment "kiali" successfully rolled out
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

6. Now deploy the bookinfo microservice in default namespace as below, also sample gateway and virtualservices rules and destination rules as well. https://github.com/istio/istio/blob/master/samples/bookinfo/platform/kube/bookinfo.yaml

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f samples/bookinfo/platform/kube/bookinfo.yaml
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$


*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f samples/bookinfo/networking/destination-rule-all.yaml
destinationrule.networking.istio.io/productpage created
destinationrule.networking.istio.io/reviews created
destinationrule.networking.istio.io/ratings created
destinationrule.networking.istio.io/details created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$


*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get all -n default
NAME                                 READY   STATUS    RESTARTS   AGE
pod/details-v1-698b5d8c98-mm4bl      2/2     Running   0          6m6s
pod/productpage-v1-bf4b489d8-td7wb   2/2     Running   0          6m5s
pod/ratings-v1-5967f59c58-5h9gc      2/2     Running   0          6m6s
pod/reviews-v1-9c6bb6658-dscrl       2/2     Running   0          6m5s
pod/reviews-v2-8454bb78d8-znv87      2/2     Running   0          6m5s
pod/reviews-v3-6dc9897554-xb2hq      2/2     Running   0          6m5s
pod/test123                          2/2     Running   0          3h18m

NAME                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/details       ClusterIP   10.44.14.222   <none>        9080/TCP   6m6s
service/kubernetes    ClusterIP   10.44.0.1      <none>        443/TCP    8d
service/productpage   ClusterIP   10.44.4.70     <none>        9080/TCP   6m5s
service/ratings       ClusterIP   10.44.15.60    <none>        9080/TCP   6m6s
service/reviews       ClusterIP   10.44.12.217   <none>        9080/TCP   6m6s

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/details-v1       1/1     1            1           6m6s
deployment.apps/productpage-v1   1/1     1            1           6m5s
deployment.apps/ratings-v1       1/1     1            1           6m6s
deployment.apps/reviews-v1       1/1     1            1           6m5s
deployment.apps/reviews-v2       1/1     1            1           6m5s
deployment.apps/reviews-v3       1/1     1            1           6m5s

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/details-v1-698b5d8c98      1         1         1       6m6s
replicaset.apps/productpage-v1-bf4b489d8   1         1         1       6m5s
replicaset.apps/ratings-v1-5967f59c58      1         1         1       6m6s
replicaset.apps/reviews-v1-9c6bb6658       1         1         1       6m5s
replicaset.apps/reviews-v2-8454bb78d8      1         1         1       6m5s
replicaset.apps/reviews-v3-6dc9897554      1         1         1       6m5s
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Now can list the all istio crds like virtualservices, gateways and destinationrules

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get virtualservices
NAME       GATEWAYS               HOSTS   AGE
bookinfo   ["bookinfo-gateway"]   ["*"]   7m44s
*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get gateways
NAME               AGE
bookinfo-gateway   7m51s
*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k get destinationrules
NAME          HOST          AGE
details       details       99s
productpage   productpage   99s
ratings       ratings       99s
reviews       reviews       99s
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

Now the application workloads graph is viewable in kiali dashboard like below.

7. Now let’s deploy a policy( PeerAuthentication) for our workloads to only accept the mutual TLS traffic in the default namespace

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "default"
spec:
  mtls:
    mode: STRICT

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ k apply -f - <<EOF
heredoc> apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: "default"
spec:
  mtls:
    mode: STRICT
heredoc> EOF
peerauthentication.security.istio.io/default created
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

8. Now test the certificates which our workloads are using while connecting to productpage

*[main][~/git/learn-istio-hardway/istio-1.16.1]$ kubectl get pod -l app=details -o jsonpath={.items..metadata.name}
details-v1-698b5d8c98-mm4bl
*[main][~/git/learn-istio-hardway/istio-1.16.1]



*[main][~/git/learn-istio-hardway/istio-1.16.1]$ kubectl exec details-v1-698b5d8c98-mm4bl -c istio-proxy -- openssl s_client -showcerts -connect productpage:9080
Can't use SSL_get_servername
depth=2 O = Istio, CN = Root CA
verify error:num=19:self-signed certificate in certificate chain
verify return:1
depth=2 O = Istio, CN = Root CA
verify return:1
depth=1 O = Istio, CN = Intermediate CA, L = localcluster
verify return:1
depth=0
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:
   i:O = Istio, CN = Intermediate CA, L = localcluster
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Feb  7 06:16:19 2023 GMT; NotAfter: Feb  8 06:18:19 2023 GMT
-----BEGIN CERTIFICATE-----
MIIEdjCCAl6gAwIBAgIQAWufOkU2XK7gRmhpAJA9yzANBgkqhkiG9w0BAQsFADBB
MQ4wDAYDVQQKDAVJc3RpbzEYMBYGA1UEAwwPSW50ZXJtZWRpYXRlIENBMRUwEwYD
VQQHDAxsb2NhbGNsdXN0ZXIwHhcNMjMwMjA3MDYxNjE5WhcNMjMwMjA4MDYxODE5
WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtnyK2MoFTUlvfIEp
DMvND0dSAHdnb3zFFQbeqO+Uq0dPEGd1cn2y38IuoXpGDuZZ0/oLnzbqzLzf/yqi
wG9FoL1jXvcbsDkbdrwLg8Y8hEZss6YQTAnElYscPjT4ZEvxzLghnu8XMoWp9Yt/
0RmPwEAaNn0Fxnhe7R2182pLJWNYGL6mfjsTAss7pnusAwvX97MTRoETLu3buaTD
n++5t7zTDm7481bppEzGphOiSHyEw1ofocBAAOnoKIRNY5pVbEqUerLyj5Hro6JK
SlrocCwrDNg8JVf0oJJheInkmsXZu2AR06OaHYDJMOC2mHBTVhcFPwnw+XboVm8N
+Aq3+wIDAQABo4GqMIGnMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEF
BQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAWgBTLflx38aDp
7g+Zh50xVA7JLrEfAjBHBgNVHREBAf8EPTA7hjlzcGlmZmU6Ly9jbHVzdGVyLmxv
Y2FsL25zL2RlZmF1bHQvc2EvYm9va2luZm8tcHJvZHVjdHBhZ2UwDQYJKoZIhvcN
AQELBQADggIBADRtBSlxCV67GCtB6ubmjGLYi2z2oLJc7ieJUueS3bg5obxzUUlB
RtHrJSrpQWOzLCZF+Kwnv9J2+hNIsuJ0MOLqrN02ZAjYMyZjyfhWqfFXemI3/w/E
qgF7O+tKDsdgRIMyZu4efhDuKkxDuZu4oqb992hVkwayFg7LnK/KxcNpXdsdL/VE
NowzII06zTXmuRgu6lfLlCZWA6O+uusp8YODKRuCMXHFLqodptqP9M4hfGDE7QRQ
FNSKiuSsTK1PvexyfxLX10E7ICkor4Ly/Fk2TjoARKRb6VbBIA2rSGnmimknn8mi
ZmhO6B/Nf1dKd4Jy2M1ioqSslmzlkTL1osCIhGcglkoT1wM7ytso21DEIE5v6x5S
+5UMJyJZDgvaniriQfxZZIx1YJCURlnZ8r0xKGguGtAAtDaKN5s9jD5LT2uyMLjl
jIspfRiarMLVJnWz2QK1y/kcDQ3JZP5387vc8QlP+xDub0n0lCsXeM/aXMXp2HIK
TptOF5agbqLsBb4B43WRoPs6fdOs89dLKqKtdrBE51Aqz9MN06QsMUc+DVz4clyw
QTFpMh4MoKjwaT21akTiymyWnsX/sBHLbVFDsCOiSBEUyIjY9TehTY3XnzGMBn+1
3nOFuBe/1Uv+Wi0pHMJFT3qbnYuMtI5AKxyXzGBzqUbMfAZRn4KVvzwv
-----END CERTIFICATE-----
 1 s:O = Istio, CN = Intermediate CA, L = localcluster
   i:O = Istio, CN = Root CA
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Feb  7 05:48:54 2023 GMT; NotAfter: Feb  6 05:48:54 2025 GMT
-----BEGIN CERTIFICATE-----
MIIFTzCCAzegAwIBAgIJAMZlrpaPBj6LMA0GCSqGSIb3DQEBCwUAMCIxDjAMBgNV
BAoMBUlzdGlvMRAwDgYDVQQDDAdSb290IENBMB4XDTIzMDIwNzA1NDg1NFoXDTI1
MDIwNjA1NDg1NFowQTEOMAwGA1UECgwFSXN0aW8xGDAWBgNVBAMMD0ludGVybWVk
aWF0ZSBDQTEVMBMGA1UEBwwMbG9jYWxjbHVzdGVyMIICIjANBgkqhkiG9w0BAQEF
AAOCAg8AMIICCgKCAgEA2VsQ6oUxf9HNTGTAP/wHHn3zUStnlVcYABPoSBYQYh/y
BOfAOc5Tck3TiHxuMWpsnOLRHUMQdH1ntR7w56W1ABVME42YjAVWKZ28npvc2xmB
vGqQmhqIlDEkAAv/RQIcghYsproTCVO8U0vgvg+NEczC91Nd637li9OCFb7da2nt
2uNL2sqJDw5V4WnQQTSEMznLGVDkewG2yCjCsWeJWuoKjZ1yURBvQMNXbZDCQLfW
WNoz1uuZmf5vNt2ovWv2p1crEBZ1dUmWT9vdtzkTZ9dvvJ+s1nWwV6K1LSIDcr+J
G4ZY1ZH8HXT69FOzRmRDZI1lJp5mf6pyXY8RKJPdpVA/tmMojMCe9NlhdSqYCaBp
DEzhZYusmiA21jOQHofhYx9y8F22ca+xrBHGhAyvY+E8ZE0TLzJmRD0r7k/9TpNd
xpwPEVhw86qo/BFKuzDQvyD3SgPMYCGbuxGKkumgbub4qml1CSlqWuvFtZ6bRJbe
+xn+Elk40rw2+reZOX+clsuKBiQpb46e/RKiTIIRF+hryOfzfCR+WZRspJCNyFiq
+imV3qRjr/96aFNakfni7pUWP6BCWPML58QkEG+sIys1IwCZ608XrL73JnTTF+Cw
cZr25NxmYKzIzFL6Zyk50EnN7MAilzrRcgLr+nzsDQBnOgieANZAadnO4ZibadsC
AwEAAaNpMGcwHQYDVR0OBBYEFMt+XHfxoOnuD5mHnTFUDskusR8CMBIGA1UdEwEB
/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgLkMCIGA1UdEQQbMBmCF2lzdGlvZC5p
c3Rpby1zeXN0ZW0uc3ZjMA0GCSqGSIb3DQEBCwUAA4ICAQBUVdo3OATfZlOWggXB
cbl99jgvAMkj9e1c4jmqHeaWNcBDcN7h0pOsGbf+Hvb46FZZtjfdULjn554P5zw4
1B+C7dpmcKoF6qFS7gfKJ/T8TaWUnM8n8KDQ27N0GW47j0PsKNmHq6S2ryYSmxnW
PD5XRNQDNmEU+ugmJvQfjnBMKFoQ2XceFKd/6YDkeNM164I4615mBTwm130jVRmV
eFkPvYKRR7TnRWq1OuJMD0AqYvpkyV0Yl6eLNlLX6ImGt2krl6Wqylhx8eJCaprw
VxeqEFjwK84ByZtSmy6XZElHMQLegfzEzPxjE05BQvEYSB03NTDc2odS0mhhOqrS
8c8joudqc1RPrTaRdZRzKMa/eLjBPC6abNbhoQqAW0mKpqjZT25OgCn1Mn4RnDWq
lAaneE/YFtqXyibX24HSCjENIRtL0UYPg2dCUYRxmrhJk6tlW+xj9eB6C2/D6XK5
Lv6WoOXixfGVm4IGrS/Wu4wdd0inqY1P6DW58MUtOVJlJFQ0Rxr4t29ZElE6OwX3
IyhACjQtl2zAzKaCQzw1hVCi8hXLtphRoUKp1L+WcYfOlG0DgIuRTMZTA8ffU19D
DBFjXocCWpOM6matbHQ9mtwpUgLMmcVU/E0OUBMdwUjEZb5Gk1mSXunXI7Gv3fjf
poLlyVuoaDgvs6CPLp5nPXnENg==
-----END CERTIFICATE-----
 2 s:O = Istio, CN = Root CA
   i:O = Istio, CN = Root CA
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: F4037B81C847F0000:error:0A00045C:SSL routines:ssl3_read_bytes:tlsv13 alert certificate required:../ssl/record/rec_layer_s3.c:1584:SSL alert number 116
eb  7 05:42:36 2023 GMT; NotAfter: Feb  4 05:42:36 2033 GMT
-----BEGIN CERTIFICATE-----
MIIFCTCCAvGgAwIBAgIJANIQcDsWNt7yMA0GCSqGSIb3DQEBCwUAMCIxDjAMBgNV
BAoMBUlzdGlvMRAwDgYDVQQDDAdSb290IENBMB4XDTIzMDIwNzA1NDIzNloXDTMz
MDIwNDA1NDIzNlowIjEOMAwGA1UECgwFSXN0aW8xEDAOBgNVBAMMB1Jvb3QgQ0Ew
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCxGKVATsPv0tsSHyysz3Qq
cChbfX1dsCgSIQa5C+lVnil30SVeXd7NQDIa73ugw5qSyzhD3WADh9EDeDE/ZqMQ
NphEd3RrQA/xNlDOamtCg9+iKqbZm8EP295CO9i+n88QlLHhrIosyiLoAOVxMGbn
ZoIlnKYiUBu/R49ACj4VJke5G8tiJHiCCqOEjMVzDGWxM1cgTZKlc6oEwn8UtvIU
GEb7NtO9XtzwxCvYfRsOaPHecexwlglq2LqXuQvKSsrgw3r5aZZkuYr3vkgqHJgm
txSllLNmonZG11iH2BFHCSHXwWb1fi0KfbVwamKdVFdqsNCauQtmUpVPkegAdVoF
6dOabLKMdXFiSmhuFv8dc4eicAmESy6ziDnHpPpBfk3R6TXeWW1DttLHCxmYzG2I
yhuIeAIlW2Kbh7cMqkFZZADbhfeuDbe8D2M4W/Wq281EaveTVEfgHF4C1Ny+RaFG
s9SL559QC5N1WH19r1M3m1VtW02lOB8qNRlwAvTthbB0s9h39A2VDeqOPxkgq6Ww
EwyXf0Zt/VnssZ4E4f5I1DbSGzmmuuUdYWr1fnHlVP45R6FCR/dAmv3PEaQvV4Oo
x8rY2XOLlfxtXWjyJ5+V5E7JQKI7PgaykD1cr4wyuuIqlKD5Vs5IfJM31Z46hAj8
wO13EKlqQjtUJLGClaVzXwIDAQABo0IwQDAdBgNVHQ4EFgQUq49L+umsYCXtcFt1
cpmpfDgslacwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAuQwDQYJKoZI
hvcNAQELBQADggIBAA4ySCi9nTR2p1Gb872a9G2XfWGQslmDyjDCEqDk+kWYoOhS
44rnB1j5RpoC+RsDV6spZ2bY8AnUy/9iwAAZZFRCXPwB3ktcDwvhEDUK1aD4ZxXB
J8txwUnB38z5Ux+w4t8TK5fiOBztQy1skfLpJJOIcO0jjt88/bU2EKfimU45yPo2
OMWisF/xI0qRY1+V6IXvalMj6lxzbhV3D1YfQ0ikOLSUQW2i+jOqO0z/NlyUlEj6
GWf6Fl7QOpjUV+mfMtDSVTi65W9FeZwt4UKaf9K6VPfMn1U+7KLa91HYlnn1tN2s
H/xc7Dgcc2c7EsxslsykgNjIYvmGCjcBg/k5waiAVqBYmRkple/8smVClymTbrGt
WAfg2kbEBZPmbOfev3RMHrAhDUphrnBn/Hd1B2bMN/q+O009wUGW2hinwASqOQT1
cQ9V0ovP6gmBWHYVhmgB7TFQMuU0JlAb20qww0l8C0NBhFn53RUcUa5Xc6ZFDbyF
g7LlhazU9x4BqkE6lSZVacsmgh/1iK+URGqEvMAzEnVkEcQ131AaSBwo/DeTeB6D
mvL62vG+rShp49CceOKLudPo3XzoBf8NLbkCvnZ/nugKEXzo0i/GYZZFmzLpvEb+
U/r/Eb+QcIXM0xw3e4604ot/7Xetax9QTTWopVW8pioGoolkkD7CogqK4FhC
-----END CERTIFICATE-----
 3 s:O = Istio, CN = Root CA
   i:O = Istio, CN = Root CA
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Feb  7 05:42:36 2023 GMT; NotAfter: Feb  4 05:42:36 2033 GMT
-----BEGIN CERTIFICATE-----
MIIFCTCCAvGgAwIBAgIJANIQcDsWNt7yMA0GCSqGSIb3DQEBCwUAMCIxDjAMBgNV
BAoMBUlzdGlvMRAwDgYDVQQDDAdSb290IENBMB4XDTIzMDIwNzA1NDIzNloXDTMz
MDIwNDA1NDIzNlowIjEOMAwGA1UECgwFSXN0aW8xEDAOBgNVBAMMB1Jvb3QgQ0Ew
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCxGKVATsPv0tsSHyysz3Qq
cChbfX1dsCgSIQa5C+lVnil30SVeXd7NQDIa73ugw5qSyzhD3WADh9EDeDE/ZqMQ
NphEd3RrQA/xNlDOamtCg9+iKqbZm8EP295CO9i+n88QlLHhrIosyiLoAOVxMGbn
ZoIlnKYiUBu/R49ACj4VJke5G8tiJHiCCqOEjMVzDGWxM1cgTZKlc6oEwn8UtvIU
GEb7NtO9XtzwxCvYfRsOaPHecexwlglq2LqXuQvKSsrgw3r5aZZkuYr3vkgqHJgm
txSllLNmonZG11iH2BFHCSHXwWb1fi0KfbVwamKdVFdqsNCauQtmUpVPkegAdVoF
6dOabLKMdXFiSmhuFv8dc4eicAmESy6ziDnHpPpBfk3R6TXeWW1DttLHCxmYzG2I
yhuIeAIlW2Kbh7cMqkFZZADbhfeuDbe8D2M4W/Wq281EaveTVEfgHF4C1Ny+RaFG
s9SL559QC5N1WH19r1M3m1VtW02lOB8qNRlwAvTthbB0s9h39A2VDeqOPxkgq6Ww
EwyXf0Zt/VnssZ4E4f5I1DbSGzmmuuUdYWr1fnHlVP45R6FCR/dAmv3PEaQvV4Oo
x8rY2XOLlfxtXWjyJ5+V5E7JQKI7PgaykD1cr4wyuuIqlKD5Vs5IfJM31Z46hAj8
wO13EKlqQjtUJLGClaVzXwIDAQABo0IwQDAdBgNVHQ4EFgQUq49L+umsYCXtcFt1
cpmpfDgslacwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAuQwDQYJKoZI
hvcNAQELBQADggIBAA4ySCi9nTR2p1Gb872a9G2XfWGQslmDyjDCEqDk+kWYoOhS
44rnB1j5RpoC+RsDV6spZ2bY8AnUy/9iwAAZZFRCXPwB3ktcDwvhEDUK1aD4ZxXB
J8txwUnB38z5Ux+w4t8TK5fiOBztQy1skfLpJJOIcO0jjt88/bU2EKfimU45yPo2
OMWisF/xI0qRY1+V6IXvalMj6lxzbhV3D1YfQ0ikOLSUQW2i+jOqO0z/NlyUlEj6
GWf6Fl7QOpjUV+mfMtDSVTi65W9FeZwt4UKaf9K6VPfMn1U+7KLa91HYlnn1tN2s
H/xc7Dgcc2c7EsxslsykgNjIYvmGCjcBg/k5waiAVqBYmRkple/8smVClymTbrGt
WAfg2kbEBZPmbOfev3RMHrAhDUphrnBn/Hd1B2bMN/q+O009wUGW2hinwASqOQT1
cQ9V0ovP6gmBWHYVhmgB7TFQMuU0JlAb20qww0l8C0NBhFn53RUcUa5Xc6ZFDbyF
g7LlhazU9x4BqkE6lSZVacsmgh/1iK+URGqEvMAzEnVkEcQ131AaSBwo/DeTeB6D
mvL62vG+rShp49CceOKLudPo3XzoBf8NLbkCvnZ/nugKEXzo0i/GYZZFmzLpvEb+
U/r/Eb+QcIXM0xw3e4604ot/7Xetax9QTTWopVW8pioGoolkkD7CogqK4FhC
-----END CERTIFICATE-----
---
Server certificate
subject=
issuer=O = Istio, CN = Intermediate CA, L = localcluster
---
Acceptable client certificate CA names
O = Istio, CN = Root CA
Requested Signature Algorithms: ECDSA+SHA256:RSA-PSS+SHA256:RSA+SHA256:ECDSA+SHA384:RSA-PSS+SHA384:RSA+SHA384:RSA-PSS+SHA512:RSA+SHA512:RSA+SHA1
Shared Requested Signature Algorithms: ECDSA+SHA256:RSA-PSS+SHA256:RSA+SHA256:ECDSA+SHA384:RSA-PSS+SHA384:RSA+SHA384:RSA-PSS+SHA512:RSA+SHA512
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 5675 bytes and written 403 bytes
Verification error: self-signed certificate in certificate chain
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 19 (self-signed certificate in certificate chain)
---
command terminated with exit code 1
*[main][~/git/learn-istio-hardway/istio-1.16.1]$

As the CA certificate used in this example is self-signed Verify return code: 19 (self-signed certificate in certificate chain) this certificate chain error returned by the open SSL command is expected.