Consider External Secret Storage

Manual Steps

Determine how the cluster was created and whether Vault-based secret encryption was enabled
- On any machine with access to your IaC or console configuration, review the cluster definition:
  - If using OCI CLI/SDK/Terraform, inspect the cluster resource for fields related to Vault / KMS key / secret encryption for etcd.
  - In the OCI Console: go to Developer Services → Kubernetes Clusters (OKE) → [your cluster] → Details and look for any section indicating Secrets encryption / OCI Vault integration at cluster creation.
- If you find a setting that explicitly enables “Kubernetes secret encryption with OCI Vault” (or similar wording), document the Vault OCID/key used and proceed to step 6 (no further remediation needed, only periodic review).
Inventory Kubernetes Secrets currently in use
- On any machine with kubectl access:
  kubectl get secrets --all-namespaces -o wide
- Identify namespaces containing highly sensitive material (credentials, tokens, keys) and note which applications depend on them. This inventory is needed if you plan to migrate to external secret storage or recreate the cluster with encryption enabled.
Decide on your target posture: external secrets vs. encrypted secrets at rest in etcd
- Based on your organization’s security requirements and current cluster lifecycle practices, choose one (or both):
  - External secrets manager pattern: Store secrets in OCI Vault or another enterprise secrets manager; expose them to workloads using a secrets operator or sidecar (e.g., external-secrets controller, Vault agent, custom controller).
  - Encrypted secrets at rest in etcd: Plan to recreate the OKE cluster with the “secrets at rest encrypted by OCI Vault” option enabled at creation time, then migrate workloads and secrets.
- Involve application owners, security, and platform teams since this affects deployment pipelines and runtime configuration.
Gather evidence and plan for external secrets integration (if chosen)
- On any machine with kubectl access, check if a secrets operator is already present:
  kubectl get pods -A | grep -i secret
- If you do not see a known operator (e.g., external-secrets, vault-agent-injector), assume you do not have external secrets in place.
- Review how workloads currently consume secrets:
  kubectl get pods -A -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\t"}{.spec.containers[*].envFrom[*].secretRef.name}{"\t"}{.spec.volumes[*].secret.secretName}{"\n"}{end}' | grep -v '^\s*$'
- Use this to estimate effort: which deployments will need to change to consume secrets from OCI Vault (or another manager) via an operator or sidecar.
Gather evidence and plan for cluster recreation with secrets encryption (if chosen)
- In the OCI Console or your IaC, confirm whether your current cluster allows toggling secrets encryption (in most OKE flows this is a cluster-creation-time choice and cannot be changed in place).
- If it cannot be changed in-place, document a migration plan:
  - Create a new OKE cluster with “Encrypt Kubernetes secrets at rest in etcd using OCI Vault” enabled, referencing an existing or new Vault key.
  - Export and review manifests from the old cluster:
    kubectl get all,configmap,secret,ingress -A -o yaml > current-cluster-export.yaml
  - Exclude or sanitize secrets from this export if your process requires that they be re-injected from OCI Vault or a separate secret pipeline rather than copied as plain Kubernetes Secrets.
Implement and validate the chosen approach
- After deploying an external-secrets solution or recreating the cluster with Vault-backed secret encryption:
  - On any machine with kubectl access, verify that workloads are running and retrieving their secrets successfully:
    kubectl get pods -A kubectl describe pod <pod-name> -n <namespace>
  - For external secrets, confirm that the controller is reconciling without errors:
    kubectl get pods -n <external-secrets-namespace> kubectl logs -n <external-secrets-namespace> <external-secrets-pod-name>
  - For a newly created, Vault-encrypted cluster, capture evidence from the OCI Console / IaC definition showing that Kubernetes secrets at rest in etcd are encrypted with OCI Vault, and retain it with your security documentation.

Using kubectl

# 1) List all Secrets in all namespaces (surface how many native K8s secrets you rely on)
# Run on: any machine with kubectl access
kubectl get secrets --all-namespaces

# 2) Show types of secrets in use (helps identify workload-managed vs manually created)
kubectl get secrets --all-namespaces -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,TYPE:.type' | sort

# 3) Inspect a sample of non-service-account secrets to understand sensitivity
# Replace <namespace> and <secret-name> with values from the previous command
kubectl get secret <secret-name> -n <namespace> -o yaml

# 4) Decode specific data keys inside a secret to understand actual contents
# List keys in a secret:
kubectl get secret <secret-name> -n <namespace> -o jsonpath='{.data}' | jq

# Decode one key (replace <key> with an entry from the previous output):
kubectl get secret <secret-name> -n <namespace> -o jsonpath='{.data.<key>}' | base64 --decode; echo

# 5) Identify Secrets that likely contain credentials (high-value for external storage)
kubectl get secrets --all-namespaces \
  -o jsonpath='{range .items[?(@.type=="Opaque")]}{.metadata.namespace}{"\t"}{.metadata.name}{"\n"}{end}'

# 6) Correlate workloads that mount or reference Secrets
# a) Pods mounting secrets as volumes:
kubectl get pods --all-namespaces -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{range .spec.volumes[?(@.secret)]}{.secret.secretName}{" "}{end}{"\n"}{end}' | sort

# b) Pods using imagePullSecrets:
kubectl get pods --all-namespaces -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{range .spec.imagePullSecrets[*]}{.name}{" "}{end}{"\n"}{end}' | sort

# c) Pods using envFrom / env that reference Secrets:
kubectl get pods --all-namespaces -o jsonpath='
{range .items[*]}
{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}
{range .spec.containers[*].envFrom[?(@.secretRef)]}{.secretRef.name}{" "}{end}
{range .spec.containers[*].env[?(@.valueFrom.secretKeyRef)]}{.valueFrom.secretKeyRef.name}{" "}{end}
{"\n"}
{end}' | sort

What to look for (indicating a potential problem for this control):

Many Opaque or other non-service-account secrets containing credentials, API keys, database passwords, or private keys when decoded in steps 3–4.
High-sensitivity data (e.g., root DB passwords, cloud provider credentials, long-lived tokens, signing keys) revealed in decoded values.
Workloads in steps 6a–6c relying heavily on such secrets directly, with no evidence in your IaC/docs that they are sourced from an external secrets manager (e.g., OCI Vault, external secrets operator).
If any of the above are true and your cluster was created without enabling etcd secret encryption using OCI Vault, that combination indicates you should consider moving to an external secrets solution and (for new clusters) enabling etcd secret encryption at creation time.

Automation

#!/usr/bin/env bash
# Check OKE CIS 4.4.2 – external / at-rest secret encryption usage
# Run on: any machine with kubectl access
# Prereqs: kubectl configured; optional: oci CLI configured for richer context

set -euo pipefail

echo "=== 1) Cluster info ==="
kubectl cluster-info || {
  echo "ERROR: Unable to reach cluster with kubectl" >&2
  exit 1
}

echo
echo "=== 2) Count of Kubernetes Secrets (by namespace and type) ==="
kubectl get secrets --all-namespaces -o json \
  | jq -r '
    .items[]
    | [.metadata.namespace, .type] 
    | @tsv' \
  | sort \
  | awk '{
      key=$1"\t"$2;
      count[key]++
    }
    END {
      printf "NAMESPACE\tTYPE\tCOUNT\n";
      for (k in count) {
        split(k, a, "\t");
        printf "%s\t%s\t%d\n", a[1], a[2], count[k];
      }
    }'

echo
echo "=== 3) Identify secrets likely holding high-value credentials ==="
echo "Heuristics: names containing password|token|key|secret|cert|credential"
kubectl get secrets --all-namespaces -o json \
  | jq -r '
    .items[]
    | select(
        (.metadata.name|test("password|token|key|secret|cert|credential"; "i"))
        or (.type != "Opaque")
      )
    | [ .metadata.namespace
      , .metadata.name
      , .type
      ] | @tsv' \
  | sort \
  | awk 'BEGIN {
           printf "NAMESPACE\tNAME\tTYPE\n"
         } { print }'

echo
echo "=== 4) Sample of Secrets content metadata (sizes only, not values) ==="
echo "This helps estimate how much sensitive data is in native Kubernetes secrets."
echo "NOTE: This does NOT show secret values."
kubectl get secrets --all-namespaces -o json \
  | jq -r '
    .items[]
    | {
        namespace: .metadata.namespace,
        name: .metadata.name,
        type: .type,
        keys: ( .data | keys ),
        size_bytes: ( .data
                      | to_entries
                      | map({k:.key, v: (.value|@base64d|length)})
                    )
      }' \
  | jq -r '
    . as $s
    | $s.size_bytes[]
    | [ $s.namespace
      , $s.name
      , $s.type
      , .k
      , .v
      ] | @tsv' \
  | sort \
  | awk 'BEGIN {
           printf "NAMESPACE\tSECRET\tTYPE\tKEY\tLENGTH_BYTES\n"
         } { print }'

echo
echo "=== 5) Detect common apps that could be migrated to external secrets ==="
echo "These controllers commonly support OCI Vault / external secret managers."
echo "Examples flagged: cert-manager, external-secrets.io, sealed-secrets, vault, external-dns, ingress controllers."
kubectl get pods --all-namespaces -o json \
  | jq -r '
    .items[]
    | {
        ns: .metadata.namespace,
        pod: .metadata.name,
        containers: (.spec.containers[].image)
      }' \
  | jq -r '
    [ .ns, .pod, .containers ] | @tsv' \
  | grep -Ei 'cert-manager|external-secrets|sealed-secrets|vault|external-dns|nginx-ingress|traefik|haproxy' \
  | sort -u \
  | awk 'BEGIN {
           printf "NAMESPACE\tPOD\tIMAGES_MATCHING_KNOWN_CONTROLLERS\n"
         } { print }'

echo
echo "=== 6) OCI / OKE specific hints (MANUAL) ==="
echo "kubectl cannot tell if etcd encryption with OCI Vault is enabled."
echo "Review in OCI Console or IaC:"
echo "- For each OKE cluster, check if 'Encrypt Kubernetes secrets in etcd using OCI Vault' (or equivalent) was enabled at cluster creation."
echo "- If you use Terraform/other IaC, inspect the cluster resource for Vault/secret-encryption settings."
echo
echo "Script complete."

How to interpret the output

Section 2:
- Large numbers of Opaque secrets across many namespaces indicate widespread use of native Kubernetes secrets, which rely on etcd at-rest protection.
- This is not inherently wrong, but all such secrets are in scope for at-rest encryption decisions.
Section 3:
- Lines showing secrets whose names imply passwords, tokens, keys, certificates, or credentials (and non-Opaque types like kubernetes.io/tls, kubernetes.io/dockerconfigjson) highlight high‑value secrets stored in etcd.
- If many critical credentials appear here, but the cluster was not created with OCI Vault-based etcd encryption and you are not using an external secrets manager, this is a concern for CIS 4.4.2.
Section 4:
- Large LENGTH_BYTES values (big keys, blobs, cert bundles) for secrets related to production systems increase the impact if etcd is not encrypted with OCI Vault.
Section 5:
- If you do not see controllers that can integrate with external secret stores (e.g., external-secrets, Vault, sealed-secrets) and you also know the cluster’s etcd is not Vault-encrypted, that suggests you are fully dependent on unencrypted-at-rest Kubernetes secrets in etcd.

Use this report together with a manual review of each OKE cluster’s configuration in the OCI Console or IaC to decide whether you need to:

Recreate the cluster with “Kubernetes secrets in etcd encrypted using OCI Vault”, and/or
Introduce an external secrets manager pattern (e.g., syncing from OCI Vault into Kubernetes).

OCI Misconfigurations

Consider External Secret Storage

More Info:

Risk Level

Address

Compliance Standards

Triage and Remediation

Remediation

​More Info:

​Risk Level

​Address

​Compliance Standards

​Triage and Remediation

​Remediation