> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Audit Logging Should Be Enabled And Shipped Off-Cluster

### More Info:

Advisory: Kubernetes API audit logging should be enabled and forwarded to an external, tamper-resistant store so control-plane activity is retained independently of the cluster.

### Risk Level

Medium

### Address

Security

### Compliance Standards

* Cloudanix Best Practice

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Manual Steps" defaultOpen="true">
        1. **Identify the OKE cluster and compartment**
           * On any machine with OCI CLI configured:
             ```sh theme={null}
             oci ce cluster list --compartment-id <OCID_OF_COMPARTMENT> --all
             ```
           * Note the `id` of the target cluster (for example, `ocid1.cluster.oc1...`).

        2. **Check whether Kubernetes API audit logging is enabled for the cluster**
           * Using OCI CLI on any machine:
             ```sh theme={null}
             oci ce cluster get --cluster-id <OCID_OF_CLUSTER> --query 'data."options"' --output json
             ```
           * In the JSON output, review any `kubernetesAudit` / `apiServer` / `logging`-related fields.
           * If the console exposes an “Audit logs” or “API server logs” toggle for OKE in the cluster’s details page, verify whether Kubernetes API audit logging is enabled there.

        3. **Verify cluster logs are being shipped to OCI Logging**
           * Using the OCI Console:
             * Go to **Observability & Management → Logging → Log Groups**.
             * In the compartment that contains the cluster, look for an OKE-related log group (for example, named with the cluster or node pool).
             * Open it and verify there is a log for control-plane or audit events (for example, “Kubernetes API Server Logs” or similar).
           * Or via OCI CLI on any machine:
             ```sh theme={null}
             oci logging log-group list --compartment-id <OCID_OF_COMPARTMENT> --all
             oci logging log list --log-group-id <OCID_OF_LOG_GROUP> --all
             ```
           * Confirm that at least one log corresponds to API server / audit events.

        4. **Confirm logs are exported off-cluster to a tamper-resistant destination**
           * In the OCI Console, for each relevant OKE control-plane / audit log:
             * Go to **Logging → Logs → `<the log>` → Actions / Configure Log**.
             * Check for an active **Service Connector** or **Log Rule** exporting to:
               * OCI Logging Analytics, or
               * OCI Object Storage bucket with restricted access, or
               * OCI Logging to external SIEM / third-party sink.
           * With OCI CLI on any machine, list service connectors:
             ```sh theme={null}
             oci sch service-connector list --compartment-id <OCID_OF_COMPARTMENT> --all
             ```
           * Inspect each connector’s `source` to confirm it pulls from the OKE log(s), and `target` to confirm it writes to an external, controlled, and access-logged destination.

        5. **If audit logging or export is missing, configure it in OCI**
           * In the OCI Console, navigate to the OKE cluster’s details page and:
             * Enable Kubernetes API / audit logging if an option exists for the cluster.
           * Then, under **Observability & Management → Logging**:
             * Create or identify a log group collecting the OKE control-plane / audit logs.
           * Under **Service Connectors**:
             * Create a new service connector with:
               * **Source:** the OKE audit/control-plane log(s).
               * **Target:** an OCI Object Storage bucket or Logging Analytics / external sink with write-only from logging services and tightly controlled read access.
           * If managing via IaC (Terraform), ensure corresponding `oci_logging_log`, `oci_sch_service_connector`, and, where supported, OKE cluster options for audit logging are defined and applied.

        6. **Re-verify logging and export after configuration**
           * Generate a few Kubernetes API events (for example, `kubectl get pods -A`) from any machine with `kubectl` access to the cluster.
           * In OCI Logging:
             * Open the OKE control-plane / audit log and confirm new entries appear matching the recent API calls.
           * In the configured external destination (Object Storage bucket, Logging Analytics, or external SIEM):
             * Confirm that newly generated audit log entries are present and that access controls on the destination prevent unauthorised modification or deletion.
      </Accordion>

      <Accordion title="Using kubectl">
        kubectl cannot enable or configure Kubernetes API audit logging for Oracle OKE clusters, because this setting is managed entirely in the Oracle Cloud Infrastructure (OCI) / OKE control-plane configuration and associated logging services. To address this finding, use the OCI Console, CLI, or IaC (Terraform/Resource Manager) as described in the Manual Steps section.
      </Accordion>

      <Accordion title="Automation">
        ```bash theme={null}
        #!/usr/bin/env bash
        #
        # audit-oke-api-logging.sh
        #
        # Purpose:
        #   Help review whether Oracle OKE clusters have API server audit logging
        #   enabled and shipped off-cluster.
        #
        # Scope:
        #   - Runs on: any machine with:
        #       * OCI CLI configured (with permissions to list & inspect OKE clusters,
        #         logging configs, log groups, and service connectors)
        #       * jq
        #
        # Notes:
        #   - This does NOT change any configuration.
        #   - This is a best‑effort helper for a MANUAL control; you still must review
        #     the output and make a judgment.
        #
        # Usage:
        #   export OCI_COMPARTMENT_OCID="ocid1.compartment.oc1..xxxxx"
        #   ./audit-oke-api-logging.sh
        #
        # Optional filters:
        #   export OKE_NAME_FILTER="prod"   # only clusters whose display-name matches this regex
        #
        set -euo pipefail

        if ! command -v oci >/dev/null 2>&1; then
          echo "ERROR: oci CLI not found in PATH" >&2
          exit 1
        fi

        if ! command -v jq >/dev/null 2>&1; then
          echo "ERROR: jq not found in PATH" >&2
          exit 1
        fi

        : "${OCI_COMPARTMENT_OCID:?ERROR: Set OCI_COMPARTMENT_OCID to the OCID of the compartment to scan}"

        OKE_NAME_FILTER="${OKE_NAME_FILTER:-}"

        echo "=== OKE API Audit Logging Review ==="
        echo "Compartment: ${OCI_COMPARTMENT_OCID}"
        [ -n "${OKE_NAME_FILTER}" ] && echo "Name filter (regex): ${OKE_NAME_FILTER}"
        echo

        # Helper: describe verdict text
        explain_verdict() {
          cat <<'EOF'
        Verdict meanings:
          - NO_CONTROL_PLANE_LOGS:
              No Logging configuration of type CONTROL_PLANE or APISERVER found.
              => API audit logging is very likely NOT enabled.
          - CONTROL_PLANE_LOGS_NO_DESTINATION:
              Cluster has control-plane logs configured, but no evidence of
              shipping to an external destination (no related service connectors).
              => Logs may be staying only in OCI Logging; assess if this meets
                 your "external, tamper-resistant" requirement.
          - CONTROL_PLANE_LOGS_WITH_DESTINATION:
              Cluster has control-plane logs AND a service connector appears to
              route them to another sink (e.g., Object Storage, Logging Analytics,
              Stream).
              => Manual review still required to confirm this meets policy (e.g.,
                 immutability, retention, access controls).
        EOF
        }

        # Cache log groups and service connectors for the compartment (and child compartments)
        echo "Fetching log groups and service connectors for analysis..." >&2

        LOG_GROUPS_JSON="$(oci logging log-group list \
          --compartment-id "${OCI_COMPARTMENT_OCID}" \
          --all \
          --compartment-id-in-subtree true \
          --query 'data[].{id:id, name:"display-name", compartment:"compartment-id"}' \
          --raw-output)"

        SERVICE_CONNECTORS_JSON="$(oci sch service-connector list \
          --compartment-id "${OCI_COMPARTMENT_OCID}" \
          --all \
          --compartment-id-in-subtree true \
          --query 'data[].{
            id:id,
            name:"display-name",
            state: "lifecycle-state",
            sourceType: "source.type",
            sourceLogGroupId: "source.logSources[0].logGroupId",
            targetKind: "target.kind",
            targetDescription: "target.description"
          }' \
          --raw-output || echo '[]')"

        echo
        echo "=== Clusters in compartment (and subcompartments) ==="
        CLUSTERS_JSON="$(oci ce cluster list \
          --compartment-id "${OCI_COMPARTMENT_OCID}" \
          --all \
          --compartment-id-in-subtree true \
          --query 'data[].{
            id:id,
            name:"name",
            lifecycle:"lifecycle-state",
            endpointConfig:"kubernetes-endpoint-config"
          }' \
          --raw-output)"

        if [ -n "${OKE_NAME_FILTER}" ]; then
          CLUSTERS_JSON="$(echo "${CLUSTERS_JSON}" | jq --arg re "${OKE_NAME_FILTER}" '[.[] | select(.name|test($re))]')"
        fi

        if [ "$(echo "${CLUSTERS_JSON}" | jq 'length')" -eq 0 ]; then
          echo "No clusters found for the given criteria."
          exit 0
        fi

        echo "${CLUSTERS_JSON}" | jq -r '.[] | "\(.name) (\(.id)) - state: \(.lifecycle)"'
        echo

        echo "=== Detailed API logging review per cluster ==="
        echo

        echo "${CLUSTERS_JSON}" | jq -c '.[]' | while read -r CL; do
          CL_ID="$(echo "${CL}" | jq -r '.id')"
          CL_NAME="$(echo "${CL}" | jq -r '.name')"
          CL_STATE="$(echo "${CL}" | jq -r '.lifecycle')"

          echo "-----"
          echo "Cluster: ${CL_NAME}"
          echo "OCID   : ${CL_ID}"
          echo "State  : ${CL_STATE}"

          # Skip non-active clusters but still show basic info
          if [ "${CL_STATE}" != "ACTIVE" ]; then
            echo "Verdict: SKIPPED_NON_ACTIVE (cluster not ACTIVE; inspect manually if needed)"
            echo
            continue
          fi

          # Get Logging configurations associated with this cluster
          LOGGING_CFGS="$(oci logging log list \
            --compartment-id "${OCI_COMPARTMENT_OCID}" \
            --all \
            --compartment-id-in-subtree true \
            --query "data[?contains(\"defined-tags\".\"Oracle-Kubernetes\".\"cluster-id\", '${CL_ID}')][]" \
            --raw-output || echo '[]')"

          CONTROL_PLANE_LOGS="$(echo "${LOGGING_CFGS}" | jq '[.[] | select(.type=="SERVICE" and (.\"configuration\".source.service=="kubernetes" or .\"configuration\".source.service=="oke"))]')"

          CP_COUNT="$(echo "${CONTROL_PLANE_LOGS}" | jq 'length')"

          if [ "${CP_COUNT}" -eq 0 ]; then
            echo "Control-plane / API-related logs: NONE detected for this cluster."
            echo "Verdict: NO_CONTROL_PLANE_LOGS"
            echo "Explanation: API audit logging is likely NOT enabled for this cluster."
            echo
            continue
          fi

          echo "Control-plane / API-related log objects:"
          echo "${CONTROL_PLANE_LOGS}" | jq -r '.[] | "- Log: \(.\"display-name\") | id: \(.id) | group: \(.\"log-group-id\") | state: \(.\"lifecycle-state\")"'

          # For each relevant log group, see if there is a service connector shipping it elsewhere
          DESTINATIONS_FOUND=0
          echo
          echo "Checking service connectors for these log groups..."

          echo "${CONTROL_PLANE_LOGS}" | jq -r '.[]."log-group-id"' | sort -u | while read -r LG_ID; do
            [ -z "${LG_ID}" ] && continue
            LG_INFO="$(echo "${LOG_GROUPS_JSON}" | jq --arg id "${LG_ID}" '[.[] | select(.id==$id)][0]')"
            LG_NAME="$(echo "${LG_INFO}" | jq -r '.name // "UNKNOWN"' 2>/dev/null || echo "UNKNOWN")"

            echo "  Log Group: ${LG_NAME} (${LG_ID})"

            # Find service connectors using this log group as source
            SC_MATCH="$(echo "${SERVICE_CONNECTORS_JSON}" | jq --arg lg "${LG_ID}" '[.[] | select(.sourceType=="logging" and .sourceLogGroupId==$lg and .state=="ACTIVE")]')"
            SC_COUNT="$(echo "${SC_MATCH}" | jq 'length')"

            if [ "${SC_COUNT}" -eq 0 ]; then
              echo "    -> NO active service connector found for this log group."
            else
              DESTINATIONS_FOUND=1
              echo "    -> Active service connectors:"
              echo "${SC_MATCH}" | jq -r '.[] | "       - \(.name) (\(.id)) | targetKind: \(.targetKind) | desc: \(.targetDescription)"'
            fi
          done

          if [ "${DESTINATIONS_FOUND}" -eq 0 ]; then
            echo
            echo "Verdict: CONTROL_PLANE_LOGS_NO_DESTINATION"
            echo "Explanation: Control-plane / API logs exist in OCI Logging, but"
            echo "             no active service connector appears to forward them"
            echo "             to an external/tamper-resistant destination."
          else
            echo
            echo "Verdict: CONTROL_PLANE_LOGS_WITH_DESTINATION"
            echo "Explanation: Control-plane / API logs are configured AND there"
            echo "             is at least one service connector forwarding the"
            echo "             log group(s) to another sink (e.g., Object Storage,"
            echo "             Logging Analytics, Stream). Manually verify that this"
            echo "             meets your 'external, tamper-resistant' requirement."
          fi

          echo
        done

        echo "==== Review Guidance ===="
        explain_verdict
        ```

        **How to interpret output (what indicates a problem)**

        Run on any machine with `oci` and `jq`:

        ```bash theme={null}
        export OCI_COMPARTMENT_OCID="ocid1.compartment.oc1..xxxxx"
        ./audit-oke-api-logging.sh
        ```

        Problem indicators to investigate and manually remediate:

        * `Verdict: NO_CONTROL_PLANE_LOGS`\
          → The script found no OKE-related control-plane/API logs. API audit logging is likely not enabled.

        * `Verdict: CONTROL_PLANE_LOGS_NO_DESTINATION`\
          → Control-plane/API logs exist only inside OCI Logging, and no active Service Connector was found that exports them. This usually means logs are not shipped to an independent, tamper-resistant store.

        `CONTROL_PLANE_LOGS_WITH_DESTINATION` is better, but still requires manual review of the destination (e.g., Object Storage bucket policies, retention, access controls) to decide whether it satisfies “external, tamper-resistant” for your environment.
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>