> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OCI OKE kubelet-config.json File Ownership Should Be root:root

### More Info:

The kubelet-config.json file should be owned by root:root. Incorrect ownership lets unprivileged users alter kubelet runtime parameters such as authentication and authorization modes.

### Risk Level

High

### Address

Compliance, Security

### Compliance Standards

* CIS OKE

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Using Console" defaultOpen="true">
        You can’t change `kubelet-config.json` ownership purely with a “button” in the OCI Console, because the file lives on each worker node’s OS. However, you can use the Console to push a startup script (cloud‑init / user data) to your node pool so that every node enforces `root:root` on boot and on replacement.

        Below are step‑by‑step instructions to do this **using only the OCI Console**, without manually SSH’ing to each node.

        ***

        ## 1. Understand where `kubelet-config.json` lives

        On OKE worker nodes (OCI‑supplied images), the kubelet config is typically under:

        * `/etc/oci/kubelet/kubelet-config.json`\
          or
        * `/var/lib/kubelet/config.yaml` (for some newer versions)

        Your security control specifically references `kubelet-config.json`, so assume path:

        ```bash theme={null}
        /etc/oci/kubelet/kubelet-config.json
        ```

        (If you know your cluster image path differs, substitute the correct one in the script below.)

        ***

        ## 2. Create a bootstrap script to fix ownership

        You’ll store this as **user data** for the node pool via the Console.

        Prepare this script locally (you’ll paste it in step 3):

        ```bash theme={null}
        #!/bin/bash
        # Ensure kubelet-config.json ownership is root:root and permissions are strict

        CONFIG_PATH="/etc/oci/kubelet/kubelet-config.json"

        if [ -f "$CONFIG_PATH" ]; then
          chown root:root "$CONFIG_PATH"
          chmod 600 "$CONFIG_PATH"
        fi

        # (Optional) Add a systemd drop-in to ensure perms stay correct on each boot

        cat >/etc/systemd/system/kubelet-perms.service <<'EOF'
        [Unit]
        Description=Ensure kubelet-config.json ownership is root:root
        After=network.target

        [Service]
        Type=oneshot
        ExecStart=/bin/bash -c '[ -f /etc/oci/kubelet/kubelet-config.json ] && chown root:root /etc/oci/kubelet/kubelet-config.json && chmod 600 /etc/oci/kubelet/kubelet-config.json'

        [Install]
        WantedBy=multi-user.target
        EOF

        systemctl daemon-reload
        systemctl enable kubelet-perms.service
        systemctl start kubelet-perms.service
        ```

        This will:

        * Set owner to `root:root`
        * Set permissions to `600`
        * Ensure on every reboot the ownership is re‑applied.

        ***

        ## 3. Add the script as cloud‑init user data to your node pool

        1. In the **OCI Console**, go to:
           * **Developer Services → Kubernetes Clusters (OKE)**.
        2. Click your cluster.
        3. Go to the **Node Pools** tab.
        4. Click the node pool you want to fix.
        5. Click **Edit** (or **Update Node Pool**).
        6. Find the **Node configuration details** / **Cloud-init script** / **User data** section:
           * If a script already exists, append the content above to the end of that script (do not remove existing logic).
           * If empty, paste the script from step 2 in full.
        7. Save / **Update Node Pool**.

        ***

        ## 4. Rotate / Recreate nodes so the script runs

        The script only runs when nodes (re)boot with the new user data.

        From the same node pool page:

        1. Use **Scale** or **Node Pool → Reboot / Terminate Nodes** to cycle nodes:
           * Safest: **cordon and drain** nodes one by one from kubectl, then terminate them so the node pool recreates them with the new user data.
        2. As each node is recreated, the script runs and fixes ownership.

        ***

        ## 5. (Optional) Verify on a node

        If you are allowed to SSH into worker nodes:

        ```bash theme={null}
        ls -l /etc/oci/kubelet/kubelet-config.json
        ```

        You should see:

        ```bash theme={null}
        -rw------- 1 root root ... /etc/oci/kubelet/kubelet-config.json
        ```

        ***

        If you tell me your exact OKE image version (Oracle Linux version / node shape) I can adjust the path or script for that specific environment.
      </Accordion>

      <Accordion title="Using CLI">
        You can’t change Unix file ownership directly with the **OCI CLI alone** (the CLI manages OCI resources, not in-guest files), but you *can* use it to run a command inside each OKE worker node to fix the ownership.

        Below are step‑by‑step instructions using **Compute Instance Run Command** via OCI CLI.

        ***

        ## 1. Prerequisites

        1. **Oracle Cloud Agent enabled** on worker nodes (OKE default worker images usually have this).
        2. Policy allowing you to use instance run command, e.g.:

        ```text theme={null}
        ALLOW group <your-group> TO USE instance-agent-command IN compartment <your-compartment>
        ```

        3. OCI CLI configured (`oci setup config`).

        ***

        ## 2. Find the worker nodes (instances) of your OKE cluster

        1. Get the node pools for your cluster:

        ```bash theme={null}
        CLUSTER_OCID="<your_cluster_ocid>"

        oci ce node-pool list \
          --compartment-id <your_compartment_ocid> \
          --cluster-id "$CLUSTER_OCID" \
          --query "data[].{id:id,\"name\":name}" \
          --output table
        ```

        2. For each node pool, list nodes and map to instance OCIDs:

        ```bash theme={null}
        NODEPOOL_OCID="<nodepool_ocid>"

        oci ce node list \
          --compartment-id <your_compartment_ocid> \
          --node-pool-id "$NODEPOOL_OCID" \
          --query "data[].{id:id,\"instance-id\":instance-id,\"lifecycle-state\":lifecycle-state}" \
          --output table
        ```

        Note the `instance-id` values – these are the Compute instances you’ll run commands on.

        ***

        ## 3. Run `chown root:root` on each node instance

        Assume path to kubelet config is `/var/lib/kubelet/config.json` or `/etc/kubernetes/kubelet/kubelet-config.json`. Use the path that applies in your environment.

        Example using `/var/lib/kubelet/config.json`:

        ```bash theme={null}
        INSTANCE_ID="<instance_ocid>"
        FILE_PATH="/var/lib/kubelet/config.json"

        oci compute instance-agent command execute \
          --instance-id "$INSTANCE_ID" \
          --command-content '{
            "content": "IyEvYmluL2Jhc2gKc2V0IC1lCgpGSUxFPiIkMSIKaWYgWCEgLWUgIiRGSUxFIjsgdGhlbgogIGVjaG8gIkZpbGUgJGZvdW5kOiAkRklMRSIgPiYyCiAgZXhpdCAwCmZpCgpjaG93biByb290OnJvb3QgIiRGSUxFIgo=}",
            --command-content-type BASE64 \
          --wait-for-state SUCCEEDED \
          --wait-interval-seconds 10 \
          --endpoint <region>-compute-iaas.oraclevcn.com \
          --variables "{\"1\": \"$FILE_PATH\"}"
        ```

        Explanation:

        * The `content` is a base64‑encoded bash script:

        ```bash theme={null}
        #!/bin/bash
        set -e

        FILE="$1"
        if [ ! -e "$FILE" ]; then
          echo "File not found: $FILE" >&2
          exit 0
        fi

        chown root:root "$FILE"
        ```

        * `--variables` passes the target file path as argument `"$1"`.

        Repeat for each `INSTANCE_ID` from step 2.

        ***

        ## 4. Verify ownership

        You can:

        * Either run another instance‑agent command to check:

        ```bash theme={null}
        oci compute instance-agent command execute \
          --instance-id "$INSTANCE_ID" \
          --command-content '{
            "content": "IyEvYmluL2Jhc2gKbHMgLWx0YXAgIiQxIgo="}' \
          --command-content-type BASE64 \
          --wait-for-state SUCCEEDED \
          --wait-interval-seconds 10 \
          --variables "{\"1\": \"$FILE_PATH\"}"
        ```

        The base64 script is:

        ```bash theme={null}
        #!/bin/bash
        ls -ltap "$1"
        ```

        * Or SSH into a node and run:

        ```bash theme={null}
        ls -l /var/lib/kubelet/config.json
        # or
        ls -l /etc/kubernetes/kubelet/kubelet-config.json
        ```

        You should see `root root` as the owner/group.

        ***

        ## 5. Make it persistent for new nodes

        For new/recreated worker nodes in OKE:

        1. Add a **cloud-init** script in your custom node image or in the node pool config that runs on boot:

        ```bash theme={null}
        #!/bin/bash
        FILE="/var/lib/kubelet/config.json"
        [ -e "$FILE" ] && chown root:root "$FILE"
        ```

        2. Or bake the correct ownership directly into a **custom image** used by the node pool.

        ***

        If you tell me your exact kubelet config path on OKE, I can give you a ready‑to‑run CLI command with the right file path only.
      </Accordion>

      <Accordion title="Using Python">
        To remediate **“OCI OKE kubelet-config.json File Ownership Should Be root:root”** using Python, you essentially need to:

        1. Find the kubelet config path on OKE worker nodes.
        2. Change its owner to `root:root` (`uid=0`, `gid=0`).
        3. Do this on every worker node (manually via SSH or automatically via a DaemonSet).

        Below are step‑by‑step instructions and a minimal Python script.

        ***

        ## 1. Identify kubelet-config.json path on OKE

        On OKE worker nodes, the kubelet config is typically at one of these paths (depending on OKE version / image):

        * `/etc/kubernetes/kubelet/kubelet-config.json`
        * `/var/lib/kubelet/config.json`
        * `/etc/oci-addons/kubelet/kubelet-config.json`

        SSH into a worker node and confirm:

        ```bash theme={null}
        sudo find /etc /var/lib -maxdepth 5 -type f -name "kubelet-config.json" 2>/dev/null
        ```

        Assume you find it at:

        ```text theme={null}
        /etc/kubernetes/kubelet/kubelet-config.json
        ```

        Use that path in the Python script.

        ***

        ## 2. Python script to set ownership to root:root

        This script:

        * Verifies file exists.
        * Sets owner to `root` and group to `root`.
        * Optionally verifies after change.

        ```python theme={null}
        #!/usr/bin/env python3

        import os
        import pwd
        import grp
        import sys

        # Adjust this if your kubelet config is in a different path
        KUBELET_CONFIG = "/etc/kubernetes/kubelet/kubelet-config.json"

        def ensure_root_ownership(path):
            if not os.path.isfile(path):
                print(f"ERROR: kubelet config file not found at: {path}")
                sys.exit(1)

            try:
                root_uid = pwd.getpwnam("root").pw_uid
                root_gid = grp.getgrnam("root").gr_gid
            except KeyError as e:
                print(f"ERROR: Cannot find 'root' user or group: {e}")
                sys.exit(1)

            st = os.stat(path)
            if st.st_uid == root_uid and st.st_gid == root_gid:
                print(f"OK: {path} already owned by root:root")
                return

            # Change ownership (requires root)
            try:
                os.chown(path, root_uid, root_gid)
                print(f"UPDATED: Set ownership of {path} to root:root")
            except PermissionError:
                print("ERROR: Permission denied. Run this script as root (sudo).")
                sys.exit(1)

            # Verify
            st2 = os.stat(path)
            if st2.st_uid == root_uid and st2.st_gid == root_gid:
                print(f"VERIFIED: {path} is now owned by root:root")
            else:
                print(f"ERROR: Failed to set ownership for {path}")
                sys.exit(1)

        if __name__ == "__main__":
            ensure_root_ownership(KUBELET_CONFIG)
        ```

        Run on node:

        ```bash theme={null}
        sudo python3 fix_kubelet_ownership.py
        ```

        ***

        ## 3. Automate across all OKE nodes (DaemonSet approach)

        If you want to avoid SSHing to each node, run the Python script in a **privileged DaemonSet** so it executes once on every node.

        ### 3.1. Build a small Python image

        Dockerfile (example):

        ```dockerfile theme={null}
        FROM python:3.11-slim

        RUN mkdir -p /opt/scripts
        COPY fix_kubelet_ownership.py /opt/scripts/fix_kubelet_ownership.py
        RUN chmod +x /opt/scripts/fix_kubelet_ownership.py

        ENTRYPOINT ["python3", "/opt/scripts/fix_kubelet_ownership.py"]
        ```

        Build and push to OCIR (or any registry):

        ```bash theme={null}
        docker build -t <region-key>.ocir.io/<tenancy-namespace>/<repo>/kubelet-fix:latest .
        docker push <region-key>.ocir.io/<tenancy-namespace>/<repo>/kubelet-fix:latest
        ```

        Adjust `KUBELET_CONFIG` in the script if your path is different.

        ### 3.2. DaemonSet YAML

        ```yaml theme={null}
        apiVersion: apps/v1
        kind: DaemonSet
        metadata:
          name: kubelet-config-ownership-fix
          namespace: kube-system
        spec:
          selector:
            matchLabels:
              app: kubelet-config-ownership-fix
          template:
            metadata:
              labels:
                app: kubelet-config-ownership-fix
            spec:
              hostPID: true
              hostNetwork: true
              containers:
              - name: fixer
                image: <region-key>.ocir.io/<tenancy-namespace>/<repo>/kubelet-fix:latest
                securityContext:
                  privileged: true
                volumeMounts:
                - name: host-etc-kubernetes
                  mountPath: /etc/kubernetes
                  readOnly: false
                # If your kubelet config is somewhere else, mount that path too.
              volumes:
              - name: host-etc-kubernetes
                hostPath:
                  path: /etc/kubernetes
                  type: Directory
              restartPolicy: Always
        ```

        Apply:

        ```bash theme={null}
        kubectl apply -f kubelet-config-ownership-fix.yaml
        ```

        Logs from one pod:

        ```bash theme={null}
        kubectl -n kube-system logs -l app=kubelet-config-ownership-fix
        ```

        Once confirmed, you can delete the DaemonSet:

        ```bash theme={null}
        kubectl -n kube-system delete daemonset kubelet-config-ownership-fix
        ```

        ***

        ## 4. Verify remediation

        On a worker node:

        ```bash theme={null}
        stat -c "%U:%G %n" /etc/kubernetes/kubelet/kubelet-config.json
        # Expected:
        # root:root /etc/kubernetes/kubelet/kubelet-config.json
        ```

        This satisfies the requirement “kubelet-config.json File Ownership Should Be root:root” in OCI OKE.
      </Accordion>

      <Accordion title="Using Terraform">
        ```hcl theme={null}
        # This misconfiguration cannot currently be fixed directly on
        # oci_containerengine_node_pool (OKE node pool) via Terraform.
        #
        # The OCI provider does not expose any argument that controls
        # file ownership of /etc/kubernetes/kubelet/kubelet-config.json
        # on worker nodes. That ownership is set inside the node image
        # / bootstrap scripts at OS level.

        # Typical node pool resource for context (no argument here can
        # change kubelet-config.json ownership):

        resource "oci_containerengine_node_pool" "example" {
          compartment_id = "OCID_OF_COMPARTMENT"
          cluster_id     = "OCID_OF_CLUSTER"
          name           = "NODE_POOL_NAME"
          kubernetes_version = "K8S_VERSION"

          node_config_details {
            size = 3
            placement_configs {
              availability_domain = "AVAILABILITY_DOMAIN_NAME"
              subnet_id           = "OCID_OF_SUBNET"
            }
          }

          node_shape = "VM.Standard3.Flex"

          node_source_details {
            source_type             = "IMAGE"
            image_id                = "OCID_OF_NODE_IMAGE"
            boot_volume_size_in_gbs = 50
          }
        }

        # To remediate:
        # - Use a custom node image or cloud-init/bootstrapping script
        #   that enforces:
        #   chown root:root /etc/kubernetes/kubelet/kubelet-config.json
        #   chmod 600 /etc/kubernetes/kubelet/kubelet-config.json
        # - Or update your existing image/agent configuration outside Terraform.

        # These are OS-level actions that must be done via the image build
        # pipeline, instance configuration, or manual/automated commands
        # on the nodes, not via Terraform arguments on the node pool.

        # Verification in Terraform:
        # - `terraform plan` will show NO changes related to kubelet-config.json
        #   because the provider does not model that file’s ownership.
        ```
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>