> ## Documentation Index > Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # OCI OKE Kubelet Should Be Allowed to Manage IPtables ### More Info: Kubelet relies on iptables (makeIPTablesUtilChains=true) to install canonical chains used by kube-proxy. Disabling this can leave NodePort and Service traffic in an inconsistent or insecure state. ### Risk Level Medium ### Address Compliance, Security ### Compliance Standards * CIS OKE ### Triage and Remediation ### Remediation To fix this in OKE you must enable kubelet’s iptables management at the **node pool** level. In OKE this is controlled through the kubelet configuration attached to the node pool. Below are the steps using the OCI Console. If your existing node pool does not allow editing kubelet settings, you’ll need to create a new node pool with the correct configuration and then migrate workloads. *** ### 1. Locate the OKE Cluster and Node Pool 1. Sign in to the **OCI Console**. 2. Open the menu → **Developer Services** → **Kubernetes Clusters (OKE)**. 3. Select the **Compartment**. 4. Click your **cluster** name. 5. Go to the **Node pools** tab and identify the node pool where the issue is reported. *** ### 2. Check If the Node Pool Kubelet Config Is Editable 1. Click the **node pool** name. 2. Look for an **Edit** button (or **Update node pool**) and then an **Advanced options** / **Kubelet configuration** section. * If you see kubelet configuration fields, proceed to the next section. * If kubelet configuration is not editable (common with some older or managed images), skip to **Step 4: Create a new node pool**. *** ### 3. Enable “Kubelet Manages iptables” in the Node Pool The exact UI text may differ slightly by region/console version, but it will be in **Kubelet configuration** or **Advanced node configuration** for the node pool: 1. In the node pool **Edit / Update** screen, expand **Kubelet configuration** or **Advanced options**. 2. Look for: * A checkbox/option like **Allow kubelet to manage iptables**, **Manage iptables**, or similar; **enable/check it**. * Or a YAML/JSON block for kubelet config. In that case, ensure the equivalent field is set, e.g.: ```yaml theme={null} kubeletConfig: makeIptablesUtilChains: true ``` or, if flags are given as extra arguments, make sure `--make-iptables-util-chains=true` is present and not overridden to `false`. 3. Save the changes: * Click **Save changes** / **Update node pool**. 4. When the node pool update completes, the nodes may be automatically cycled; if not, you may need to: * Manually **reboot** nodes, or * **Terminate** nodes in that pool to let OKE recreate them with the new kubelet config. *** ### 4. If Kubelet Config Is Not Editable: Create a New Node Pool If you cannot change kubelet configuration on the existing pool: 1. From the cluster’s **Node pools** tab, click **Create node pool**. 2. Use the same: * **VCN**, **Subnets**, **Image**, **Shape**, and **Placement** as the existing pool (unless you intentionally want to change them). 3. In the **Kubelet configuration / Advanced options** section: * Enable **Allow kubelet to manage iptables** (or similar). * Or add the kubelet config that ensures iptables management is enabled: ```yaml theme={null} kubeletConfig: makeIptablesUtilChains: true ``` 4. Complete creation of the node pool and wait until all nodes show as **Active**. *** ### 5. Migrate Workloads to the New Node Pool On the cluster (kubectl): 1. Label or taint the **new** node pool nodes if needed to match scheduling rules of your workloads. 2. **Cordon and drain** old node pool nodes: ```bash theme={null} kubectl cordon kubectl drain --ignore-daemonsets --delete-emptydir-data ``` Repeat for all nodes in the old node pool. 3. Confirm all pods are running on the **new** node pool. 4. In the OCI Console, once you’re sure the old pool is unused, **Delete** the old node pool. *** ### 6. Optional: Ensure No OS Firewall Overrides iptables On your node images (if customized): * Ensure services like `firewalld` or `ufw` are **disabled** or configured so they do not overwrite kubelet-managed iptables rules. *** After nodes are recreated or updated with the new configuration, re-run your security/benchmark scan; the “Kubelet should be allowed to manage iptables” finding should clear. In Oracle Container Engine for Kubernetes (OKE), there is **no OCI or OKE setting (and no OCI CLI flag)** that directly toggles “Kubelet Should Be Allowed to Manage IPtables.” That behavior is controlled by **kubelet startup flags on the worker nodes**, not by an OKE cluster or node pool property. In OKE’s managed node pools, kubelet is started by Oracle’s bootstrap scripts, and by default it already uses the standard iptables management flags. If you are seeing a security finding saying “Kubelet Should Be Allowed to Manage IPtables,” it usually means: * You are using **custom images or custom bootstrap scripts** for your worker nodes, and/or * You have modified the kubelet systemd unit or kubelet arguments so that it is **not** allowed to manage iptables (for example, removing or changing the `--make-iptables-util-chains` behavior). Because of that, this cannot be remediated with a single `oci` CLI command. Instead, you must fix it in your node image / bootstrap and then roll your node pool. Below are the practical remediation steps, using OCI CLI where possible: *** ### 1. Confirm you are using custom images / bootstrap Use OCI CLI to inspect your node pool: ```bash theme={null} oci ce node-pool get --node-pool-id ``` Look for: * `nodeSourceDetails` → custom image OCID * Any custom `userData` / cloud-init or other bootstrap mechanism you manage. If you are using Oracle-provided images and haven’t modified bootstrap, OKE already configures kubelet correctly; the finding may be a false positive or scanner misconfiguration. *** ### 2. Fix your kubelet configuration on the node image / bootstrap You must ensure kubelet is started **with iptables management enabled**. On Linux nodes this is typically in: * `/etc/systemd/system/kubelet.service` or * `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`\ or a similar systemd unit / drop‑in file, depending on your image. On the base image or in your bootstrap script: 1. Open the kubelet unit or drop‑in configuration. 2. Ensure the kubelet `ExecStart` line includes the standard iptables option (the default in kubeadm is `--make-iptables-util-chains=true`; some distributions now omit it because it’s true by default, which is fine as long as you didn’t explicitly disable it). 3. Remove or correct any flag that would prevent kubelet from managing iptables. Example (on your custom image): ```bash theme={null} sudo mkdir -p /etc/systemd/system/kubelet.service.d/ cat << 'EOF' | sudo tee /etc/systemd/system/kubelet.service.d/20-iptables.conf [Service] Environment="KUBELET_EXTRA_ARGS=--make-iptables-util-chains=true" EOF sudo systemctl daemon-reload sudo systemctl restart kubelet ``` Bake this into your **custom image** or your **cloud-init / bootstrap script** so that all future nodes come up correctly. *** ### 3. Update your node pool to use the fixed image (OCI CLI) After creating a new custom image with corrected kubelet configuration: ```bash theme={null} oci ce node-pool update \ --node-pool-id \ --node-source-details '{"sourceType": "IMAGE", "imageId": ""}' \ --force ``` Wait for the update to complete: ```bash theme={null} oci ce node-pool get --node-pool-id \ --query 'data."lifecycle-state"' --raw-output ``` *** ### 4. Rotate / recycle nodes to pick up the new configuration For each node in the node pool: ```bash theme={null} oci ce node-pool get --node-pool-id \ --query 'data.nodes[].id' --raw-output ``` For each node OCID, terminate it (OKE will recreate it using the updated image): ```bash theme={null} oci ce node-pool-evict-nodes \ --node-pool-id \ --nodes '[""]' \ --is-decrement-size false ``` Repeat until all nodes have been recycled. *** ### 5. Verify on a new node SSH into a newly created node from that node pool and check kubelet flags: ```bash theme={null} ps aux | grep kubelet | grep -v grep ``` Confirm that kubelet either: * Has `--make-iptables-util-chains=true`, or * Does not have any flag that disables iptables chain management. You can also inspect systemd: ```bash theme={null} systemctl cat kubelet ``` *** ### Key point There is **no direct OCI CLI switch** like “allow kubelet to manage iptables” for OKE.\ The only reliable remediation is: 1. Fix kubelet configuration in your node image / bootstrap. 2. Update the node pool to use the corrected image (via OCI CLI). 3. Rotate nodes so they come up with the fixed kubelet configuration. For OKE the kubelet flag that matters for this control is: ```bash theme={null} --make-iptables-util-chains=true ``` You need to ensure this flag is present (and not set to `false`) in the kubelet arguments on all worker nodes, then restart kubelet. Below is a simple, Python‑based approach you can automate. *** ## 1. High‑level steps 1. Get the worker node public IPs (from OKE / OCI). 2. SSH to each worker node. 3. Update kubelet config to include `--make-iptables-util-chains=true`. 4. Reload systemd and restart kubelet. 5. Verify kubelet has the correct flag. *** ## 2. Example Python script (SSH‑based remediation) This example uses: * `oci` SDK to discover node public IPs from the node pool. * `paramiko` to SSH and patch the kubelet configuration. > Adjust file paths if your OKE worker node uses a different kubelet drop‑in file (common ones are shown). ```python theme={null} import oci import paramiko import io import re # ========= USER INPUTS ========= compartment_id = "" cluster_id = "" node_pool_id = "" # or loop all nodepools ssh_username = "opc" # typical for Oracle Linux ssh_key_path = "/path/to/private/key" # private key used to access worker nodes # kubelet systemd drop-in file (commonly used in OKE nodes; adjust if needed) KUBELET_DROPIN_PATH = "/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf" # ========================================= def get_node_public_ips(config, node_pool_id): ce_client = oci.container_engine.ContainerEngineClient(config) compute_client = oci.core.ComputeClient(config) # 1. List nodes in the nodepool nodes = ce_client.list_node_pool_nodes(node_pool_id).data public_ips = [] for node in nodes: # node is a NodePoolNode instance_id = node.instance_id # 2. Get instance VNIC attachments vnics = oci.pagination.list_call_get_all_results( oci.core.ComputeClient(config).list_vnic_attachments, compartment_id=compartment_id, instance_id=instance_id ).data for att in vnics: vnic = oci.core.VirtualNetworkClient(config).get_vnic(att.vnic_id).data if vnic.public_ip: public_ips.append(vnic.public_ip) return list(set(public_ips)) def ssh_connect(ip, username, key_path): key = paramiko.RSAKey.from_private_key_file(key_path) client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) client.connect(ip, username=username, pkey=key) return client def get_file(ssh_client, path): sftp = ssh_client.open_sftp() f = sftp.file(path, mode='r') content = f.read().decode('utf-8') f.close() sftp.close() return content def put_file(ssh_client, path, content): sftp = ssh_client.open_sftp() f = sftp.file(path, mode='w') f.write(content) f.flush() f.close() sftp.close() def ensure_kubelet_iptables_flag(content): """ Look for kubelet args line and ensure --make-iptables-util-chains=true is present. """ lines = content.splitlines() new_lines = [] changed = False kubelet_arg_pattern = re.compile(r'(^.*kubelet.*)$') for line in lines: m = kubelet_arg_pattern.search(line) if m: # if flag is present and set to false, change it to true; if missing, add it if '--make-iptables-util-chains=' in line: if '--make-iptables-util-chains=false' in line: line = line.replace('--make-iptables-util-chains=false', '--make-iptables-util-chains=true') changed = True else: # add flag if line.strip().endswith('"'): # inside a systemd ExecStart with quotes line = line[:-1] + ' --make-iptables-util-chains=true"' else: line = line + ' --make-iptables-util-chains=true' changed = True new_lines.append(line) # If we never matched the kubelet exec line, no change is done; caller can handle. return "\n".join(new_lines), changed def run_command(ssh_client, cmd): stdin, stdout, stderr = ssh_client.exec_command(cmd) rc = stdout.channel.recv_exit_status() out = stdout.read().decode() err = stderr.read().decode() return rc, out, err def main(): # Load OCI config (default from ~/.oci/config) config = oci.config.from_file() public_ips = get_node_public_ips(config, node_pool_id) print(f"Found worker node public IPs: {public_ips}") for ip in public_ips: print(f"\nProcessing node {ip} ...") ssh = ssh_connect(ip, ssh_username, ssh_key_path) try: # Backup file original = get_file(ssh, KUBELET_DROPIN_PATH) backup_path = KUBELET_DROPIN_PATH + ".bak" put_file(ssh, backup_path, original) print(f"Backed up kubelet config to {backup_path}") # Patch content new_content, changed = ensure_kubelet_iptables_flag(original) if not changed: print("No change needed; kubelet already allowed to manage iptables.") else: put_file(ssh, KUBELET_DROPIN_PATH, new_content) print("Updated kubelet drop-in with --make-iptables-util-chains=true") # Reload systemd and restart kubelet for cmd in [ "sudo systemctl daemon-reload", "sudo systemctl restart kubelet", "sudo systemctl status kubelet --no-pager --full" ]: rc, out, err = run_command(ssh, cmd) print(f"Command: {cmd}\nExit: {rc}\nOUT:\n{out}\nERR:\n{err}") finally: ssh.close() if __name__ == "__main__": main() ``` *** ## 3. Verification From your workstation: ```bash theme={null} kubectl get nodes -o wide # pick a node, then: kubectl debug node/ -it --image=busybox # inside debug pod: ps aux | grep kubelet | grep make-iptables-util-chains # you should see: --make-iptables-util-chains=true ``` If you want, I can adapt the script to: * Work across all nodepools in a cluster, or * Use a DaemonSet inside the cluster instead of SSH. ```hcl theme={null} # As of the current OCI Terraform provider, the OKE node pool resource # (oci_containerengine_node_pool / oci_oke_nodepool) does NOT expose # any arguments to manage kubelet flags such as makeIPTablesUtilChains. # # This specific finding therefore cannot be remediated via Terraform today. # You must change this on the node pool itself using the OCI Console or OCI CLI: # - Console: Container clusters (OKE) -> your Cluster -> Node pools -> # select NODE_POOL_NAME -> Kubelet configuration -> enable # “Allow Kubelet to manage iptables (makeIPTablesUtilChains=true)” -> Save. # # - OCI CLI (example): # oci ce node-pool update \ # --node-pool-id OCID_OF_NODE_POOL \ # --kubelet-config '{"makeIPTablesUtilChains": true}' # # No Terraform change is possible until Oracle exposes kubelet configuration # for node pools in the oci_containerengine_node_pool resource. # terraform plan should show no changes for this setting, because it is # currently unmanaged / unexposed in Terraform. ```