> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OCI Monitoring Should Have Anomalous GPU Usage Alarm Configured

### More Info:

Monitor for anomalous GPU instance provisioning. Attackers frequently deploy high-cost, high-compute GPU instances for illicit cryptocurrency mining after compromising cloud accounts

### Risk Level

Medium

### Address

Compliance, Security

### Compliance Standards

* APRA CPS 234 (Australia)
* AWS Well Architected Framework
* BSI C5 (Germany)
* Brazil LGPD
* CCPA / CPRA (California)
* CIS Critical Security Controls v8
* CMMC 2.0
* CSA Cloud Controls Matrix v4
* DPDPA
* Digital Operational Resilience Act (EU)
* ISO/IEC 27017
* ISO/IEC 27018
* ISO/IEC 27701
* KSA PDPL
* MAS Technology Risk Management (Singapore)
* MITRE ATT\&CK (Cloud)
* NIS2 Directive
* NIST SP 800-171
* NYDFS 23 NYCRR 500
* SWIFT Customer Security Controls Framework
* Sarbanes-Oxley IT General Controls
* UK NCSC Cyber Assessment Framework

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Using Console" defaultOpen="true">
        Below are concise, console‑only steps to configure an anomalous GPU usage alarm in OCI Monitoring.

        ***

        ### 1. Confirm GPU Metrics Are Available

        1. In OCI Console, open **Observability & Management → Monitoring → Service Metrics**.
        2. In **Compartment**, select the compartment where your GPU instances run.
        3. Set:
           * **Metric namespace**: `oci_computeagent`
           * **Resource group**: often `gpu` (or leave as “All” if unsure).
        4. In **Metric name**, look for GPU metrics such as:
           * `GpuUtilization`
           * `GpuMemoryUtilization`
           * `GpuTemperature`
        5. Select a metric and verify you see recent data points in the chart.\
           If there is no data, ensure the **Oracle Cloud Agent (Management Agent)** with GPU plugins is enabled on the instance.

        ***

        ### 2. Create an Alarm for Anomalous GPU Usage

        1. Go to **Observability & Management → Monitoring → Alarms**.
        2. Click **Create alarm**.

        **Basic configuration:**

        3. **Name**: e.g., `gpu-anomaly-usage-alarm`.
        4. **Description**: e.g., `Alarm when GPU utilization behaves anomalously`.
        5. **Compartment**: choose the compartment of the GPU metrics.
        6. **Metric namespace**: `oci_computeagent`.
        7. **Resource group**: `gpu` (or as confirmed in step 1).

        **Choose the metric and anomaly rule:**

        8. Under **Metric name**, select `GpuUtilization` (or the specific GPU metric you want to monitor).
        9. For **Dimensions**, optionally scope to:
           * Specific instance: `resourceId = <instance-OCID>`
           * Or other relevant dimensions if needed.
        10. Under **Trigger rule (Alarm condition)**, select:
            * **Rule type**: `Anomaly` (or `Dynamic threshold / Anomaly detection` if phrased that way in your tenancy).
        11. Configure anomaly condition:
            * **Statistic**: `avg` (or `max` depending on your policy).
            * **Period**: e.g., `1 minute` or `5 minutes`.
            * **Anomaly model / Baseline**: leave as **Automatic** unless you have specific training data.
            * **Sensitivity**: set to **Medium** (adjust later based on noise).

        ***

        ### 3. Set Alarm Threshold & Evaluation

        1. Configure when to fire:
           * Condition: e.g., `Anomaly score > 0` or the UI’s default anomaly trigger (often “When metric is anomalous”).
           * **Trigger delay / trigger after**: e.g., **3 consecutive periods** to avoid noise.
        2. **Severity**: choose (e.g., `Critical` or `Warning` as per your policy).

        ***

        ### 4. Configure Notifications

        1. Under **Destination**, select or create an **OCI Notifications** topic:
           * If needed, click **Create topic** and then add **Subscriptions** (email, Slack via HTTPS, PagerDuty, etc.) in **Developer Services → Notifications**.
        2. In the alarm:
           * Choose **Notification topic**: select the topic you created.

        (Optionally, configure **Repeat notification** and **Suppression** windows if supported and needed.)

        ***

        ### 5. Finalize and Test

        1. Click **Create alarm**.
        2. Confirm the alarm appears in **Alarms** list with `OK` or `FIRING` state.
        3. To test:
           * Intentionally increase GPU load (e.g., run a GPU‑heavy workload) to cause anomalous behavior versus normal baseline.
           * Verify that the alarm transitions to **FIRING** and that you receive the notification.

        ***

        This completes remediation: OCI Monitoring now has an anomalous GPU usage alarm configured through the OCI Console.
      </Accordion>

      <Accordion title="Using CLI">
        Below is a minimal, CLI‑only way to set up an “anomalous GPU usage” alarm in OCI Monitoring. I’ll assume Linux/macOS shell; on Windows, adjust quoting.

        ***

        ## 1. Prereqs

        1. Ensure the instance has GPU metrics published:
           * It must be a GPU shape.
           * Compute Instance Agent must be running (default on OCI images).

        2. Your OCI user must have policies like:
           ```text theme={null}
           allow group <group-name> to read metrics in compartment <compartment-name>
           allow group <group-name> to manage alarms in compartment <compartment-name>
           allow group <group-name> to use ons-topics in compartment <compartment-name>
           ```

        3. OCI CLI configured:
           ```bash theme={null}
           oci setup config
           ```

        ***

        ## 2. Identify GPU Metric and Namespace

        List available GPU metrics in the instance’s compartment:

        ```bash theme={null}
        COMPARTMENT_OCID="<compartment-ocid>"

        oci monitoring metric list \
          --compartment-id "$COMPARTMENT_OCID" \
          --namespace oci_computeagent \
          --query "data[?contains(name, 'Gpu') || contains(name, 'GPU')]" \
          --all
        ```

        Typical GPU utilization metric is under namespace `oci_computeagent` with a name like:

        * `GpuUtilization` or similar (use exactly what you see in the output).

        Note the:

        * `namespace` (e.g., `oci_computeagent`)
        * metric `name` (e.g., `GpuUtilization`)
        * dimensions (e.g., `resourceId`, `gpuIndex`)

        ***

        ## 3. Create a Notification Topic (for the Alarm)

        If you already have a topic OCID, skip to step 4.

        ```bash theme={null}
        TOPIC_NAME="gpu-alerts-topic"

        oci ons topic create \
          --compartment-id "$COMPARTMENT_OCID" \
          --name "$TOPIC_NAME" \
          --description "GPU anomalous usage alerts" \
          --query "data.id" \
          --raw-output
        ```

        Save the returned OCID as:

        ```bash theme={null}
        TOPIC_OCID="<topic-ocid-from-above>"
        ```

        Optionally subscribe an email:

        ```bash theme={null}
        oci ons subscription create \
          --compartment-id "$COMPARTMENT_OCID" \
          --topic-id "$TOPIC_OCID" \
          --protocol EMAIL \
          --endpoint you@example.com
        ```

        Then confirm the email.

        ***

        ## 4. Build an “Anomalous GPU Usage” Query

        Two common ways (pick one):

        ### 4.1. Simple “High GPU Utilization” (practical anomaly proxy)

        Example: GPU utilization > 90% for 5 minutes on any GPU in a compartment:

        ```bash theme={null}
        METRIC_NAMESPACE="oci_computeagent"
        METRIC_NAME="GpuUtilization"     # replace with your metric name

        QUERY_TEXT="${METRIC_NAME}[1m]{resourceId = \"*\"}.mean() > 90"
        ```

        You can restrict to a specific instance:

        ```bash theme={null}
        INSTANCE_OCID="<gpu-instance-ocid>"
        QUERY_TEXT="${METRIC_NAME}[1m]{resourceId = \"${INSTANCE_OCID}\"}.mean() > 90"
        ```

        ### 4.2. If you want deviation from baseline (pseudo‑anomaly)

        Some tenants use a rule like “> 2× the 1‑hour rolling average”. OCI Monitoring doesn’t yet have a native anomalyDetection() function generally available in all regions, so you usually approximate anomaly with a high threshold or derivative‑type rules. If your tenancy has advanced query features, adapt accordingly; otherwise use 4.1.

        ***

        ## 5. Create the Alarm via OCI CLI

        ```bash theme={null}
        ALARM_NAME="Anomalous-GPU-Usage"
        ALARM_DESCRIPTION="Alert when GPU utilization is anomalously high."
        SEVERITY="CRITICAL"      # or MAJOR, MINOR, WARNING
        PENDING_DURATION="PT5M"  # condition must hold 5 minutes

        oci monitoring alarm create \
          --compartment-id "$COMPARTMENT_OCID" \
          --display-name "$ALARM_NAME" \
          --metric-compartment-id "$COMPARTMENT_OCID" \
          --namespace "$METRIC_NAMESPACE" \
          --query-text "$QUERY_TEXT" \
          --severity "$SEVERITY" \
          --is-enabled true \
          --destinations "[\"$TOPIC_OCID\"]" \
          --pending-duration "$PENDING_DURATION" \
          --body '{
            "messageFormat": "RAW",
            "resolution": "1m",
            "notificationVersion": "1.0"
          }' \
          --query "data.id" \
          --raw-output
        ```

        Notes:

        * `--namespace` must match the namespace from step 2.
        * `--query-text` is the full alarm expression (no extra `where`/`select` syntax, just Monitoring query).
        * `pending-duration` is ISO‑8601 (e.g., `PT5M` = 5 minutes).

        ***

        ## 6. Update or Fix an Existing Alarm (if misconfigured)

        List alarms:

        ```bash theme={null}
        oci monitoring alarm list \
          --compartment-id "$COMPARTMENT_OCID" \
          --all \
          --query "data[?displayName=='${ALARM_NAME}']"
        ```

        Get the alarm OCID:

        ```bash theme={null}
        ALARM_OCID="<alarm-ocid>"
        ```

        Update the query to the correct GPU anomaly condition:

        ```bash theme={null}
        oci monitoring alarm update \
          --alarm-id "$ALARM_OCID" \
          --query-text "$QUERY_TEXT" \
          --namespace "$METRIC_NAMESPACE" \
          --metric-compartment-id "$COMPARTMENT_OCID" \
          --is-enabled true
        ```

        ***

        If you paste your actual `metric list` output (for GPU) and your tenancy’s region, I can give you the exact `QUERY_TEXT` line you should use.
      </Accordion>

      <Accordion title="Using Python">
        Below is a practical way to remediate this in OCI using Python by:

        1. Making sure GPU metrics are available.
        2. Creating a Monitoring Alarm that fires on anomalous GPU usage.

        ***

        ## 1. Ensure GPU metrics are available

        1. Use a GPU shape (e.g. `VM.GPU3.1`, `BM.GPU4.8`, etc.).
        2. Ensure the **Oracle Cloud Agent** is enabled on the instance.
           * In OCI Console:\
             Compute → Instances → your instance → Oracle Cloud Agent → ensure it’s enabled and monitoring plugins are running.
        3. Confirm GPU metrics exist:
           * Monitoring → Metrics Explorer
           * Namespace: usually `oci_computeagent`
           * Look for metrics like `GpuUtilization`, `GpuMemoryUtilization`, etc.

        Once metrics are visible, you can create an alarm programmatically.

        ***

        ## 2. Concept for “anomalous” GPU usage

        OCI doesn’t have a built‑in anomaly engine on GPU metrics. A practical approach for “anomalous GPU usage” is:

        * Alarm if **GPU utilization is too high** for too long (e.g. potential abuse / runaway job).
        * Optionally add another alarm if utilization is **too low** during expected workload hours.

        Example high-usage rule:

        ```text theme={null}
        GpuUtilization[1m]{resourceId = "ocid1.instance.oc1..xxxx"} > 90
        ```

        ***

        ## 3. Python setup

        Install and configure the OCI Python SDK:

        ```bash theme={null}
        pip install oci
        ```

        Configure credentials (config file in `~/.oci/config`):

        ```ini theme={null}
        [DEFAULT]
        user=ocid1.user.oc1..xxxxx
        fingerprint=aa:bb:cc:dd:...
        key_file=/path/to/oci_api_key.pem
        tenancy=ocid1.tenancy.oc1..xxxxx
        region=us-phoenix-1
        ```

        ***

        ## 4. Python code to create a GPU anomaly alarm

        This example creates a **high GPU utilization alarm** on a specific instance.

        ```python theme={null}
        import oci
        from oci.monitoring.models import CreateAlarmDetails

        # 1. Load config
        config = oci.config.from_file("~/.oci/config", "DEFAULT")

        # 2. Clients
        monitoring_client = oci.monitoring.MonitoringClient(config)
        notifications_client = oci.ons.NotificationControlPlaneClient(config)

        # 3. Variables – UPDATE these for your environment
        compartment_id = "ocid1.compartment.oc1..xxxx"
        gpu_instance_id = "ocid1.instance.oc1.phx.xxxxx"
        topic_id = "ocid1.onstopic.oc1.phx.xxxxx"   # OCI Notifications topic OCID for email/Slack/etc.
        display_name = "Anomalous GPU Usage Alarm"
        severity = "CRITICAL"
        # Namespace for metrics coming from the Compute Agent:
        metric_namespace = "oci_computeagent"
        metric_name = "GpuUtilization"

        # 4. Define the alarm query
        # Example: GPU utilization > 90% for 5 out of 5 minutes
        # Adjust threshold and duration to your needs.
        query = (
            f"{metric_name}[1m]{{resourceId = \"{gpu_instance_id}\"}}.mean() > 90"
        )

        alarm_details = CreateAlarmDetails(
            display_name=display_name,
            compartment_id=compartment_id,
            metric_compartment_id=compartment_id,
            namespace=metric_namespace,
            query=query,
            severity=severity,
            destinations=[topic_id],
            is_enabled=True,
            # alarm conditions
            pending_duration="PT5M",       # condition must hold for 5 minutes
            resolution="PT1M",             # evaluate every 1 minute
            body="High GPU usage detected on instance.",
            message_format="TEXT",
            repeat_notification_duration="PT30M",  # re-notify every 30 minutes if still firing
        )

        # 5. Create the alarm
        response = monitoring_client.create_alarm(alarm_details)
        print("Created alarm OCID:", response.data.id)
        ```

        ***

        ## 5. Optional: “Low GPU usage” (expected workload hours)

        If “anomalous” for you means “GPU is unexpectedly idle during work”, you can add another alarm:

        ```python theme={null}
        low_gpu_query = (
            f"{metric_name}[5m]{{resourceId = \"{gpu_instance_id}\"}}.mean() < 10"
        )

        low_usage_alarm = CreateAlarmDetails(
            display_name="Low GPU Usage During Expected Hours",
            compartment_id=compartment_id,
            metric_compartment_id=compartment_id,
            namespace=metric_namespace,
            query=low_gpu_query,
            severity="WARNING",
            destinations=[topic_id],
            is_enabled=True,
            pending_duration="PT15M",
            resolution="PT5M",
            body="Low GPU usage detected during expected workload period.",
            message_format="TEXT",
        )

        resp = monitoring_client.create_alarm(low_usage_alarm)
        print("Created low-usage alarm OCID:", resp.data.id)
        ```

        You can further filter by `availabilityDomain`, `displayName`, or custom tags in the query if you want one alarm for a group of GPU instances instead of a single `resourceId`.

        ***

        If you share your exact GPU metrics (namespace/metric names as seen in Metrics Explorer) and how you define “anomaly” (thresholds, times), I can adjust the query and Python code precisely.
      </Accordion>

      <Accordion title="Using Terraform">
        ```hcl theme={null}
        resource "oci_monitoring_alarm" "anomalous_gpu_usage_alarm" {
          # OCID of the compartment where GPU instances run
          compartment_id = "OCI_COMPARTMENT_OCID"

          # OCID of the compartment that owns the metric (often same as compartment_id)
          metric_compartment_id = "OCI_METRIC_COMPARTMENT_OCID"

          display_name = "anomalous-gpu-usage"

          # Example query: alert if average GPU utilization exceeds a threshold over 5 minutes.
          # Replace GPU_METRIC_NAME and DIMENSION_FILTERS to match your GPU metric/shape filters,
          # and GPU_UTILIZATION_THRESHOLD with the numeric threshold you require.
          #
          # Example with a common pattern (adjust as needed for your tenancy):
          #   "GpuUtilization[5m]{shape = \"VM.GPU*\"}.mean() > 80"
          query = "GPU_METRIC_NAME[5m]{DIMENSION_FILTERS}.mean() > GPU_UTILIZATION_THRESHOLD"

          # How long the condition must be met before triggering (e.g., "5m", "10m")
          pending_duration = "5m"

          # Notification topic(s) for the alarm
          destinations = [
            "OCI_NOTIFICATIONS_TOPIC_OCID",
          ]

          severity = "CRITICAL"

          is_enabled  = true
          is_suppressed = false

          # Optional: human-readable description
          body = "Alarm for anomalous GPU usage: triggered when GPU utilization breaches the configured threshold."

          # Optional: keep notifications per metric dimension separate
          is_notifications_per_metric_dimension_enabled = true

          # Optional: how often to repeat notifications while the alarm is firing
          repeat_notification_duration = "60m"

          message_format = "PRETTY_JSON"
        }
        ```

        Changing this alarm’s query, thresholds, or destinations is an in-place update and does not force replacement of other resources (no outage risk beyond normal alarm behavior changes).

        To verify, `terraform plan` should show either creation of `oci_monitoring_alarm.anomalous_gpu_usage_alarm` or an in-place update (`~`) of the existing alarm with the new `query` (and any other changed fields).
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>