Skip to content

Audit Logging

kcp extends Kubernetes audit events with workspace-specific annotations that help identify which workspace (logical cluster) each request belongs to. This is essential for multi-tenant environments where you need to track and audit access across different workspaces.

Audit Event Annotations

kcp automatically adds the following annotations to all audit events:

kcp.io/path

The canonical workspace path (e.g., root:consumer:production). This is the human-readable hierarchical path that uniquely identifies the workspace.

tenancy.kcp.io/workspace

The workspace cluster name (logical cluster identifier). This is the internal cluster name used by kcp.

kcp.io/cluster

Alias for the workspace cluster name. Contains the same value as tenancy.kcp.io/workspace.

Example Audit Event

{
  "kind": "Event",
  "apiVersion": "audit.k8s.io/v1",
  "level": "Request",
  "auditID": "5684337a-48d6-4491-aed2-a0bca6fcda2b",
  "stage": "RequestReceived",
  "requestURI": "/api/v1/namespaces/default/configmaps?limit=500",
  "verb": "list",
  "user": {
    "username": "kcp-admin",
    "uid": "e6741a4d-fc7c-44c5-b5ec-9357b44b7e0b",
    "groups": [
      "system:kcp:admin",
      "system:authenticated"
    ]
  },
  "sourceIPs": [
    "127.0.0.1"
  ],
  "userAgent": "kubectl/v1.30.1 (darwin/arm64) kubernetes/6911225",
  "objectRef": {
    "resource": "configmaps",
    "namespace": "default",
    "apiVersion": "v1"
  },
  "requestReceivedTimestamp": "2025-12-09T16:03:44.214758Z",
  "stageTimestamp": "2025-12-09T16:03:44.214758Z",
  "annotations": {
    "kcp.io/cluster": "16iai06e7ob717ht",
    "kcp.io/path": "root:consumer:production",
    "tenancy.kcp.io/workspace": "16iai06e7ob717ht"
  }
}

The example audit event above would be generated by running the following command:

kubectl --server=https://127.0.0.1:6443/clusters/root:consumer:production get configmap -n default

This command lists configmaps in the default namespace within the root:consumer:production workspace, which triggers the audit event with the workspace annotations shown in the example.

Cross-Workspace Audit Logging

The workspace path annotations (kcp.io/path, tenancy.kcp.io/workspace) are especially important when accessing resources across workspaces via APIExport and APIBinding. When you claim resources from another workspace through an APIBinding, audit events are generated for the consumer workspace, allowing you to track which workspace is accessing which resources.

Setting Up Cross-Workspace Event Logging

This example demonstrates how to set up cross-workspace access and verify that audit logs include workspace path annotations.

Step 1: Create Workspaces

Create a provider workspace to export APIs and a consumer workspace to consume them:

# Create provider workspace
kubectl ws create root:monitoring

# Create consumer workspace
kubectl ws create root:team-a-production

Step 2: Create APIResourceSchema and APIExport

In the provider workspace, create an APIResourceSchema and APIExport:

# Switch to provider workspace
kubectl ws root:monitoring

# Create APIResourceSchema
cat <<EOF | kubectl apply -f -
apiVersion: apis.kcp.io/v1alpha1
kind: APIResourceSchema
metadata:
  name: v1.monitoringevents.monitoring.kcp.io
spec:
  group: monitoring.kcp.io
  names:
    kind: MonitoringEvent
    plural: monitoringevents
  versions:
    - name: v1
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
      served: true
      storage: true
EOF

# Create APIExport with permission claim for events
cat <<EOF | kubectl apply -f -
apiVersion: apis.kcp.io/v1alpha1
kind: APIExport
metadata:
  name: events-monitoring
spec:
  latestResourceSchemas:
    - v1.monitoringevents.monitoring.kcp.io
  permissionClaims:
    - group: ""
      resource: "events"
      identityHash: "events"
      verbs: ["get", "list", "watch"]
EOF

Step 3: Create APIBinding in Consumer Workspace

Switch to the consumer workspace and create an APIBinding:

# Switch to consumer workspace
kubectl ws root:team-a-production

# Create APIBinding
cat <<EOF | kubectl apply -f -
apiVersion: apis.kcp.io/v1alpha1
kind: APIBinding
metadata:
  name: events-monitoring
spec:
  reference:
    workspace:
      path: root:monitoring
      exportName: events-monitoring
  permissionClaims:
    - group: ""
      resource: "events"
      identityHash: "events"
      verbs: ["get", "list", "watch"]
      state: Accepted
EOF

Step 4: Configure Audit Policy

Create an audit policy that captures cross-workspace requests. The policy should include rules for the resources you want to audit:

apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  - level: Request
    resources:
      - group: "monitoring.kcp.io"
        resources: ["monitoringevents"]
      - group: ""
        resources: ["events"]

Start kcp with audit logging enabled:

kcp start \
  --audit-policy-file=./audit-policy.yaml \
  --audit-log-path=./kcp.audit \
  --audit-log-format=json

Step 5: Access Resources via Virtual Workspace

Access resources from the consumer workspace using the virtual workspace endpoint:

# Get cluster names from Workspace resources
# Switch to root workspace to query child workspaces
kubectl ws :root
PROVIDER_CLUSTER=$(kubectl get workspace monitoring -o jsonpath='{.spec.cluster}')
CONSUMER_CLUSTER=$(kubectl get workspace team-a-production -o jsonpath='{.spec.cluster}')

# Access via virtual workspace endpoint
# Replace <kcp-server> with your kcp server address (e.g., 192.168.1.230:6443)
kubectl --server=https://<kcp-server>/services/apiexport/${PROVIDER_CLUSTER}/events-monitoring/clusters/${CONSUMER_CLUSTER} \
  get monitoringevents

Example:s59r6w07v3evzwjw

# Set variables for cluster names from Workspace resources
# Switch to root workspace to query child workspaces
kubectl ws :root
CLUSTER_NAME=$(kubectl get workspace monitoring -o jsonpath='{.spec.cluster}')
CONSUMER_CLUSTER=$(kubectl get workspace team-a-production -o jsonpath='{.spec.cluster}')

# Access monitoring events via virtual workspace
kubectl --server=https://192.168.1.230:6443/services/apiexport/${CLUSTER_NAME}/events-monitoring/clusters/${CONSUMER_CLUSTER} \
  get monitoringevents

Note: events-monitoring in the URL path is the name of the APIExport created in Step 2.

Step 6: Verify Audit Logs

Check the audit log file to verify that events include workspace path annotations:

# View recent audit events with workspace annotations
cat kcp.audit | jq -r 'select(.annotations."kcp.io/path" != null) | {
  timestamp: .requestReceivedTimestamp,
  verb: .verb,
  resource: .objectRef.resource,
  workspace_path: .annotations."kcp.io/path",
  workspace_cluster: .annotations."tenancy.kcp.io/workspace",
  user: .user.username
}'

Example output showing cross-workspace access:

{
  "kind": "Event",
  "apiVersion": "audit.k8s.io/v1",
  "level": "Request",
  "auditID": "d71ac943-774d-495c-a605-99acc117974f",
  "stage": "ResponseComplete",
  "requestURI": "/apis/monitoring.kcp.io/v1/monitoringevents?limit=500",
  "verb": "list",
  "user": {
    "username": "kcp-admin",
    "uid": "9d8d8c2e-06e6-4ac5-a093-bdd53519a890",
    "groups": ["system:kcp:admin", "system:authenticated"]
  },
  "objectRef": {
    "resource": "monitoringevents",
    "apiGroup": "monitoring.kcp.io",
    "apiVersion": "v1"
  },
  "annotations": {
    "kcp.io/cluster": "dnwjm29bmxcaz9x7",
    "kcp.io/path": "root:team-a-production",
    "tenancy.kcp.io/workspace": "dnwjm29bmxcaz9x7"
  }
}

Notice that the kcp.io/path annotation shows root:team-a-production (the consumer workspace), even though the resource is being accessed from the provider workspace's APIExport. This allows you to track which workspace is consuming which resources.

Enabling Audit Logging

To enable audit logging in kcp, you need to provide an audit policy file and configure where audit logs should be written. Audit logging is disabled by default.

Basic Setup

Enable audit logging by starting kcp with the following flags:

kcp start \
  --audit-policy-file=/path/to/audit-policy.yaml \
  --audit-log-path=/path/to/kcp.audit \
  --audit-log-format=json

Required Flags

  • --audit-policy-file: Path to the audit policy YAML file that defines which events should be logged
  • --audit-log-path: Path where audit logs will be written
  • --audit-log-format: Output format (json or legacy). JSON format is recommended for easier parsing

Example Audit Policy

Create an audit policy file (e.g., audit-policy.yaml) to define what events to log:

apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  - level: Request
    verbs: ["*"]

This example policy logs all requests at the Request level. For production environments, you may want to be more selective about what gets logged to manage log volume.

Complete Example

Here's a complete example of starting kcp with audit logging enabled:

# Create audit policy file
cat > audit-policy.yaml <<EOF
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
  - level: Request
    verbs: ["*"]
    resources:
      - group: ""
        resources: ["*"]
EOF

# Start kcp with audit logging
kcp start \
  --audit-policy-file=./audit-policy.yaml \
  --audit-log-path=./kcp.audit \
  --audit-log-format=json

Configuration

Audit logging in kcp uses the standard Kubernetes audit policy configuration. For detailed information on configuring audit policies, including different log levels and filtering options, see the Kubernetes audit documentation.

Notes

  • The kcp.io/path annotation is only populated when the LogicalCluster informer is available. In cache-server deployments, this annotation may be empty.
  • All annotations are added to both RequestReceived and ResponseComplete audit event stages.
  • The workspace annotations are automatically added to all audit events, regardless of the audit policy configuration.