Shard

Shard CRD schema reference (group operator.kcp.io)

Shard is the Schema for the shards API

Full name:: shards.operator.kcp.io
Group:: operator.kcp.io
Singular name:: shard
Plural name:: shards
Scope:: Namespaced
Versions:: v1alpha1

Version v1alpha1

Example CR

apiVersion: operator.kcp.io/v1alpha1
kind: Shard
metadata:
  labels:
    app.kubernetes.io/name: kcp-operator
    app.kubernetes.io/managed-by: kustomize
  name: secondary-shard
spec:
  etcd:
    endpoints:
      - http://etcd-shard.default.svc.cluster.local:2379
  rootShard:
    ref:
      name: shard-sample
  auth:
    serviceAccount:
      # if you want controllers to access particular workspaces using service accounts
      # serviceAccounts must be enabled, on all shards and the front-proxy.
      enabled: true
  logLevel:
    verbosityLevel: 4
  deploymentTemplate:
    spec:
      template:
        spec:
          hostAliases:
            # add a hardcoded DNS override to the same IP as in v1alpha1_frontproxy.yaml.
            - ip: "10.96.100.100"
              hostnames:
                - "example.operator.kcp.io"

Properties

.apiVersion

string

object

ConfigMap is a reference to the ConfigMap containing the audit policy file which is mounted into the container.

.spec.audit.policy.configMap.key

string Required

Key in the data.

.spec.audit.policy.configMap.name

string Required

Name of the object.

array

Optional: DropGroups configures groups to be dropped before forwarding requests to Shards.

.spec.auth.dropGroups[*]

string

.spec.auth.oidc

object

Optional: OIDC configures OpenID Connect Authentication.

.spec.auth.oidc.caFileRef

object

Optionally provides a reference to a secret that contains a CA bundle for the OIDC issuer. This is useful when the OIDC issuer is not publicly trusted.

.spec.auth.oidc.caFileRef.key

string

Key is the key in the secret that contains the CA file. Defaults to “ca.crt”.

.spec.auth.oidc.caFileRef.name

string Required

Name is the name of the secret that contains the CA file.

.spec.auth.oidc.clientID

string Required

ClientID is the OIDC client ID configured on the issuer side for this kcp instance.

.spec.auth.oidc.clientSecret

string

array

Optional: PassOnGroups configures groups to be passed on before forwarding requests to Shards

string

The API version of the authentication.k8s.io TokenReview to send to and expect from the webhook.

.spec.authorization

object

.spec.authorization.allowPaths

array

A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the ‘core’ kubernetes server. If specified, completely overwrites the default of [/healthz,/readyz,/livez].

.spec.authorization.allowPaths[*]

string

.spec.authorization.order

array

A list of authorizers that should be enabled, allowing administrator rearrange the default order. The default order is: [AlwaysAllowGroups,AlwaysAllowPaths,RBAC,Webhook]

.spec.authorization.order[*]

string

.spec.authorization.webhook

object

Authorization Webhook configuration.

.spec.authorization.webhook.allowPaths

array

A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the ‘core’ kubernetes server. If specified, completely overwrites the default of [/healthz,/readyz,/livez].

Deprecated: Use spec.authorization.allowPaths instead. This field is kept for backward compatibility but can not be used at the same time as spec.authorization.allowPaths.

object

.spec.deploymentTemplate.metadata.annotations

object

object

Labels is a key value map to be copied to the Pod.

array

An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[].preference.matchExpressions[].values[*]

string

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[*].preference.matchFields

array

A list of node selector requirements by node’s fields.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[].preference.matchFields[]

object

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[].preference.matchFields[].key

string Required

The label key that the selector applies to.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[].preference.matchFields[].operator

string Required

Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution[].preference.matchFields[].values

array

An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.

array

An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[].matchExpressions[].values[*]

string

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[*].matchFields

array

A list of node selector requirements by node’s fields.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[].matchFields[]

object

A node selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[].matchFields[].key

string Required

The label key that the selector applies to.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[].matchFields[].operator

string Required

Represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.

.spec.deploymentTemplate.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[].matchFields[].values

array

An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.

string Required

operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.

.spec.deploymentTemplate.spec.template.spec.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[].podAffinityTerm.labelSelector.matchExpressions[].values

array

values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.

.spec.deploymentTemplate.spec.template.spec.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution[].podAffinityTerm.namespaceSelector.matchExpressions[].key

.spec.deploymentTemplate.spec.template.spec.affinity.podAffinity.requiredDuringSchedulingIgnoredDuringExecution[].namespaceSelector.matchExpressions[].key

.spec.deploymentTemplate.spec.template.spec.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[].podAffinityTerm.namespaceSelector.matchExpressions[].key

.spec.deploymentTemplate.spec.template.spec.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[].namespaceSelector.matchExpressions[].key

.spec.deploymentTemplate.spec.template.spec.hostAliases[*]

object

HostAlias holds the mapping between IP and hostnames that will be injected as an entry in the pod’s hosts file.

.spec.deploymentTemplate.spec.template.spec.hostAliases[*].hostnames

array

Hostnames for the above IP address.

array

values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[].labelSelector.matchExpressions[].values[*]

string

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].labelSelector.matchLabels

object

matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is “key”, the operator is “In”, and the values array contains only “value”. The requirements are ANDed.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].matchLabelKeys

array

MatchLabelKeys is a set of pod label keys to select the pods over which spreading will be calculated. The keys are used to lookup values from the incoming pod labels, those key-value labels are ANDed with labelSelector to select the group of existing pods over which spreading will be calculated for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. MatchLabelKeys cannot be set when LabelSelector isn’t set. Keys that don’t exist in the incoming pod labels will be ignored. A null or empty list means only match against labelSelector.

This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default).

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[].matchLabelKeys[]

string

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].maxSkew

integer Required

MaxSkew describes the degree to which pods may be unevenly distributed. When whenUnsatisfiable=DoNotSchedule, it is the maximum permitted difference between the number of matching pods in the target topology and the global minimum. The global minimum is the minimum number of matching pods in an eligible domain or zero if the number of eligible domains is less than MinDomains. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 2/2/1: In this case, the global minimum is 1. | zone1 | zone2 | zone3 | | P P | P P | P | - if MaxSkew is 1, incoming pod can only be scheduled to zone3 to become 2/2/2; scheduling it onto zone1(zone2) would make the ActualSkew(3-1) on zone1(zone2) violate MaxSkew(1). - if MaxSkew is 2, incoming pod can be scheduled onto any zone. When whenUnsatisfiable=ScheduleAnyway, it is used to give higher precedence to topologies that satisfy it. It’s a required field. Default value is 1 and 0 is not allowed.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].minDomains

integer

MinDomains indicates a minimum number of eligible domains. When the number of eligible domains with matching topology keys is less than minDomains, Pod Topology Spread treats “global minimum” as 0, and then the calculation of Skew is performed. And when the number of eligible domains with matching topology keys equals or greater than minDomains, this value has no effect on scheduling. As a result, when the number of eligible domains is less than minDomains, scheduler won’t schedule more than maxSkew Pods to those domains. If value is nil, the constraint behaves as if MinDomains is equal to 1. Valid values are integers greater than 0. When value is not nil, WhenUnsatisfiable must be DoNotSchedule.

For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same labelSelector spread as 2/2/2: | zone1 | zone2 | zone3 | | P P | P P | P P | The number of domains is less than 5(MinDomains), so “global minimum” is treated as 0. In this situation, new pod with the same labelSelector cannot be scheduled, because computed skew will be 3(3 - 0) if new Pod is scheduled to any of the three zones, it will violate MaxSkew.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].nodeAffinityPolicy

string

NodeAffinityPolicy indicates how we will treat Pod’s nodeAffinity/nodeSelector when calculating pod topology spread skew. Options are: - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations.

If this value is nil, the behavior is equivalent to the Honor policy. This is a beta-level feature default enabled by the NodeInclusionPolicyInPodTopologySpread feature flag.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].nodeTaintsPolicy

string

NodeTaintsPolicy indicates how we will treat node taints when calculating pod topology spread skew. Options are: - Honor: nodes without taints, along with tainted nodes for which the incoming pod has a toleration, are included. - Ignore: node taints are ignored. All nodes are included.

If this value is nil, the behavior is equivalent to the Ignore policy. This is a beta-level feature default enabled by the NodeInclusionPolicyInPodTopologySpread feature flag.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].topologyKey

string Required

TopologyKey is the key of node labels. Nodes that have a label with this key and identical values are considered to be in the same topology. We consider each as a “bucket”, and try to put balanced number of pods into each bucket. We define a domain as a particular instance of a topology. Also, we define an eligible domain as a domain whose nodes meet the requirements of nodeAffinityPolicy and nodeTaintsPolicy. e.g. If TopologyKey is “kubernetes.io/hostname”, each Node is a domain of that topology. And, if TopologyKey is “topology.kubernetes.io/zone”, each zone is a domain of that topology. It’s a required field.

.spec.deploymentTemplate.spec.template.spec.topologySpreadConstraints[*].whenUnsatisfiable

string Required

WhenUnsatisfiable indicates how to deal with a pod if it doesn’t satisfy the spread constraint. - DoNotSchedule (default) tells the scheduler not to schedule it. - ScheduleAnyway tells the scheduler to schedule the pod in any location, but giving higher precedence to topologies that would help reduce the skew. A constraint is considered “Unsatisfiable” for an incoming pod if and only if every possible node assignment for that pod would violate “MaxSkew” on some topology. For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same labelSelector spread as 3/1/1: | zone1 | zone2 | zone3 | | P P P | P | P | If WhenUnsatisfiable is set to DoNotSchedule, incoming pod can only be scheduled to zone2(zone3) to become 3/2/1(3/1/2) as ActualSkew(2-1) on zone2(zone3) satisfies MaxSkew(1). In other words, the cluster can still be imbalanced, but scheduler won’t make it more imbalanced. It’s a required field.

.spec.etcd

object Required

Etcd configures the etcd cluster that this shard should be using.

.spec.etcd.endpoints

array Required

Endpoints is a list of http urls at which etcd nodes are available. The expected format is “https://etcd-hostname:2379”.

string

.spec.extraVolumeMounts

array

Optional: ExtraVolumeMounts defines additional volume mounts that should be added to the shard container. Each entry’s name must match the name of a volume defined in ExtraVolumes (or one of the volumes the operator manages).

.spec.extraVolumeMounts[*]

object

VolumeMount describes a mounting of a Volume within a container.

.spec.extraVolumeMounts[*].mountPath

string Required

Path within the container at which the volume should be mounted. Must not contain ‘:’.

.spec.extraVolumeMounts[*].mountPropagation

string

mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10. When RecursiveReadOnly is set to IfPossible or to Enabled, MountPropagation must be None or unspecified (which defaults to None).

.spec.extraVolumeMounts[*].name

string Required

This must match the Name of a Volume.

.spec.extraVolumeMounts[*].readOnly

boolean

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.

.spec.extraVolumeMounts[*].recursiveReadOnly

string

.spec.extraVolumes[].downwardAPI.items[].resourceFieldRef.resource

string Required

Required: resource to select

.spec.extraVolumes[*].emptyDir

object

emptyDir represents a temporary directory that shares a pod’s lifetime. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir

.spec.extraVolumes[*].emptyDir.medium

string

medium represents what type of storage medium should back this directory. The default is “” which means to use the node’s default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir

.spec.extraVolumes[*].emptyDir.sizeLimit

sizeLimit is the total amount of local storage required for this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. The default is nil which means that the limit is undefined. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir

.spec.extraVolumes[*].ephemeral

object

ephemeral represents a volume that is handled by a cluster storage driver. The volume’s lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed.

Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim).

Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod.

Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information.

A pod can use both types of ephemeral volumes and persistent volumes at the same time.

.spec.extraVolumes[*].ephemeral.volumeClaimTemplate

object

Will be used to create a stand-alone PVC to provision the volume. The pod in which this EphemeralVolumeSource is embedded will be the owner of the PVC, i.e. the PVC will be deleted together with the pod. The name of the PVC will be <pod name>-<volume name> where <volume name> is the name from the PodSpec.Volumes array entry. Pod validation will reject the pod if the concatenated name is not valid for a PVC (for example, too long).

An existing PVC with that name that is not owned by the pod will not be used for the pod to avoid using an unrelated volume by mistake. Starting the pod is then blocked until the unrelated PVC is removed. If such a pre-created PVC is meant to be used by the pod, the PVC has to updated with an owner reference to the pod once the pod exists. Normally this should not be necessary, but it may be useful when manually reconstructing a broken cluster.

This field is read-only and no changes will be made by Kubernetes to the PVC after it has been created.

Required, must not be nil.

.spec.extraVolumes[*].ephemeral.volumeClaimTemplate.metadata

object

May contain labels and annotations that will be copied into the PVC when creating it. No other fields are allowed and will be rejected during validation.

.spec.extraVolumes[*].ephemeral.volumeClaimTemplate.spec

object Required

The specification for the PersistentVolumeClaim. The entire content is copied unchanged into the PVC that gets created from this template. The same fields as in a PersistentVolumeClaim are also valid here.

.spec.extraVolumes[*].ephemeral.volumeClaimTemplate.spec.accessModes

array

accessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1

array

values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.

array

targetWWNs is Optional: FC target worldwide names (WWNs)

.spec.extraVolumes[].fc.targetWWNs[]

string

.spec.extraVolumes[*].fc.wwids

array

wwids Optional: FC volume world wide identifiers (wwids) Either wwids or combination of targetWWNs and lun must be set, but not both simultaneously.

object

image represents an OCI object (a container image or artifact) pulled and mounted on the kubelet’s host machine. The volume is resolved at pod startup depending on which PullPolicy value is provided:

Always: the kubelet always attempts to pull the reference. Container creation will fail If the pull fails.
Never: the kubelet never pulls the reference and only uses a local image or artifact. Container creation will fail if the reference isn’t present.
IfNotPresent: the kubelet pulls if the reference isn’t already present on disk. Container creation will fail if the reference isn’t present and the pull fails.

object

ClusterTrustBundle allows a pod to access the .spec.trustBundle field of ClusterTrustBundle objects in an auto-updating file.

Alpha, gated by the ClusterTrustBundleProjection feature gate.

ClusterTrustBundle objects can either be selected by name, or by the combination of signer name and a label selector.

Kubelet performs aggressive normalization of the PEM contents written into the pod filesystem. Esoteric PEM features such as inter-block comments and block headers are stripped. Certificates are deduplicated. The ordering of certificates within the file is arbitrary, and Kubelet may change the order over time.

.spec.extraVolumes[].projected.sources[].clusterTrustBundle.labelSelector

object

Select all ClusterTrustBundles that match this label selector. Only has effect if signerName is set. Mutually-exclusive with name. If unset, interpreted as “match nothing”. If set but empty, interpreted as “match everything”.

.spec.extraVolumes[].projected.sources[].clusterTrustBundle.labelSelector.matchExpressions

array

matchExpressions is a list of label selector requirements. The requirements are ANDed.

.spec.extraVolumes[].projected.sources[].clusterTrustBundle.labelSelector.matchExpressions[*]

object

A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.

.spec.extraVolumes[].projected.sources[].clusterTrustBundle.labelSelector.matchExpressions[*].key

string Required

key is the label key that the selector applies to.

.spec.extraVolumes[].projected.sources[].clusterTrustBundle.labelSelector.matchExpressions[*].operator

string Required

operator represents a key’s relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.

.spec.extraVolumes[].projected.sources[].clusterTrustBundle.labelSelector.matchExpressions[*].values

array

values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.

keyring is the path to key ring for RBDUser. Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it

.spec.extraVolumes[*].rbd.monitors

array Required

monitors is a collection of Ceph monitors. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it

integer

Level sets the verbosity level for the component. Higher values mean more verbose logging. This corresponds to the -v flag in kcp components.

.spec.replicas

integer

Replicas configures how many instances of this shard run in parallel. Defaults to 2 if not set.

.spec.resources

object

Resources overrides the default resource requests and limits.

.spec.resources.claims

array

Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container.

Labels is a key value map to be copied to the target Kubernetes Service.

.spec.serviceTemplate.spec

object

.spec.serviceTemplate.spec.clusterIP

string

.spec.serviceTemplate.spec.type

string

Service Type string describes ingress methods for a service

.spec.shardBaseURL

string

ShardBaseURL is the base URL under which this shard should be reachable. This is used to configure the external URL. If not provided, the operator will use kubernetes service address to generate it.

.status

object

ShardStatus defines the observed state of Shard

string