Information about Nutanix Kubernetes Platform (Part 3 of many)

The Nutanix Kubernetes Platform (or NKP) allows you to configure one or more infrastructure providers. Of course you are able to configure Nutanix, but you are also able to configure AWS, GCP and AWS.Even EKS and AKS are configurable by you.

In this post we will discuss universal configurations as well as Nutanix specific configurations.

As discussed in part 2 of these serie there are some prerequisites , as well for NKP, as well for the infrastructure. You can read about them on the Nutanix support portal.

Prism Central credentials, roles and permissions

You need Prism Central credentials for Nutanix Infrastructure and NKPfunctionality. These credentials are used for:

• To manage the cluster, such as listing subnets and other infrastructure, and to create VMs in Prism Central, which the Cluster API Provider Nutanix Cloud Infrastructure (CAPX) infrastructure provider uses.
• To manage persistent storage used by Nutanix Container Storage Interface (CSI) providers.
• To discover node metadata used by the Nutanix Cloud Cost Management (CCM) provider.

The credentials are required to authenticate with Prism Central API’s. At the moment, CAPX supports two mechanisms for supplying the credentials. You can inject the credentials into the CAPX manager deployment or you can use workload specific credentials.

When provisioning Kubernetes clusters with NKP on Nutanix infrastructure, you need a pre-defined role that contains the minimum permissions needed for NKP to provide proper access to deploy clusters. You can assign a pre-defined role by creating an authorization policy. Prism Central provides pre-defined permissions for the Kubernetes Infrastructure Provisions role.

The Kubernetes Infrastructure Provisions role has the following permissions. See also this link.

Infrastructure Role	Permission Granted
AHV VM	Create Virtual Machine Create a Virtual Machine Disk Delete Virtual Machine Delete Virtual Machine Disk Update Virtual Machine Update Virtual Machine Project View Virtual Machine
Category	Create Or Update Name Category Create Or Update Value Category Delete Name Category Delete Value Category View Name Category View Value Category
Category Mapping	Create Category Mapping Delete Category Mapping Update Category Mapping View Category Mapping
Cluster	View Cluster Create Image Delete Image View Image
Project	View Project
Subnet	View Subnet

Prism Central Resources

For cluster creation, you will need Prism Central resources such as subnet IPs and a storage container. When creating a storage container, ensure that your storage container name matches the name of the cluster that you plan to deploy.

Migrating VMs from VLAN to OVN

Migrating VMs from VLAN basic to OVN VLAN is not done through atlas_cli, which is recommended by other projects in Nutanix.

The existing VLAN implementation is basic VLAN. However, advanced VLAN uses OVN as the control plane instead of Acropolis. The subnet creation workflow is from Prism Central rather than Prism Element. Subnet creation can be done using APIs or through the UI.

Nutanix Base OS Image

The base OS image is used by NKP Image Builder (NIB) to create a custom image. When you create a base OS image, you can:

• Create your custom image for Rocky Linux 9.4 or Ubuntu 22.04
• Use the pre-built Rocky Linux 9.4 image downloaded from the portal. See the screenshot below. (Also a NKP Node OS Image (Rocky Linux) for AHV is available.)

Any NKP license tier can use this pre-built Rocky Linux. Only with Starter you have to use this image.

Universal configurations

Universal configurations for NKP are shared amongst all infrastructure providers and are related to environment variables, flags for cluster creation, local registries, and so on. See the table of some universal configurations.

Name of configuration	Description
Container Runtime Engine (CRE)	A container runtime engine is required to install NKP and a bootstrap cluster. NKP supports both Podman and Docker container engines for cluster creation.
HTTP or HTTPS Proxy	HTTP or HTTPS are proxy value strings that list a set of proxy servers, URLs, or wildcard addresses specific to your environment. When creating an NKP cluster in a proxied environment, you need to specify proxy settings for the following: – Bootstrap cluster – Cluster API components – NKP Kommander components
Output Directory Flag	When creating a cluster, the –output-directory flag can be used to organize the cluster configuration into individual files. Organizing individual files in this manner helps with editing and managing the cluster configuration.
Cluster API Components Customization	The objects in a cluster are custom resources defined by Cluster API (CAPI) components and they belong to three different categories. – Cluster A cluster object references infrastructure-specific and control plane objects. It describes the infrastructure-specific properties that include region, VPC ID, subnet IDs, and security group rules required by the Pod network implementation. – Control Plane A KubeadmControlPlane object describes the control plane, which is the group of machines that run the Kubernetes control plane components. These components include etcd distributed database, API server, core controllers, and scheduler. The object describes the configuration for these components and refers to an infrastructure-specific object that represents the properties of all control plane machines. – Node Pool A node is a physical or virtual server. A node pool is a collection of nodes with identical properties. For example, a cluster might have one node pool with large memory capacity and another with graphics processing unit (GPU) support.
Registry and Registry Mirrors	A registry is a centralized location that stores all images used for Kubernetes applications. – Registry Mirrors Registry mirrors are local copies of images from a public registry that follow (or mirror) the file structure of a public registry. – Container Repositories Container repositories are a collection of related container images1. – Container Registries Container registries are collections of container repositories and can also offer API paths and access rules.
Bastion Host	The bastion VM hosts the installation of the NKP Konvoy bundles, images, and the container runtime engine or other local registry needed to create and operate your cluster.
Load Balancers	In a Kubernetes cluster, depending on the direction of traffic flow, there are two kinds of load balancing: – Internal load-balancing for traffic within a Kubernetes cluster. – External load-balancing for traffic coming from outside the cluster.

1. A container image has everything that the software might need to run, including code, resources, and tools. Container repositories store container images for setup and deployment and allow you to manage, pull, and push images during cluster operations. ↩︎

Infrastructure Providers

Infrastructure providers like Nutanix are used to make resources available to your NKP clusters. As a result, NKP requires provider-specific details to connect, provision, and manage clusters.

As said before, other infrastructure can be used as an infrastructure provider.

To add a Nutanix infrastructure provider, the following steps have to be carried out.

Step 1

Select a workspace from the Workspace drop-down menu and select Infrastructure Providers in the left pane. Then, click +Add Infrastructure Provider.

Note that, depending on your requirement, you can create an infrastructure provider at the Global level or the Workspace level. If you intend to use a single infrastructure provider with multiple workspaces, then add your infrastructure at the Global level. However, if you intend for different workspaces to use different infrastructure providers, add them at the individual workspace level.

Step 2

In the Add Infrastructure Provider window, select Nutanix as your provider and enter a name in lowercase (uppercase is not supported), Prism Central endpoint and credentials, and trust bundle. You use the trust bundle to make sure the copnnection to Prism Central is secure and trusted.

If you don’t add a trust bundle, you will be prompted.

Step 3

After the infrastructure provider is added, it will be listed on the Infrastructure Providers page.

Other infrastructure Providers

Adding a Nutanix infrastructure provider is quite simple. In the Nutanix Kubernetes Platform guide you will find some more information about adding other infrastructure providers. Also this additions to NKP are quite easy to manage.

• Adding an AWS Infrastructure provider with a User Role
• Adding an Azure Infrastructure provider
• Adding a vSphere Infrastructure Provider

Using Workspaces

Workspaces are a collection of Clusters that maintain a similar configuration. Certain configuration are automatically federated to those clusters. With workspaces you are able to delegate control to different teams.
Clusters assigned to a workspace will receive the applications and RBAC policies that are applied to the workspace.

When the cluster is deployed, global and default workspaces are created automatically. You can add more morespaces from the Global Workspace in the NKP UI. To do so, select Global from the Workspace drop-down menu, and click Workspaces on the left pane. In the Workspaces window, click + Create Workspace.

In the Create Workspace, you have to enter a name. Specifying an ID / Namespace is optional. It will be automatically generated, but it may be more convenient to use an ID that is easy to remember.
Description and annotations are also optional.

Finally, click create to the workspace to be created.

Deleting a workspace

To delete a workspace, select Global from the Workspace drop-down menu. On the workspaces dashboard, click on the ellipsis to the right of the workspace you want to delete, and click Delete.
You are only able to delete a workspace if all the clusters in the workspace have been deleted or detached.

Next Post will be “Creating and managing Workload Clusters with NKP”. Stay tuned