Deploying a vSphere Kubernetes Service (VKS) Supervisor with Ansible |

Table of Contents

Today’s blog post will focus on how to deploy a vSphere Kubernetes Service Supervisor Cluster using Infrastructure-as-Code (IaC) tools.

First, we will briefly compare Terraform and Ansible in this context, and then we will explore the actual automation workflow implemented with Ansible.

Terraform Considerations

Terraform is widely used for declarative infrastructure management, but certain limitations exist when working with vSphere Supervisor Clusters.

For example, the Tier-0 Gateway, a required parameter for a Supervisor Cluster, is not explicitly configurable in the Terraform provider. Testing environments with multiple Tier-0 Gateways showed that Terraform could not determine which gateway to use, causing deployment failures.

This highlights that while Terraform is powerful for many scenarios, it may not be the best choice when fine-grained control over vCenter or NSX configurations is required.

Implementing Automation with Ansible

To overcome these limitations, the deployment workflow was implemented with Ansible, which allows direct API interactions and full control over the Supervisor Cluster creation process.

Before creating playbooks, API calls against vCenter were tested using the vCenter Developer Center, confirming reliable communication with the vCenter API and providing a solid foundation for automation.

Ansible Playbooks

Two key Ansible playbooks were developed:

One for deploying the Supervisor Cluster - createSV.yaml
Another for creating the Namespaces within that cluster - createNS.yaml (loops task_seq_ns.yaml)

Both Playbooks collect their data from a supervisor manifest file: supervisor.yaml

supervisor.yaml

apiVersion: v1
kind: vSphereSupervisorCluster
metadata: 
  name: 'VKS-Cluster-01'
  vsphere_url: 'https://vcenter.example.com'
  vsphere_username: '<base64 encoded vsphere user>'
  vsphere_password: '<base64 encoded vsphere password>'
spec:
  clusterName: 'cluster-01'
  storagePolicyName: 'Storage Policy'
  tierZeroName: 'example-tier-0'
  sizeHint: 'SMALL'
  networks:
    masterManagementNetwork:
      addressRange:
        subnetMask: '255.255.255.0'
        startingAddress: '10.0.0.10'
        gateway: '10.0.0.1'
        addressCount: 5
      network: 'network-1'
    pods:
      address: '10.60.0.0'
      prefix: '20'
    services:
      address: '10.70.0.0'
      prefix: '24'
    ingress:
      address: '10.80.0.0'
      prefix: '24'
    egress:
      address: '10.90.0.0'
      prefix: '24'
    masterNTPServers: 
      - '10.0.0.1'
    masterDNS:
      - '10.0.0.1'
    workerDNS:
      - '10.0.0.1'
    networkProvider: 'NSXT_CONTAINER_PLUGIN'
  namespaces:
    - name: 'namespace-1'
      description: 'ansible-created-namespeace'
      type: 'USER' #USER or GROUP
      user: 'user-1'
      domain: 'vsphere.local'
      role: 'EDIT' #EDIT or VIEW
      storageLimit: '10240'
    - name: 'namespace-2'
      description: 'ansible-created-namespeace'
      type: 'user-2'
      user: 'administrator'
      domain: 'vsphere.local'
      role: 'EDIT' #EDIT or VIEW
      storageLimit: '10240'
    - name: 'namespace-3'
      description: 'ansible-created-namespeace'
      type: 'USER' #USER or GROUP
      user: 'user-3'
      domain: 'vsphere.local'
      role: 'EDIT' #EDIT or VIEW
      storageLimit: '10240'

createSV.yaml

- name: Bootstrap vSphere Supervisor Cluster
  hosts: localhost
  vars_files:
    - supervisor.yaml

  tasks:     

    - name: Create vSphere Session
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/com/vmware/cis/session'
        method: POST
        url_username: '{{ metadata.vsphere_username | b64decode }}'
        url_password: '{{ metadata.vsphere_password | b64decode }}'
        force_basic_auth: true
        validate_certs: false
      register: session

    - name: Show if session failed
      ansible.builtin.fail: 
        msg: 'Login failed. Please provide the correct username + password'
      when: session.status != 200
    
    - name: Show status code
      ansible.builtin.debug:
        msg: 
          - 'Status_Code: {{ session.status }}'
          - 'Session_ID: {{ session.json.value }}'
        verbosity: 2

    - name: Get Cluster ID
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/vcenter/cluster?filter.names={{ spec.clusterName }}'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: vsphere_cluster

    - name: Set fact vSphere Cluster ID
      ansible.builtin.set_fact:
        vsphere_cluster_id: '{{ vsphere_cluster.json.value[0].cluster }}'

    - name: Get Storage Policies
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/vcenter/storage/policies'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: vsphere_storage_policies

    - name: Show Storage Policies
      ansible.builtin.debug:
        var: vsphere_storage_policies
        verbosity: 2
    
    - name: Get Storage Policy ID
      ansible.builtin.set_fact:
        vsphere_storage_policy_id: "{{ (vsphere_storage_policies.json.value | selectattr('name', 'equalto', spec.storagePolicyName ))[0].policy }}"
    
    - name: Show vSphere Storage Policy ID
      ansible.builtin.debug:
        var: vsphere_storage_policy_id
        verbosity: 2

    - name: Get Virtual Distributed Switch ID
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/api/vcenter/namespace-management/distributed-switch-compatibility?cluster={{ vsphere_cluster_id }}&compatible=true'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: vsphere_vds

    - name: Show Virtual Distributed Switch ID
      ansible.builtin.debug:
        var: vsphere_vds.json[0].distributed_switch
        verbosity: 2

    - name: Set fact Virtual Distributed Switch ID
      ansible.builtin.set_fact:
        vsphere_vds_id: '{{ vsphere_vds.json[0].distributed_switch }}'
    
    - name: Get NSX Edge Cluster ID
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/api/vcenter/namespace-management/edge-cluster-compatibility?cluster={{ vsphere_cluster_id }}&compatible=true&distributed_switch={{ vsphere_vds_id | urlencode }}'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: nsx_edge_cluster

    - name: Show NSX Edge Cluster ID
      ansible.builtin.debug:
        var: nsx_edge_cluster.json[0].edge_cluster
        verbosity: 2

    - name: Set fact NSX Edge Cluster ID
      ansible.builtin.set_fact:
        nsx_edge_cluster_id: '{{ nsx_edge_cluster.json[0].edge_cluster }}'
    
    - name: Get Master Network Portgroup
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/vcenter/network?filter.names={{ spec.networks.masterManagementNetwork.network }}'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: vsphere_master_network_portgroup_response
    
    - name: Show Master Network Portgroup ID
      ansible.builtin.debug:
        var: vsphere_master_network_portgroup_response.json.value[0].network
        verbosity: 2
    
    - name: Set fact Master Network Portgroup ID
      ansible.builtin.set_fact:
        vsphere_master_network_portgroup_id: '{{ vsphere_master_network_portgroup_response.json.value[0].network }}'

    - name: Get Tier-0
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/api/vcenter/namespace-management/nsx-tier0-gateways?distributed_switch={{ vsphere_vds_id | urlencode }}'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: nsx_tier_zero_gateways
    
    - name: Set fact Tier-0 Gateway
      ansible.builtin.set_fact:
        nsx_tier_zero_gateway_id: "{{ (nsx_tier_zero_gateways.json | selectattr('display_name', 'equalto', spec.tierZeroName) | first).tier0_gateway }}"  
    
    - name: Show Tier-0 Gateway ID
      ansible.builtin.debug:
        var: nsx_tier_zero_gateway_id
        verbosity: 2
  
    - name: Get Supervisor Cluster
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/api/vcenter/namespace-management/clusters/{{ vsphere_cluster_id }}'
        method: GET
        validate_certs: false
        status_code:
          - 200
          - 404
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: supervisor_cluster
    
    - name: Show Message if Supervisor Cluster already exists
      ansible.builtin.debug:
        msg: 'Supervisor Cluster already exists.'
      when: supervisor_cluster.status == 200

    - name: Create Supervisor Cluster
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/api/vcenter/namespace-management/clusters/{{ vsphere_cluster_id }}?action=enable'
        method: POST
        validate_certs: false
        status_code:
          - 200
          - 204
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
        body_format: json
        body:
          image_storage:
            storage_policy: '{{ vsphere_storage_policy_id }}'
          ncp_cluster_network_spec:
            nsx_edge_cluster: '{{ nsx_edge_cluster_id }}'
            nsx_tier0_gateway: '{{ nsx_tier_zero_gateway_id }}'
            pod_cidrs:
              - address: '{{ spec.networks.pods.address }}'
                prefix: '{{ spec.networks.pods.prefix }}'
            egress_cidrs:
              - address: '{{ spec.networks.egress.address }}'
                prefix: '{{ spec.networks.egress.prefix }}'
            ingress_cidrs:
              - address: '{{ spec.networks.ingress.address }}'
                prefix: '{{ spec.networks.ingress.prefix }}'
            cluster_distributed_switch: '{{ vsphere_vds_id }}'
          master_management_network:
            mode: 'STATICRANGE'
            network: '{{ vsphere_master_network_portgroup_id }}'
            address_range:
              subnet_mask: '{{ spec.networks.masterManagementNetwork.addressRange.subnetMask }}'
              starting_address: '{{ spec.networks.masterManagementNetwork.addressRange.startingAddress }}'
              gateway: '{{ spec.networks.masterManagementNetwork.addressRange.gateway }}'
              address_count: '{{ spec.networks.masterManagementNetwork.addressRange.addressCount }}'
          master_NTP_servers: '{{ spec.networks.masterNTPServers }}'
          ephemeral_storage_policy: '{{ vsphere_storage_policy_id }}'
          service_cidr:
            address: '{{ spec.networks.services.address }}'
            prefix: '{{ spec.networks.services.prefix }}'
          size_hint: '{{ spec.sizeHint }}'
          master_DNS: '{{ spec.networks.masterDNS }}'
          worker_DNS: '{{ spec.networks.workerDNS }}'
          network_provider: '{{ spec.networks.networkProvider }}'
          master_storage_policy: '{{ vsphere_storage_policy_id }}'
      when: supervisor_cluster.status != 200

    - name: Drop vSphere session
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/com/vmware/cis/session'
        method: DELETE
        url_username: '{{ metadata.vsphere_username | b64decode }}'
        url_password: '{{ metadata.vsphere_password | b64decode }}'
        force_basic_auth: true
        validate_certs: false
        status_code:
          - 200
          - 401

createNS.yaml

- name: Create Namespaces
  hosts: localhost
  vars_files:
    - supervisor.yaml

  tasks:     

    - name: Create vSphere Session
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/com/vmware/cis/session'
        method: POST
        url_username: '{{ metadata.vsphere_username | b64decode }}'
        url_password: '{{ metadata.vsphere_password | b64decode }}'
        force_basic_auth: true
        validate_certs: false
      register: session

    - name: Show if session failed
      ansible.builtin.fail: 
        msg: 'Login failed. Please provide the correct username + password'
      when: session.status != 200
    
    - name: Show status code
      ansible.builtin.debug:
        msg: 
          - 'Status_Code: {{ session.status }}'
          - 'Session_ID: {{ session.json.value }}'
        verbosity: 2

    - name: Get Cluster ID
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/vcenter/cluster?filter.names={{ spec.clusterName }}'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: vsphere_cluster

    - name: Set fact vSphere Cluster ID
      ansible.builtin.set_fact:
        vsphere_cluster_id: '{{ vsphere_cluster.json.value[0].cluster }}'
  
    - name: Get Storage Policies
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/rest/vcenter/storage/policies'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: vsphere_storage_policies

    - name: Show Storage Policies
      ansible.builtin.debug:
        var: vsphere_storage_policies
        verbosity: 2
    
    - name: Set fact Storage Policy ID
      ansible.builtin.set_fact:
        vsphere_storage_policy_id: "{{ (vsphere_storage_policies.json.value | selectattr('name', 'equalto', spec.storagePolicyName ))[0].policy }}"
    
    - name: Show vSphere Storage Policy ID
      ansible.builtin.debug:
        var: vsphere_storage_policy_id
        verbosity: 2

    - name: Get Supervisor Cluster
      ansible.builtin.uri:
        url: '{{ metadata.vsphere_url }}/api/vcenter/namespace-management/clusters/{{ vsphere_cluster_id }}'
        method: GET
        validate_certs: false
        headers:
          vmware-api-session-id: '{{ session.json.value }}'
      register: supervisor_clusters
    
    - name: Loop Namespaces
      include_tasks: task_seq_ns.yaml
      loop: '{{ spec.namespaces }}'
      loop_control:
        loop_var: item

task_seq_ns.yaml

- name: Get Namespace
  ansible.builtin.uri:
    url: '{{ metadata.vsphere_url }}/api/vcenter/namespaces/instances/{{ item.name }}'
    method: GET
    validate_certs: false
    status_code:
      - 200
      - 404
    headers:
      vmware-api-session-id: '{{ session.json.value }}'
  register: namespace

- name: Show Message if Namespace already exists
  ansible.builtin.debug:
    msg: 'Namespace: {{ item.name }} already exisiting.'
  when: namespace.status == 200

- name: Create Namespace
  ansible.builtin.uri:
    url: '{{ metadata.vsphere_url }}/api/vcenter/namespaces/instances'
    method: POST
    validate_certs: false
    status_code:
      - 200
      - 204
    headers:
      vmware-api-session-id: '{{ session.json.value }}'
    body_format: json
    body: 
      cluster: '{{ vsphere_cluster_id }}'
      namespace: '{{ item.name }}'
      description: '{{ item.description }}'
      access_list:
        - role: '{{ item.role }}'
          subject_type: '{{ item.type }}'
          subject: '{{ item.user }}'
          domain: '{{ item.domain }}'
      storage_specs:
        - limit: '{{ item.storageLimit }}'
          policy: '{{ vsphere_storage_policy_id }}'
  when: namespace.status != 200

These playbooks were fully tested and verified to work as expected. This approach allows for repeatable and consistent deployments of Supervisor Clusters and Namespaces, entirely managed through code. You can find the official repository on github: vsphere-supervisor-as-code.

Conclusion

Using Ansible to automate the deployment of a vSphere Supervisor Cluster provides a flexible, reliable, and reproducible solution. While Terraform remains a strong tool for declarative infrastructure, its current provider limitations make Ansible the better choice for scenarios that require direct API control and precise configuration management.

This automation workflow ensures that Supervisor Clusters can be deployed consistently, laying the groundwork for fully codified management of Kubernetes infrastructure on vSphere.