Understanding VMware NSX Control Plane

In this post we'll discuss about NSX control plane in detail. I'll describe the  NSX controllers and functions along with NSX Controller workload distribution. We'll also identify the user world agent in the control plane and the control plane component interactions.


NSX Control Plane

The NSX Control plane runs in the VMware NSX Controller cluster. NSX Controller is an advanced distributed state management system that provides control plane functions for NSX logical switching and routing functions. It is the central control point for all logical switches within a network and maintains information about all hosts, logical switches (VXLANs), and distributed logical routers.






About NSX Controller

NSX Controller provides several benefits:
  • VXLAN and logical routing network information distribution to ESXi hosts
  • Clustering for scale out and high availability
  • Workload distribution among NSX Controllers cluster nodes
  • Maintenance of tables for VXLAN and DLRs
  • Removal of VXLAN dependency on multicast routing and protocol independent multicast in the physical network
  • Suppression of ARP broadcast traffic in VXLAN networks.
NSX Controller is an advanced distributed state management system that controls virtual networks and overlay transport tunnels.


NSX Controller Functions

NSX Controller clusters are responsible for managing various tables for the following logical components:
  • Logical switches
  • DLRs
Additionally, NSX Controllers clusters are responsible for updating ESXi host on the state of logical network components.



NSX Controller Workload Distribution

The NSX Controller cluster must perform the following operations:

  • Dynamically distribute workloads across all available NSX Controller cluster nodes
  • Redistribute workloads when a cluster member is added
  • Have the ability to sustain failure of any cluster node
  • Perform the workload distribution so that it is transparent to applications

To achieve these functions, NSX Controller uses sharding distribute workloads across NSX controller cluster nodes.

Sharding is used to distribute workloads across NSX Controller cluster nodes. Sharding is the action of dividing the NSX Controller workloads into different shards so that each NSX Controller instance has an equal portion of work.



Sharding Assignment

When NSX Controller cluster is made aware of the objects beign created, it performs the following tasks:
  1. For a given cluster role, create a number of shards.
  2. Define objects that are to be sharded.
  3. Assign objects to their shards.

These shards are assigned to the different NSX Controller instances in that cluster. The master for a role decides whitch NSX Controller instances are assigned to which shard in the cluster.



Sharding Distribution

Sharding automatically distributes workloads across NSX Controller cluster nodes. Each numbered box on the shard represents shards that the master uses to divide the workloads. The logical switch master divides the logical switches into shards and assigns these shards to different NSX COntroller instances. The master for logical router also divides the logical routers into shards and assigns these shards to different NSX Controller instances.



If a request comes in on router shard 3, that shard is told to connect to third NSX Controller instance. If a request comes in on logical switch shard 2, that request is processed by the second NSX Controller instances.



Shard Redistribution

When an NSX Controller cluster node fails, the controller-clustering service redistributres shards among the remaining nodes.
Shard redistribution occurs in the following situations:

  • Creation of NSX Controller cluster
  • Increase in the number of available NSX Controller nodes in the cluster
  • Reduction in the number of NSX Controller nodes in the cluster



Control Plane Security

The control plane is secured with SSL encryption by using certificates that are managed by NSX Manager:
  • All NSX Controller communication is protected with SSL encryption over the management network.
  • NSX Manager creates and installs self-signed certificates to each ESXi host and NSX Controller cluster.
  • Mutual authentication of NSX entities occurs by verifying certificates.


Control Plane User World Agents: netcpa and vsfwd

netcpa has following responsibilities:
  • It uses SSL to secure the communication with NSX Controller instances.
  • It mediates between NSX Controller instances and the hypervisor kernel module, except the distributed firewall.
  • It sends information about virtual machine netowork connectivity, MAC address, and IP address to NSX Controller.
  • It retrieves configuration from NSX Manager through the message bus agent vsfwd.

Control plane agent (netcpa) is a TCP (SSL) client that communicates with the controller using the control plane protocol. netcpa might connect to multiple controllers. It communicates with the message bus client (vsfwd) to retrieve control plane related information from NSX Manager.



Management and Control Plane Agent Interactions

User workls agents are deployed by NSX Manger on ESXi hosts through ESX Agent Manager during host preperation. Each ESXi host runs two user world agents (UWA) that include the message client (vsfwd) and control plane agent (netcpa).

Comments

Post a Comment

Popular posts from this blog

Specifying SSH port in Ansible Inventory

There may be some instances where you set a custom port for SSH on your network device. If ssh port for hosts is different than the default port 22, it can be specified in the inventory file with colon (:) after hostname. #vi inventory # Inventory file for Ansible   [XE] ios-xe-mgmt.cisco.com:8181 ios-xe-mgmt-latest.cisco.com:8181   [XR] sbx-iosxr-mgmt.cisco.com:8181
Read more

Ansible-Playbook to display output of multiple show commands (using stdout_lines with Loop)

 In this playbook, we we'll see how we can get display of multiple show commands in stdout_lines format. We can make use of loops (or with_items) for submitting multiple commands, but debug output with stdout_lines does not gives the formatted result as it would give for single command. So in case of multiple commands, we can debug the output of each command separately in stdout_lines format.
Read more

Filtering Routes in BGP using Route-maps and Prefix-list

Image
Order of preference of attributes in BGP The order of preference varies based on whether the attributes are applied for inbound updates or outbound updates. For inbound updates the order of preference is:     route-map     filter-list     prefix-list, distribute-list For outbound updates the order of preference is:     prefix-list, distribute-list     filter-list     route-map NOTE: The attributes prefix-list and distribute-list are mutually exclusive, and only one command (neighbor distribute-list or neighbor prefix-list) can be applied to each inbound or outbound direction for a particular neighbor. Scenario: We own the AS500 and advertising a network block of 192.0.2.0/24 and 180.179.179.0/16 to two different ISPs. Objectives: Configure router R1 to establish eBGP neighbor relationship with ISP1. Configure router R1 to establish eBGP neighbor relationship with ISP2. Advertise  192.0.2.0/24 ...
Read more

Ansible Playbook for Network OS Upgrade with pre and post checks

You have 100s of network switches or routers that you need to upgrade. How much time would it take for you to do the upgrades? There are a lot number of sub-tasks involved while upgrading IOS image of a Cisco router or a switch. This time of upgradation can be reduced through automation from various Enterprise Configuration Management tools that also have ability to upgrade network OS. Though these tools give an easy to use graphical interface, but this requires you to have appropriate license and also restricts you to customize your upgrade process.
Read more

VMware NSX Traffic Flow — East-West & North-South

Image
Understanding how traffic is flowing in NSX environment is an important aspect to successfully maintain and troubleshoot networks having NSX. In this post we'll understand hop-by-hop flow of traffic in East-West and North-South directions. East-West: VMs on Same Subnet, Same Host VM-1 has IP address 172.16.20.6 and VM-2 has IP address 172.16.20.7 VM-1 vNIC è Logical Switch (Segment ID 5002) è VM-2 vNIC
Read more

Bypassing Proxy Server in Google Chrome

Image
There may be times when you might want to use Google Chrome and do not wish to use the proxy settings for which you are bound to use in Internet Explorer because of your company policies. If you try to change proxy settings in Google Chrome, by default it will open the System Internet Properties dialogue box, and most of the times, change in proxy setting options is restricted by domain group policy. So here is an alternate way to remove or disable proxy settings in Google Chrome. 1. Right click the GoogleChrome.exe and create it's shortcut. 2. Open the properties of the shortcut of GoogleChrome.exe created in previous step. 3. Here you have to edit the Target of this shortcut. You have to add   --proxy-server=""  after the existing path. You are done. Now enjoy the web browsing without any restrictions.
Read more

Export or Backup Azure Network Security Groups into CSV using PowerShell

There could be many use cases where you may want to export Network Security Groups into CSV. You might have question, how to export or backup Azure Network Security Groups into CSV. Here is the PowerShell script that you can use to export Azure Network Security Groups into CSV using PowerShell script. This script will export Network Security Group along with rules of all Active subscriptions into a CSV.
Read more

Ansible-playbook for backing up running config of Cisco IOS

This ansible-playbook can be used to backup running configuration from Cisco IOS devices. You can refer to my earlier post Getting Started with your first ansible-playbook for Network Automation  to know about the parameters used in this playbook. Inventory file # Inventory file for Ansible [XE] ios-xe-mgmt.cisco.com:8181 ios-xe-mgmt-latest.cisco.com:8181 [XR] sbx-iosxr-mgmt.cisco.com:8181 [all:vars] ansible_network_os=ios Playbook --- - name: Define Parameters   hosts: XE   gather_facts: no   connection: network_cli   tasks:    - name: backup the config      ios_config:       backup: yes      register: backup_config    - name: Store the config to directory      copy:       src: "{{ backup_config.backup_path }}"       dest: "/tmp/backups/{{ inventory_hostname }}" Running the playbook [prashant@Prashant-VM01 ~]$ ansible-playbook play03.yml -i /home/prasha...
Read more