• info@lab2prod.com.au
  • Australia
Multi-Cloud GSLB using NSX Advanced Load Balancer|LAB2PROD

Multi-Cloud GSLB using NSX Advanced Load Balancer|LAB2PROD

A guide to configuring a Global Server Load Balancer (GSLB) between an on premise NSX-T enabled datacenter and Amazon Web Services (AWS)

A feature that was previously lacking in the NSX-T stack was the ability to properly perform DNS based load balancing, a feature that is extremely useful when an application spans multiple datacenters and requires load to be balanced across regions, especially when not running stretched layer 2 networks.
With the addition of NSX Advanced Load Balancer (NSX-ALB), formerly known as Avi Vantage, this is no longer the case.  In this article I will go through the process of configuring the required components of NSX-T Datacenter, NSX-ALB, AWS and Microsoft DNS to be able to balance load across multiple regions.
Before getting started, it would make sense to describe the lab topology to make it easier to visualize.  Below is a diagram of the layout.

nsx-t aws topology diagram

A quick overview;

  • This lab is using a single Microsoft AD/DNS on a VM IP
  • There is a single NSX-T Manager at IP
  • There is an NSX-ALB/ Avi Vantage controller at IP (on-prem) and one at (AWS)
  • There is a VPC private subnet in AWS with range, this is where all workloads and management appliances will sit (NSX-ALB/ Avi Vantage controller and service engines)
  • The on-prem subnet is used for all management interfaces
  • The network is an NSX-T overlay segment that is dynamically advertised to the upstream device using BGP and is where the on-prem virtual services and workload will sit

To begin first deploy two NSX-ALB/ Avi Vantage Controllers, this article does not walk through the process of deploying the NSX-ALB/ Avi Vantage Controllers, those details can be found in the below links;

Make sure you do not skip the step of configuring the T0, T1, segments with DHCP and route redistribution.  It is good practice to have all the VIPs you anticipate creating on their on segment and workload on another, however this is not mandatory.
Next we will start configuring the NSX-ALB/ Avi Vantage Controllers, this includes DNS profiles, virtual services, service engines, service engine groups, pools and GSLB services.

Configuring DNS and Virtual Services in NSX Advanced Load Balancer (NSX-ALB) for the GSLB Service

  1. Log in to the first controller, in my case it is (avi-ctr01-.shank.com) and click on the 3 lines in the top left, then navigate to templates, and click on “IPAM/DNS Profiles” in the “Profiles” tabnsx-alb configure DNS and Profile
  2. Click on “Create” in the top right, then “DNS Profile”, fill out the highlighted fields below changing them to suit your environment.nsx-alb configure DNS profile for GSLB
  3. Next click on the 3 lines in the top left again and select “Applications”, we will be creating the first virtual service that will host the DNS service to respond to DNS queries.

  4. Click on “Create Virtual Service” and then “Advanced setup”

    nsx-alb create the virtual service

  5. If you followed the guides linked earlier to set up AWS and NSX-T integration you should now be presented with a choice of “Default-Cloud” and in my case “NSX Cloud” (this will vary depending on what you called it).  Select your “NSX Cloud” and next.

  6. The next two steps will walk through the initial page (Step 1: Settings) of the virtual service configuration

    nsx-alb define virtual service parameters

  7. The second half of the settings page is below, these settings must be configured for the DNS Virtual Service that we are creating now.

    nsx-alb configure virtual service settings

  8. Continue clicking “Next” until you get to “Step 4: Advanced”, here the only setting that we will change is the “SE Group”, if you click on the drop down now you will notice you only have the option of the “Default” group.  We will NOT be using this for our DNS virtual service.  Select “Create SE Group”

    nsx-alb create service engine (SE) group

  9. There are a  fair  few settings on this page so please pay attention to the options being configured.  The fields being edited in this page are based on minimal requirements and best practices.  These configuration steps will be broken down into two 2 steps.  On the basic settings page you must enter a “Service Engine Group Name”, if you wish you can select “Real Time Metrics”, more information regarding what each metric is for can be found here; Avi Vantage Analytics Overview.  It is recommended that you do not share the Service Engine that the DNS virtual service runs on with other virtual services, I will show you how doing this can also cause false positives in a later step.  Once complete, click on the “Advanced” tab.

    nsx-alb configure Service Engine (SE) group settings

  10. On this page, the only setting I change is the one in the screen shot, which will result in the Service Engines that get deployed having “dns” prefixed to their name.  Click save and finish up the Service Engine Group creation.  nsx-alb se group name prefix setting

  11. Once you are back in the “New Virtual Service” creation page, the “Service Engine Group” you just created will be populated in the “SE Group” field.  Click “Next” and then “Save”

  12. Once the service engines have deployed and communicated back to the controller to get their config, the virtual service status should be green like below.  Checking the Service Engines in vCenter is also good to do at this point.

    nsx-alb healthy service engines

  13. Next we create another virtual service or (VIP) for the web servers in this datacenter that will be load balanced.  The steps will be similar to the previous steps, changing the name to reflect the use-case, eg. web.  Also there is no need to change the service port, in my case I will be balancing across two web servers on port 80.  This may be different for your use case.  

  14. We do however need to create a pool, this will be a pool of web servers with which the load will be balanced across.  To do this on the first page of creating the virtual service, click the drop down menu under pool and create new.

    nsx-alb create pool

  15. The next two steps will walk through creating the pool for the web server pool, the settings in the screenshot below reflect the requirements for my environment, replace the values with what is required in your environment.  Once this part is completed, click on next.

    nsx-alb define pool settings

  16. On the next page “Step 2: Servers”, is where you define which servers will be part of this pool, there are a few ways to define them, in this case I am using their IP address.  Click save and complete the virtual service creation wizard, keeping all remaining options as default.

    nsx-alb define pool settings

  17. The summary of settings you require for the web server virtual service (or whatever type of pool you are creating, the port may need to change in your environment) are;

    nsx-alb settings required for virtual service

  18. You should now have 2 virtual services configured and in a healthy state.

    nsx-alb virtual service summary green healthy

  19. Repeat steps 1 through 17 for the second region or site, if all went well you should have two virtual services created in either site, 2 service engine groups and both should have their DNS services configured.

NSX Advanced Load Balancer – NSX-ALB: GSLB Service Configuration

The GSLB service operates by first linking both regions together, that is the NSX-T on-prem Datacenter and AWS.  One of the sites NSX-ALB/ Avi Vantage Controller clusters will have to be the leader and the other will be the follower.  All configuration for the GSLB service will be configured on the leader,  if required the leader can be failed over to the other site.  Once the sites have been linked and the GSLB service has been properly configured, the leader replicates data to the follower so they remain in sync. Later in this post we will cover checking the DNS service across each controller and ensuring load is being balanced across all available servers.

  1. Use  the 3 lines in the top right and navigate to Infrastructure, GSLB and then Site Configuration, select Add New Site.  Here you will first associate the leader controller.  As can be seen in the below picture.nsx-alb create new GSLB
  2. Configure the DNS Virtual Service settings as below, clicking save once completensx-alb configure DNS virtual service for GSLB
  3. Repeat the above two steps to add the secondary region, in this case the AWS controller cluster.  Once this process is complete both sites should be available and have a green status symbol.  When configuring the second controller, there will be a check box called “Active Member” select this and it will then give you the ability to create the DNS Virtual Services for your second region.nsx-alb site configuration health status
  4. Now go to Applications, GSLB Services, click on Create and Advanced Setup.  Here we are creating the actual GSLB service, that is the DNS name to which clients will be resolving and then being redirected to a backend pool member.  Change the settings in the screenshot to values relevant to your environment.  First we define the service name and in the next step we will create and associate the pools.

    nsx-alb GSLB service creation

  5. Scroll down till you see GSLB Pool and then click on Add Pool, once again match the settings to your requirements.  Once this section is completed, scroll down and you will find another “Add GSLB Pool Member”.  Select this and fill out the details for the second region.  Make sure you do NOT set the virtual service as the DNS virtual service, you are mapping the second server pool that you created.  Click Done and then Save on the next window. 

    nsx-alb configure new gslb pool
  6. In my case I can now see web1.lab.shank.com, click on your GSLB service and you should be able to check the site status.  If you recall earlier when I mentioned that having a single Service Engine doing both DNS and your typical load balancing caused red herring errors, the below picture reflects this.  The reason this occurs, from what I gather is because the same service engine group is hosting the DNS service it has troubles resolving itself.  This is why I figure it is best practice to have the DNS service hosted on its on virtual servicensx-alb gslb pool health status 1
  7. I have noticed that sometimes regardless of having the DNS virtual service separate it still does not go green and changing the health monitor around can sometimes work.  Keeping in mind the overall status and GSLB service is still up and functioning with the red status symbols.

    nsx-alb gslb pool health status 2

Configuring MS DNS for Delegation to the NSX Advanced Load Balancer – NSX-ALB DNS Virtual Services

In order for DNS queries for *.lab.shank.com to be resolved by the DNS Virtual Services both on-prem and in AWS, we need to create a delegation and let MS DNS where to send those queries.

  1. Open DNS Manager and create two new A  records, in my case I created dns-aws.shank.com and dns-vs.shank.com (with the DNS Viirtual IP address of each).  If you are using new subnets in your environment, ensure you create add them in AD Sites and Services as well, or else you will not be generating a PTR record when creating the A records.nsx-alb microsoft DNS configuration
  2. Now right click anywhere in the white space in the same screen and select “New Delegation”

    new dns delegation for nsx-alb

  3. Enter the domain you wish to delegate, in my case as was seen earlier it is lab.shank.com

    microsoft zone creation for nsx-alb GSLB
  4. On the next screen we will be adding in the two NS records which will be both DNS Virtual Service IP’s, which can now also be resolved by entering the A record name you created earlier.  You will get timeouts, but they will still work.  Repeat for the remaining Virtual Service.

    complete NS record configuration for NSX-ALB GSLB services in microsoft DNS
  5. That should be all that you need for the DNS side of things.

In the final section we will test out the new service we have just created and do some verification and troubleshooting!

NSX Advanced Load Balancer – NSX-ALB: Verification and Troubleshooting

Lets start by verifying the application can be resolved and the webpages load, in order to show that different VIPs are being resolved upon each refresh, I have installed a chrome extension called Website IP.  This extension shows the IP address of the page that you are viewing in the bottom right.

Test1: Loading the URL site.lab.shank.com

For the first test I will resolve the NSX-ALB GLSB service site.lab.shank.com that I created earlier in this article.  This will demonstrate that the client (my browser), sends a DNS query to my domain controller, which then forwards the request to one of the DNS Virtual services we created in NSX-ALB/ Avi Vantage. 

nsx-alb advanced load balancer gslb testing

So on the first attempt you can see that the query was forwarded and resolved by the VIP, and this is the on-prem VIP.  Let’s test again to see if we get the AWS VIP.

nsx-alb advanced load balancer gslb testing

In the above screenshot we can see the IP address that was hit was, which is the AWS VIP that was created.  You may notice in your tests that you rarely if ever get redirect to the second VIP, this may mean you have to adjust your DNS service TTL’s in NSX-ALB/ Avi Vantage GSLB service configuration.  For my testing purposes I dropped the TTL to 1 as can be seen in the image below.

nsx-alb advanced load balancer gslb dns round robin TTL

Test 2: Client NSlookups

In these tests I will be performing NSlookups from my Windows 10 client to site.lab.shank.com to show round robin working.

nsx-alb advanced load balancer nslookup from client

The above output from command prompt shows nslookup being run twice, one right after the other.  The result shows that first the on-prem VIP is hit and then second is the AWS VIP.

Test 3: Checking DNS records on the DNS Virtual Services

In order to check this, open up a SSH session to both NSX-ALB/ Avi Vantage controllers, once logged in type shell and then show virtualservice ‘virtualServiceName’ dnstable.  In the image below you can see 3 A records.  One for the GSLB service, one for the DNS Virtual Service itself and  one for the webserver VIP (the pool of webservers).  Notice you can also see the lowered TTL in the TTL column that was associated to the GSLB service in NSX-ALB/ Avi Vantage.

nsx-alb advanced load balancer gslb checking DNS virtual services for DNS records

Verifying Traffic in the Management Console

Log into the NSX-ALB/ Avi Vantage management console, you can do this by browsing to https://aviControllerFDQN. I like to log into both controllers so I can see traffic being hit at both sites.
Once logged in, navigate to Application > Virtual Services and then select the server pool virtual service and not the DNS virtual service.  From here you can click on logs and then you will be able to see the hits on the VIP, the queries and the responses.
The below screen shot is from the on-prem controller and the highlighted sections show the details pertaining to the first test we did.

nsx-alb advanced load balancer gslb checking avi controller for analytics of the VIP

Below is a screen shot of the AWS VIP being hit.

nsx-alb advanced load balancer gslb checking avi controller for VIP analytics


So by now you should be somewhat familiar with the process of deploying NSX-ALB/ Avi Vantage, integrating it with NSX-T Datacenter and AWS.  Keep in mind the second region can be another NSX-T cloud or on-prem cloud, such as a vSphere environment.  

We have also walked through creating and assigning DNS templates for use with the GSLB service, creating Virtual Services, both for DNS name resolution and standard server load balancing and finally creating the GSLB service for load balancing across two regions.  If you have any questions or require anything clarified further, feel free to leave a comment!

1 thought on “Multi-Cloud GSLB using NSX Advanced Load Balancer|LAB2PROD

Leave a Reply

Your email address will not be published. Required fields are marked *