Cloud and Datacenter Management Blog

Microsoft Hybrid Cloud blogsite about Management

Leave a comment

UPDATE : Hybrid Cloud with NVGRE WhitePaper #CloudOS #SCVMM #WAPack

HybridCloud NVGRE Update v5

This white paper is meant to show you a real world deployment of a fabric infrastructure that is supporting network virtualization with NVGRE for hybrid cloud computing, together with Windows Azure Pack.

Read this great whitepaper first when you start with Microsoft Private Cloud and Hybrid Cloud computing made by my fellow MVP’s :

You can follow them on Twitter :


Kristian Nese, Cloud & Datacenter Management MVP @KristianNese
Flemming Riis, Cloud & Datacenter Management MVP @FlemmingRiis
Stanislav Zhelyazkov, Cloud & Datacenter MVP @StanZhelyazkov
Marc van Eijk, Azure MVP @_marcvaneijk
Daniel Neumann, Cloud & Datacenter Management MVP @neumanndaniel
Stanislav Zhelyazkov, Cloud & Datacenter Management MVP

You can download this Great White Paper on Hybrid Cloud with NVGRE & Windows Azure Pack here

Thank you for this real world WhitePaper guys ! 😉

Leave a comment

Awesome Free E-Book Building a Virtualized Network Solution #sysctr #SCVMM #Hyperv #NVGRE #SDN

Building Virtualized Networking Solutions

Microsoft System Center: Building a Virtualized Network Solution
Mitch Tulloch with Nigel Cain, Alvin Morales, Michel Luescher, Damian Flynn
February 2014
136 pages

This book is geared to private and hybrid cloud architects preparing to design and build a virtualized network solution based on Windows Server 2012 and System Center 2012 SP1, or later. Written by experts on the Microsoft System Center team and with Microsoft MVP Mitch Tulloch as series editor, this title focuses on architecture and design.

You can download “Building a Virtualized Network Solution” here

AWESOME Free E-Book 🙂


#HyperV Network Virtualization technical details and Gateway Architecture #SCVMM #Cloud #SDN

S2S Hybridcloud

In Hyper-V Network Virtualization (HNV), a customer is defined as the “owner” of a group of virtual machines that are deployed in a datacenter. A customer can be a corporation or enterprise in a multitenant public datacenter, or a division or business unit within a private datacenter. Each customer can have one or more VM networks in the datacenter, and each VM network consists of one or more virtual subnets


Generic Routing Encapsulation figure 1.

Generic Routing Encapsulation This network virtualization mechanism uses the Generic Routing Encapsulation (NVGRE) as part of the tunnel header. In NVGRE, the virtual machine’s packet is encapsulated inside another packet. The header of this new packet has the appropriate source and destination PA IP addresses in addition to the Virtual Subnet ID, which is stored in the Key field of the GRE header.

The Virtual Subnet ID allows hosts to identify the customer virtual machine for any given packet, even though the PA’s and the CA’s on the packets may overlap. This allows all virtual machines on the same host to share a single PA, as shown in Figure 1.

Sharing the PA has a big impact on network scalability. The number of IP and MAC addresses that need to be learned by the network infrastructure can be substantially reduced. For instance, if every end host has an average of 30 virtual machines, the number of IP and MAC addresses that need to be learned by the networking infrastructure is reduced by a factor of 30.The embedded Virtual Subnet IDs in the packets also enable easy correlation of packets to the actual customers.

With Windows Server 2012 and later, HNV fully supports NVGRE out of the box; it does NOT require upgrading or purchasing new network hardware such as NICs (Network Adapters), switches, or routers. This is because the NVGRE packet on the wire is a regular IP packet in the PA space, which is compatible with today’s network infrastructure.

Windows Server 2012 made working with standards a high priority. Along with key industry partners (Arista, Broadcom, Dell, Emulex, Hewlett Packard, and Intel) Microsoft published a draft RFC that describes the use of Generic Routing Encapsulation (GRE), which is an existing IETF standard, as an encapsulation protocol for network virtualization. For more information, see the following Internet Draft: Network Virtualization using Generic Routing Encapsulation. As NVGRE-aware becomes commercially available the benefits of NVGRE will become even greater.

Here you can read more on Microsoft Technet about Hyper-V Network Virtualization technologies

Hyper-V Network Virtualization Gateway Architectural Guide :

SCVMM2012R2 Design

System Center 2012 R2 Virtual Machine Manager Figure 2.

In the VMM model the Hyper-V Network Virtualization Gateway is managed via a PowerShell plug-in module. Partners building Hyper-V Network Virtualization gateways need to create a PowerShell plug-in module which physically runs on the VMM server. This plug-in module will communicate policy to the gateway. Figure 2 shows a block diagram of VMM managing a Hyper-V Network Virtualization deployment. Note that a partner plug-in runs inside the VMM server. This plug-in communicates to the gateway appliances. The protocol used for this communication is not specified here. The partner may determine the appropriate protocol. Note that VMM uses the Microsoft implementation of WS-Management Protocol called Windows Remote Management (WinRM) and Windows Management Instrumentation (WMI) to manage the Windows Server 2012 hosts and update network virtualization policies.

Cross Premise Gateway

Cross Premises Gateway Figure 3.

The Hybrid Cloud scenario enables an enterprise to seamlessly expand their on-premises datacenter into the cloud. This requires a site to site VPN tunnel. This can be accomplished with Windows Server 2012 as the host platform and a per tenant Windows Server 2012 guest virtual machine running a Site To Site (S2S) VPN tunnel connecting the cloud datacenter with various on-premise datacenters. Windows Server 2012 S2S VPN supports IKEv2 and configuration of remote policy can be accomplished via PowerShell/WMI. In addition Windows Server 2012 guest virtual machines support new network interface offload capabilities that enhance the performance and scalability of the gateway appliance. These offload capabilities are discussed below in the Hardware Considerations section.

Figure 3 shows a scenario where Red Corp and Blue Corp are customers of Hoster Cloud. Red Corp and Blue Corp seamlessly extend their datacenter into Hoster Cloud has deployed Windows Server 2012 based per tenant virtual machine gateways allowing Red Corp and Blue Corp to seamlessly extend their on-premise datacenters. In Figure 10 there is no requirement that Red Corp or Blue Corp run Windows Server 2012 S2S VPN, only that the customer’s on premise S2S VPN support IKEv2 to interact with corresponding Windows Server 2012 S2S virtual machines running on HostGW.

Figure 9 shows the internal architecture for HostGW. Each Routing Domain requires its own virtual machine. The technical reason for this is that a vmnic can only be associated with a single Virtual Subnet (VSID) and a VSID can only be part of a single routing domain. The VSID switch port ACL does not support trunking of VSIDs. Therefore the simplest way to provide isolation is with a per tenant (Routing Domain) gateway virtual machine.

Each of the virtual machines is dual homed which means they have two virtual network interfaces. One of the virtual network interfaces has the appropriate VSID associated with it. The other virtual network interface has a VSID of 0 which means traffic is not modified by the WNV filter. The Windows Server 2012 virtual machine is running RRAS and using IKEv2 to create a secure tunnel between Hoster Cloud and the customer’s on premise gateway.

HybridCloud VM Gateway

Hybrid Cloud with Windows Server 2012 based Per Tenant VM Gateways Figure 4.

Figure 4 shows the architecture where VMM is managing a Hyper-V Network Virtualization deployment. The partner has a plug-in that runs in the VMM server. When using Windows Server 2012 as a Hyper-V Network Virtualization gateway appliance a local management process running in Windows is required as the end point for this communication from the plug-in running in the VMM server. This is how the plug-in is able to communicate network virtualization policy to the WNV filter running on HostGW

You can Read more on Microsoft Technet about Hyper-V Network Virtualization Gateway Architecture here


HOTFIX : Windows Server 2012 R2 NVGRE Gateway #SCVMM #NVGRE


After some hours or days, a Windows Server 2012 R2 Network Virtualization that uses Generic Routing Encapsulation (NVGRE) gateways cannot establish outgoing UDP or TCP connections intermittently.

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that described in this article. Apply this hotfix only to systems that are experiencing the problem described in this article. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a “Hotfix Download Available” section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft website:

Note The “Hotfix Download Available” form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.


To apply this hotfix, you must be running Windows Server 2012 R2.

Registry information

To apply this hotfix, you do not have to make any changes to the registry.

Restart requirement

You might have to restart the computer after you apply this hotfix.

Hotfix replacement information

This hotfix does not replace a previously released hotfix.

NVGRE Hotfix download


UPDATE on Microsoft NVGRE Gateway in #SCVMM 2012 R2 : Adding a New Private #Cloud and Tenant

With the WhitePaper Hybrid Cloud with NVGRE (WSSC 2012 R2)  we made a isolated Blue Cloud tenant with the NVGRE Gateway in System Center 2012 R2 Virtual Machine Manager :

You can find that blogpost here

Now we have to add a new private Cloud ” vWorkspace” where the Virtual Machines are NOT isolated and use the NVGRE Gateway too :

First we make a new Virtual Machine Network.

VM Network 1

Here you make the connectivity with the NVGRE Gateway

VM Network 2Here we make the Network Address Translation and we made one rule for port 80.

VM Network 3Created also an Green Tenant IP-Pool

Then we made a Hyper-V Port Profile and the Port Classification for the Green Tenant

Port ProfileHyper-V Port Profile for the Green Tenant.

Port ClassificationPort Classification for the Green Tenant

Then we modified the following switches

Production SwitchAdding the Virtual port to the logical production Switch.

Stand Alone SwitchAnd adding the Virtual Port of the Green Tenant to the Stand-Alone Switch.

Here you see the properties of the vWorkspace Private Cloud

Cloud Properties 1We add the Management Logical network and the PANetwork

Cloud Properties 2Here we set the port classification of this private Cloud.

And now we are ready to go and make an GreenTenantVM which is using the NVGRE Gateway :

VM PropertiesOne NIC in Management and the Other in Production in Tenant Green virtual network

This virtual Machine is now presenting a Website portal for VDI via NVGRE Gateway NAT Rule port 80

VWorkspace sitevWorkspace VDI Portal

And here you see the properties of the NVGRE Gateway in Microsoft System Center 2012 R2 Virtual Machine Manager

NVGRE Gateway PropertiesNow we have two routing domains and Virtual Subnets.