mountainss Cloud and Datacenter Management Blog

Microsoft SystemCenter blogsite about virtualization on-premises and Cloud


Leave a comment

NEW via #MSFTConnect 2017 Microsoft #Azure Databrick

Today at Microsoft Connect(); we introduced Azure Databricks, an exciting new service in preview that brings together the best of the Apache Spark analytics platform and Azure cloud. As a close partnership between Databricks and Microsoft, Azure Databricks brings unique benefits not present in other cloud platforms. This blog post introduces the technology and new capabilities available for data scientists, data engineers, and business decision-makers using the power of Databricks on Azure.

Azure Databricks Preview

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

Read more on Microsoft Docs what Microsoft Azure Databrick is

Quickstart: Get started with Azure Databricks using the Azure portal :

This quickstart shows how to create an Azure Databricks workspace and an Apache Spark cluster within that workspace. Finally, you learn how to run a Spark job on the Databricks cluster.

Creating Clusters

In Databricks, you can create two different types of resources:
Standard Clusters: Databricks’ standard clusters have lot of configuration options to customize and fine tune your Spark jobs. You can learn more about standard clusters below.
Serverless Pools (BETA): With serverless pools, Databricks’ auto-manages all the resources and you just need to provide the range of instances required for the pool. Serverless pools support only Python and SQL. Serverless pools also auto-configures the resources with right Spark configuration. Visit Serverless Pools to know more about them.

Read more on the Microsoft Azure Blog here:  A Technical Overview of Azure Databricks after Microsoft Connect() 2017.

Advertisements


Leave a comment

#Microsoft Azure Virtual Datacenter Guidance Whitepaper Available #Cloud #Security #Azure

Overview Azure Virtual Datacenter is an approach to making the most of the Azure cloud platform’s capabilities while respecting your existing security and networking policies. When deploying enterprise workloads to the cloud, IT organizations and business units must balance governance with developer agility. Azure Virtual Datacenter provides models to achieve this balance with an emphasis on governance. Deploying workloads to the cloud introduces the need to develop and maintain trust in the cloud to the same degree you trust your existing datacenters. The first model of Azure Virtual Datacenter guidance is designed to bridge that need through a locked-down approach to virtual infrastructures. This approach isn’t for everyone. It’s specifically designed to guide enterprise IT groups in extending their on-premises infrastructure to the Azure public cloud. We call this approach the trusted datacenter extension model. Over time, several other models will be offered, including those that allow secure Internet access directly from a virtual datacenter.

In the Azure Virtual Datacenter model, you can apply isolation policies, make the cloud more like the physical datacenters you know, and achieve the levels of security and trust you need. Four components any enterprise IT team would recognize make it possible: software-defined networking, encryption, identity management, and the Azure platform’s underlying compliance standards and certifications. These four are key to making a virtual datacenter a trusted extension of your existing infrastructure investment. Central to this model is the idea that your cloud infrastructure has isolation boundaries that can be thought of as your corporate namespace. Think of it as your isolated cloud within Azure. Within this virtual boundary, security controls, network policies, and compliance come together, providing you with an IT infrastructure on Azure capable of securely integrating cloud resources with your existing on-premises datacenter. You can deploy new virtual workspaces in the virtual datacenter much as you would deploy additional capacity to your physical datacenter. These virtual workspaces are self-contained

Environments where workloads can run independently, and workload teams can get workspace specific access. Workspaces enable teams to build solutions and manage workloads with great freedom while adhering to the overall access and security policies defined in the central IT infrastructure. This guide is intended for enterprise IT architects and executives. Using the lens of the physical datacenter, the guide discusses an approach to designing secure, trusted virtual datacenters on the Azure platform. Azure Virtual Datacenter is not a specific product or service but rather a way to think about cloud infrastructures. It offers proven practices and guidance to help smooth your migration to the cloud. At the end of this guide, you can learn about the upcoming Virtual Datacenter Automation guidance. This guidance includes a collection of scripts and Azure Resource Manager templates that will help you build an Azure Virtual Datacenter using the trusted extension model.

You can download this Awesome Microsoft whitepaper Azure Virtual Datacenter here


Leave a comment

Microsoft Azure #CloudShell Overview with #Bash CLI 2.0 and #Powershell #Azure #DevOps

Azure Powershell in the Portal

Azure Cloud Shell is an interactive, browser-accessible shell for managing Azure resources. It gives you the flexibility of choosing the shell experience that best suits the way you work. Linux users can opt for a Bash experience, while Windows users can opt for PowerShell.

At the left corner you can change from Powershell Cmd to Bash

Bash with Azure CLI 2.0 

Features
Browser-based shell experience
Cloud Shell enables access to a browser-based command-line experience built with Azure management tasks in mind. Leverage Cloud Shell to work untethered from a local machine in a way only the cloud can provide.

Choice of preferred shell experience
Azure Cloud Shell gives you the flexibility of choosing the shell experience that best suits the way you work. Linux users can opt for a Bash experience, while Windows users can opt for PowerShell.

Pre-configured Azure workstation
Cloud Shell comes pre-installed with popular command-line tools and language support so you can work faster.

View the full tooling list for Bash experience and PowerShell experience.

Automatic authentication
Cloud Shell securely authenticates automatically on each session for instant access to your resources through the Azure CLI 2.0 or Azure PowerShell cmdlets.

Connect your Azure File storage
Cloud Shell machines are temporary and as a result require an Azure Files share to be mounted as clouddrive to persist your $Home directory. On first launch Cloud Shell prompts to create a resource group, storage account, and file share on your behalf. This is a one-time step and will be automatically attached for all sessions. A single file share can be mapped and will be used by both Bash and PowerShell in Cloud Shell.

I like Microsoft Azure CLI 2.0 in the Cloud Shell and here you find a CLI 2.0 command line reference guide.

In the following step-by-step examples ( in Bash and Powershell ) you will see how easy it is to make an Azure Container Instance in the Cloud.
we begin with starting Bash Shell from the Azure Portal.

When you run Cloud Shell for the first time it will create a Cloud Drive of 5GB.

Cloud Shell machines are temporary and as a result require an Azure Files share to be mounted as clouddrive to persist your $Home directory. On first launch Cloud Shell prompts to create a resource group, storage account, and file share on your behalf. This is a one-time step and will be automatically attached for all sessions. A single file share can be mapped and will be used by both Bash and PowerShell in Cloud Shell.

Create Storage for your CloudDrive

A locally-redundant storage (LRS) account and Azure Files share can be created on your behalf. The Azure Files share will be used for both Bash and PowerShell environments if you choose to use both. Regular storage costs apply.

  • Cloud Shell runs on a temporary machine provided on a per-session, per-user basis
  • Cloud Shell times out after 20 minutes without interactive activity
  • Cloud Shell can only be accessed with a file share attached
  • Cloud Shell uses a the same file share for both Bash and PowerShell
  • Cloud Shell is assigned one machine per user account
  • Permissions are set as a regular Linux user (Bash)

az container create -h

With this command you see the options to create a Container Instance in Microsoft Azure Cloud.

You see also some examples to learn from

az group create –name MyResourceGroup –location eastus

We now created a resource group in the East US location of Azure for our Container.

az container create –name mycontainer –image microsoft/aci-helloworld –resource-group MyResourceGroup –ip-address public

We now Created an Azure Container Instance.

ProvisioningState Succeeded

az container list –output table

The result of your Azure Container Instance

To see how your Azure Container Instance is doing, you can read the logs.

az container logs –name mycontainer –resource-group MyResourceGroup

When your Azure Container Instance was for testing, you can delete the instance by :

az container delete –name mycontainer  –resource-group MyResourceGroup

az container list –output table

I got two Azure Container Instances running, and now you can see that mycontainer instance is deleted.
This was just an simple example by using Bash in the Azure Portal with CLI 2.0 commands. Of course there are a lot of Azure Solutions to play with:

In the following step-by-step example we will use Azure Powershell from the portal instead of Bash :

$PSVersionTable.PSVersion

Microsoft Azure Powershell via the Portal is using version 5.1 Build 14393 Revision 1480 in my example.
Of course there is also an Azure Powershell reference guide online

New-AzureRmResourceGroup -Name MyResourceGroup -Location EastUS

We now have created the Resource Group with Azure Powershell for the Container Instance.

New-AzureRmContainerGroup -ResourceGroupName MyResourceGroup -name mycontainer -image microsoft/iis:nanoserver -OsType Windows -IpAddressType Public

Get-AzureRmContainerGroup -ResourceGroupName MyResourceGroup -Name mycontainer

You will see that the New Azure Container Instance is provisioned.

Get-AzureRmContainerGroup

And the IIS is running in the Azure Container Instance.

Of course you don’t have your laptop always with you, but Microsoft Azure has an Awesome Mobile App to work with.

 

Mobile Azure Powershell via the App

Mobile Azure Bash via the App

Here you can get the Microsoft Azure Mobile App

Here are some handy links to use with Microsoft Azure Cloud Shell :

Deploy resources with Resource Manager templates and Azure CLI

Deploy resources with Resource Manager templates and Azure PowerShell

Overview of Azure Cloud Shell (Preview)

Microsoft Azure Cloud Roadmap


Leave a comment

What’s New in Windows Server 2016 version 1709 #Winserv #Hyperv #Containers

Application containers and micro-services

  • The Server Core container image has been further optimized for lift-and-shift scenarios where you can migrate existing code bases or applications into containers with minimal changes, and it’s also 60% smaller.
  • The Nano Server container image is nearly 80% smaller.
    • In the Windows Server Semi-Annual Channel, Nano Server as a container base OS image is decreased from 390 MB to 80 MB.
  • Linux containers with Hyper-V isolation

For more information, see Changes to Nano Server in the next release of Windows Server and Windows Server, version 1709 for developers.

Modern management

Check out Project Honolulu for a simplified, integrated, secure experience to help IT administrators manage core troubleshooting, configuration, and maintenance scenarios. Project Honolulu includes next generation tooling with a simplified, integrated, secure, and extensible interface. Project Honolulu includes an intuitive all-new management experience for managing PCs, Windows servers, Failover Clusters, as well as hyper-converged infrastructure based on Storage Spaces Direct, reducing operational costs.

Compute

Nano Container and Server Core Container: First and foremost, this release is about driving application innovation. Nano Server, or Nano as Host is deprecated and replaced by Nano Container, which is Nano running as a container image.

For more information about containers, see Container Networking Overview.

Server Core as a container (and infrastructure) host, provides better flexibility, density and performance for existing applications under a modernization process and brands new apps developed already using the cloud model.

VM Load Balancing is also improved with OS and Application awareness, ensuring optimal load balancing and application performance. Storage-class memory support for VMs enables NTFS-formatted direct access volumes to be created on non-volatile DIMMs and exposed to Hyper-V VMs. This enables Hyper-V VMs to leverage the low-latency performance benefits of storage-class memory devices.

Storage-class memory support for VMs enables NTFS-formatted direct access volumes to be created on non-volatile DIMMs and exposed to Hyper-V VMs. This enables Hyper-V VMs to leverage the low-latency performance benefits of storage-class memory devices. Virtualized Persistent Memory (vPMEM) is enabled by creating a VHD file (.vhdpmem) on a direct access volume on a host, adding a vPMEM Controller to a VM, and adding the created device (.vhdpmem) to a VM. Using vhdpmem files on direct access volumes on a host to back vPMEM enables allocation flexibility and leverages a familiar management model for adding disks to VMs.

Virtualized Persistent Memory (vPMEM) is enabled by creating a VHD file (.vhdpmem) on a direct access volume on a host, adding a vPMEM Controller to a VM, and adding the created device (.vhdpmem) to a VM. Using vhdpmem files on direct access volumes on a host to back vPMEM enables allocation flexibility and leverages a familiar management model for adding disks to VMs.

Container storage – persistent data volumes on cluster shared volumes (CSV). In Windows Server, version 1709 as well as Windows Server 2016 with the latest updates, we’ve added support for containers to access persistent data volumes located on CSVs, including CSVs on Storage Spaces Direct. This gives the application container persistent access to the volume no matter which cluster node the container instance is running on. For more info, see Container Storage Support with Cluster Shared Volumes (CSV), Storage Spaces Direct (S2D), SMB Global Mapping.

Container storage – persistent data volumes with SMB global mapping. In Windows Server, version 1709 we’ve added support for mapping an SMB file share to a drive letter inside a container – this is called SMB global mapping. This mapped drive is then accessible to all users on the local server so that container I/O on the data volume can go through the mounted drive to the underlying file share. For more info, see Container Storage Support with Cluster Shared Volumes (CSV), Storage Spaces Direct (S2D), SMB Global Mapping.

Security and Assurance

Windows security baselines have been updated for Windows Server and Windows 10. A security baseline is a group of Microsoft-recommended configuration settings and explains their security impact. For more information, and to download the Policy Analyzer tool, see Microsoft Security Compliance Toolkit 1.0.

Network encryption enables you to quickly encrypt network segments on software-defined networking infrastructure to meet security and compliance needs.

Host Guardian Service (HGS) as a shielded VM is enabled. Prior to this release, the recommendation was to deploy a 3-node physical cluster. While this ensures the HGS environment is not compromised by an administrator, it was often cost prohibitive.

Linux as a shielded VM is now supported.

For more information, see Guarded fabric and shielded VMs overview.

Storage

Storage Replica: The disaster recovery protection added by Storage Replica in Windows Server 2016 is now expanded to include:

  • Test failover: the option to mount the destination storage is now possible through the test failover feature. You can mount a snapshot of the replicated storage on destination nodes temporarily for testing or backup purposes. For more information, see Frequently Asked Questions about Storage Replica.
  • Project Honolulu support: Support for graphical management of server to server replication is now available in Project Honolulu. This removes the requirement to use PowerShell to manage a common disaster protection workload.

 

SMB:

  • SMB1 and guest authentication removal: Windows Server, version 1709 no longer installs the SMB1 client and server by default. Additionally, the ability to authenticate as a guest in SMB2 and later is off by default. For more information, review SMBv1 is not installed by default in Windows 10, version 1709 and Windows Server, version 1709.
  • SMB2/SMB3 security and compatibility: Additional options for security and application compatibility were added, including the ability to disable oplocks in SMB2 for legacy applications, as well as require signing or encryption on per-connection basis from a client. For more information, review the SMBShare PowerShell module help.

 

Data Deduplication:

  • Data Deduplication now supports ReFS: You no longer must choose between the advantages of a modern file system with ReFS and the Data Deduplication: now, you can enable Data Deduplication wherever you can enable ReFS. Increase storage efficiency by upwards of 95% with ReFS.
  • DataPort API for optimized ingress/egress to deduplicated volumes: Developers can now take advantage of the knowledge Data Deduplication has about how to store data efficiently to move data between volumes, servers, and clusters efficiently.

Remote Desktop Services (RDS)

RDS is integrated with Azure AD, so customers can leverage Conditional Access policies, Multifactor Authentication, Integrated authentication with other SaaS Apps using Azure AD, and many more. For more information, see Integrate Azure AD Domain Services with your RDS deployment.

For a sneak peek at other exciting changes coming to RDS, see Remote Desktop Services: Updates & upcoming innovations

Networking

Docker’s Routing Mesh is supported. Ingress routing mesh is part of swarm mode, Docker’s built-in orchestration solution for containers. For more information, see Docker’s routing mesh available with Windows Server version 1709.

New features for Docker are available. For more information, see Exciting new things for Docker with Windows Server 1709.

Windows Networking at Parity with Linux for Kubernetes: Windows is now on par with Linux in terms of networking. Customers can deploy mixed-OS, Kubernetes clusters in any environment including Azure, on-premises, and on 3rd-party cloud stacks with the same network primitives and topologies supported on Linux without the need for any workarounds or switch extensions.

Core network stack: Several features of the core network stack are improved. For more information about these features, see Core Network Stack Features in the Creators Update for Windows 10.

  • TCP Fast Open (TFO): Support for TFO has been added to optimize the TCP 3-way handshake process. TFO establishes a secure TFO cookie in the first connection using a standard 3-way handshake. Subsequent connections to the same server use the TFO cookie instead of a 3-way handshake to connect with zero round trip time.
  • CUBIC: Experimental Windows native implementation of CUBIC, a TCP congestion control algorithm is available. The following commands enable or disable CUBIC, respectively.

netsh int tcp set supplemental template=internet congestionprovider=cubic

netsh int tcp set supplemental template=internet congestionprovider=compound

  • Receive Window Autotuning: TCP autotuning logic computes the “receive window” parameter of a TCP connection. High speed and/or long delay connections need this algorithm to achieve good performance characteristics. In this release, the algorithm is modified to use a step function to converge on the maximum receive window value for a given connection.
  • TCP stats API: A new API is introduced called SIO_TCP_INFO. SIO_TCP_INFO allows developers to query rich information on individual TCP connections using a socket option.
  • IPv6: There are multiple improvements in IPv6 in this release.
    • RFC 6106 support: RFC 6106 which allows for DNS configuration through router advertisements (RAs). You can use the following command to enable or disable RFC 6106 support:

netsh int ipv6 set interface <ifindex> rabaseddnsconfig=<enabled | disabled>

  • Flow Labels: Beginning with the Creators Update, outbound TCP and UDP packets over IPv6 have this field set to a hash of the 5-tuple (Src IP, Dst IP, Src Port, Dst Port). This will make IPv6 only datacenters doing load balancing or flow classification more efficient. To enable flowlabels:

netsh int ipv6 set flowlabel=[disabled|enabled] (enabled by default)

netsh int ipv6 set global flowlabel=<enabled | disabled>

  • ISATAP and 6to4: As a step towards future deprecation, the Creators Update will have these technologies disabled by default.
  • Dead Gateway Detection (DGD): The DGD algorithm automatically transitions connections over to another gateway when the current gateway is unreachable. In this release, the algorithm is improved to periodically re-probe the network environment.
  • Test-NetConnection is a built-in cmdlet in Windows PowerShell that performs a variety of network diagnostics. In this release we have enhanced the cmdlet to provide detailed information about both route selection as well as source address selection.

Software Defined Networking

  • Virtual Network Encryption is a new feature that provides the ability for the virtual network traffic to be encrypted between Virtual Machines that communicate with each other within subnets that are marked as “Encryption Enabled”. This feature utilizes Datagram Transport Layer Security (DTLS) on the virtual subnet to encrypt the packets. DTLS provides protection against eavesdropping, tampering and forgery by anyone with access to the physical network.

Windows 10 VPN

  • Pre-Logon Infrastructure Tunnels. By default, Windows 10 VPN does not automatically create Infrastructure Tunnels when users are not logged on to their computer or device. You can configure Windows 10 VPN to automatically create Pre-Logon Infrastructure Tunnels by using the Device Tunnel (prelogon) feature in the VPN profile.
  • Management of Remote Computers and Devices. You can manage Windows 10 VPN clients by configuring the Device Tunnel (prelogon) feature in the VPN profile. In addition, you must configure the VPN connection to dynamically register the IP addresses that are assigned to the VPN interface with internal DNS services.
  • Specify Pre-Logon Gateways. You can specify Pre-Logon Gateways with the Device Tunnel (prelogon) feature in the VPN profile, combined with traffic filters to control which management systems on the corporate network are accessible via the device tunnel.

Release Notes: Important Issues in Windows Server, version 1709


Leave a comment

Cross-Blogpost Microsoft #AzureStack Developer Kit (ASDK) in the Classroom #MVPbuzz for #Education

Azure Stack in the Classroom for Education

 


Leave a comment

Watch all those Awesome Microsoft #MSIgnite 2017 video sessions #Azure #AzureStack #MSOMS

Empower IT and developer productivity with Microsoft Azure with @scottgu

Microsoft Azure virtual machine infrastructure innovation and automation

Microsoft Azure Stack Development Kit and why it matters

Manage hybrid cloud and transform your workplace with PowerShell and Azure Automation

See here all the Microsoft Ignite 2017 video sessions

Thank you Microsoft and MVP’s for those Awesome sessions at Ignite 2017