When you install Azure Virtual Machines or Kubernetes Clusters in the Microsoft Cloud, It’s important to monitor your workload and keep your IT department in Control for the Business. Metric alerts in Azure Monitor work on top of multi-dimensional metrics. These metrics could be platform metrics, custom metrics, popular logs from Azure Monitor converted to metrics and Application Insights metrics.
IT Department of a company has most of the time different teams with each having it’s own responsibility of workloads in the Microsoft Cloud. For example, the Servicedesk is supporting the Business and they like to see if all the Services are up and running for the Business. The Infrastructure Team wants the same, but on deep level components of the Services like Memory, Network, Storage, CPU, Performance, Availability and more. The Technical Application Team is interested if the application is running and working with all the Interfaces, Databases, and/or Azure Pipelines.
Each Team can build there own Azure Dashboard(s) in the Microsoft Cloud.
Here I Have made an easy example of my Windows Server 2019 Virtual Machines and my Azure Kubernetes Cluster in One Microsoft Azure Dashboard :
You can Start from Azure Monitor Metrics
Or you can Start from the Virtual Machine Blade here.
When you have your Azure Monitor metrics ready with the right information then you can create it in your Azure Dashboard for your Team.
Select another Dashboard.
Create your Own Dashboard.
Now we have the first VM with CPU percentage in the Azure Dashboard.
Here I have added More Virtual Machines to the Same Metric Chart.
When you have Azure Kubernetes Cluster to monitor :
From here you can Add Container Insights information into your Azure Dashboard :
Adding Azure Monitor Container Insights of KubeCluster01
The Azure Monitor Container Insights logs for your Dashboard information, with Pin to Dashboard.
Azure Monitor for containers is a feature designed to monitor the performance of container workloads deployed to either Azure Container Instances or managed Kubernetes clusters hosted on Azure Kubernetes Service (AKS). Monitoring your containers is critical, especially when you’re running a production cluster, at scale, with multiple applications.
Azure Monitor for containers gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring from Kubernetes clusters, these metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux and stored in your Log Analytics workspace.
Azure Stack HCI solutions are available for customers who want to run virtualized applications on modern hyperconverged infrastructure (HCI) to lower costs and improve performance. Azure Stack HCI solutions feature the same software-defined compute, storage, and networking software as Azure Stack, and can integrate with Azure for hybrid capabilities such as cloud-based backup, site recovery, monitoring, and more.
Adopting hybrid cloud is a journey and it is important to have a strategy that takes into account different workloads, skillsets, and tools. Microsoft is the only leading cloud vendor that delivers a comprehensive set of hybrid cloud solutions, so customers can use the right tool for the job without compromise.
As enterprise environments now span on-premises to the cloud, customers look to leverage the innovation in Azure services using their on-premises tools. To enable this, Microsoft has integrated System Center with a set of management services in Azure to augment the on-premises tools.
With Service Map integration with System Center Operations Manager (SCOM), you can automatically create distributed application diagrams in Operations Manager (OM) that are based on the dynamic dependency maps in Service Map.
With Azure Management Pack, you can now view perf and alert metrics in SCOM, integrate with web application monitoring in Application Insights, and monitor more PaaS services, such as Azure Blob Storage, Azure Data Factory, etc.
Virtual Machine Manager (VMM) 2019 enables simplified patching of VMs by integrating with Azure Update Management.
What is New in Microsoft System Center 2019
Read here what is new on Microsoft System Center 2019 for your IT Management :
New features in DPM 2019
See the following sections for detailed information about the new features/feature updates supported in DPM 2019.
Modern Backup Storage (MBS) was introduced in System Center Data Protection Manager (DPM) 2016 to deliver 50% storage savings, 3X faster backups, and more efficient, workload-aware storage. DPM 2019 introduces further performance improvements with MBS resulting in 50-70% faster backup with Windows Server 2019. This article provides the upgrade information for System Center 2019 – Data Protection Manager (DPM).
New features in Operations Manager 2019
See the following sections for detailed information about the new and updated features in System Center 2019 – Operations Manager. Features and updates introduced in Operations Manager version 1801 and 1807 are included in version 2019. Plan your Upgrade to SCOM 2019
Service Map integration
Service Map automatically discovers application components on Windows and Linux systems and maps the communication between services. It automatically builds a common reference map of dependencies across your servers, processes, and third-party services. Integration between Service Map and System Center Operations Manager allows you to automatically create distributed application diagrams in Operations Manager that are based on the dynamic dependency maps in Service Map.
Microsoft Azure Sentinel delivers intelligent security analytics and threat intelligence across the enterprise, providing a single solution for alert detection, threat visibility, proactive hunting, and threat response.
Collect data at cloud scale across all users, devices, applications, and infrastructure, both on-premises and in multiple clouds.
Detect previously undetected threats, and minimize false positives using Microsoft’s analytics and unparalleled threat intelligence.
Investigate threats with artificial intelligence, and hunt for suspicious activities at scale, tapping into years of cyber security work at Microsoft.
Respond to incidents rapidly with built-in orchestration and automation of common tasks.
In the following step-by-step guide you get a global overview of Azure Sentinel :
When you have your Azure Sentinel Solutions in place with alerting rules and telemetry and analytics is coming to your workspace, Hunting is the next Threat management tool :
Azure sentinel Hunting
Working with Tags and Collaborate with Teammates
Launch Investigations and Bookmark
Working with Azure Notebooks for Azure Sentinel
Welcome to the Azure Sentinel repository! This repository contains out of the box detections, exploration queries, hunting queries, dashboards and playbooks to help you get ramped up with Azure Sentinel and provide you security content to secure your environment and hunt for threats. You can also submit any issues or feature requests as you onboard to Azure Sentinel. For questions and feedback, please contact AzureSentinel@microsoft.com
Get started from here to Configure your Azure Sentinel Environment
Choose your Data Collections for Azure Sentinel Security
Lot of Choice already Build-in for you.
From here you can make your own Azure Sentinel Analytics Alert Rules.
Alert Rules
Create Alert rules with the right mappings, triggers, and scheduling, response automation.
Add your own playbooks for your Security
Unlock the power of AI for security with Machine Learning
Machine Learning in Azure Sentinel is built-in right from the beginning. We have thoughtfully designed the system with ML innovations aimed to make security analysts, security data scientists and engineers productive. One such innovation is Azure Sentinel Fusion built especially to reduce alert fatigue.
Building your Full Screen Dashboard for Monitoring
More information about Azure Sentinel Intelligent Security :
Learn Azure in a Month of Lunches breaks down the most important Azure concepts into bite-sized lessons with exercises and labs—along with project files available in GitHub—to reinforce your skills. Learn how to:
Use core Azure infrastructure and platform services—including how to choose which service for which task.
Plan appropriately for availability, scale, and security while considering cost and performance.
Integrate key technologies, including containers and Kubernetes, artificial intelligence and machine learning, and the Internet of Things.
When you have your Hybrid Cloud Enterprise Design ready in a Microsoft HUB-Spoke model and your Security in place, you can do your optimize on your Azure workloads and keep up-to-date for your compliancy. Microsoft Azure Security Center can support you in Security and Compliancy (GDPR). Here you see my former blogposts about Microsoft Azure HUB-Spoke model architecture and Security by design :
Security in software is always on the move and changing in this world, when you think you are ready something has changed already. That’s why I love Microsoft Azure Security Center to keep you posted and giving you advise on Security but also on Compliancy.
From here you see a high-level overview of these new possibilities in Microsoft Azure Security Center :
Security Center Overview
Microsoft Azure Security Center is working with the following navigation menu’s on the left :
General
Policy & Compliance
Resource Security Hygiene
Advanced Cloud Defense
Threat Protection
Automation & Orchestration
Microsoft Azure Secure Score Dashboard
Microsoft Azure Security Center is working with Overall Secure Score. In my Test LAB we have some work to do 😉
The Azure secure score reviews your security recommendations and prioritizes them for you, so you know which recommendations to perform first. This helps you find the most serious security vulnerabilities so you can prioritize investigation. Secure score is a tool that helps you assess your workload security posture. Improve your secure score in Azure Security Center
Azure Security Center Recommendations
Microsoft Azure Security Center gives you advise to make your Security Score higher and you can improve immediately.
Open Subnet without NSG.
From here you can Enable a Network Security Group (NSG) on the Subnet and make your network more secure.
Creating NSG from Azure Security Center.
A subnet with NSG.
Azure Security Center Advise on Disk Encryption
Description on Applying Disk Encryption on your Virtual Machines
General Information, with Impact and Implementation Cost.
Threats, what can happen when you don’t implement the security.
Security is a on-going process 24 hours -365 days to monitor, analyze, and prevent security issues. Working on Compliancy for your Business and making your own Security policies is important. Microsoft Azure Security Center can support you in this journey. When you Optimize your Azure workloads or make new solutions in Azure, keep it secure with Microsoft Azure Security Center.
When you are implementing your Microsoft Azure Design like a HUB-Spoke model you have to deal with security of your Azure environment (Virtual Datacenter). One of them are Network Security Groups to protect your Virtual networks and make communication between Azure subnets possible in a Secure Azure Virtual Datacenter.
You really have to plan your Azure Virtual networks and implement it by Architectural Design. Now I’m writing about Azure Network Security Groups which is important, but there are more items to deal with like :
Naming Conventions in your Azure Virtual Datacenter
Azure Subscriptions ( who is Owner, Contributor, or Reader? )
Azure Regions ( Where is my Datacenter in the world? )
Azure VNET and Sub-Nets ( IP-addresses )
Security of your Virtual Networks ( Traffic filtering, Routing )
How to Manage Microsoft Azure Network Security Groups (NSG) ?
IMPORTANT: Before you start with Azure Network Security Groups, test every ARM JSON Script first in your Dev-Test Azure Subscription before you do production. Talk with your Cloud Administrators, because when you implement Infrastructure as Code (IaC) and work with ARM Templates you can delete manual settings in NSG’s for example, which will give you troubles like no protocol communication between subnets.
When you start new in Microsoft Azure, It’s easy to make your Azure security baseline for all of your Network Security Groups (NSG’s) by Azure Resource Manager (ARM) templates.
When you have a Microsoft Azure HUB-Spoke model with for example four Azure Subscriptions and a lot of Azure Virtual Networks – Subnets, you got a lot of NSG’s to manage and you don’t want to manage those manually. So there are different ways to manage Azure Network Security Groups via ARM Templates. For example :
ARM Templates from the Azure Portal
Make your ARM Baseline template.
Edit your parameters and Deploy.
Here you saw a standard Virtual Machine Deployment, but you can add of course all of your Azure Resource Manager templates here including your NSG Base Line template. In this way your deployments are documented ( Scripts).
Azure DevOps Services is a cloud service for collaborating on code development. It provides an integrated set of features that you access through your web browser or IDE client. The features are included, as follows:
Git repositories for source control of your code
Build and release services to support continuous integration and delivery of your apps
Agile tools to support planning and tracking your work, code defects, and issues using Kanban and Scrum methods
Many tools to test your apps, including manual/exploratory testing, load testing, and continuous testing
Highly customizable dashboards for sharing progress and trends
Built-in wiki for sharing information with your team
The Azure DevOps ecosystem also provides support for adding extensions and integrating with other popular services, such as: Campfire, Slack, Trello, UserVoice, and more, and developing your own custom extensions.
Choose Azure DevOps Services when you want the following results:
Quick set up
Maintenance-free operations
Easy collaboration across domains
Elastic scale
Rock-solid security
You’ll also have access to cloud load testing, cloud build servers, and application insights.
Azure DevOps Repo for your Templates
From here you can make your Infrastructure as Code (IaC) Pipelines together with your Cloud Administrator Team 😉
When you have your Azure DevOps Private Repository in place and you like to work with Visual Studio for example, you can connect to your Repo and Check-in your NSG ARM Script but Deploy with Visual Studio to your Azure Virtual Datacenter.
So you see there are enough ways to deploy ARM Templates and this is not all, because you can also use Azure Cloudshell for example or other CLI command-line interfaces. But now we want to set the NSG Baseline for our Azure Subscription. A good start is to see the possibilities in the JSON scripting for Network Security Groups. Here you find the settings and explanation of Azure Components.
By Default is Internet available in a NSG ! So here you see that Internet is not allowed only the AzureCloud West Europe resources because some Azure SDK Component work via ” Public internet” ( Microsoft IP-Addresses). (RDP protocol is marked and not set in this example for Security reasons)
Internet by Default Rules, so you must set your security Rules !
Conclusion :
You really have to implement Azure Security by Design, make your Base-line with ARM Templates in a Private Repo for your Azure Network Security Groups with the Correct RBAC Configuration for your Cloud Administrator Team. Don’t make them manually and do settings manually when you have a lot of NSG’s ! Versions of your ARM templates are documented in your Repository 😉
Test Always first in a Dev-Test Azure Subscription or in Azure DevOps with a Test plan before you implement in Production.
Microsoft Azure Hybrid Cloud Architecture HUB-Spoke Model
Microsoft Azure Hub-Spoke model
This blogpost about Microsoft Azure Hub-Spoke model by Enterprise Design 2 of 4 “Lift and Shift” is part of a Datacenter transition to Microsoft Azure Intelligent Cloud. It’s talking about Azure Architecture, Security, Assessment, Azure Policy, and implementation of the design. Here you find the first blogposts :
When creating a building, scaffolding is used to create the basis of a structure. The scaffold guides the general outline and provides anchor points for more permanent systems to be mounted. An enterprise scaffold is the same: a set of flexible controls and Azure capabilities that provide structure to the environment, and anchors for services built on the public cloud. It provides the builders (IT and business groups) a foundation to create and attach new services keeping speed of delivery in mind. Read more hereI did the “Lift and Shift” between quotes because it’s important to follow the process workflow to be successful in your Datacenter transition to the Microsoft Azure Cloud.
App Migration to Azure: Your options explained by Jeremy Winter
The Azure Migrate service assesses on-premises workloads for migration to Azure. The service assesses the migration suitability of on-premises machines, performs performance-based sizing, and provides cost estimations for running on-premises machines in Azure. If you’re contemplating lift-and-shift migrations, or are in the early assessment stages of migration, this service is for you. After the assessment, you can use services such as Azure Site Recovery and Azure Database Migration Service, to migrate the machines to Azure.
In your datacenter you got all kind of different workloads and solutions like :
Hyper-V Clusters
VMware Clusters
SQL Clusters
Print Clusters
File Clusters
Web Farm
Two or three tiers solutions
Physical Servers
Different Storage solutions
When you do your Datacenter Assessment it’s important to get your workloads visible, because “Lift and Shift” with Azure Site Recovery (ASR) of a Virtual Machine is an different scenario then SQL database migration to Azure. That’s why Microsoft has different tooling like :
To get your dependencies in your Datacenter on the map, Microsoft has Azure Service Maps.
Service Map automatically discovers application components on Windows and Linux systems and maps the communication between services. With Service Map, you can view your servers in the way that you think of them: as interconnected systems that deliver critical services. Service Map shows connections between servers, processes, inbound and outbound connection latency, and ports across any TCP-connected architecture, with no configuration required other than the installation of an agent.
This is very handy to get insides of your Datacenter communication workloads.
Installation example of Hyper-V Virtual Machines with ASR
In the following step-by-step guide we will install the Azure Site Recovery Agent on a Hyper-V host and migrate a virtual machine to Microsoft azure in a “Lift and Shift” way.
First create a Recovery Services Vault => Click Add.
Then you go to your new created Recovery Vault and click on Getting started for Site Recovery. => Prepare infrastructure and follow the steps.
When you have selected Hyper-V VM to Azure, the next step is the ASR Deployment Planner tool kit. Here you find more information on Azure Site Recovery Deployment Planner user guide for Hyper-V-to-Azure production deployments.
Then in step 3 you will make your Hyper-V Site in Microsoft azure with the Right Hyper-V Servers.
Give your Hyper-V Site the right name, especially when you have a lot of Hyper-V Clusters with Different workloads.
Here is where the registration begins with the Azure Site Recovery (ASR) Agent installation on your Hyper-V Host.
Follow the five steps and make sure your Hyper-V Node can access Azure via secure port 443(https) via Proxy or firewall rules.
Install as Administrator the AzureSiteRecoveryProvider.exe file on the Hyper-V host.
Click on Next
Choose your Installation location and Click on Install.
The Azure Site Recovery agent is installed and need to be registered with your Azure Recovery Vault.
For this you need the key file from the Azure portal to download at step 4. Click on Register.
Browse to your downloaded key file from the Azure Portal Recovery Vault and click on Next.
When you have a proxy you can select that, otherwise select Next.
Now your Azure ASR Agent on Hyper-V is registered with your Azure Site Recovery Vault.
In the Azure Portal you will see your Hyper-V Node, in my Demo LAB it’s WAC01.MVPLAB.LOCAL.
In the next step you can choose an existing Storage account, or a new one with different specifications.
Check also after storage your network in azure.
In this step we create the replication policy.
Set your own settings.
The Replication policy is added to the configuration.
When you click on OK the Infrastructure is done.
We are now going to enable the replication :
Select your Source and location.
here you select your target Storage account, Resource Group and Network.
The connections are made between Hyper-V, ASR Vault and Storage.
Select the Virtual Machine(s) from the Hyper-V host to replicate for migration with ASR
Configure the properties.
Click on OK
From here the Replication will begin from Hyper-V Host to Azure 🙂
Azure Sire Recovery Replication Job status.
Replicated item(s)
To make your recovery plan and do the failover for migration to azure, you have to wait until the first replication is done for 100%.
Azure Site Recovery Plan for failover (Migration)
Make recovery Plan.
Click OK
The Target in the recovery plan can only be selected when the first replication is done.
Overview of the Azure Site Recovery Migration failover.
From the Hyper-V Host you can pause or see the replication health status.
Azure Migrate Virtual Machines using Azure Site Recovery video with Microsoft Jeff Woolsey
Microsoft Azure Data Migration Assistant
To migrate your SQL Backend to Microsoft Azure, use this step-by-step instructions help you perform your first assessment for migrating to on-premises SQL Server, SQL Server running on an Azure VM, or Azure SQL Database, by using Data Migration Assistant.
Conclusion :
“Lift and Shift” Migration of your complete datacenter exists of different scenarios for your workloads to Microsoft Azure. With that said, Microsoft has for each scenario tooling available to get the job done. It’s all about a good Architectural Design, Security in place, People and process to get your Intelligent Azure Cloud up and running for your Business.
Next Blogpost Microsoft Azure Hub-Spoke model by Enterprise Design 3 of 4 : SQL assessment and Data Migration to Azure
“Humans are susceptible to social engineering. Machines are susceptible to tampering. Machine learning is vulnerable to adversarial attacks. Singular machine learning models can be “gamed” leading to unexpected outcomes.”
In this talk, they compare the difficulty of tampering with cloud-based models and client-based models. Then discuss how they develop stacked ensemble models to make machine learning defenses less susceptible to tampering and significantly improve overall protection for customers. They talk about the diversity of base ML models and technical details on how they are optimized to handle different threat scenarios. Lastly, they describe suspected tampering activity they have witnessed using protection telemetry from over half a billion computers, and whether mitigation worked.
Take action on Alerts
Make your Own rules based on Alerts.
Select another Dashboard.
Cloud and Datacenter Management Blog 