People, Process, and Technology to continually provide value to customers.
While adopting DevOps practices automates and optimizes processes through technology, it all starts with the culture inside the organization—and the people who play a part in it. The challenge of cultivating a DevOps culture requires deep changes in the way people work and collaborate. But when organizations commit to a DevOps culture, they can create the environment for high-performing teams to develop.
My name is James van den Berg and I’m a MVP in Cloud and Datacenter Management on my DevOps journey as an IT Infrastructure Guy managing datacenters on-prem and in the Microsoft Azure Cloud. Today It’s not only a Virtual Machine or a Website to deploy for your customers, it’s much more then that like :
Time to market, deploy your solution fast without waiting on dependencies because you automated your process with a CI CD Pipeline.
Security and Monitoring to keep you in Controle.
Working together with different Teams who are each responsible for a part of the solution.
Start Creating Azure Kubernetes Cluster for your Containers.
Managed Azure Kubernetes Service (AKS) makes deploying and managing containerized applications easy. It offers serverless Kubernetes, an integrated continuous integration and continuous delivery (CI/CD) experience, and enterprise-grade security and governance. As a hosted Kubernetes service, Azure handles critical tasks like health monitoring and maintenance for you. The Kubernetes masters are managed by Azure. You only manage and maintain the agent nodes. As a managed Kubernetes service, AKS is free – you only pay for the agent nodes within your clusters, not for the masters. In the following steps you can see the different ways for creating Azure Kubernetes Cluster via the Azure Portal, or via Azure Cloud Shell, or via Azure Resource Template. When the Microsoft Azure Kubernetes Cluster is running, then I will explain the different ways for deploying container workloads on AKS. When your workload is running on Azure Kubernetes Services, you also have to monitor your Container workloads with Azure Monitor Container Insights to keep in Controle. Let’s start with installing Azure Kubernetes Services (AKS)
Installing Azure Kubernetes Cluster via the Portal.
To begin you need of course a Microsoft Azure Subscription and you can start for free here
Basics information of the Azure Kubernetes Cluster
To Create the Azure Kubernetes Cluster, you have to follow these steps and type the right information in the Portal:
Basics
Scale
Authentication
Networking
Monitoring
Tags
Review + Create
At the basics screen you select the right Azure Subscription and the Resource Group. You can create a New Resource Group or one you already made.
At Cluster details, you give your Cluster a name and select the Kubernetes version.
Here you select the Kubernetes Node size for your Container workload and the number of nodes.
You can start a Cluster already with One node, but choose to start with the right size for your workloads.
When you click on Change size, you can choose your nodes to do the job. 😉
Select the right Size node
Then we go to step 2 and that is Scale.
2. Scale options in Azure Kubernetes Cluster
Here you have two options :
Virtual Nodes
VM Scale sets (Preview)
To quickly deploy workloads in an Azure Kubernetes Service (AKS) cluster, you can use virtual nodes. With virtual nodes, you have fast provisioning of pods, and only pay per second for their execution time. In a scaling scenario, you don’t need to wait for the Kubernetes cluster autoscaler to deploy VM compute nodes to run the additional pods. Virtual nodes are only supported with Linux pods and nodes. More information here about Virtual Nodes
To create an AKS cluster that can use multiple node pools, first enable two feature flags on your subscription. Multi-node pool clusters use a virtual machine scale set (VMSS) to manage the deployment and configuration of the Kubernetes nodes. With this Preview feature you can run Linux Containers and Windows Containers on the same Cluster. More information here about VM Scale sets (Preview)
3, Authentication
The service principal is needed to dynamically create and manage other Azure resources such as an Azure load balancer or container registry (ACR). To interact with Azure APIs, an AKS cluster requires an Azure Active Directory (AD) service principal. More information about the Service Principal can be found here
Azure Kubernetes Service (AKS) can be configured to use Azure Active Directory (Azure AD) for user authentication. In this configuration, you can sign in to an AKS cluster by using your Azure AD authentication token.
Cluster administrators can configure Kubernetes role-based access control (RBAC) based on a user’s identity or directory group membership. More information about RBAC for AKS
4. Networking
Configuring the virtual Networks for your Azure Kubernetes Cluster is important for the right IP range but later on also for the Network Security Groups (NSG).
Here you see an example of the Kubernetes NSG which is connected to the Internet by Default after installation, you can deep dive into security but be careful which settings you do here because Microsoft resources must have access to service the Azure Kubernetes Cluster.
NSG created after installation is finished
NSG Rule set Inbound and outbound
In a container-based microservices approach to application development, application components must work together to process their tasks. Kubernetes provides various resources that enable this application communication. You can connect to and expose applications internally or externally. To build highly available applications, you can load balance your applications. More complex applications may require configuration of ingress traffic for SSL/TLS termination or routing of multiple components. For security reasons, you may also need to restrict the flow of network traffic into or between pods and nodes.
Best practices for network connectivity and security in Azure Kubernetes Service (AKS):
Keep Azure Monitoring Enabled and Connect to your Log Analytics workspace or create a new workspace for Container monitoring of your Azure Kubernetes Cluster.
Azure Monitor for containers is a feature designed to monitor the performance of container workloads deployed to either Azure Container Instances or managed Kubernetes clusters hosted on Azure Kubernetes Service (AKS). Monitoring your containers is critical, especially when you’re running a production cluster, at scale, with multiple applications.
Azure Monitor for containers gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs are also collected. After you enable monitoring from Kubernetes clusters, metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux. Metrics are written to the metrics store and log data is written to the logs store associated with your Log Analytics workspace.
6. Tags
When you build more Azure Kubernetes Clusters for different departments or teams you can TAG your Clusters for organizing your billing and security for example. Here you find more information about tagging.
After this you click on the last step Review and Create The Azure portal will do a validation of your Azure Kubernetes Cluster settings, and when it’s validated you hit Create. But when you want more Automation, you can download the JSON ARM template first and use that.
Installing Azure Kubernetes Cluster via Cloud Shell
Azure Cloud Shell AKS CLI
Azure hosts Azure Cloud Shell, an interactive shell environment that you can use through your browser. Cloud Shell lets you use either bash or PowerShell to work with Azure services. You can use the Cloud Shell pre-installed commands to run the code in this article without having to install anything on your local environment.
Here you see an Example of AKS CLI with Auto Scaler with max count of nodes 😉
Installing Azure Kubernetes Cluster via Template
Create Azure Kubernetes Cluster via Template in the Portal
Now you have your Microsoft Azure Kubernetes Cluster (AKS) running in the Cloud, you want to deploy your Container workloads on the Cluster. In the following steps you see different deployments.
Deploy Container workload with Azure DevOps Project
Deployment Center
First you select your repository where your source code is of your workload.
Set the information right and click Next.
Simple example Click Next
Create a Container Registry.
Building Pipeline with Azure DevOps.
Here you see the Building in Microsoft Azure DevOps.
Build, test, and deploy in any language, to any cloud—or on-premises. Run in parallel on Linux, macOS, and Windows, and deploy containers to individual hosts or Kubernetes.
Here you find all the information about Microsoft Azure DevOpsfor your workloads, code and Deployments.
Deploying Container workload completed with Azure DevOps.
Deploy Container Workloads via Visual Studio Code
When you download and install Visual Studio Code on your computer, you can install the Azure Kubernetes extension for VSCode.
Here you see Microsoft Visual Studio Code connected with my Azure subscription where my Azure Kubernetes Cluster is running. With the standard Helm Repository packages for deployment to your AKS Cluster. Here you see a WordPress yaml file which I deployed to the Kubernetes Cluster on Azure.
Just Select your Package and Install on Azure Kubernetes.
From here you can into the Container and read the logs.
I’m using Visual Studio Code a lot for Azure Kubernetes but also for Docker Containers and images.
Making Azure ARM JSON templates and this great for Infrastructure as Code.
Azure Monitoring with Container Insights
In One Dashboard you can see the Status of all your Clusters
Azure Monitor Container Insights Live View
Because we installed Azure Monitor for Containers on the Microsoft Azure Kubernetes Cluster, we can live see what is happening inside the Kubernetes Cluster with the containers. This is a great feature when you have a issue with a Container for troubleshooting fast and see what is happening.
Conclusion
Microsoft Azure Kubernetes Cluster is fast and easy to manage. You can upgrade your Cluster without downtime of your Container workload. With Azure Monitor for Containers you can see what’s happening inside the container and you can set alerts when something went wrong. This keeps you in Controle of the solution. With Deployment center alias Azure DevOps Projects you can deploy your workload via Azure DevOps Pipeline and work on versioning, testplans, Azure DevOps repo and work together with a Team on the following releases. Working with Azure Kubernetes Multi node pools with Linux and Windows on the same Cluster is possible. Try it yourself and start with a Proof of Concept for your Business.
First of all Thank you for following me and Sharing Microsoft Cloud and Datacenter Management content on Social Media 🙂 Sharing & Learning Together is Better.
Welcome 577 New Followers on Twitter of the 5904 Followers 🙂
More then 2.807.000 Tweet impressions in One year !
Started with Friday is MVPbuzz Day for Education to get Azure Cloud in the Classroom, working together with Teachers and Students in my Free time.
Working with Microsoft Learn in Teams for the Students.
Meetings and Speaking for Education, all about Azure and AzureStack Technologies.
Conferences, like the Global MVP Summit 2018, DevOps Amsterdam, Community Group meetings.
Microsoft Ignite, Microsoft Build, Microsoft Connect events.
Almost every week Microsoft Product Group Intervention (PGI) sessions Online.
Sharing the News every Day via Twitter, Facebook, LinkedIn, Microsoft Tech Community, Blog
But what is coming in 2019 ?
Rocking with Azure in the Classroom !
I will continue every day sharing knowledge with the Community and continue my Free work on MVPbuzz Friday for Education to get Azure Cloud Technology in the Classroom for Teachers and Students.
The trend I see for 2019 is more Infrastructure and Security by Code with Microsoft Azure DevOps
and of course you have to be in Control with Microsoft Azure Monitor
I will write a blogpost in January 2019 about Microsoft Azure Hub-Spoke model by Enterprise Design 4 of 4: Optimize your Azure Workload.
More Items in 2019 to come :
Microsoft Azure Security Center for Hybrid IT
Windows Server 2019 in combination with Azure Cloud Services.
More on Containers in the Cloud
Azure Stack and ASDK
Integration with Azure Cloud.
API Management
Azure DevOps Pipelines and Collabration
Azure IoT for Smart Cities and Buildings combined with AI Technology
2019 will be a Great year again with New Microsoft Technologies and Features for your business.
Azure DevOps Services is a cloud service for collaborating on code development. It provides an integrated set of features that you access through your web browser or IDE client. The features are included, as follows:
Git repositories for source control of your code
Build and release services to support continuous integration and delivery of your apps
Agile tools to support planning and tracking your work, code defects, and issues using Kanban and Scrum methods
Many tools to test your apps, including manual/exploratory testing, load testing, and continuous testing
Highly customizable dashboards for sharing progress and trends
Built-in wiki for sharing information with your team
The Azure DevOps ecosystem also provides support for adding extensions and integrating with other popular services, such as: Campfire, Slack, Trello, UserVoice, and more, and developing your own custom extensions.
When you are implementing your Microsoft Azure Design like a HUB-Spoke model you have to deal with security of your Azure environment (Virtual Datacenter). One of them are Network Security Groups to protect your Virtual networks and make communication between Azure subnets possible in a Secure Azure Virtual Datacenter.
You really have to plan your Azure Virtual networks and implement it by Architectural Design. Now I’m writing about Azure Network Security Groups which is important, but there are more items to deal with like :
Naming Conventions in your Azure Virtual Datacenter
Azure Subscriptions ( who is Owner, Contributor, or Reader? )
Azure Regions ( Where is my Datacenter in the world? )
Azure VNET and Sub-Nets ( IP-addresses )
Security of your Virtual Networks ( Traffic filtering, Routing )
How to Manage Microsoft Azure Network Security Groups (NSG) ?
IMPORTANT: Before you start with Azure Network Security Groups, test every ARM JSON Script first in your Dev-Test Azure Subscription before you do production. Talk with your Cloud Administrators, because when you implement Infrastructure as Code (IaC) and work with ARM Templates you can delete manual settings in NSG’s for example, which will give you troubles like no protocol communication between subnets.
When you start new in Microsoft Azure, It’s easy to make your Azure security baseline for all of your Network Security Groups (NSG’s) by Azure Resource Manager (ARM) templates.
When you have a Microsoft Azure HUB-Spoke model with for example four Azure Subscriptions and a lot of Azure Virtual Networks – Subnets, you got a lot of NSG’s to manage and you don’t want to manage those manually. So there are different ways to manage Azure Network Security Groups via ARM Templates. For example :
ARM Templates from the Azure Portal
Make your ARM Baseline template.
Edit your parameters and Deploy.
Here you saw a standard Virtual Machine Deployment, but you can add of course all of your Azure Resource Manager templates here including your NSG Base Line template. In this way your deployments are documented ( Scripts).
Azure DevOps Services is a cloud service for collaborating on code development. It provides an integrated set of features that you access through your web browser or IDE client. The features are included, as follows:
Git repositories for source control of your code
Build and release services to support continuous integration and delivery of your apps
Agile tools to support planning and tracking your work, code defects, and issues using Kanban and Scrum methods
Many tools to test your apps, including manual/exploratory testing, load testing, and continuous testing
Highly customizable dashboards for sharing progress and trends
Built-in wiki for sharing information with your team
The Azure DevOps ecosystem also provides support for adding extensions and integrating with other popular services, such as: Campfire, Slack, Trello, UserVoice, and more, and developing your own custom extensions.
Choose Azure DevOps Services when you want the following results:
Quick set up
Maintenance-free operations
Easy collaboration across domains
Elastic scale
Rock-solid security
You’ll also have access to cloud load testing, cloud build servers, and application insights.
Azure DevOps Repo for your Templates
From here you can make your Infrastructure as Code (IaC) Pipelines together with your Cloud Administrator Team 😉
When you have your Azure DevOps Private Repository in place and you like to work with Visual Studio for example, you can connect to your Repo and Check-in your NSG ARM Script but Deploy with Visual Studio to your Azure Virtual Datacenter.
So you see there are enough ways to deploy ARM Templates and this is not all, because you can also use Azure Cloudshell for example or other CLI command-line interfaces. But now we want to set the NSG Baseline for our Azure Subscription. A good start is to see the possibilities in the JSON scripting for Network Security Groups. Here you find the settings and explanation of Azure Components.
By Default is Internet available in a NSG ! So here you see that Internet is not allowed only the AzureCloud West Europe resources because some Azure SDK Component work via ” Public internet” ( Microsoft IP-Addresses). (RDP protocol is marked and not set in this example for Security reasons)
Internet by Default Rules, so you must set your security Rules !
Conclusion :
You really have to implement Azure Security by Design, make your Base-line with ARM Templates in a Private Repo for your Azure Network Security Groups with the Correct RBAC Configuration for your Cloud Administrator Team. Don’t make them manually and do settings manually when you have a lot of NSG’s ! Versions of your ARM templates are documented in your Repository 😉
Test Always first in a Dev-Test Azure Subscription or in Azure DevOps with a Test plan before you implement in Production.
End-to-end Pipelines for Automating Microsoft Azure Deployments
Overview :
Imagine a fully automated, end-to-end pipeline for your cloud deployments—one that encompasses and automates everything:
• Source code repos.
• The build and release iterations.
• Agile processes supported by continuous integration and continuous deployment (CI/CD)
• Security and governance.
• Business unit chargebacks.
• Support and maintenance.
Azure services and infrastructure-as-code (IaC) make control plane automation very achievable. Many enterprise IT groups dream of creating or unifying their disparate automation processes and supporting a common, enterprise-wide datacenter control plane in the cloud that is integrated with their existing or new DevOps workflows. Their development environments may use Jenkins, Azure DevOps Services (formerly Visual Studio Team Services), Visual Studio Team Foundation Server (TFS), Atlassian, or other services. The challenge is to automate beyond the CI/CD pipeline to the management and policy layers. From a planning and architecture standpoint, it can seem like an overwhelming program of interdependent systems and processes. This guide outlines a planning process that you can use for automated support of your cloud deployments and DevOps workflows beyond the CI/CD pipeline. The Azure platform provides services you can use, or you can choose to work with third-party or open source options. The process is based on real-world examples that we have deployed with enterprise customers on Azure.
This whitepaper was authored by Tim Ehlen. It was edited by Nanette Ray. It was reviewed by AzureCAT.
Azure Cloud Shell is an interactive, browser-accessible shell for managing Azure resources. It provides the flexibility of choosing the shell experience that best suits the way you work. Linux users can opt for a Bash experience, while Windows users can opt for PowerShell. But now with Project Monaco my favorite tool Visual Studio Code is integrated in Microsoft Azure Cloud Shell 🙂 Awesome for Infrastructure as Code (IaC) like I did with Terraform
Create a Terraform configuration file
In this section, you create a file that contains resource definitions for your infrastructure.
Create a new file named main.tf.
Copy following sample resource definitions into the newly created main.tf file:
resource “azurerm_resource_group” “test” {
name = “acctestrg”
location = “West US 2”
}
The Azure DevOps Project presents a simplified experience where you bring your existing code and Git repository, or choose from one of the sample applications to create a continuous integration (CI) and continuous delivery (CD) pipeline to Azure. The DevOps project automatically creates Azure resources such as a new Azure virtual machine, creates and configures a release pipeline in VSTS that includes a build definition for CI, sets up a release definition for CD, and then creates an Azure Application Insights resource for monitoring.
Infrastructure as Code (IaC) gives you benefits like :
Consistency in naming conventions of Azure components
Working together in the same way with your company policies
Reusability of Templates
Automatic documentation and CMDB of deployments in your repository
Rapid deployments
Flexibility and Scalability in code for Azure Deployments
As an Large Enterprise Company you don’t want to Click and Type in the Azure Portal with lot of employees to get the job done in a consistent way. The changes and deployments will be different in time because people can make mistakes. For Developers it’s important to make your building process before you publish your application, so why not for DevOps and ITpro to do the same thing for Infrastructure.
In the following step-by-step guide you will learn how to make a Microsoft Azure DevOps Project and make a CI/CD Pipeline deploying a virtual machine with your ASP.net Application.
When you have your prerequisites in place you can start with the following steps :
Search for DevOps at All Services in the Azure Portal
Select .NET and Click on Next
You can see where you are in the flow of creating your CI/CD Pipeline, when you need a Azure SQL Database for your ASP.net application you can select Add a Database (Option). This will provide you Azure SQL as a Service (PaaS).
Database-as-a-Service
(I didn’t Choose for SQL)
In this step select Virtual Machine and click Next
From here you can create a VSTS account or your Existing account of Visual Studio Team Services. After selecting VSTS you can manage your Azure settings and by clicking on Change you can select the Azure options.
Select the Virtual Machine you need for your Application.
Here you see the Deployment Running
Important for Infrastructure as Code (IaC), the Deployment template can be saved into the library and / or you can download it for reusability or make your own policies into the template.
When you save it into the Azure Library you get the release notes and who’s the publisher
In the Microsoft Azure DevOps Project main Dashboard you will see the status of your CI/CD Pipeline and that release is in progress or not. On the right-side of the Dashboard you see the Azure resources like the Application endpoint, the Virtual Machine and Application Insights for monitoring. When the CI/CD Pipeline deployment is succeeded you can browse to your ASP.net Application.
Your Application.
Your Virtual Machine Running and in the Monitoring.
The Microsoft Azure DevOps Project CI/CD Pipeline is Completed.
Application Insights is an extensible Application Performance Management (APM) service for web developers on multiple platforms. Use it to monitor your live web application. It will automatically detect performance anomalies. It includes powerful analytics tools to help you diagnose issues and to understand what users actually do with your app. It’s designed to help you continuously improve performance and usability. It works for apps on a wide variety of platforms including .NET, Node.js and J2EE, hosted on-premises or in the cloud. It integrates with your DevOps process, and has connection points to a variety of development tools. It can monitor and analyze telemetry from mobile apps by integrating with Visual Studio App Center and HockeyApp.
You can drill down into the error to see what is happening.
Azure Application Insights topology
Application Insights is aimed at the development team, to help you understand how your app is performing and how it’s being used. It monitors: Request rates, response times, and failure rates – Find out which pages are most popular, at what times of day, and where your users are. See which pages perform best. If your response times and failure rates go high when there are more requests, then perhaps you have a resourcing problem. Dependency rates, response times, and failure rates – Find out whether external services are slowing you down. Exceptions – Analyse the aggregated statistics, or pick specific instances and drill into the stack trace and related requests. Both server and browser exceptions are reported. Page views and load performance – reported by your users’ browsers. AJAX calls from web pages – rates, response times, and failure rates.
User and session counts. Performance counters from your Windows or Linux server machines, such as CPU, memory, and network usage. Host diagnostics from Docker or Azure. Diagnostic trace logs from your app – so that you can correlate trace events with requests. Custom events and metrics that you write yourself in the client or server code, to track business events such as items sold or games won.
You can also drill down into Microsoft Azure Log Analytics and run your analytics queries to get the right information you want for troubleshooting. More information on Azure Log Analytics and queries is on MSFT docs.
From App Insight we see it was an Exception error
Because the Azure DevOps Project is connected with VSTS you can follow the Build and Release here to and you got your documentation of the CI/CD Pipeline.
From here you can work with your Developers and DevOps and manage the User and Groups security in de CI/CD Pipeline for the next Build. Working together to build innovative apps via VSTS from one Dashboard :
VSTS Dashboard
Next day you see it was one time error and the Pipeline is running Fine 😉
Invest in your CI/CD Pipeline and make your own environment is important before you deploy into Azure production for your business. Make your ARM Templates and Code in repositories like Git or VSTS. When you have this all in place your are more in control of your consistent Deployments and Changes in the Azure Cloud. I hope this blogpost is useful for you and your Company. Start today with Infrastructure as Code (IaC) and get the benefits 😉
Cloud and Datacenter Management Blog 