Cloud and Datacenter Management Blog

Microsoft Hybrid Cloud blogsite about Management


Leave a comment

#Microsoft Data Deduplication Overview and DPM Storage #Winserv #sysctr #SCDPM

Data deduplication involves finding and removing duplication within data without compromising its fidelity or integrity. The goal is to store more data in less space by segmenting files into small variable-sized chunks (32–128 KB), identifying duplicate chunks, and maintaining a single copy of each chunk. Redundant copies of the chunk are replaced by a reference to the single copy. The chunks are compressed and then organized into special container files in the System Volume Information folder. The result is an on-disk transformation of each file as shown in Figure 1. After deduplication, files are no longer stored as independent streams of data, and they are replaced with stubs that point to data blocks that are stored within a common chunk store. Because these files share blocks, those blocks are only stored once, which reduces the disk space needed to store all files. During file access, the correct blocks are transparently assembled to serve the data without calling the application or the user having any knowledge of the on-disk transformation to the file. This enables administrators to apply deduplication to files without having to worry about any change in behavior to the applications or impact to users who are accessing those files.

DedupOn-disk transformation of files during data deduplication

After a volume is enabled for deduplication and the data is optimized, the volume contains the following:

  •  Unoptimized files. For example, unoptimized files could include files that do not meet the selected file-age policy setting, system state files, alternate data streams, encrypted files, files with extended attributes, files smaller than 32 KB, other reparse point files, or files in use by other applications (the “in use” limit is removed in Windows Server 2012 R2).
  • Optimized files. Files that are stored as reparse points that contain pointers to a map of the respective chunks in the chunk store that are needed to restore the file when it is requested.
  • Chunk store. Location for the optimized file data.
  • Additional free space. The optimized files and chunk store occupy much less space than they did prior to optimization.

To cope with data storage growth in the enterprise, administrators are consolidating servers and making capacity scaling and data optimization key goals. Data deduplication provides practical ways to achieve these goals, including:

  •  Capacity optimization. Data deduplication stores more data in less physical space. It achieves greater storage efficiency than was possible by using features such as Single Instance Storage (SIS) or NTFS compression. Data deduplication uses subfile variable-size chunking and compression, which deliver optimization ratios of 2:1 for general file servers and up to 20:1 for virtualization data.
  • Scale and performance. Data deduplication is highly scalable, resource efficient, and nonintrusive. It can process up to 50 MB per second in Windows Server 2012 R2, and about 20 MB of data per second in Windows Server 2012. It can run on multiple volumes simultaneously without affecting other workloads on the server. Low impact on the server workloads is maintained by throttling the CPU and memory resources that are consumed. If the server gets very busy, deduplication can stop completely. In addition, administrators have the flexibility to run data deduplication jobs at any time, set schedules for when data deduplication should run, and establish file selection policies.
  •  Reliability and data integrity. When data deduplication is applied, the integrity of the data is maintained. Data Deduplication uses checksum, consistency, and identity validation to ensure data integrity. For all metadata and the most frequently referenced data, data deduplication maintains redundancy to ensure that the data is recoverable in the event of data corruption.
  • Bandwidth efficiency with BranchCache. Through integration with BranchCache, the same optimization techniques are applied to data transferred over the WAN to a branch office. The result is faster file download times and reduced bandwidth consumption.
  • Optimization management with familiar tools. Data deduplication has optimization functionality built into Server Manager and Windows PowerShell. Default settings can provide savings immediately, or administrators can fine-tune the settings to see more gains. One can easily use Windows PowerShell cmdlets to start an optimization job or schedule one to run in the future. Installing the Data Deduplication feature and enabling deduplication on selected volumes can also be accomplished by using an Unattend.xml file that calls a Windows PowerShell script and can be used with Sysprep to deploy deduplication when a system first boots.

Plan to Deploy Data Deduplication

Install and Configure Data Deduplication

Monitor and Report for Data Deduplication

Deduplicating Microsoft System Center 2012 R2 DPM storage :

Business benefits
Using deduplication with DPM can result in large savings. The amount of space saved by deduplication when optimizing DPM backup data varies depending on the type of data being backed up. For example, a backup of an encrypted database server may result in minimal savings since any duplicate data is hidden by the encryption process. However backup of a large Virtual Desktop Infrastructure (VDI) deployment can result in very large savings in the range of 70-90+% range, since there is typically a large amount of data duplication between the virtual desktop environments. In the configuration described in this topic Microsoft ran a variety of test workloads and saw savings ranging between 50% and 90%.

Recommended deployment
To deploy DPM as a virtual machine backing up data to a deduplicated volume Microsoft recommend the following deployment topology:

  • DPM running in a virtual machine in a Hyper-V host cluster.
  • DPM storage using VHD/VHDX files stored on an SMB 3.0 share on a file server.
  • For this example deployment Microsoft configured the file server as a scaled-out file server (SOFS) deployed using storage volumes configured from Storage Spaces pools built using directly connected SAS drives. Note that this deployment ensures performance at scale.

DeDup DPM storage

Note the following:

  • This scenario is supported for DPM 2012 R2
  • The scenario is supported for all workloads for which data can be backed up by DPM 2012 R2.
  • All the Windows File Server nodes on which DPM virtual hard disks reside and on which deduplication will be enabled must be running Windows Server 2012 R2 with Update Rollup November 2014.

Sizing Volumes for Data Deduplication in Windows Server

More information on Deduplicating DPM storage you can find here

Microsoft System Center DPM Blog

Dedup


Leave a comment

#Microsoft System Center 2012 R2 DPM Scale-Out Protection #sysctr #SCDPM #Azure #HybridCloud

ScaleOut DPM 2012 R2 RU4High Level Design System Center 2012 R2 Backup with Azure

In this fast Hybrid Cloud world where a lot is changing, backing up your data is important for Enterprise organisations.
With System Center 2012 R2 Data Protection Manager you can do a lot. I made a High Level Design of our
System Center 2012 R2 DPM POD’s ( Dell PowerEdge R620 Server with MD1200 SAS Storage Enclosures ).
For the first time we skipped to buy a Tape Library for long time protection, we now backup to Microsoft Azure Backup Vault storage for long time protection. So we have no tape management any more.

When you have more System Center 2012 R2 Data Protection Manager Servers in your Datacenter, you can think of Scale-Out protection where you are more flexibel in your Datacenter to save your data.

Scale Out

Here you can read more about System Center 2012 R2 DPM Scale-Out protection

System Center Data Protection Manager (DPM) offers backup to Azure for long term protection by leveraging Azure Backup. The first step in configuring DPM-A (DPM backup to Azure) is to register the DPM server to the Azure Backup vault. This article explains how the registration process is greatly simplified and enhanced with Update rollup UR4 for DPM.
Till Update Rollup 3 (UR3), users were required to generate a self-signed certificate and upload this certificate to the backup vault through the portal. During the registration process, the certificate was used to register the server to the backup vault. While this process ensured security, it was a bit overwhelming.
With UR4, registration of the DPM server to the Azure Backup vault is simplified without compromising on the security by using a concept called “vault credential”. A vault credential is an authentication entity that can be downloaded from the backup vault on the portal. The vault credential is valid for 48 hours from the downloaded time and can only be used during the registration process to authenticate the server with the backup vault.

Here you can read more on Simplified workflow to configure backup to Azure from DPM

To Protect SQL Server 2014 using DPM 2012 R2

Two Powerful ways to use Azure to Back Up your Enterprise

teched_cloudos_componentsKeep thinking of  “how to save your Data”


1 Comment

#MVPBUZZ CloudOS Roadshow : System Center 2012 R2 DPM and #WindowsAzure Backup Vault Storage #sysctr

Friday, March 7, 2014 we had a Great event at Microsoft Netherlands called MVP CloudOS Roadshow :

MVP CloudOS RoadShow 2

MVP CloudOS RoadShow 1

Here you can find the information about the MVP CloudOS Roadshow

I spoke about Hybrid Cloud with Microsoft System Center 2012 R2 Data Protection Manager and Windows Azure Backup Storage in the Cloud of Microsoft.
Here you find the step-by-step guide for backup to Windows Azure and eliminate Tape Management :

What you need before you begin is the following :

  • A Microsoft Windows Azure subscription for the Backup Vault in the Cloud.
    When you don’t have a Windows Azure Subscription you can try it first here :Try Windows Azure
  • And On-Premisses you need a System Center 2012 R2 DPM Server with a secure Internet Connection.

First you have to make an Certificate on the DPM Server :

SDK 81

Download the SDK for Windows 8.1 here (You only need to install the SDK)

We need MakeCert.exe located in the following directory after installing the SDK kit for the Self-Signed Certificate.

Makecert

Copy Makecert.exe to the DPM Server to make the Self-Signed Certificate.
CommandRun as AdminOpen Command Prompt Run as Administrator and go to the directoy where Makecert.exe is located.
Type the following command for example :

makecert.exe -r -pe -n CN=AzureBackup -ss my -sr localmachine -eku 1.3.6.1.5.5.7.3.2 -len 2048 -e 01/01/2016 AzureBackup.cer

You can find more information about makecert.exe here

Now we have created the self-signed certificate, we will make the Backup Vault in Windows Azure :

Login on Windows Azure Management Portal : https://manage.windowsazure.com

Create Backup Vault 1Give the Backup Vault a Name and Region for the Storage and Click on Create Vault.

When the Windows Azure Backup Vault is created you can upload the self-signed Certificate.

Manage Certificate DPMUpload your certificate.

After this you have to download the Backup agent for System Center 2012 R2 DPM.

Download Backup AgentDownload agent from Windows Azure Backup Vault for the DPM Server.

When the Agent is installed go to the console of System Center 2012 R2 DPM.

DPM1Go to Management and Click on Register

DPM2Browse to the certificate

DPM3Select the self-signed Certificate AzureBackup

DPM4Fetching Backup Vaults from Windows Azure.

DPM5Select your Backup Vault

DPM6Select your proxy server for the secure Internet connection

DPM7Here you can manage your Internet Bandwidth

DPM8Set here your recovery folder settings.

DPM9Set your Encryption settings and Click on Register

DPM10DPM Registration with Windows Azure Backup Vault is Completed.

From here we can configure the protection groups with Windows Azure Backup Vault :

DPM11Edit Protection Group PG_FileVolume

DPM12Select the Shares or Disk Volumes.

DPM13Select your Online Protection Goals.

DPM14Update Group and the backup will be made to the Cloud Storage of Windows Azure.

DPM15

BackupVault Azure 1Here you can see your Usage Overview in Windows Azure

BackupVault Azure 2Here you see the Registered DPM Server for the Backup Vault in Windows Azure.