How to Use Data Deduplication in Windows Server 2012 R2

One of the more useful features of Windows Server 2012 and Windows Server 2012 R2 is native data deduplication. Although deduplication features have existed in storage hardware for years, the release of Windows Server 2012 marks the first time that Microsoft has allowed deduplication to occur at the operating system level.

Before you can use the deduplication feature, you will have to install it. To do so, open Server Manager and then choose the Add Roles and Features command from the Manage menu. When the Add Roles and Features Wizard launches, navigate through the wizard until you reach the Add Roles screen. Expand the File and Storage Services role, and then expand the File and iSCSI Services container and select Data Deduplication, as shown in Figure 1. Click Next on the remaining screens and then click Install to install the necessary components. When the process completes, click Close.


Deduplication is performed on a per-volume basis. To do duplicate a volume, open the Server Manager and select the Volumes container. Next, right click on a volume and choose the Configure Data Deduplication command from the resulting shortcut menu, as shown in Figure 2.


At this point the Deduplication Settings dialog box will appear, as shown in Figure 3. You can enable data deduplication by simply selecting the Enable Data Deduplication check box and clicking OK. However, there are a couple of other settings on this dialog box that are worth paying attention to.


The first such setting is the Duplicate Files Older Than setting. The deduplication mechanism in Windows is post process. In other words, deduplication does not happen in real time. Instead, a scheduled process performs the deduplication at a later time. The reason why Microsoft gives you the option of waiting until a file is a few days old before it is be duplicated is because the deduplication process consumes system resources such as CPU cycles and disk I/O. You really don’t want to waste these resources on deduplicating temporary files. Making sure that a file is at least a few days old before it is deduplicated is a great way to avoid wasting system resources.

Another setting that is worth paying attention to is the File Extensions to Exclude setting. The basic idea behind this setting is that some types of files cannot be deduplicated because they are already compressed. This includes things like zip files, and compressed media files such as MP3 files. The File Extensions to Exclude setting lets you avoid wasting system resources by preventing Windows from trying to do duplicate files that most likely will not benefit from the deduplication process. Similarly, if you have folders containing compressed files you can exclude those folders from the deduplication process.

Finally, there is an option to set the deduplication schedule. You should configure the deduplication process to occur outside of peak hours of operation.

Of course this raises the question of the hardware resources that are required in order to perform data deduplication. The minimum supported configuration is a single processor system with 4 GB of RAM and a SATA hard disk. According to Microsoft, a deduplication job needs one CPU core and about 350 MB of RAM. Such a system could theoretically run a single deduplication job that would be capable of processing about 100 GB per hour. Higher-end systems can be duplicate multiple volumes simultaneously. The theoretical limit is that ninety volumes can be deduplicated simultaneously. In reality however, seventeen volumes at a time is a more realistic expectation from today’s hardware.

It is also worth noting that not every volume type can be deduplicated. Windows Server cannot deduplicate a system volume or a boot volume. Furthermore, the volume cannot reside on removable media and it must not be formatted as ReFS. Cluster shared volumes also cannot be deduplicated.

As I alluded to earlier, there are certain data types that can benefit from the deduplication process more than others. However, there are some types of data that should not be deduplicated. For example, you should not attempt to deduplicate a volume containing files that are constantly open or that change frequently. Similarly, Microsoft does not support deduplicating volumes containing Hyper-V virtual hard disks (for production VMs), although Windows Server 2012 R2 supports the deduplication of Hyper-V-based virtual desktops. You should also avoid deduplicating any volume containing files that are near 1 TB in size.

The biggest restriction with regard to data deduplication is that you cannot deduplicate volumes containing Exchange Server or SQL Server databases. If you attempt to do duplicate these volumes, there is a very real chance that you will corrupt the databases. Although not explicitly spelled out by Microsoft support policies, I recommend that you avoid deduplicating any volume containing a database. Many database applications expect to have control over the way the database pages are stored. Introducing deduplication when the database application expects to have full control over the underlying storage can result in corruption.

The Windows Server native deduplication feature does a great job of helping to conserve physical storage. Even so, it is important to properly plan for deduplication prior to implementing it because there are a number of situations in which the use of deduplication is not appropriate.

Continue Reading

How to Install Squid (Caching / Proxy) on CentOS 7

Squid is a caching and forwarding web proxy. It is most often used in conjunction with a traditional LAMP stack (Linux, Apache, MySQL, PHP), and can be used to filter traffic on HTTP, FTP, and HTTPS, and increase the speed (thus lower the response time) for a web server via caching.

Pre-Flight Check
These instructions are intended specifically for installing Squid on a single CentOS 7 node.
I’ll be working from a Liquid Web Core Managed CentOS 7 server, and I’ll be logged in as root.

Step #1 Install Squid
First, clean-up yum:

yum clean all

As a matter of best practice we’ll update our packages:

yum -y update

Installing Squid and related packages is now as simple as running just one command:

yum -y install squid

Configure Squid to Start on Boot
And then start Squid:

systemctl start squid

Be sure that Squid starts at boot:

systemctl enable squid

To check the status of Squid:

systemctl status squid

To stop Squid:

systemctl stop squid

To access squid settings access : /etc/squid/

My basic config:

http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost


# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 3128

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 10000 16 256
cache_replacement_policy heap LFUDA
cache_swap_low 90
cache_swap_high 95
maximum_object_size_in_memory 100 MB
cache_dir aufs /var/spool/squid 40000 16 256
cache_mem 40000 MB

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

# Add any of your own refresh_pattern entries above these.

That is all!

Continue Reading

Install Cacti (Monitoring tool) on ubuntu 15.04 / 15.10 server.

Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices.

Features Graphs

Unlimited number of graph items can be defined for each graph optionally utilizing CDEFs or data sources from within cacti.
Automatic grouping of GPRINT graph items to AREA, STACK, and LINE[1-3] to allow for quick re-sequencing of graph items.
Auto-Padding support to make sure graph legend text lines up.
Graph data can be manipulated using the CDEF math functions built into RRDTool. These CDEF functions can be defined in cacti and can be used globally on each graph.
Support for all of RRDTool’s graph item types including AREA, STACK, LINE[1-3], GPRINT, COMMENT, VRULE, and HRULE.

Data Sources

Data sources can be created that utilize RRDTool’s “create” and “update” functions. Each data source can be used to gather local or remote data and placed on a graph.
Supports RRD files with more than one data source and can use an RRD file stored anywhere on the local file system.
Round robin archive (RRA) settings can be customized giving the user the ability to gather data on non-standard timespans while store varying amounts of data.

Data Gathering

Contains a “data input” mechanism which allows users to define custom scripts that can be used to gather data. Each script can contain arguments that must be entered for each data source created using the script (such as an IP address).
Built in SNMP support that can use php-snmp, ucd-snmp, or net-snmp.
Ability to retrieve data using SNMP or a script with an index. An example of this would be populating a list with IP interfaces or mounted partitions on a server. Integration with graph templates can be defined to enable one click graph creation for hosts.
A PHP-based poller is provided to execute scripts, retrieve SNMP data, and update your RRD files.


Graph templates enable common graphs to be grouped together by templating. Every field for a normal graph can be templated or specified on a per-graph basis.
Data source templates enable common data source types to be grouped together by templating. Every field for a normal data source can be templated or specified on a per-data source basis.
Host templates are a group of graph and data source templates that allow you to define common host types. Upon the creation of a host, it will automatically take on the properties of its template.

Graph Display

The tree view allows users to create “graph hierarchies” and place graphs on the tree. This is an easy way to manage/organize a large number of graphs.
The list view lists the title of each graph in one large list which links the user to the actual graph.
The preview view displays all of the graphs in one large list format. This is similar to the default view for the 14all cgi script for RRDTool/MRTG.

User Management

User based management allows administrators to create users and assign different levels of permissions to the cacti interface.
Permissions can be specified per-graph for each user, making cacti suitable for co location situations.
Each user can keep their own graph settings for varying viewing preferences.

Preparing your system

Before installing cacti you need to make sure you have installed Ubuntu 15.04 LAMP server.

Install Cacti on ubuntu 15.04 / 15.10 server

Open the terminal and run the following command

sudo apt-get install cacti-spine

The above command starts the cacti installation and you should see the first as php path change select ok and press enter


Now select the webserver you want to use (in my case it is apache2)


Cacti database configurations select yes


Enter database admin user password


Mysql application password for cacti


confirm the password


Now that Cacti is installed, we can start the configuration process on it.

Configuring cacti

Point your web browser towards http://YOURSERVERIP/cacti/install/ to start the initial setup and click next


Select new install option and click next


In the following screen you need to make sure you have all the required paths are correct and click on finish


Now login to Cacti with the default admin/admin, and change the password to something more sensible


After login in to Cacti you should see similar to the following screen



That is all!

Continue Reading

How to Install Plugin Mirage in Cacti


Cacti version 0.8.8+ It may work on previous versions, but we haven’t tested against them.
PIA version 3.1

Download Mirage Plugin 1.2.0

Dowload Mega URL:!joBCSajS
Key: !5xOf-FjcDAkkS3K2TkAkTA


Move the folder mirage to CACTI_HOME/plugins/
Ensure permission are correct (CACTI_HOME/plugins/mirage)

Install Mirage through Cacti Plugin Management /Enable Mirage pluging through Cacti Plugin Management


Review and save Mirage settings


After installation it should generate a file called ” mirage_poller_output.log ” within the / var / www / html / cacti / log /

That is all!

Continue Reading

Easy Samba installation on RHEL/CentOS 7

Samba is a client/server system that implements network resource sharing for Linux and other UNIX computers. With Samba, UNIX files and printers can be shared with Windows clients and vice versa. Samba supports the Session Message Block (SMB) protocol. Nearly all Windows computers include SMB support with their internal network subsystems (NetBIOS in particular).
With an appropriately-configured Samba server on Linux, Windows clients can map drives to the Linux filesystems. Likewise, theSamba client on UNIX can connect to Windows shares by their UNC name. Although differences among various operating systems (such as filesystem naming conventions, end-of-line conventions, and authentication) can limit interoperability, Samba offers a generally serviceable mechanism for resource sharing on a heterogenous network.
In this tutorial we will show you how to install and configure Samba server on RHEL and CentOS 7 linux.

Install and configure Samba on Rhel/CentOS 7
To install samba packages enter following command:

#yum install samba samba-client samba-common -y

Now configure samba edit the file /etc/samba/smb.conf

#mv /etc/samba/smb.conf /etc/samba/smb.conf.bkp
#vi /etc/samba/smb.conf

and paste following line:

workgroup = WORKGROUP
server string = Samba Server %v
netbios name = centos
security = user
map to guest = bad user
dns proxy = no
#============================ Share Definitions ==============================
path = /samba/anonymous
browsable =yes
writable = yes
guest ok = yes
read only = no

Save the smb.conf file and restart the service:

#mkdir -p /samba/anonymous
#systemctl enable smb.service
#systemctl enable nmb.service
#systemctl restart smb.service
#systemctl restart nmb.service

Add these Iptables rules, so that samba will work perfectly:

#firewall-cmd --permanent --zone=public --add-service=samba
#firewall-cmd --reload

Change permission for samba share:

#chmod -R 0755 anonymous/
#chown -R nobody:nobody anonymous/

Further we need to allow the selinux for the samba configuration as follows:

#chcon -t samba_share_t anonymous/

Now you can access the Centos 7.0 sharing in windows as follows, go to the Run prompt and type \centos :


Acesse \\centos


Now anonymous user can browse & create new text documents:


Secured samba server

For this I will create a group smbgrp & user rasho to access the samba server with proper authentication

#useradd rasho
#groupadd smbgrp
#usermod -a -G smbgrp rasho
#smbpasswd -a rasho
[root@localhost]# smbpasswd -a rasho
Retype new SMB password: REPEAT YOUR SAMBA PASS
Added user rasho.

Create a new share, set the permission on the share:

#mkdir /home/secure
#chown -R rasho:smbgrp /home/secure/
#chmod -R 0770 /home/secure/
#chcon -t samba_share_t /home/secure/

Again edit the configuration file as :

#vi /etc/samba/smb.conf

Add the newly created samba share in smb.conf file:

path = /home/secure
valid users = @smbgrp
guest ok = no
writable = yes
browsable = yes


Restart the samba service:

#systemctl restart smb.service
#systemctl restart nmb.service

Now at windows machine check the folder now with the proper credentials


Open samba sharing


Create new text documents:image7

That is all!


Continue Reading