Data cleansing is the process of fixing any and all data quality issues arising in a dataset. It is alternately known as data cleaning, data washing, or data scrubbing.
Data cleansing, performed by specialized software programs, scans datasets looking for errors in data quality. Data quality dimensions are defined in many ways by different organizations, but generally include most of the following factors:
- Accuracy – Is the information correct? Is the customer’s name spelled correctly, or are there typos?
- Completeness – Is any part of a value missing? Does the telephone number have the right number of digits?
- Uniqueness – Is any customer’s information entered in the database more than once?
- Consistency – Does the data translate correctly between its collection and storage?
- Timeliness – Is the data sufficiently current to provide value to the company?
- Integrity – Is the data correctly linked, or will further analysis create uniqueness issues?
- Conformity – Is each data value recorded in the same format?
Data cleansing software searches for any of these errors using various statistical models and algorithms. It then repairs the information where possible and flags other data points for the analyst overseeing the process. Information is verified for accuracy, glaring mistakes are manually repaired, and any data values that cannot be repaired without further information are eliminated from analysis. In effect, data cleansing is similar to running a spell check on a word document. The process can and should be run consistently to ensure data is up to date and accurate.
Data cleansing is different from data purging. In purging data, old and useless entries are removed to make room for new data. Data cleansing focuses on maximizing accuracy and ensuring efficiency.
Data cleansing safeguards against corrupt data. Companies that have changing data (and that’s everyone) need to consistently and systematically check the information they have to ensure consistency. This is especially important when merging databases. Washing data can result in increased revenue, streamlined business practices and decision making, and improved management and analysis. Celsius, a leading telecom equipment manufacturer, was able to increase its contacts by 84 percent after cleansing its data, and increased the number of valid emails by 307 percent.
Companies need to conduct thorough cleansings of their data frequently. Failure to do so can lead to misapplication of resources, resulting in loss of time and money. For example, consider how often people change their mailing addresses, emails, and other company and individual details. With incorrect contact information, companies waste resources trying to communicate with customers who have moved on. Data changes often, and companies need to make sure they are on top of changes that occur.
But data cleansing has more applications than just organizing contact information. Those who choose not to clean their data and check for quality may have costly and disastrous outcomes in other areas as well. For example, in 1998 NASA launched the Mars Climate Orbiter space probe. Upon the atmospheric entry of Mars, it disintegrated. This was due to discrepancy in the measurement units NASA was using to monitor the mission. The total cost was $655 million and a lot of embarrassment. Without data cleansing, companies can misinterpret survey data and thus customer insights, get different outcomes from analysis of the same data, and skew results on transactional data.
Data Cleansing at MaritzCX
Because data quality is so important to ensuring optimal outcomes, MaritzCX uses top-quality data-cleansing software for its programs, and provides that same quality to its customers.
MaritzCX believes organizations should be able to see, sense and act on the experiences and desires of every customer, at every touch point, as it happens. We help organizations increase customer retention, conversion and lifetime value by ingraining customer experience intelligence and action systems into the DNA of business operations. MaritzCX is the combination of the Allegiance award-winning CX platform and Maritz Research strategic consulting services.