Published in General

The Beautiful Big Data

What can Big Data mean for your Voice of Customer (VoC) software platform and program?

Each year, Mary Meeker, of the VC firm Kleiner Perkins Caufield Beyers, publishes a comprehensive and always interesting 100+ page deck on Internet Trends.

Meeker’s 2014 edition has a section on Big Data where she lists the following six trends:

1. Uploadable, findable, shareable, real-time data
2. Sensors use rising rapidly.
3. Processing costs falling rapidly, while cloud rises
4. Beautiful new user interfaces, aided by data-generating consumers – helping make data usable/useful
5. Data mining / analytics tools improving & helping find patterns
6. Early emergence of data/pattern driven problem solving

Note that “more data” is not listed as a trend in 2014. More data is just the norm. It’s always been the norm. We’re simply creating more of it more rapidly. The whole Volume, Velocity, and Variety thing that defined Big Data years ago. At Allegiance we feel that Big Data is all about Big Value, no matter the volume. It’s a healthy spirit and curiosity for what one can do with the data that matters. Big Data’s new tools, algorithms and platforms can help illuminate patterns and find new value in any data set.

The early Big Data platforms were invented to support Google’s and Facebook’s massive data sets. If you think those are fast, imagine what these Big Data platforms can do for your Voice of Customer program which likely has a much smaller data footprint.

Keep in mind that even though your VoC program may only have 10k, or 100k, or 10 million records, your VoC platform vendor is handing many such clients on a cloud based architecture. The volume of records across all of the vendor’s clients adds up quickly into “Big Data” type volume levels.

Allegiance’s platform is pioneering the path in the VoC space in regards to Meeker’s Internet Trends on Big Data. Meeker’s 2014 trends were actually a type of validation for us. Here’s why:

1. Uploadable, findable, shareable, real-time data

Allegiance can collect and import a variety of data types including survey, operational, financial, social media, web analytics, attachments and more. Allegiance Engage standard reporting and dashboards allow one to filter and search for any data in your program. Data and report sharing is handled via exports to Office documents or scheduled for email or FTP delivery as our users configure them to be delivered. New inbound data is immediately available for monitoring and analysis.

2. Sensors use rising rapidly

Our client’s customer experience operational data is pre-populated with each survey program. These operational and financial data are generated by business systems such as CRM, POS, PMS, ERP, but some of the data comes from a growing variety of sensors and related software. Ranging from the bar code reader at the retail store, to web analytics sensors measuring web behavior, to hardware sensors that upload product usage and activity information, sensors are becoming lower in cost, more pervasive, and more connected.

The cash register is perhaps the most important “sensor” of all, and is why many of our customers include a variety of business metrics in their pre-populated data or append new business data downstream of the experience event.

Sensors can also augment the customer experience. Upon waking up, my wearable wristband computer detects the event and turns on my coffee maker.

3. Processing costs falling rapidly, while cloud rises

Modern Big Data tools are based on Open Source technology. To you, this means low cost and continuous innovation by anyone. These technologies allow your VOC platform vendor to take advantage of a large & growing network of users and expertise. As a result, your VOC vendor’s engineering team is trained quicker, debugs quicker, and takes less time to add value in the form of new features.

Big data technology also makes us future proof. By using distributed, open source, big data tools, we can easily scale our platform to meet any data volume or performance requirement that our clients bring today and tomorrow. We no longer need to try to squeeze performance out of a limited vertical reporting platform and can instead focus our engineering efforts on finding value in the data.

4. Beautiful new user interfaces, aided by data-generating consumers – helping make data usable/useful

Expanded data volumes are provoking innovation in data visualization and usability. Your basic web browser is now powerful enough to generate very dynamic vector based graphics. This means more interactive reporting and drill down. It means your VoC staff or thousands of front line managers can have engaging data visualizations which are tailored to their needs and easy to consume.

As for usability, Big Data technology is fast. That’s it. It’s Fast. Our Dashboards and Reporting tools product, which recently launched, have unprecedented performance due to the Open Source technologies like Hadoop, Hive, and Elasticsearch which power it. Google made it clear a long time ago that if it’s fast, people will use it more. That’s certainly true here.

Old problems with latency are eliminated with Open Source Big Data tech. For example, in recent tests on our Dashboard 2, for a survey with 100 thousand records, the latency associated with the data query to build a chart runs between 10ms and 200ms (ten thousandths of a second to 200 thousandths) depending on the complexity of the chart. Hence, even if a dashboard has ten reports on it, the entire dashboard completes its data query operation in under two seconds.

Now imagine your VOC platform vendor running thousands of dashboards, each with multiple reports, at the same time. The distributed nature of computing that’s inherent to Hadoop and related Big Data technologies helps ensure each of those thousands of users has a speedy user experience.

5. Data mining / analytics tools improving & helping find patterns

Today’s computational power to organize and interpret data is unprecedented. In some ways, data mining and machine learning are like human learning by example. But at a massive scale and speed. It’s interesting that many of the data mining and machine learning algorithms in use today were actually developed by brilliant mathematicians decades ago, before there was a computer or software readily at hand to experiment.

Data mining allows one to include a wide variety of feedback and operational data in the analysis. The old assumptions and requirements of only-likert scales in the analysis, normal distributions or linear relationships are no longer necessary. Your customers and real life are “non-linear”, and we need tools like data mining which fundamentally think this way.

Allegiance’s Spotlight is the first Data Mining tool to be fully integrated into an online enterprise class Voice of Customer platform. Spotlight automatically searches your data for patterns which drive your program’s key metrics. Spotlight is simple to setup, very fast, and presents results very simply. In plain English.

6. Early emergence of data/pattern driven problem solving

Unless your data is totally random, there are likely to be plenty of patterns hidden in it. A comprehensive data mining technology such as Spotlight’s patent pending mining algorithm can find these patterns very quickly.

For our customers, it was “early” 2 years ago when we pioneered this area of pattern discovery. Today, our clients use Spotlight to solve problems and uncover actionable insights across both their feedback and operational data.

The Big Data trends presented by Meeker are already a day to day routine for Allegiance clients who aim to find new advantages from their data. Advantages for their staff who will enjoy the fast reporting’s user experience–and advantages from newly discovered patterns which help create great and unforgettable customer experiences. Allegiance is hard at work to be ahead of Meeker’s Big Data trends of 2015.

I want to thank our Allegiance engineering team for their contributions to this article.

Al Nevarez – VP Data Science, Allegiance