Big Data vs. Smart Data, when size does not matter

Gaetano Masi
Chief Growth Officer @KPI6

The term Big Data was first used in 1990 by John Mashey, PhD in Computer Science at Pennsylvania State University and, to date, has become one of the most important buzzwords for companies around the world. Gathering information provides them with valuable insights into their customers and their industries, enabling them to do predictive analysis

The amount of data we produce is boundless. But making sense of millions (perhaps billions) of conversations can be time-consuming and difficult without powerful technology, especially when this data is not structured. This is often the case with online text data in the form of articles, news, social media posts, and comments on forums or blogs.

Such is the complexity of this analysis that often companies are pervaded by discouragement.

Without structure, Big Data are undisciplined and unusable, they are just a mass of unrelated information that would take too long to be understood, and even in this case, may not prove useful. 

However, this information can become intelligent data if it is effectively superimposed; there are platforms, such as KPI6, that can make life easier for marketing teams.


But what is Big Data?


Big Data is a huge amount of data, both unstructured and structured, collected on a daily basis; once filtered and transformed into Smart Data, a cleaner and more precise form of data, it makes it possible to get useful insights for a more efficient decision-making.

Big Data and Smart Data are not two worlds in conflict, on the contrary! They are in opposition to each other, but are two directly related elements.

Big Data are commonly described using the five Vs: value, variety, volume, velocity, veracity. With Smart Data, a reduction in the "volume" occurs. Variety can be reduced or not, depending on the screening process applied to filter the information. Value, velocity and veracity (accuracy) should increase as the volume decreases.

Smart Data focuses on the actionable value obtained from human involvement in the creation, processing and consumption of data, in order to improve the human experience. It is the result of a process that, from Big Data, provides deeper information about a specific context.


Let's take some examples


  • In the world of Financial Services:

Big Data: the list of transactions carried out by the customers of a bank

Smart Data: fraud detection, or credit worthiness

  • In the world of Retail:

Big Data: the list of products purchased with a fidelity card

Smart Data: personalized and geo-localized offers

  • In the world of Telco:

Big Data: information on voice traffic volumes in certain geographical areas

Smart Data: anomaly detection for infrastructures diagnostic.


Smart Data gives meaning to Big Data, thanks to algorithms for classification that let you focus on the actionable value of a specific context. They can become a tailor-made product with a high level of customization, that understands the complexity of the real world.


Social Big Data


How can the analysis of conversations, on the web and on social media, help companies?

Today, the web is packed with an enormous amount of data and information; just think that every minute, 900,000 people log into Facebook, 500,000 Tweet, 65,000 post on Instagram and 700,000 hours of videos are watched on YouTube.

How can companies make use of this data? Many tools and solutions allow you to monitor public conversations on social networks, with the aim of identifying signals, trends, crises by summarizing the data collected in meaningful, actionable analysis, in line with the needs of the users.

Many solutions on the market have found themselves at the turn of the evolution occurring in the world of Social Big Data. They are focused on collecting data, and the results they provide hardly satisfy the specific contexts of their end users. What are the key points to consider in order to carry out an analysis with Social Media Intelligence platforms such as KPI6?

  • Brief

Don’t fear the word brief: the collection of useful information to structure a successful strategy is necessary. To define a perimeter of analysis and a subject, the brand or the product; to identify sources, choose the country and languages, understand the competitive scenario and the market’s benchmark. Boolean logic can help narrowing down only the necessary information. 

  • Definition of KPIs

The next step is to identify the desired scenario and use-case, to define the rules for segmentation and then to outline the KPIs needed to achieve the objectives.

  • Analysis

What are the most frequent analyses?

All those that can be done on texts, hashtags, emojis, studying the peaks of conversations, distribution, trends and related topics, to understand who has been talking about a specific topic and how.

Once the data has been collected, it's time to abstract value.

Most common use cases


What are the most common use cases and how can Big Data and Smart Data help us? 

Brand Reputation: the collection of brand mentions over time (Big Data) to understand the trends and identify ideal customers (Smart Data).

Competitive intelligence: the collection of a competitor’s mentions, conversations on the industry (Big Data) and ROI analysis on active campaigns about the market presence (Smart Data).

Content Strategy: the collection of textual data, images, and data on engagement (Big Data), analysis of the Keywords to plan an editorial strategy (Smart Data).

Crisis Management: the collection of “dangerous” conversations to the brand (Big Data) with the extraction of hater profiles and automatic alerting on the most relevant threats (Smart Data).

In the future, data collection will be even easier and by 2021, according to Ventana Research Assertions, 66% of the analysis will allow us to discover not only what happened and why, but they will also tell us what we should do about it