In this article, we’ll explore what Big Data is, how it works in practice, and how businesses can apply it. This post is part of a series that I’m writing about Industry 4.0, where we give answers to questions such as: “What is IoT?“, “How to create a datadriven organization?“, and “What is Industry 4.0?“.

What is Big Data?

So, what is Big Data? In short, it refers to sets of data that are too large or complex to be processed, analyzed, or used with standard methods, according to Oxford Learner’s Dictionaries.

Big Data relies on digitally available data, which involves quantities, characters, or symbols that are operated on by computers, stored, transmitted as electrical signals, and recorded on magnetic, optical, or mechanical media.

Can we divide different Types of Big Data?

Big Data can be categorized into three different types: structured Big Data, unstructured Big Data, and semi-structured data.

Structured Big Data

Structured data refers to all data that can be stored, accessed and processed in a predefined format, such as data in tables in a database. When dealing with large amounts of structured data, we refer to it as structured Big Data.

Unstructured Big Data

On the other hand, unstructured data is data whose format or structure is unknown. In addition to being large in volume, unstructured data presents challenges in processing. A typical example of unstructured data is a data source that contains simple text files, images, and videos, such as Google search results. Search results for a search term may contain both text, images, and videos.

Semi-Structured Big Data

Semi-structured Big Data contains both types of data. It refers to data that is structured in a fixed format, but not defined, for example, in a table. An example of this is data in an XML file where the format needs to be defined in Excel.

What are the main Characteristics of Big Data?

Big Data is characterized by the properties of Volume, Variety, Velocity, and Variability.

Volume

The name Big Data derives from the large volume of data. The quantity of data is a crucial factor in determining the value of a dataset.

Variety

The variety of Big Data is determined by both the nature of the data (structured/unstructured) and the sources of the data. While the sources originally consisted of spreadsheets and databases, the variety of Big Data can be expanded to include data in the form of emails, photos, monitoring equipment, PDFs, audio, and so on.

Velocity

Velocity refers to the speed at which data is generated from various sources such as business processes, application logs, networks, social media sites, sensors, mobile devices, and more. This generates an enormous and continuous stream of data at high speed.

How does Big Data work in practice and how can it be applied in busines?

Here are some practical examples:

Emergency information

In New York City, 911 emergency information is enriched with Big Data. Through partnerships with organizations such as Apple, Android, and Uber, relevant data from a patient’s phone and wearables can be sent to emergency services during crises. This includes GPS location and real-time sensor data, allowing emergency services to respond more quickly and effectively.

Netflix – House of Cards

The former hit Netflix series House of Cards was created with the help of Big Data. Netflix’s analysis showed that viewers of the older British series House of Cards also frequently watched movies directed by David Fincher starring Kevin Spacey. Based on this, they could predict that a combination of these three factors would lead to a hit series. Years later, Big Data determines not only which movies and series Netflix invests in but also how series are presented to subscribers. Based on viewing history (including the points at which users pause), the thumbnails that appear on the “Popular on Netflix” homepage are determined.

Skupos: convenience for convenience stores

The American Skupos platform collects transaction data from 7,000 different convenience stores. This amounts to billions of transactions annually, which give store owners insight into determining best-sellers and advising on orders by location. Meanwhile, distributors can predict demand and brands can analyze a constant stream of product data.

View all our Data Analytics and AI/ML Solutions