Big Data - Unleashing its Potential for Organizational Success
Posted in Operations & IT Articles, Total Reads: 945
, Published on 20 February 2015
“Data! Data! Data! I can’t make bricks without clay!”
(Sir Arthur Conan Doyle)
The above quote by Sherlock Holmes gives us an appreciation that data is the building block for any investigation and making decisions.
With this we understand that decisions based on data is not an new concept. Basically it dates back to over 6000 years when Mesopotamians used it for manual accounting and recording of agricultural produce. In modern times, data gets recorded in computers and other hi-end technological systems. Recording of data takes place at the most of the simplest of the tasks like purchase of groceries to the most complex form of utility( e.g. smart phones).
This count is growing each day and every second and hence categorized as Big Data. Currently these large units of data inventory have grown beyond the ability to be managed and analyzed with traditional data processing tools. The initial idea was to control the size of this information to free up the memory of the computers for a new set of growing data. This led to the introduction of new technologies like Google’s MapReduce and open-source Yahoo’s Hadoop and more. Data collected was used by many internet companies to gain significant competitive advantage in the market. As time passes by, companies have to change their business models to survive and hence this massive change can be categorized as data-driven technological revolution of which we are all a part of.
How a data qualify as a Big Data?
Douglas Laney of Gartner Research originated the concept of 3V which serve as pre-requisites for data to be qualified as Big data. He introduced 3V – Velocity, Variety and Volume.
1) Velocity – This indicates the speed at which the data is created, stored, analysed and visualized. Companies in the past used to run processes in the night when Operations was shut down and the databases were updated by the next day morning. In the Big Data era, data is attempted to be created and stored in real time or in near real time with the help of latest wireless / wired technologies.
2) Variety – In the past, data used to be in structured format that fits into columns and rows. Now data comes in many different formats and sometimes even in complex structured formats. E.g. Data from RSS Feeds (Rich Site Summary Feeds) are categorized as semi-structured data which come in standardized xml formats that allow compatibility with many different programs. This in turn helps users to aggregate data from many sites.
3) Volume –According to a study by IDC 2011, there will be 50 times of data created in 2020 as against what is collected in 2011 (1.8 zettabytes of data). At the speede at which data is being collected, it’s easy to predict that this amount will increase every two years. With experience in the data analytics field, every day brings forth a new perspective. That said, many organizations have added more dimensions to the Big Data pre-requisites in the form of:
4) Veracity – Data and analysis at that high speed of data collection may lead to inaccuracies and incorrect data. This can potentially result in major flaws in decision making and can potentially result in organization wide loss.
5) Variability – Data is variable in nature and hence different types of algorithms are needed to treat them to ensure stability and near nil-variance
6) Visualization – Any data needs to be converted into meaningful information. Visualisation supports in achieving this balance where all the large sets of data and its respective variables have to be interpreted in a manner which is easy to read and comprehend. This can also be in the form of complex graphs or charts.
7) Value – Conversion of data into information when used for decision making and other insights for organisational growth yields a huge value in return.
How Walt Disney used the V-concept and improved its Service offerings?
Annually 100mn visitors visit Walt Disney Parks across the world and each visitor generates a lot of data which Walt Disney’s systems capture. In 2013, Walt Disney introduced the concept of “Magic Band” which is an improved ticketing system for it’s’ guests. Visitors wearing this wristband can use the facilities like Theme and waterpark tickets, hotel room keys, food & beverage payment options, taking photos and transferring images to the user account and many more.
This system operates on a wireless tracking communication system which can also track the location of the visitor within the Walt Disney World. An immense effort was put in by Disney to launch this system, over 60000 employees being trained, free Wi-fi system installed in sections of the Park to capture real-time information along with the information collected on the user’s Smartphones. Its estimated that Disney underwent a cost of $800mn to run this program.
Initially beginning its data collection through open-source tools to keep the costs down, Disney opted for a paid tool which is more reliable in dealing with large data sets when its’ traditional systems and technologies had failed.
Not compromising the privacy levels of its users, this system allows Disney to collect massive amount of sensitive and valuable data about its visitors and Disney uses this data to make valuable decisions for its visitors for improving its service offerings. Introducing it in Orlando, Disney has seen immense results and is on its way to expand this system to all its theme parks across the globe.
How do Disney and other Companies arrive at a Big Data Business Case?
Like Disney and other companies, there is no set operating process in building a business case for Big Data projects however there are some critical elements on what a business case will cover:
1) Background: The summary on the aspects of the business which will get touched in this project and how it aligns with the overall organization objectives
2) Cost – Benefit Analysis: Cost benefit analysis needs to be measured for such projects and align it to the business or the needs of the process.
3) Project scoping: This is highly important as its commonly observed that the scope keeps expanding during the course of the project and thereby affecting the overall costs budgeted for the project.
4) Risk: Calculating risk is one of the primary things in such project however it’s the most complex task to fulfil. Risks such as technology issues, regulatory and privacy issues can disrupt the whole project.
How to build a Project team?
Finding and hiring talented workers with analytical skills is the first step in creating an effective data analytics team. Any Big Data Project team needs to have a mix of the following:
1. Business intelligence – All the applications and technologies contain data and all the project team members will require access to this data. BI can also provide factual information which can help team make sound business decisions.
2. Data mining – Huge data sets are commonly archived and data mining techniques are used to determine patterns and to analyse the data from different dimensions suiting the business requirements.
3. Statistical tools – Big Data consists of huge data sets and there is always a need to analyse the data on sample basis. This in turn helps in arriving at predictive analysis and testing.
4. Data Modeling – Data modeling refers to the forming and documenting the existing process that occur in an software application design and development. During the course of the Project, Data modeling experts provide their knowledge on building a robust software.
5. Predictive analysts – Any analysis is not complete without forecasting and prediction. Predictive analysts are needed to provide their knowledge on the risks and opportunities for growth in new markets.
Such large-scale project can not only have a huge improvement on the organizational growth but it also has a high impact on the society.
How will different industries use Big Data?
Big Data is expected to bring a lot of benefits to the consumers in the form of improved services, more user-friendly systems and thereby resulting in a more transparent dealing of services.
Big Data can affect all industries starting from Banking, Healthcare, Consumer goods, Information Technology and also the Government services. Gartner predicts that BIG data development will drive up IT spending to $232 billion by the end of 2016. All this is possible only when all organization and the governments are able to fully start using Big Data and reap its benefits.
With rise in social networking websites, consumers have found a platform to connect with each other and this helps the organizations to capture a lot of data with the help of data analysts. In a nutshell, there is data everywhere and hence it becomes highly important to make use of it for the benefit of the organization and the overall economy.
This article has been authored by Vidya Krishnan from ANZ
1) Think Bigger: Developing a Big Data strategy for your business: by Mark Van Rijmenam
2) Big Data Analytics: Turning Big Data into Big Money: by Frank J. Ohlhorst
3) Big Data – A Revolution That Will Transform How We Live, Work and Think: by Viktor Mayer-Schonberger & Kenneth Cukier
If you are interested in writing articles for us, Submit Here