Future of Big Data

Posted in Operations & IT Articles, Total Reads: 1940 , Published on 12 May 2013

The term ‘data’ comes from the Latin word datum meaning ‘something given’. “Big data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. To understand the extent of big data today, the following statistics can be considered – 235 terabytes data was collected by the US Library of Congress by April 2011, 30 billion pieces of content is shared on Facebook every month (Maynika, et al., 2011).

Information has gone from being scare to superabundant, bringing with it new benefits and problems like exceeding the available storage space (Figure 1), difficulty in ensuring data security and protecting privacy. To handle this data deluge a new class of employees called data scientists have emerged who work on this data to extract relevant information (Economist, 2010).


Figure 1. Chart showing information created and available storage

The abundance of data is primarily linked to the Internet of Things (Chui, Loffler, & Roberts, 2010) and the mobile phone revolution. Also, there has been a tremendous increase in the number of people who have access to information. Between 1990 and 2005 more than 1 billion people worldwide entered the middle class family sector. As they get richer they become more literate, which fuels information growth, notes Mr Cortada. (Economist, 2010). And quite naturally in this background the statement of Walmart CIO that, “Data is becoming the raw material for business” is very relevant.

Tomorrow’s enterprise will be majorly based around the Internet of Things (Chui, Loffler, & Roberts, 2010) and that leaves the enterprise with a mountain of data to handle which essentially is big data. Since data is becoming the most valuable resource, for enterprise to develop a sustained competitive advantage, best application of data needs to me ensured (Penrose, 1959). So, the challenge before the enterprise now, is to convert this data into knowledge. Knowledge is considered as “justified true belief” (Nonaka & Takeuchi, 1995) and is modeled as an unambiguous, reducible and easily transferable construct. According to Eisenhardt and Santos (2000), knowledge is conceptualized as a resource that can be acquired, transferred or integrated to achieve sustained competitive advantage (Eisenhardt & Santos, 2000). Developing this knowledge which is heterogeneous in nature owing to the methodology through which the knowledge is developed from data will provide the enterprise a competitive advantage and superior corporate performance (Decarolis & Deeds, 1999) (Winter & Szulanski, 1999). The interpretations drawn using analysis of big data will be significantly based out of intuition which is developed based on tacit knowledge and the knowledge-based approach of strategy considers this tacit knowledge as the most strategic resource of the firm (Kogut and Zander, 1992; Grant, 1996). This tacit knowledge is difficult to imitate and relatively immobile, it can constitute the basis of sustained competitive advantage (Grant, 1996; Decarolis and Deeds, 1999; Gupta and Govindarajan, 2000). Studies also show that significant productivity growth can be achieved through big data across different sectors of economy (Figure 2) (Maynika, et al., 2011).


Figure 2. Productivity growth through big data in different sectors of economy

According Bierly and Chakrabarti (1996) study, sustained profits differentials were found with regard to two types of enterprises – innovators and explorers. Innovators are aggressive knowledge developers, achieving high levels of internal and external knowledge acquisition, focusing on both radical and incremental innovation, and applying knowledge very quickly. Explorers were characterized by very radical knowledge evolution and average values on the other dimensions (Bierly & Chakrabarti, 1996). Tomorrow’s enterprise essentially needs to be innovators or explorers when it comes to handling big data so that sustained profit differentials are achieved. The use big data has already shown successes in the domains of healthcare, public sector administration, retail and manufacturing (Maynika, et al., 2011).

Some examples of enterprises using big data as a means for competitive advantage:

  • Web-based companies, such as Amazon.com, eBay, and Google are experimenting with aspects like, where to place buttons on a Web page and sequence of content display in order to determine what will increase sales and user engagement.
  • Financial institutions like Capital One, are using big data analysis methods for segmenting credit card customers for tailoring products based on individual risk profiles of investors.
  • United Kingdom’s Tesco gathers transaction data on its customers through a loyalty card program and uses that to make decisions on pricing, promotions and shelf allocation.
  • Fresh direct, the online store makes decisions based on real-time data relating to online transactions and customer service interaction to adjust its prices and promotional offers. (which is termed as “Now-casting” (Bollier, 2010))
  • Ford Motor, PepsiCo and Southwest Airlines analyze consumer sentiment and brand image through the posts that are made in Facebook and Twitter. (Bughin, Chui, & Maynika, 2010)

Key challenges that enterprises face with regard to big data:

  • Need for software to apply increasingly sophisticated techniques
  • Develop platform that will enable to generate, communicate, share and access large volumes of data
  • Shortage of the analytical and managerial talent to derive relevant inputs from big data
  • Developing mechanisms to ensure that privacy and data security concerns are addressed
  • Defining clear property rights over data creation and ensuring necessary safeguards to prevent violations (Maynika, et al., 2011)

Big data has developed as a radically new kind of “knowledge infrastructure” that has led to data being considered as a factor of production today. The implications that it has on enterprises is immense as has been noted above. Enterprises need to develop mechanisms to extract valuable data which will be a source of sustained competitive advantage. I would like to conclude with a quote of Hal Varian, Chief Economist at Google “To make money, you’ve got to predict two things—what’s going to happen and what people think is going to happen. You only make money by beating that spread.” – which is essential obtained through big data (Bollier, 2010).

The article has been authored by By: SREEVAS S. (FPM student)  from IIM Kozhikode.


Bierly, P., & Chakrabarti, A. (1996). Generic Knowledge Strategies in the U.S. Pharmaceutical Industry. Strategic Management Journal, 123-135.

Bollier, D. (2010). The Promise and Peril of Big Data. Washington D.C.: The Aspen Institute.

Bughin, J., Chui, M., & Maynika, J. (2010, August). Clouds, big data and smart assets: Ten tech-enabled business trends to watch. Mckinsey Quarterly.

Chui, M., Loffler, M., & Roberts, R. (2010). The Internet of Things. Mckinsey Quarterly.

Data. (n.d.). Retrieved January 31, 2013, from Wikipedia: http://en.wikipedia.org/wiki/Data

Decarolis, D. M., & Deeds, D. (1999). The Impact of Stocks and Flows of Organizational Knowledge on Firm Performance: An Empirical Investigation of the Biotechnology Industry. Strategic Management Journal, 953-968.

Economist. (2010, February 25). Data, data everywhere. Retrieved January 31, 2013, from Economist: http://www.economist.com/node/15557443

Eisenhardt, K., & Santos, F. (2000). Knowledge-Based View:A New Theory of Strategy? In A. Pettigrew, H. Thomas, & R. Whittington, Handbook of Strategy and Management. CA: Sage Publications.

Grant, R. M. (1996). Toward a Knowledge-Based Theory of the Firm. Strategic Management Journal, 109-122.

Gupta, A. K., & Govindarajan, A. (2000). Knowledge Flow within Multinational Corporations. Strategic Management Journal, 473-496.

Kogut, B., & Zander, U. (1992). Knowledge of the Firm, Combinative Capabilities and the Replication of Technology. Organization Science, 383-397.

Maynika, J., Chui, M., Briwn, B., Bughin, J., Dobbs, R., Roxburgh, C., et al. (2011). Big data: The next frontier for innovation, competition and productivity. McKinsey & Company.

Nonaka, I., & Takeuchi, H. (1995). The Knowledge-Creating Company. New York: Oxford University Press.

Penrose, E. T. (1959). The Theory of Growth of the firm. London: Basil Blackwell.

Winter, S. G., & Szulanski, G. (1999). Replication as Strategy. Working Paper - Presented at Academy of Management Conference.


If you are interested in writing articles for us, Submit Here