Big Data-What's the Future ?

Posted in Operations & IT Articles, Total Reads: 1141 , Published on 21 May 2013

Big Data is here. The analysts and research personnel have made it clear that mining machine generated data is essential to future success.  Embracing new technologies and techniques along with providing a fast and reliable path to business adoption holds the key to an organisation’s attainment of goals.  Adopting appropriate tools to capture and organize a wide variety of data types from different sources is the way to derive the real business value from Big data. This would enable an ease in analysis within the context of all the enterprise data.

Integration of Big Data capabilities with existing infrastructure and BI investments is the need of the hour. Aligning new operational and management capabilities with standard IT, building for enterprise scale and resilience, unifying the database and developing paradigms as we embrace Open Source are some short term and long term measures. Lastly, expanding the IT governance to include a Big Data centre of excellence to ensure business alignment, growing the skilled workforce, managing Open Source tools and technologies, sharing knowledge, establishing standards, and managing best practices need to be taken care of as well.

The term BIG DATA has created a buzz among the enterprises in recent years, but behind the hype there's a simple story. For years, companies have been making business decisions based on transactional data stored in databases. Beyond that critical data, however, is a potential huge chunk of non-traditional and weakly structured data: social media, email etc. that can be extracted for useful information. Reductions in the cost of storage and power have made it possible to collect this data -which would have been thrown away only a few years ago.  As a result, the firms are looking to include non-traditional yet potentially very valuable data with their traditional enterprise data in their business intelligence analysis.

The potential of data-driven decision-making is now being acknowledged broadly, and there is growing interest for the notion of Big Data. Factors like Heterogeneity, timeliness, scale and privacy problems with Big Data obstruct progress at all phases of the pipeline that can create value from data. Much data today is not in structured format; for example, tweets and blogs are unstructured pieces of text, while images and video are designed for storage and display, but not for functions like search: transforming such content into a structured format for later analysis is a major task. Big Data is changing the world, and it is opening up a world of possibilities for operational executives.  New technologies and evolving analytics-based solutions are changing the nature of workforce management. With quantifiable metrics and an emphasis on the strategic impact of more productive employees, results can be drastically optimized. Suitable investment in Big Data will lead to a new surge of fundamental technological advances that will be embodied in the next generations of Big Data management and analysis systems



Source: Big Data For the Future: Unlocking the Predictive Power of the Web, Staffan Truvé, PhD


With terabytes of data per company and limited number of employees, data has now been one of the crucial factors of production alongside labour and capital. Thus, there occurs a need to tap the opportunity. Using big data would lead to value creation in several ways. First, big data can create significant value by making information transparent and functional at much higher frequency. Second, as firms store more transactional digital data, they can collect more reliable and accurate performance monitoring details on everything like product inventories to sick days, and therefore boost performance. Leading organisations analysing and collecting data to conduct controlled experiments to improve decision-making; while other firms are using data for basic low-frequency forecasting to high-frequency nowcasting to fine-tune their business knobs just in time. Third, big data would segment the customer base so narrowly which would help them serve better for their tailor made requirements. Fourth, detailed analytics can significantly improve decision-making. Finally, big data would be used for evolution and development of the next generation of products and services. Using big data would be one of the key bases for growth and competition among the firms, wherein the organisations would strive hard to capture the potential value. Growth in productivity and Consumer surplus are also anticipated to be the key consequences of using big data. Issues like privacy, security and intellectual property would have to be addressed while capturing the full potential of big data. Firms would not only have to adopt the appropriate technology and skilled workforce to use big data, but would also have to streamline their processes and structure their workflows to make the most of big data.

The Article has been authored by Ashish Himtani, IIM Indore.


  • Challenges and Opportunities with Big Data, Philip Bernstein, Microsoft, Elisa Bertino, Purdue Univ
  • Big Data analytiCS, IDC go-to-market services
  • Big Data For the Future:Unlocking the Predictive Power of the Web, Staffan Truvé, PhD
  • Big data: The next frontier for innovation, competition and productivity, Mckinsey&Company


If you are interested in writing articles for us, Submit Here