Big Data Analytics through Machine Learning

Posted in Operations & IT Articles, Total Reads: 306 , Published on 01 February 2017

In the last few years, big data analytics has taken a priority role in ensuring growth for businesses, understanding customers & exploring new opportunities. With World’s data doubling every two years while the cost of storing it declining at roughly the same rate; every company is now transforming into a data company capable of analysing large complex data and delivering faster and more accurate insights within reasonable cost. Also, machine learning has been a critical aspect of business. But the latest opportunity for companies is to have Big data Analytics through Machine Learning.

Image: pixabay

For Instance, Tesla has collected 780 million miles of driving data, and they are adding another million every 10 hours. Ultimately this data will be the basis for their autonomous, self-driving car they plan to release in 20181.


Why Machine Learning is preferred for Big Data Analytics?

Machine Learning is the science of getting computers to act without being explicitly programmed. It is preferred for big data analytics because:

Classical statistics model relies on human experts to formulate and test relationship between a set of input and output variables, this process is based on a number of underlying assumptions and the model becomes ill suited when the volume of data is too large because the task of formulating relationship becomes extremely cumbersome.

However in case of machine learning, it flips this entire process and starts with the output variable itself; subsequently uncovering the factors which are driving this outcome. As a result, machine learning is free from limitations of human scale thinking and it has the ability to discover patterns buried in data which can be very difficult to extract from classical models

Apart from this machine learning models have the ability to automatically adjust and improve over time which classical statistical models do not have.


Data Science Leveraging Machine Learning in Various Sectors

1. Agricultural sector


Leveraging Machine learning in Agriculture

1. Precision Farming: Precision Farming allows farmers, a way to deal with every field and field section variations. Using various sensors one can extract information about Soil moisture, Earth density, Air Quality, Temperature of a big farm over a period of time and machine learning can be used to:

• Identify best Suited Crops for different seasons which will provide maximum yield

• Identify the optimum level of fertilizers to be used, Seeds to be planted per acre to obtain maximum yield.

2. Accurate Monsoon Prediction: Over the years various statistical models developed in India have been unable to predict monsoon accurately .An analysis of ten years’ forecast data shows that the IMD’s June-July forecast got the ‘rainfall range’ wrong 60 % of the times3. Hence there is an urgent need for better monsoon prediction models using machine learning technology which can process huge amount of data variables like Historical data of Sea Surface and Land Surface Air Temperatures, Presence of El Nino/El Nina, Cloud Formation Patterns and predict outcome with a much better accuracy.

Banking Sector


Leveraging Machine learning in Banking

1. Improve the profitability of Banks : Through Machine Learning, analysis of basic Information of Customers like Age, Occupation, Salary, Spending Pattern, Savings Pattern, Pattern of Usage of Credit and Debit card can result in :

• Identification of common pool of customers which are most profitable for the Banks and developing customer acquisition strategies with special focus on acquiring those customers.

• Prediction of the probability of a customer defaulting on loan payments based on their past records and taking adequate measures to minimize default risk will reduce the Non-Performing Assets of the Banks.

• Identification of purchasing pattern of individual users and raising alarms in case of deviation from that pattern, thus customising fraud detection at individual level which will reduce instances of false alarm as well improve the efficiency of existing fraud detection models.

2. Improved Customer Experience: Data analytics through machine learning can be used to improve customer experience as in :

• Replacing traditional Password login with Face/Speech Recognition will make the banking experience more efficient.

• Mapping the latest Bank offering/Financial products with segment of customers most likely to use that product will not only improve the hit ratio for banks but its customers will stop getting irrelevant offers/suggestions.

3. Product Development: Analysis of huge amount of Customer feedback data and voice data from call centres with the help of machine learning can play an important role in identification of new Financial Products/Services based on the changing Needs of customers.

Defence Sector


Leveraging Machine learning in Defence

1. Extracting Intelligence Information from Social media Applications: Machine Learning Technology can be used to extract relevant intelligence information from social networks like twitter, facebook, Instagram which can act as a supplement to existing piece of intelligence information, Provide real time information about the probability of occurrence of an unnecessary events at any place, Provide useful data to government in making strategies against enemy countries.

2. Use of Drones: Advanced machine learning algorithms can help drones/unmanned ground vehicles in navigating through unknown territory with the help of pattern analysis and filter relevant information from ad hoc data gathered through surroundings using sensors attached to drones.

Healthcare Sector


Leveraging Machine learning in Healthcare

1. Risk profiling of Patients & Better Diagnosis of illness: Using basic Information of Patient along with his/family medical history machine learning can be used for:

• Predicting which diseases a person is most likely to suffer from in future and recommending necessary changes in lifestyle to minimize those risks.

• Analysing combination of patient’s data simultaneously to come up with better accuracy of diagnosis and recommending optimum dosage amount of prescribed drugs, in the process reducing the number of redundant tests, possible side effects of medicine and reduced cost of care.

2. Expedition of Drug Testing Process: Machine learning technology can be used to speed up drug testing process by mapping the effect of drugs in human cells under all possible set of biological conditions at lower cost and better efficiency.

Limitations of Machine Learning

Some of the limitations of machine learnings are:

1. It is not guaranteed that machine learning algorithm will always work in every case imaginable. It requires careful understanding of the problem at hand in order to apply right machine algorithm.

2. Some machine algorithm requires a lot of training data, in some cases it might be cumbersome to work with or collect such large amount of data.

3. Deep Learning algorithms (Branch of Machine learning) can be made to totally change their response even to small changes made to the input data, In other words Deep Neural Networks can easily make fool of themselves by mislabelling a lion as a bus.

This article has been authored by Ankit Thakur from XIMB




3. The Hindu

4. IDC, Mckinsey Global Institute analysis





If you are interested in writing articles for us, Submit Here

Share this Page on: