By now most of you must have heard the term ‘Big Data’ and have had the question ‘What is Big Data’, ‘Why is it called Big Data’ and ‘Why is this important to me or a company?’ There are many explanations for this but the simple answer to the question, of what Big Data is.
“Big data is a set of high volume, high velocity and high variety data, which needs innovative ways to process and analyse to bring out sensible decision-making set of information.”
Simply, a set of Big Data will not be able to store in your traditional relational Database management system and you will not be able to extract information by executing simple SQL queries. Understanding the following will enable you to visualize what this is all about
HIGH VOLUME –The amount or the volume of data for specific criteria would be very high. The source of the data can be machine generated and the amount of data generated. Collection of data from a large sensor network, smart metering for electricity and water will generate a huge volume of data and the growth will be exponential.
Example: Large social media networks have to store multiple peta bytes of data per day!
HIGH VELOCITY –The speed which the data is generated is high. This can be small chunks of data but the number of occurrence against time will be quite high.
Example: Collection of data from a large sensor network from multiple locations - monitoring of seismic activities, data from humidity, temperature and wind sensors will generate data every second.
HIGH VARIETY – The types of data will also be different, simply unstructured as the multiple sources will provide the data in many different formats.
Examples:
Various formats, types and structures
Text, numerical, images, audio, video, sequences, time series, social media data, multi-dim arrays, etc.
Static data vs. Streaming data
A single application can be generating or collecting many types of data
Alright, now that we know what Big Data is and how to identify this trending word in the market - we should now look into why it is important to process these data. It’s quite simple. As we process the structured data and come into conclusions to make decisions, we can process and analyse this - Big Data and take decisions. The difference is that Big Data will provide more detailed and varied insights into the specific scenario as needed.
Let’s discuss some simple examples.
We are all familiar with general health problems which affect us and what we do is, to do certain checkups on regular intervals and get physician or a specialist’s advice or medicine as applicable. As you are aware, for a critical patient; those tests are done at more frequent intervals and a patient at ICU, receives multiple measurements like blood pressure, breathing patterns and ECG etc., which are taken continuously. Imagine if the doctor can use these kinds of data from multiple patients and analyze them using modern tools by combining some environmental social economic factors as well. The same kind of analysis can be used to understand underneath factors for many more health hazards or to identify certain outbreaks by collecting data from large groups of identified patients or from general public from certain areas.
Following are some business use cases for which the Big Data Analytics can be used to drive incremental revenue, improve operational efficiencies and reduce risks. In most instances, analytics around a customer has provided deeper understanding of the customer sentiment so the selling will be highly targeted towards the actual needs of individual customer.
TELCO – Todays telecommunication service providers are quite concerned on customer satisfaction, as it is directly related to their revenue. By understanding customers behaviour, the services subscribed to, the services used, usage patterns, etc., the service provider can improve both, the customer satisfaction and also increase revenue through upselling and cross selling.
SUPERMARKETS – Buying patterns can be analysed for each customer by transactions based on the purchased products, quantities, occasion, etc. This way, opportunity arises of upselling for each customer and to improve customer satisfaction. Because of the various options available in this era of innovation, the potential of customer doing business with a specific seller is quite high.
ONLINE BUSINESS – Buying or browsing patterns for a narrower customer base can be created with Big Data analytics allowing the business they serve to assist them frequently.
As mentioned above, a major area which Big Data analytics are getting highlighted recently is the healthcare sector. The extracting and analysing relevant past clinical data and other related factors; medical professionals can improve health of patients and even avoid preventable deaths to a larger extent. Not only improving the quality of life, these types of analysis will help predicting any potential epidemics, find cures and even wipe out certain diseases from the planet.
How can we get it done?
Well there can be multiple initiations depending on the area we want to start. When it comes to individual health, there are multiple wearable sensors which are being used and even connect your smartphone which collects and records data of your health. From blood pressure and heart rate - tracking your movements with wearable devices will collect the data fulltime and store for future analysis. Once this data is analysed with genetic data, living conditions and socio-economic factors and even with through social media, a vivid picture will provide you with potential areas which you need to take action on.
Going one step further, we would be able to compare and contrast individual data with many other records to identify patterns and anomalies which would highlight potential risks. Likewise, our modern researchers are providing options for patients to have medicine which are designed and produced for individuals based on individual profiles which is created using the captured data and analysing them.
Yet another area which we must mention is the insurance industry - a high risk involved sector which will definitely benefit by taking decisions based on Big Data analytics. Insurance companies will mostly benefit by understanding fraudulent claims and the ability to reject such claims at early stages. By analysis of past data from various source systems, an insurance company can benefit greatly to reject or increase the premium, even at the initial engagement period.
I guess now it’s a good time to answer the questions whether Sri Lanka can benefit using Big Data. The Answer - a definite Yes.
A year back Big Data analytics it seemed would take another five or six years more to be applied into our day today activities. Luckily this is happening much sooner, as many institutions have taken the initiative to incorporate Big Data analytics into their decision making process. A good example would be the social media analytics - many service providers, retailers and online sellers do use the social media activities and feedback of their customers and potential buyers in their decision making.
When it comes to Big Data, it is also required to know few things about security and privacy of the data being used. Security of Big Data can be categorized into a couple of main areas.
Firstly, considering the vast amount of information collected there is a challenge of enforcing standard security safeguards to the data captured or received from various sources. When large volumes of data with varied formats received as streams of data from large scale platforms which are spread across various geographic regions the attack surface gets expand and the classical security measures will not be able to
do much.
Secondly, the new can of worms will open up related to privacy of data. There will be questions related to the ethical nature where it is difficult to draw a fine line.
(To be continued with Part 2 of ‘Can Big Data benefit enterprises in Sri Lanka?)
(Chamara Rupasinghe is the Head of Information Security Business, Just in Time Group)