In math lessons at school, you constantly worked with data: add, multiply, divide in your head or in a column. Perhaps you also keep the family budget in a notebook or in a spreadsheet – enter information and use simple formulas: find the amounts, differences, averages. That is, you perform data processing, and mostly manually. When there are few of them, it is relatively easy to cope with such tasks.
Big data is when there is really a lot of information: there is no clear boundary, but usually we are talking about gigabytes, if not about terabytes. These arrays can come from many sources at once: online stores and social networks, industrial quality management systems, video surveillance systems, IoT devices.
The density of the data can also be different: some systems take measurements every hour, others several times per second. Accordingly, the amount of information is different: from a few kilobytes to hundreds of gigabytes.
Working with big data manually is difficult: it is time-consuming, expensive, and inefficient. Therefore, for the analysis of such arrays, automatic processing tools are used.
Why does a business need to analyze data
Imagine you are running a grocery store. How do you know what the buyer wants? Ask him – and you will hear what products he purchases more often, at what time he usually goes shopping.
But a lot of details will remain behind the scenes. For example, it is analysts who know how full shelves, bad weather, background music affect purchases.
All this and other data can be collected and analyzed. This will help the supermarket to arrange the goods so that the buyer stays in the sales area for as long as possible and pays attention to the necessary offers, and to revise the work schedule of cashiers in order to reduce queues at the checkout. By learning more about the interests of its customers, the store will be able to optimize procurement and logistics. As a result, revenue will increase and expenses will decrease.
You can find a use for big data in any area:
1. In factories, a computer vision system monitors workers. The system will notice if someone has forgotten about the helmet and will remind you of the safety rules.
2. In banks, big data analysis dictates the terms of loans and deposits, reveals hacker attacks and suspicious transactions.
3. Big data drives cities too. Smart traffic lights reduce traffic jams, computer vision looks for criminals in the crowd. Analysts use consulting services in data science before building a new road or center of public services, changing the route of the bus.
How the job of a data analyst differs from a data scientist
In simple situations, you can do without big data analysis and use banal logic. For example, if you notice that customers with children in the store often purchase certain cookies, then you can simply put baby juice next to it and thereby increase sales.
But in practice, things are usually much more complicated. For example, how to create an optimal package of services for a mobile operator and determine the price that will be affordable for the subscriber and will bring the maximum benefit to the company?
The analyst can structure and process data on the mobile market, existing packages and subscriber costs. He will formulate and test hypotheses, find patterns and draw conclusions: he will offer a specific composition of the package and its price.
More complex tasks, as well as the search for non-obvious patterns in the data, is already being dealt with by another specialist – a data scientist. So, you may not even suspect that the purchases are related. Or that the routes of cars on Tuesday and Wednesday are different, so traffic jams are formed in different areas – although, it would seem, these are normal weekdays.
