The definition of Big Data
As the Internet development, data is being generated and transmitted rapidly. Internet browsing, using apps, and using simple mobile devices are constantly producing user’s data. The data created in this way are very diverse and are updated rapidly and transmitted continuously. The collection of large and complex data obtained from these data sources is called Big Data. Big Data is too vast and complex to be managed by traditional data processing software. (Rai, 2022) Therefore, special big data software and architecture solutions called big data platforms have been created to manage vast data sets. Big data management technologies are specifically designed to handle the transmission of vast amounts of data of various kinds at very high speeds. The big data platform created in this way consists of various servers, databases, and business intelligence tools. This generally helps data experts analyze data and find its trends and patterns.
These days, big data is applied and utilized in almost all industries. Companies and organizations around the world have used big data to improve insights and make efficient decisions. As a result, business operations can be improved and valuable information can be used to provide better customer service. Companies that use big data effectively are likely to gain a potential competitive edge over those that do not because they can quickly access valuable information and make efficient business decisions. (Botelho, 2022)
However, not all data can be easily used and analyzed for use. In order to utilize the transmitted data, it is necessary to convert the received data and organize the data. Understanding how raw data can be processed is critical to using big data. This is because it becomes data only when valuable information can be extracted and converted into available among the numerous information obtained. (Allen, 2021)
Types of Big Data
(The three types of Big Data)
In order to convert data to be available, it is necessary to understand the types of data. Big data can be classified into three ways
- Structured data
This is kind of big data. Structured data is the easiest to work with because of its fixed format. Due to this data type is already an organized number, it is easy to collect information through programs. In general, since related data are processed and stored in equal formats, access is possible smoothly. Highly organized information is stored in the database. These databases structure and filter the initial data collected for the purpose of performing a specific analysis. Therefore, this is an easy type to analyze because little preparation is required to process the data. This type includes the following. (Allen, 2021)
- Names
- Addresses
- Debit/Credit card numbers
- Geolocation
- Billing
- Contact
- Unstructured data
This is data that has no specific format or structure at all. Only about 5% of the total data is structured data. Therefore, the majority of data types are composed of unstructured data formats. The reason for this type is that almost everything users do on a computer generates unstructured data. In order to obtain useful information, data experts must be able to interpret the obtained data well. Extracting and converting only valuable information from unstructured data is very complex and time-consuming. However, the most important thing than other tasks is to organize and make unstructured data available. Exampled unstructured data include the following. (Allen, 2021)
- Video
- Audio
- Image files
- Social media posts
- Semi-structured data
Semi-structured data is a type that includes both the characteristics of structured and unstructured data. It is not strictly organized like structured data, but it is partially organized or has loose frames. Therefore, this type is easier to analyze data than unstructured data. Most semi-structured data are converted into unstructured data with metadata attached. Metadata contains essential tags or information that make up the data. Therefore, using this type of data through appropriate extraction and transformation can obtain valuable and useful quantitative data. Exampled Semi-structured data include the following. (Wolff, 2020)
- CSV, XML, and JSON
- HTML
- Web pages
- Electronic Data Interchange (EDI)
The characteristics of Big Data
(The three characteristics of big data)
When people think about what big data is, the first characteristic that people can think of is size. Other properties besides size have recently emerged. In 2001, Garner analyst Doug Laney proposed three characteristics of big data. It is called the three V’s: Volume, Variety, and Velocity. His proposed characteristics have become a frame for explaining big data.
- Volume
Volume represents the size of the data. Big data is a vast collection of data. A huge amount of data is generated every day from various resources such as social media platforms, business processes, and networks. Existing data is measured in familiar sizes, such as megabytes and gigabytes. However, big data is stored in the same size as terabytes, petabytes, and zeta bytes. At Berkeley School of information, a comparison was made to understand the difference in volume. One gigabyte corresponds to a seven-minute video with HD quality. On the other hand, one terabyte may store 1500 CDs or 220 DVDs. This amounts to about 16 million Facebook Photos. One zeta byte is equivalent to 250 billion DVDs. According to a report by EMC, the digital world size is doubling every two years. Data storage will increase to accommodate growing data sets. Therefore, it is very difficult to accurately define the volume of big data. (Gandomi and Haider, 2014)
- Variety
Advances in technology have enabled organizations and companies to use various types of data. Types of data include structured data, semi-structured data, and unstructured data. Spreadsheets or relational databases are typical examples of structured data. However, unstructured data makes up 95% of all data. Today, various types of data such as e-mail, photos, videos, and audio are provided. Since this kind of data is not organized, additional techniques for analysis are needed. Semi-structured and unstructured data that could not be used well in the past are not new. However, new data analysis and management technologies that help organizations or companies make good use of data for business processes are innovative. Data utilization, which has become more diverse than in the past, helps to make efficient decisions in running a company. (Gandomi and Haider, 2014)
- Velocity
Velocity includes the generation rate of data to the time required for analysis. As the use of digital devices such as smartphones and sensors spread, data generation speed that was not seen in the past occurred. As a result, the need for a plan to process and analyze data in real-time is increasing. Real-time information deteriorates or loses value in a short time. Therefore, immediate data processing is essential to achieve the best results. Traditional data management systems cannot process vast amounts of data in real-time. Big data management technologies enable immediate data management. Thus, these technologies are being used here. (Gandomi and Haider, 2014)
The advantages and disadvantages of big data
All technologies and machines have advantages and disadvantages. No matter how good the technology is, it has its drawbacks and risks. Big data also has pros and cons. The risks of big data sometimes weaken some of the benefits. In order for companies to make good use of big data, they must properly understand the drawbacks and risks of this technology and find solutions to minimize them.
The advantages of big data
- Increase productivity
A large amount of data from various sources helps companies know what their customers need. With big data, organizations can update the functions and services of existing products to provide new functions and services to customers. This technology helps to improve productivity and consequently maintain customers and increase corporate sales. (Harvey, 2018)
- Identify potential risks
Organizations such as financial firms and banks use big data to detect fraud. Data experts can detect strange patterns using machines that utilize big data. Non-common patterns may be inconsistent information or incorrect fraudulent information. Discovering the dangers of fraud is very important for organizations such as financial companies and banks. Besides financial companies, firms in all occupations can use big data to detect and deal with potential threats in a short time. This can increase the reliability of the company by providing better services to customers. (Harvey, 2018)
- Efficient decision making
Companies use big data to identify what consumers want and don’t want. Customer feedback is important because it can collect a lot of valuable data. Many organizations have been able to use this technology to make accurate and efficient decisions. Big data can use past data to predict the future. Therefore, companies can use this technology to refer to the direction they need to move forward. Firms can increase their competitiveness within the industry through efficient decision-making. (Harvey, 2018)
- Acquire and maintain customers
Interactions such as customer feedback are very important for all businesses because they are effectively available for marketing. The digital traces of customers allow firms to know a lot of things, such as consumer tendencies, preferences, and requirements. Big data provides companies with vast amounts of information, enabling them to come up with customized marketing ideas for their target. Targeted marketing stimulates customers’ emotions and empathy, increasing satisfaction, and corporate loyalty. This ultimately guarantees a company’s revenue growth. (Harvey, 2018)
Difficulty using big data
- Lack of technology and technicians
According to Atscale’s survey, the difficulty in using big data was the lack of technology to analyze big data and data scientists and big data experts. The method of analyzing big data is different from analyzing general data. Big data analysis requires professional knowledge and skills, so the supply is insufficient than the demand for experts. (Harvey, 2018)
- The high expense to use
Organizations and companies must hire professional personnel and maintain and manage hardware and software to use big data. These days, open-source technology has been developed that can greatly reduce the cost of the software. However, processing and analyzing big data frequently exceed the planned budget. Moreover, the salaries of data scientists and big data experts are high in the data field. Therefore, it can be expensive to hire experts and continue to do analysis work. (Harvey, 2018)
- Security Risks
Many companies collect vast amounts of information for big data analysis. The collected information also includes sensitive data such as people’s personal information. Such data should be protected through thorough security. However, cyber attackers such as hackers are likely to attack companies that use big data to access sensitive data. Cyber security attacks are quite threatening to companies because sensitive information can be leaked. (Harvey, 2018)
- Compliance with regulations.
Another tricky issue for using big data is compliance with government regulations. Most of the important and sensitive information, such as people’s personal information, is in a company’s big data repository. Therefore, companies and organizations, which is using big data, must process and store data in accordance with government regulations or industry standards. The amount of data transmitted per day is enormous. As the volume of big data increases, data management will become more difficult. (Harvey, 2018)
Big data applications
- Healthcare
The healthcare industry has access to vast amounts of data. Big data is having a big impact on the healthcare sector and changing many healthcare systems. Wearable devices and sensors collect data from people in need of medical care and provide it to medical institutions. This technology helps medical professionals quickly access and analyze patients’ medical information efficiently. Big data is used by the healthcare sector for the following purposes. (Venkatram and Mary. A, 2017)
- Disease prediction and prevention
- Telemedicine
- Electronic health record system
- Efficient medical decision support system through research comparison
- Banking and Insurance
Today, the banking and securities industries are using big data to capture illegal trading activities in financial markets. Big data can monitor financial activities such as credit card holders’ purchasing patterns to capture abnormal movements in suspected fraudulent transactions. In addition, the target customer’s website use and transaction data are analyzed for marketing in this industry. Big data is used by the financial sector for the following purposes. (Venkatram and Mary. A, 2017)
- Fraud detection
- Risk management
- Identify and predict customers
- Manufacturing
Like other industries, big data enables efficient business in the manufacturing industry. Good use of a lot of information related to the manufacturing industry can improve product quality and energy efficiency. Companies that use data well will be able to gain the upper hand in competition within the industry. Big data is used by the manufacturing sector for the following purposes. (Venkatram and Mary. A, 2017)
- Supply chain management
- Inventory management
- Sensors based operation
Conclusion
In the era of the 4th Industrial Revolution, technologies such as artificial intelligence, the Internet of Things, and robots have begun to emerge. Data has focused the attention of governments and companies around the world as a source for developing these technologies. Various industries are building data platforms, and actively introducing data-related systems to secure more data than in the past and utilize it. Most developed countries have almost completed the construction of data systems for business automation. As demand for big data increases rapidly, the size of the market is also expected to continue to grow. As the market size of big data grows, this is having a big impact on people’s lives.
This technology provides customized information for each people in the personalized modern society. Even now, personalization services are provided that analyze individual data on portals and platforms and recommend content that suits people’s tastes. For example, most users who use Netflix watch videos through recommended content services. This service analyzes users’ tastes, selects and classifies data, and recommends content suitable for users. In the future, as the amount of data increases and the quality increases, the technology to analyze individuals will improve. Then, personalization services will develop more and more, helping virtual assistants who know me better than me to predict the future through data and make efficient decisions. Artificial intelligence will also develop rapidly based on high-quality data. The more high-quality data is used for machine learning, the better artificial intelligence will be. Big data must help protect humanity from crises and make the most efficient decisions by increasing the accuracy of future predictions in complex and diversifying modern societies.
But there are also some risks to using this technology. Big data includes not only general information but also private data, so there are problems such as leakage of personal information and invasion of privacy. There is also a risk of using incorrect data or improperly using it because of collecting and useing large amounts of data. Misused data can lead to poor results by providing inappropriate services to those who want to use it by governments or companies. To prevent this problem from occurring, the risks associated with the use of technology should be strictly managed. There are some problems, but big data must be a revolutionary technology that will change the future of mankind. If experts make the most of the benefits of this technology, people will live more convenient and affluent life.
Reference
Allen, R. (2021). Types of Big Data | Understanding & Interacting With Key Types. [online] SelectHub. Available at: https://www.selecthub.com/big-data-analytics/types-of-big-data-analytics/#:~:text=Big%20data%20is%20classified%20in [Accessed 6 September. 2022].
Wolff, R. (2020). What Is Semi-Structured Data & How to Analyze It. [online] MonkeyLearn Blog. Available at: https://monkeylearn.com/blog/semi-structured-data/ [Accessed 6 September. 2022].
Gandomi, A. and Haider, M. (2014). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), pp.137–144. doi:10.1016/j.ijinfomgt.2014.10.007.
Rai, A. (2022). What is Big Data – Characteristics, Types, Benefits & Examples 2019. [online] upGrad blog. Available at: https://www.upgrad.com/blog/what-is-big-data-types-characteristics-benefits-and-examples/ [Accessed 7 September. 2022].
Botelho, B. (2022). What is Big Data and Why is it Important? [online] SearchDataManagement. Available at: https://www.techtarget.com/searchdatamanagement/definition/big-data [Accessed 7 September. 2022].
Harvey, C. (2018). Big Data Pros and Cons. [online] Datamation. Available at: https://www.datamation.com/big-data/big-data-pros-and-cons/ [Accessed 8 September. 2022].
Venkatram, K. and Mary. A, G. (2017). Review on Big Data & Analytics – Concepts, Philosophy, Process and Applications. Cybernetics and Information Technologies, 17(2), pp.3–27. doi:10.1515/cait-2017-0013.
