Big Data Projects

Exam Focus

This topic review is a very broad overview of the usage of big data analysis for financial forecasting. Candidates should understand (1) the terminology used and the processes involved in big data projects, (2) the requirements and limitations of the techniques discussed, and (3) how to evaluate a model’s performance.

INTRODUCTION

Big data is characterized by the three Vs of volume, variety, and velocity:

• Volume refers to the quantity of data. Big data refers to a huge volume of data. Variety refers to data sources.
• Big data is collected from various sources: user generated, traditional transactional, emails, images, clickstreams, and so on. The collection of various data presents tremendous opportunities as well as concerns, such as privacy protection.
• Velocity refers to the speed with which the data is created and collected (e.g., all the social media postings created during a specified time interval).

When used for generating inferences, an additional characteristic, the veracity or validity of the data, needs to be considered. Not all data sources are reliable, and the researcher has to separate quality from quantity to generate robust forecasts.

Structured data (e.g., balance sheet data for companies) is neatly organized in rows and columns. Unstructured data (e.g., text from SEC filings) is unorganized, and the machine learning (ML) algorithm has to sift through the noise to pick out information.