Data Analysis Steps

To illustrate the steps involved in analyzing data for financial forecasting, we will use an example of a consumer credit scoring model in the following five steps:

1. Conceptualization of the modeling task. This step requires us to define the problem at hand, the output of the model, how the model will be used and for whom, and whether the model will be embedded in existing (or new) business processes. In our example, the purpose of the model is to accurately measure the credit risk for a borrower.
2. Data collection. For financial forecasting, usually structured, numeric data is collected from internal and external sources. Credit scoring models may use past repayment history, employment history, income, and other relevant variables about a borrower.

The researcher has to determine which sources (internal or external) to use to collect this data.

1. Data preparation and wrangling. This step involves cleaning the data set and preparing it for the model. Cleaning the data set includes addressing any missing values or verification of any out of range values.

Preprocessing data may involve aggregating, filtering, or extracting relevant variables. For the credit scoring model, rules may be used to fill in the gaps where data is either missing or deemed to be inaccurate.

1. Data exploration. This step involves feature selection and engineering as well as initial (i.e., exploratory) data analysis. In a credit scoring model, several variables may be combined to form an ability to pay score.
2. Model training. This step involves determining the appropriate ML algorithm to use, evaluating the algorithm using a training data set, and tuning the model. The choice of the model depends on the nature of the relationship between the features and the target variable.

These five steps are iterative. Depending on the quality of the output, the researcher can go back and tweak any of the steps involved to improve the model. For example, the researcher may revisit the data exploration step to reengineer the features selected for the model.

The steps involved need to be modified in order to analyze unstructured, text based data. For example, suppose the researcher wants to also incorporate a borrower’s social media posts in the determination of credit scores. The first four steps would then be modified as follows:

1. Text problem formulation. The analyst will determine the problem and identify the exact inputs and output of the model. The analyst also has to determine how the output will be used.
2. Data collection (curation). This is determining the sources of data to be used (e.g., web scouring, specific social media sites). If using supervised learning, annotating a reliable target variable is also necessary (e.g., specifying which text is associated with negative or positive scoring of credit risk).
3. Text preparation and wrangling. This requires preprocessing the stream(s) of unstructured data to make it usable by traditional structured modeling methods.
4. Text exploration. This involves test visualization as well as text feature selection and engineering.

The output of a model using unstructured data may then be used in isolation, or combined with other structured variables as an input into another model.

This critical step involves cleansing and organizing raw data for use in a model, and takes most of the project’s time and resources. Once a problem is defined, appropriate data to be collected is identified with the help of domain experts. Data collection involves downloading data from internal and external sources.

While accessing a database, appropriate caution must be exercised to ensure data validity. README files associated with a database usually contain information about how, what, and where data is stored. External data can also be obtained from third party vendors using an application programming interface (API).

External data comes with a price, but saves time and money in the data wrangling step. One issue with using external data in financial forecasting is that other market participants can use the same data, diminishing the firm’s proprietary advantage.

Data cleansing deals with reducing errors in the raw data. For structured data, errors in raw data include:

• Missing values. Invalid values (i.e., data is outside of a meaningful range). Inaccurate values.
• Non uniform values due to use of wrong format or unit of measurement. Duplicate observations.

Data cleansing is accomplished via automated, rules based algorithms as well as human intervention. Metadata (summary data) for the raw data set may serve as a starting point for error identification. Observations with erroneous values that cannot be cleansed would be dropped.

Data wrangling involves preprocessing data for model use. Preprocessing includes data transformation and scaling.

Data transformation types include:

• Extraction (e.g., extracting number of years employed based on dates provided).
• Aggregation, which involves consolidating two related variables into one, using appropriate weights. Filtration, which involves removing irrelevant observations.
• Selection, which involves removing features (i.e., data columns) not needed for processing. Conversion of data of diverse types (e.g., nominal, ordinal).

Outliers in the data set can be identified via statistical techniques (e.g., any value further than three standard deviations from the mean) and replaced with algorithm determined values. Alternatively, the observation may be deleted. One approach to removing outliers is known as trimming, whereby the highest and lowest x% of observations are excluded.

In winsorization, extreme values may be replaced by the maximum value allowable for that variable.

Conversion of data features to a common unit of measurement is known as scaling. Some ML algorithms (e.g., neural networks, SVM) require features to be homogenous (i.e., feature values in the same range). Two common methods of scaling are normalization and standardization.

Normalization scales variable values between 0 and 1.

Standardization centers the variables at 0 and scales them as units of standard deviations from the mean.

A standardized variable has a normal distribution with a mean of 0 and a standard deviation of

1. A standardized variable value of +1.22 is interpreted as having a value of 1.22 standard deviations above its mean.

Unlike normalization, standardization is not sensitive to outliers, but assumes that the variable is normally distributed.

Unstructured, text based data is more suitable for human use rather than for processing by a computer. For analysis, unstructured data has to be converted into structured data. Text processing is the cleansing and preprocessing of text based data.

Text Preparation or Cleansing

Text cleansing involves the following steps:

1. Remove HTML tags. Text collected from web pages has embedded HTML tags, which may need to be removed before processing. A regular expression (regex) is a text string used to identify characters in a particular order.
2. Remove punctuations. Text analysis usually does not need punctuations, so these need to be removed as well. Some punctuations (e.g., %, $, ?) may be needed for analysis, and if so, they are replaced with annotations (i.e., textual expressions) for model training.
3. Remove numbers. Digits are removed or replaced with annotations to let the ML program know that a number is present, but its value is not important in the analysis. If the value of a number is important for analysis, such values are first extracted via text applications.
4. Remove white spaces. Extra formatting related white spaces (e.g., tabs, indents) do not serve any purpose in text processing and are removed.

Text Wrangling (Preprocessing)

Cleansed text is then normalized using the following steps:

1. Lowercasing. So as to not discriminate between market and Market.
2. Removal of stop words. In some ML applications, stop words such as the, is, and so on do not carry any semantic meaning; hence, they are removed to reduce the number of tokens in the training data.
3. Stemming. This is a rules based algorithm that converts all variations of a word into a common value. For example, integrate, integration, and integrating are all assigned a common value of integrat.

While stemming makes the text confusing for human processing, it is ideally suited for machines.

1. Lemmatization. This involves the conversion of inflected forms of a word into its lemma (i.e., morphological root). Lemmatization is similar to stemming, but is more computationally advanced and resource intensive.

In text wrangling, a token is a word, and tokenization is the process of splitting a sentence into tokens. For example, consider the sentence, “It is a beautiful day.” This may be assigned five tokens: (1) it, (2) is, (3) a, (4) beautiful, and (5) day.

After the data is cleansed and normalized, a bag of words (BOW) procedure is applied, which simply collects all the words or tokens without regard to the sequence of occurrence. A document term matrix is then used to convert the unstructured data into structured data. In this matrix, each text document is a row, and the columns are represented by tokens.

The cell value represents the number of occurrences of a token in a document (i.e., row).

If the sequence of text is important, N grams can be used to represent word sequences. A two word sequence is a bigram, a three word sequence is trigram, and so forth. Consider the sentence, “The market is up today.”

Bigrams of this sentence include “the_market,” “market_is,” “is_up,” and “up_today.” BOW is then applied to the bigrams instead of the original words. N gram implementation will affect the normalization of the BOW because stop words will not be removed.