AU2018101524A4 - Stock prediction research based on finiancial news by svm - Google Patents

Abstract

Based on the marketing situation, as for the stock prediction, we need to include the influence of policies, industry information and investor opinions on stocks' rise and fall. We want to get useful information from high-frequency words in market related, thus predicting the immediate changes in the stock market more accurately by the machine learning. (1)Manipulation of the collected information from individual stock news and industry news. Labeled (stock rise or fall) textual data sorted by date is produced. (2)Analyzing the news data with Jieba Chinese analyzer. (3), Transforming news into eigenvectors by The TfidfVectorizer which is utilized to extract the features of textual data. (4)Utilizing the SVM model in scikit-learn package to classify eigenvectors. The data set is assorted into training set and testing set with undergoing cross-validation. In fact, we can take advantage of the basic methods of TF-IDF and SVM, using both texting mining and stock prediction. Stock Trade News News -p date No DATE SATE NEWS the saie p xp: Chinese date? t 420140905 -1.1 Trexts date ~ tag/label nw Dailyh Sock Daily News Figure 2

Description

TITLE

STOCK PREDICTION RESEARCH BASED ON FINIANCIAL

NEWS BY SVM

FIELD OF THE INVENTION

Based on the machine learning, text mining of stock news related market, and exploring the impact of news on stock value changes to predict stocks.

BACKGROUND OF THE INVENTION

The stock market is a thermometer of a country or region’s economic and financial activities. It reflects capital operation status, industrial development trend and economic heat to a certain level or extent. Therefore, understanding and forecasting the stock market is beneficial to control the financial economic market and adjust the national macro-control.

Based on the effective market hypothesis, one of the classic investment theories, in the stock market with sound laws, good functions, high transparency and full competition, all valuable information has been timely, accurately and fully reflected in the stock price trend. But in fact, the premise of the efficient market hypothesis is very rational, but investors are not completely rational. Investors not only occasionally depart from rationality, but often depart from rationality in the same way.

It shows that due to the complexity and freedom of the market, human cognition of the stock market volatility law is a challenging world-class problem. With the advent of the technological age, machine-learning methods have penetrated into the financial economy. Machine learning uses data or past experience to optimize the performance standards of computer programs. Therefore, the method can be used to understand the causality or relationship between the environment and itself through the processing of complex data and the analysis of redundant information. In fact, the artificial intelligence-based stock forecasting method—the nonlinear system analysis method and the combined prediction method—is more widely used. These methods rely on the index and characteristics of stocks, using neural networks integration technology or support vector machines to conduct stock forecasting, which can get rid of the constraints of linear mathematical relations and time constraints.

These analytical methods based on the stock price reflect the supply and demand relationship of the market on the whole. The deep learning framework of stock forecasting is also based on income optimization. But based on behavioral finance, it’s historically acknowledged that the stock market is alive, whose changes and processes are determined by human psychology. The stock market is largely a reflection of human nature. Many phenomena in the stock market do not conform to scientific principles and established logic.

Based on the above situation, people need to try to add various uncertain random factors, such as investor sentiment, policy influence, industry opinions, etc., to stock forecasting indicators, instead of assuming that capital market participants are computers, who predict the analytical method at the level of pure technical analysis. The psychology and behavior of irrational investors will affect the stock market situation, so that stock prices will be separated from their own value. Irrational market transactions can bring market liquidity. At the same time, the fluctuation of investor sentiment is related to abnormal industrial adjustment income. As a result of above facts, text becomes a more complex but more comprehensive information carrier than numbers and symbols. The surge in user engagement and interaction among these emerging social networks and traditional portals has accelerated the size effect of textual data, which in turn has led to a large increase in the number and level of electronic information, resulting in a shift from the “information scarcity” era to “information overload” era. News, media, social software and stock-related forums include abundant financial data and information of investors’ ease for emotion. What we need to do is not only to obtain information, but how to extract information quickly and efficiently from massive information so as to achieve effective management and utilization.

Based on the impact of investor sentiment on the rise and fall of stocks, we will make full use of the comprehensiveness of text information, the objective efficiency of machine learning and its high ability of dealing with problems, and explore the accuracy of the model based on the combination of TF-IDF and SVM for stock prediction.

SUMMARY OF THE INVENTION

The present invention is a well trained model to predict the rise and fall of stocks by news. There’re several parts in the model, including a usable Chinese text tokenizer, a vocabulary representing text features generated by TF-IDF method, a text vectorizer based on TF-IDF method and a SVM classiffier with well-tuned parameters. And there’re 4 main processes to train the model: Data Processing, Word Segmentation, Text

Features Extraction and SVM Classification.

In Data Processing, we search plenty of news from the Internet about stocks, including single stock news and trade news, as well as the rise and fall of stocks corresponding to the news date. Then, for each stock, getting all dates of the news related to it and combine single stock news and trade news.

In Word Segmentation, we import ‘jieba’ Tokenizer, an efficient and convenient Chinese tokenizer, to deal with the large amount of Chinese news text data. We set a series of judgement to try to avoid the negative effect of unrelated segments given by ‘jieba’ , such as if the segment is in the Chinese stop-word list, in which words always don’t express effective meanings; if the segment is a number, regardless of the pure number, decimal or percentage; if the segment contain common Chinese surnames and so on. Note the segmentation result as corpus.

In Text Features Extraction, we adopt TF-IDF method to calculate the weight of each segment and then construct a vocabulary of words with the highest weight as the characteristics of the corpus. As a result, we could gain the TF value and IDF value of each segment, and the product of TF value and IDF value, called TF-IDF value, would be used to transform corpus to feature matrix.

In SVM Classification, we choose SVM model to work with the problem because of its good performance in high dimensional space because the feature vectors are usually high dimensional, such as 4,000 dimensions or more. We split the feature vector set into two parts: train set and test set to train the model and help debug parameters of SVM model. In order to examine the accuracy of the predict result given by the model, we collected some other stock news as configuration files to detect the model and plotted the accuracy results.

DESCRIPTION OF THE DRAWINGS

The appended drawings are only for the purpose of description and explanation but not for limitation, wherein:

Figure. 1: Overview of the whole process.

Figure 2: Manipulation of the collected information from individual stock news and industry news. Fabeled (stock rise or fall) textual data sorted by date is produced.

Figure 3: Analyzing the news data with Jieba Chinese analyzer. A multi-step processing is applied to remove disruptions such as numbers, names and common stop words.

Figure 4: Introducing scikit-leam package. The TfidfVectorizer is utilized to extract the features of textual data, transforming news into eigenvectors. The number of features is controlled by the input parameters max_df, min_df and max_features.

Figure 5: Utilizing the SVM model in scikit-leam package to classify eigenvectors. The data set is assorted into training set and testing set by a ratio of 9:1, and undergoes cross-validation. The average prediction precision serves as the evaluation standard of optimization to adjust the kernel function of SVM model and parameter C and Gamma.

DESCRIPTION OF PREFERRED EMBODIMENT

In order that the present model can be more readily understood, reference will now be made to the accompanying drawing to illustrate the specific process of the present model.

Step A: Data Processing

Find the news of daily individual stock and the changing situation of corresponding stocks from some Chine financial websites, 1 represents stocks rise and -1 represents stocks fall.

First of all, extract individual stock news for each stock by stock code. Then, connect all the news for that day to one text for each stock by date. After that, select the industry news that affects the stock's changing situation on that day according to the date that stocks go rise or down, and combine with individual stock news into a complete text. When the data processing of all stocks is completed, all texts are integrated into a text data set for later models.

Data pattern is shown in the figure:

Trade news in the corresponding period is shown in the figure:

Step B: Word Segmentation

Import the Chinese word segmentation tokenizer and input the result of the previous step, then use precise word segmentation to obtain the word segmentation results. After that, do multi-step judgment of the result and remove the useless words.

Import Chinese stop-word list, and check the result of text segmentation. If the segment belongs to the stop list, then delete this word; if not, decide if the word is a number, including numbers with decimals and with symbols such as percentage. If none of above has happened, then determine whether the word is a person's name through determine if the word contains a Chinese surname. If it is, then delete the word; if not, the word can enter the word segmentation result corpus.

Step C: Text Feature Extraction

Use TF-IDF method for feature extraction.

Some significant concepts in TF-IDF method:

Term Frequency (the TF value)

In a given document, term frequency refers to the frequency at which a given word appears in the document. This number is the normalization of the term count to prevent it from being biased towards long files, for the same word may have a higher number of words in a long document than in a short one, regardless of the word's importance.

For a verb t in a given document, tfd4 can be represented as

tftq represents the frequency of the word t in document d, ndt represents the number of times the word t appears in document d.

Document Frequency (the DF value) dft represents the number of files containing the word t

Inverse Document Frequency (the IDF value)

Inverse docment frequency is a measure of the general importance of a word.

The idf of a certain word is

\D\ represents the total number of documents in corpus. The purpose of plus 1 is to prevent the words appear in all documents be totally ignored, that is idf #0.

In order to avoid dividing by zero, there is another formula that numerator and denominator both add 1, which is set parameter , that is suppose a document contains all terms.

TF-IDF (the TF-IDF value) TF-IDF means TF multiply IDF. This is a term weighting commonly used in information retrieval, which is also common in document classification.

The formula of tf-idf is as follows:

Import TfidfVectorizer from SK-Feam, and set up parameters max_df, min_df, max_features, where max_df and min_df represents a proportion of documents, integer absolute counts, and max_features represent the max size of the vocabulary generated by TfidfVectorizer.

Import the result of word segmentation, and let the vectorizer automatically generate the text features — a vocabulary, in which the

word’s incidence rate is in the range of the mindf to maxdf. Then obtain the sparse representation of TF-IDF feature matrix by calculating the TF value and IDF value, then transform the sparse array to a normal array.

Step D: SVM Classification

For the prediction of stocks with two labels (1 represents stocks rise and -1 represents stocks fall), use SVM to classify the feature vectors.

Here’s the basic theory of SVM:

Support Vector Machine (SVM) is a sorting algorithm in the field of machine learning, which plays a significant role compared with other algorithms. As a classical algorithm, SVM is more convenient to use and has many advantages. It can not only solve the linear inseparable case, but also reduce the computational complexity when the volume of data set becomes larger.

SVM principles are divided into varies parts, such as Soft Interval Maximization, Lagrangian Duality, Optimization Problem Solving, Kernel Function, Sequence Minimum Optimization SMO. In the model of predicting stocks, the basic principle is Soft Interval Maximization. The core idea of it is to maximize the spacing between the support vectors and the separated hyperplane. Therefore, the following derivations are supposed to be performed.

The distance between point x and hyperplane (w, b) is

The formula of hyperplane is w x + b = 0·

Thus, the geometric interval from point X to hyperplane (w, b) is

xi represents the data i, yi represents the target variable corresponding to the data i, which has two values +1 and -1.

Therefore, when the data i is correctly classified, the value of y is the same as the value of w*x+b, and the geometric interval is positive. When misclassified, the value of y is opposite to the value of w*x+b, and the geometric interval is negative.

Fig.® The value symbol of the sample data about w*x+b

Define the smallest geometrical interval as Y _ m5nK

Thus, the objective function of the interval maximization problem is mas 7, and following the constraints

Build the following transformation:

Then the target function is converted to

, the corresponding constraints becomes

Build the following transformation:

Then theh objective function and constraint condition can be changed into

Since the exponential change of w and b do not affect the formula of hyperplane

Thus,

Therefore, the objective function and constraint conditions of the final interval maximization are as follows:

However, considering the real data in real life, there are some outliers, which do not satisfy the constraint derived above As shown in figure (2), the point A is outlier specific point Fig.(2) Outlier specific point

To solve this problem, introduce a relaxation variable for each sample point, so that constraint cindition becomes

Ej > 0·

Add a variable to the outlier constraint, so that it can satisfy the condition that it is greater than or eqeal to 1

Then the corresponding target variable becomes C represents penalty parameters

The purpose of the penalty parameter is to minimize the target variable, that is the maximum geometrical interval, and to minimize the relaxed variable.

The target function that adds the slack variable is Soft Interval

Maximazation.

Import SVC from SK-Leam, let X represents eigenvector group and Y represents labels.

Then split X and Y into two parts: train set train_X , train_Y and test set test_X and test_Y.

Input train set to SVM models for fitting, and after fitting the model, use test set to test trained models. Input test_X to the model, the result of output is predicted_Y. Compared predicted_Y and test_Y, obtain the accuracy of classification model.

Repeatedly split the feature vector set to adjust the parameters of SVM: kernel function of SVM classificier, penalty coefficient C and common coefficient gamma.

Adjust kernel, C and gamma to gain the optimal model with highest accuracy, then evaluate the model by using cross validation.

EXAMPLE 1

It is reported that, in China, it is a socialist economic system with Chinese characteristics based on macroeconomic regulation and control. However, with the progress of globalization of the economy, China's economic model has increasingly emphasized the importance of the market for resource allocation. Based on the above situation, as for the stock prediction, we need to include the influence of policies, industry information and investor opinions on stocks' rise and fall. We can use the combination of TF-IDF and SVM models to get useful information from high-frequency words in market related, thus predicting the immediate changes in the stock market more accurately. For example, China recently launched a tax for being single and a second-child fund policy, which will

affect the stock value of listed insurance companies, and will further affect the volatility of insurance-related stocks by affecting investor sentiment and choice. And the influencing factors of this part can be added to the stock prediction through this method. EXAMPLE 2

With the use increase of public Internet forums by the users, we can get public opinions on optimization of a certain product or a company, or the specific opinions on the planned implementation of a certain policy of the state. We can take advantage of the basic methods of TF-IDF and SVM, to obtain the public’s opinion on a certain incident from more numerous and richer data out of the restrictions of the limited options in ordinary polls. For example, the specific pricing of the real estate market, to some extent, depends on the purchase willing of the consumers before the opening. In other word, we can use text mining to get more accurately prediction on the consumers’ willing of purchase.

Claims (2)

1. A stock prediction models based on news related stock, which is the advantage of this model is not only following the big data, but also making use of machine learning to screen useful information in complex and massive information, it is also because we will break away from the constraints of simple numbers and add the influence about industry news and national policies in the stock forecasting; this means that we can conduct multi-complicated stock forecasting through text mining.

2. A model as claim 1, which is accomplished the mission of text mining by combining, include TF-IDF and SVM.