Hindawi

International Transactions on Electrical Energy Systems

Volume 2022, Article ID 1197534, 11 pages
https://doi.org/10.1155/2022/1197534

Research Article
News Text Classification Method Based on the GRU_CNN Model

Lujuan Deng,1 Qingxia Ge,1 Jiaxue Zhang,1 Zuhe Li ,1,2 Zeqi Yu,1 Tiantian Yin,1
and Hanxue Zhu1
1
School of Computer and Communication Engineering, Zhengzhou University of Light Industry,
Zhengzhou 450002, Henan, China
2
Henan Key Laboratory of Food Safety Data Intelligence, Zhengzhou University of Light Industry,
Zhengzhou 450002, Henan, China

Correspondence should be addressed to Zuhe Li; zuheli@zzuli.edu.cn

Received 25 June 2022; Revised 25 July 2022; Accepted 16 August 2022; Published 31 August 2022

Academic Editor: Raghavan Dhanasekaran

Copyright © 2022 Lujuan Deng et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The convolutional neural network can extract local features of text but cannot capture structure information or semantic re-
lationships between words, and a single CNN model’s classification performance is low, whereas GRU can effectively extract
semantic information and global structure relationships of text. To address this problem, this paper proposes a news text
classification method based on the GRU_CNN model, which combines the advantages of CNN and GRU. The model first trains
word vectors as the embedding layer with the Word2vec model and then extracts semantic information from text sentences with
the GRU model. Following that, this model employs the CNN model to extract crucial semantic information features and finally
completes the classification through the Softmax layer. The experimental results reveal that the GRU_CNN hybrid model
outperforms single CNN, LSTM, and GRU models in terms of classification effect and accuracy.

1. Introduction makers need to have enough knowledge and understanding

of a certain category field, but this method necessitates a
With the rise of various news clients such as NetEase News, significant amount of manpower, material wealth, and fi-
Toutiao, and Tencent News, news resources have become nancial support. What is more, the subjectivity of manual
readily available. However, how to accurately locate infor- rule design will also cause differences in classification. Text
mation from existing news resources and find information categorization in natural language processing is gradually
that can meet their own needs is becoming a problem worth being substituted by automatic classification by machine
studying at present. In addition, news classification is the rather than manual classification since the emergence of
basis of news website classification navigation, personalized machine learning. Support vector machine (SVM) [5],
recommendation, etc. Therefore, efficient and accurate K-nearest neighbor (KNN) [6], Naive Bayes (NB) [7], lo-
screening and classification of news content have very im- gistic regression (LR) [8], and decision tree [9] are a few
portant practical significance, so how to achieve efficient and machine learning algorithms that are commonly employed
accurate classification has become a nodus [1]. in text classification. However, such traditional machine
Text classification is indeed the fundamental work learning models usually require manually designed features
among many natural language processing applications, and and feature combinations, which are highly subjective and
it can support tasks including topic classification [2], sen- task-specific. At the same time, these models can only extract
timent analysis [3], and question answering system [4]. At shallow features. The model is prone to overfitting and
present, there have been many studies on text classification. inefficiency caused by the large amount of data, which ul-
The early research was mainly based on manual rules and timately affects the classification accuracy. Due to the
knowledge engineering methods, which require manual popularity of deep learning, many neural network models
processing of data and the participation of experts. The rule- have been extensively applied in the production of text
2 International Transactions on Electrical Energy Systems

categorization in recent years. Deep neural network models requires a lot of calculations during training due to its
have superior feature extraction ability to typical machine numerous parameters and the complexity of the calculations
learning algorithms, making them ideally suited for appli- between each gate.
cations in the area of text categorization [10]. Chung et al. proposed the GRU model [17] on this basis,
The primary goal of this study is to address the problem which simplified the structure and training parameters of
of Chinese news text classification using deep learning LSTM and improved the efficiency of training. However,
models. As GRU can efficiently extract the semantic in- GRU is unable to compute in parallel and is still unable to
formation and global structural information of text, and as resolve the gradient vanishing problem entirely.
CNN cannot effectively extract the structural information of Graves and Schmidhuber [18] applied the bidirectional
text and the semantic relationship of sentences, the classi- LSTM model for the first time to solve the categorization
fication accuracy of a single CNN model is low. Therefore, problem and achieved better classification results than the
this paper combines the benefits of the CNN model and the unidirectional LSTM model. However, compared with
GRU model to propose the GRU_CNN model for the LSTM, BiLSTM has more parameters and is more difficult to
classification of news text. calculate. Cao et al. [19] utilized the BiGRU model to cat-
egorize Chinese text by synthesizing the context of the ar-
2. Related Work ticle. The model is simple, with a few parameters and fast
convergence.
Among deep learning classification models, there are two Li and Dong [20] employed CNN to extract text’s local
main types of common categories, namely, convolutional features and the BiLSTM to extract text’s global features in
neural network and recurrent neural network, which have order to fully capitalize on the benefits of the features ob-
many applications in text categorization. tained by the two models and enhance the model’s ability to
Kim [11] first proposed a text categorization model classify data.
based on CNN in 2014, which converts words into fixed- In conclusion, a single neural network model often has
length word vectors through Word2vec as the input of CNN the problem of low classification accuracy. This research
and then uses multisize convolution to check word vector provides a news text categorization strategy based on the
convolution. Finally, pooling and classification are per- GRU_CNN model in order to get a better classification
formed. The key advantage of CNN is that it has a quick performance. This model extracts more accurate text fea-
training speed and can efficiently extract local text features, tures, and its effectiveness has been demonstrated through
but the pooling layer will lose a lot of vital information and experiments.
overlook the association between the local and the whole.
After Kim proposed the model, Zhang and Wallace [12] 3. GRU_CNN Model
also proposed a text categorization method based on CNN
and performed numerous comparative experiments under The whole process of the GRU_CNN hybrid model is as
various hyperparameter settings. Besides, they also offered follows: Firstly, input the news text data, preprocess the news
parameter tuning advice and have some experience with text, and train the text using the Word2vec model to produce
hyperparameter configuration. a word vector comprising the text’s overall information.
Johnson and Tong proposed the deep pyramid con- Secondly, to extract contextual semantic information from
volutional neural network (DPCNN) [13], which has low the text, the vector representation of words obtained by
complexity and excellent categorization power. This model Word2vec is fed into the GRU model for semantic infor-
primarily investigates word-level CNN and improves ac- mation extraction. Then, the output results of the GRU
curacy by deepening the depth of the network. DPCNN can model are input into the CNN model for further semantic
extract long-distance text dependencies by constantly feature extraction. Finally, the final word vector is input to
deepening the network, but the computational complexity the Softmax layer for text classification. Figure 1 depicts the
will increase as well, which makes practical applications GRU_CNN model’s structure.
problematic. Yao et al. proposed a graph-based convolu-
tional neural network (GCN) [14], which is more effective
for small datasets categorization. GCN’s limitations of 3.1. Word2vec Embedding Layer. This paper uses the classic
flexibility and scalability are its main drawbacks. Word2vec [21] model for word embedding, and the
Mikolov et al. [15] employed the recurrent neural net- Word2vec of the Gensim toolkit is used to train word
work model to achieve text categorization. RNN can ac- vectors, which is simple and fast. The Word2vec model is
commodate inputs of arbitrary length, and the model size divided into three layers: input, hidden, and output. The
does not grow with the input length. However, the RNN model’s input is a one-hot vector; that is, the textual in-
model relies on learning features for a long time, which is formation is represented by 􏼈x1 , x2 , . . . , xV 􏼉. Assuming the
prone to the problem of gradient dispersion. size of the vocabulary is V and the dimension of each hidden
Hochreiter and Schmidhuber proposed a long short- layer is N, the input to the hidden layer is represented by a
term memory network (LSTM) [16] based on RNN, which matrix W of size V × N. Similarly, word vectors can be
makes up for the classic RNN’s poor learning performance obtained by connecting the hidden and output layers via a
for long-distance sentences as well as the problem of gra- N × V matrix W′ . Figure 2 depicts the Word2vec model’s
dient disappearance or gradient explosion. However, LSTM structure.
International Transactions on Electrical Energy Systems 3

GRU

x1

softmax
GRU
x2

xn-1 GRU
xn

GRU

Word2vec Max Fully

Input GRU Convolution Output
embedding Pooling Connected
Layer Layer Layer Layer
Layer Layer Layer
Figure 1: GRU_CNN model.

prone to the problem of gradient dispersion. In order to

overcome the shortcomings of RNN, many variant models
x1 y1 of the recurrent neural network have been proposed, among
h1 which the two most popular models are LSTM and GRU.
x2 y2
The input gate, memory unit, forget gate, and output gate
x3 h2 y3 are the four main components of an LSTM. The important
features of the text are retained through the “memory” and
“forgetting” of the input vector, while the largely pointless
hi
xj yk stuff is removed. However, as the volume of texts grows, the
WV×N W′N×V issue that LSTM network training takes a long time to
hN complete is gradually revealed. This is because of the high
number of parameters and the relatively difficult calculation
xv yV between the gates. To address the shortcomings of the LSTM
model, a simpler neural network model GRU is presented.
Figure 2: Word2vec model. The GRU model combines the forget gate and the input gate
into an update gate based on LSTM.
The Word2vec model is analogous to a simple neural As a variant of RNN, gated recurrent unit network
network that mainly includes CBOW and Skip-Gram (GRU) uses a reset gate and update gate to decide how to
models. The CBOW model primarily foretells the present discard and update information, which successfully solves
value based on the context, which is equivalent to taking a the problem of long-term dependence. GRU contains fewer
word out of context and letting you infer its meaning. The parameters and is easier to calculate than LSTM, which can
Skip-Gram model primarily forecasts the context using the enhance training efficiency to a certain level while achieving
current word, which is the same as giving you a word and the same effect as LSTM. This paper uses the GRU model to
asking you to predict which words might come before and obtain the context semantic information of text. Figure 4
after it. CBOW is better suited to smaller datasets, whereas depicts the GRU model’s structure.
Skip-Gram performs better in large corpora. Figure 3 depicts The update gate is used to determine the impact of the
the structure of the CBOW and Skip-Gram models. This previously hidden layer state on the current layer. The larger
paper uses the Skip-Gram model for training, and Skip- the value in the update gate, the greater the impact on the
Gram is more sensitive to low-frequency words. current layer at the previous moment. Formula (1) illustrates
the update gate’s computation process.
Um � σ WU •􏼂hm−1 , xm 􏼃􏼁. (1)
3.2. GRU Layer. Recurrent neural network (RNN) [22] has
been proved by many scholars to be suitable for deep The reset gate is used to remove the invalid information
learning tasks of text processing. But when faced with longer at the last moment. The smaller the reset gate value, the more
texts, it is difficult for the gradient of the letter sequence to invalid information is deleted. Formula (2) illustrates the
backpropagate to the earlier semantic information, and it is reset gate’s computation process.
4 International Transactions on Electrical Energy Systems

Input Projection Output Input Projection Output

W(t-2) W(t-2)

SUM
W(t-1) W(t-1)

W(t) W(t)

W(t-1) W(t-1)

W(t+2) W(t+2)

CBOW Model Skin-gram Model

Figure 3: CBOW model and Skip-Gram model.

neural network model with good performance and

hm
fast training speed. Figure 5 depicts the CNN model’s
hm-1
structure.
The function of the convolution layer is to extract local
1- features within a set window range and use the convolution
Cm Um kernel to convolute the input vectors to obtain the feature
~ output.
σ σ hm
The pooling layer further integrates the local features of
tanh
the text extracted by the convolution layer, shrinks the
feature map, and enhances the computation speed. At the
same time, it obtains global information from the feature
values, improves the robustness of the extracted features,
Xm and controls overfitting issues. Generally, two approaches
are used: Mean pooling and Max pooling. Mean pooling is to
Figure 4: GRU model. calculate the average value of column vectors, and Max
pooling is to directly extract the maximum level of column
Cm � σ WC •􏼂hm−1 , xm 􏼃􏼁. (2) vectors. This article uses the Max pooling operation.
The fully connected layer further abstracts its eigen-
The current state computation method is presented in values, taking all the outputs of the pooling layer as input,
the following formulas: where each neural unit is connected to each unit of the
pooling layer and transforms the pooling layer vector into a
h􏽥m � tanh W•􏼂Cm ⊙ hm−1 , xm 􏼃􏼁, (3) long vector by activating function ReLu, mapping the text
from the feature space to the marker space.
hm � Um ⊙ h􏽥m + 1 − Um 􏼁 ⊙ hm−1 . (4) The CNN model is utilized to extract semantic
information features in this paper. After obtaining the
Among them, WU , WC , and W represent the weight contextual semantic information extracted from the GRU
matrix of the GRU, xm is the input data, hm is the current layer, it is input to the CNN layer for extracting semantic
hidden state of the model, the input of the previous state is features.
hm−1 , h􏽥m is the candidate active state, Um and Cm represent
update and reset gates, respectively, ⊙ represents the
Hadamard product, that is, the elements of the corresponding
4. Experiment and Analysis
position are multiplied, and σ represents the sigmoid 4.1. Dataset Introduction. The news text dataset used in this
function. article is Cnews, which is a subset of the THUCNews dataset.
10 of these categories were used, each with 6,500 entries, for
a total of 65,000 news data. The specific categories and
3.3. CNN Layer. Convolutional neural network (CNN) was divisions of Cnews dataset are shown in Table 1.
originally employed to solve image processing problems and
later applied to natural language processing. It has grown in
popularity as among the most widely utilized deep learning 4.2. Comparison Experiment Design. In order to test the
models. Among text categorization models based on deep hybrid neural network model’s effectiveness, several classical
learning, the convolutional neural network is a shallow models are selected for comparative experiments.
International Transactions on Electrical Energy Systems 5

FeatureMap

Max-pooling

Text category

Convolution fully
kernels connected
layer

Word vector Convolution Poooling Softmax

layer layer layer
Figure 5: CNN model.

Table 1: Distribution of Cnews dataset. Table 2: Parameters.

Data category Training set Test set Validation set Parameter names Values
Sports 5000 1000 500 Convolution kernel size (2, 3, 4)
Finance 5000 1000 500 Word vector dimension 100
Real estate 5000 1000 500 The number of hidden units 128
Home 5000 1000 500 Dropout 0.5
Education 5000 1000 500 learning_rate 0.001
Technology 5000 1000 500 Optimizer Adam
Fashion 5000 1000 500 Batch size 64
Politics 5000 1000 500 Epoch 6
Games 5000 1000 500
Entertainment 5000 1000 500

(1) LR: logistic regression is a machine learning cate- 4.4. Evaluation Index. The experiment employs the accuracy
gorization method for estimating the likelihood of of text categorization as its evaluation index in order to
something. examine the effectiveness of the model put forth in this
study. The accuracy rate indicates the proportion of correctly
(2) NB: Naive Bayes is a machine learning categorization
predicted samples to the total samples, and the calculation
algorithm that leverages Bayes’ theorem.
formula is shown in
(3) CNN: it is the most basic convolutional neural
network for text categorization. 1 K
accuracy � 􏽥 i|.
􏽘 |yi � y (5)
(4) LSTM: it is the most basic long and short memory K i�1
network for text categorization.
In formula (5), K represents the sample capacity of the
(5) GRU: it is the most basic gated recurrent unit net- sample set, yi represents the label of the sample xi , and y􏽥i
work for text categorization. is used as the prediction result. When yi and y 􏽥 i are
(6) LSTM + CNN: firstly, the LSTM model is utilized to consistent, the value of the precision rate is 1; otherwise,
extract the contextual semantic information from the the value is 0.
input text, and then the CNN model is used to extract
the feature of the semantic information. Finally,
complete classification through the Softmax layer. 4.5. Analysis of Experimental Results. The error and accu-
racy of GRU_CNN model obtained by training on the
4.3. Parameters. The experiment presented in this paper is training set are shown in Figure 6. During the experiment,
based on the TensorFlow framework, and the model’s pa- the accuracy and loss values are output on the training set
rameter settings determine how well the model trains. Ta- and validation set every 100 batches and recorded and
ble 2 displays the GRU_CNN model’s specific parameter saved. Continuously train the model, saving the best-
settings. performing model for testing on the test set. We can see
6 International Transactions on Electrical Energy Systems

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
step_100

step_1000
step_1300
step_1500
step_1700
step_1900
step_2100
step_2300
step_2500
step_2700
step_2900
step_3100
step_3300
step_3500
step_3700
step_3900
step_4100
step_4300
step_4500
step_300
step_500
step_700
step_900

accuracy
loss
Figure 6: Error and accuracy results of model training.

that the model’s accuracy on the training set has achieved in this experiment is Adam. The Adam optimization algo-
good results. rithm has high computational efficiency and fast conver-
In the experiment, the number of epochs can affect the gence speed. In order to make the algorithm more efficient,
model’s performance. This study discusses the model clas- different learning rate values are selected for experiments.
sification effect under different epoch numbers through Table 5 and Figure 9 illustrate the experimental outcomes. As
experiments. Table 3 and Figure 7 illustrate the experimental shown by the experimental results, the model’s accuracy is
outcomes. The experimental results show that the number of highest when Adam’s corresponding learning rate is 0.001.
epochs required by different models to achieve the optimal Therefore, the learning rate of the Adam optimizer in this
effect varies. The GRU_CNN model reaches the highest case is 0.001.
accuracy rate when the epoch number is 6. The numbers of Word vector dimension is also an important parameter.
epochs at which CNN, LSTM, GRU, and LSTM + CNN The larger the dimension, the more feature information the
achieve the best results are 3, 2, 4, and 5, respectively. model can learn, and the greater the risk of the model
Figure 6 shows that the model’s performance first rises and overfitting. However, the lower the word vector dimension,
then falls with the increase of the number of epochs, which the greater the risk of the model underfitting. Therefore, the
means that, before reaching the best effect of the model, the appropriate word vector dimension is also important for the
ability of model learning characteristics will continue to model’s training effect. This paper selects different dimen-
increase as the number of epochs increases. After the best sions of the word vectors to train. Table 6 and Figure 10
results, the model will show an overfitting phenomenon so illustrate the experimental outcomes. As can be observed
that the model’s performance will decrease. from the experimental findings that all models’ accuracy
During the training process of the model, the dropout reaches the highest in dimension 100, this paper chooses 100
method is introduced into the model in this experiment. The dimensions to experiment.
value of dropout is also an important parameter, and an Figure 11 illustrates the experimental results of all
appropriate value can enable the model to converge better, models on the Cnews dataset. For proving the performance
avoid overfitting, and improve its performance. Therefore, of the GRU_CNN model, several classical models are se-
we choose different dropout values for training. The dropout lected for comparative experiments. In the classification
values set in this experiment are [0.2, 0.3, 0.4, 0.5, 0.6, 0.7], model based on machine learning, logistic regression (LR)
respectively. The optimal dropout values are selected based and Naive Bayes (NB) are selected for comparative exper-
on the training results of the model. Table 4 and Figure 8 iments. In the classification model based on deep learning,
illustrate the experimental outcomes. The findings show that this paper selects a single CNN, LSTM, and GRU model, as
only the LSTM model has the best effect when the dropout well as a hybrid model LSTM_CNN for comparative ex-
value is 0.7, and the other models have the best effect when periments. The experimental results are selected to compare
the dropout value is 0.5. At this time, the dropout value can the best effect of each model.
effectively prevent the model from overfitting on the premise As can be noticed from the outcomes of the experi-
of ensuring accuracy. Therefore, the dropout value of the ments, the GRU_CNN model has the best classification
model in this paper is set to 0.5. effect and the highest accuracy on the Cnews dataset.
During the course of updating parameters by gradient Among the traditional machine learning classification
backpropagation in the neural network, the optimizer used algorithm models, the classification accuracy of logistic
International Transactions on Electrical Energy Systems 7

Table 3: Table of epoch experimental results.

Epoch CNN (%) LSTM (%) GRU (%) LSTM + CNN (%) GRU + CNN (%)
1 93.96 94.42 95.5 95.28 96.48
2 95.26 96.3 94.62 95.56 96.94
3 96.9 96.06 94.54 97.26 97.22
4 96.24 95.42 95.9 97.24 97.04
5 96.84 96 94.62 97.78 97.1
6 96.22 96.24 95.88 97.18 97.86
9 96.91 95.68 94.3 97.4 97.2
10 94.26 94.8 94.28 97.56 96.84

99

98

97
accuracy (%)

96

95

94

93

92
1 2 3 4 5 6 9 10

epoch number

CNN LSTM+CNN
LSTM GRU+CNN
GRU
Figure 7: The accuracy curve of each model under different epoch numbers.

Table 4: Table of dropout experimental results.

Dropout CNN (%) LSTM (%) GRU (%) LSTM + CNN (%) GRU + CNN (%)
0.2 95.94 95.72 95.88 96.32 94.32
0.3 96.42 94.3 95.54 97.28 95.92
0.4 95.16 95.3 94.64 95.98 95.94
0.5 96.9 96.3 95.9 97.78 97.86
0.6 95.14 95.38 93.18 97.42 97.38
0.7 96.44 96.78 93.4 97.38 97.52

regression (LR) on the Cnews dataset is superior to that of account, the GRU_CNN model this study proposes per-
the Naive Bayes model (NB), and the classification ac- forms better.
curacy of all deep learning models is higher than that of
traditional machine learning classification models, which
is also the reason why deep learning models are popular 4.6. Ablation Experiment. In order to verify the effectiveness
nowadays. In the deep learning model, the hybrid neural of using Word2vec to train word vectors as the embedding
network model has a greater categorization accuracy than layer, this paper first selects CNN, LSTM, and GRU as the
the single deep learning model, and the CNN model has a comparative experimental model and compares the exper-
better categorization effect than the LSTM and GRU imental results of Word2vec_CNN, Word2vec_LSTM, and
models, as shown by the results. Although the classifica- Word2vec_GRU. The experimental results are shown in
tion accuracy of the GRU_CNN model and the Table 7.
LSTM_CNN model is not significantly different, the GRU It can be seen that using Word2vec to train the word vector
model has fewer parameters and is easier to calculate than as the embedding layer can effectively optimize the vector
the LSTM model. As a result, when everything is taken into representation of the input text, so as to obtain a better training
8 International Transactions on Electrical Energy Systems

99

98

97

accuracy (%) 96

95

94

93

92
0.2 0.3 0.4 0.5 0.6 0.7
dropout number

CNN LSTM+CNN
LSTM GRU+CNN
GRU
Figure 8: The accuracy curves of each model under different dropout values.

Table 5: Table of learning rate experimental results.

learning_rate CNN (%) LSTM (%) GRU (%) LSTM + CNN (%) GRU + CNN (%)
0.1 92.1 10 10 10.02 10
0.01 94.44 93.94 10 96.18 10
0.001 96.9 96.78 95.9 97.78 97.86
0.0001 90.58 92.06 91.88 95.46 95.08
0.00001 20.92 80.86 82 85.16 83.06

100
90

80
70
accuracy (%)

60
50
40
30

20
10
0.1 0.01 0.001 0.0001 0.00001

learning_rate numbers

CNN LSTM+CNN
LSTM GRU+CNN
GRU
Figure 9: The accuracy curves of each model under different learning rates.

effect. Compared with CNN, the accuracy of Word2vec_CNN Next, in order to verify the effectiveness of the
has increased by 1.23%. Compared with LSTM, the accuracy of Word2vec_GRU_CNN model proposed in this paper in the
Word2vec_LSTM has increased by 3.32%. Compared with text classification task, the Word2vec_CNN and Word2-
GRU, the accuracy of Word2vec_GRU has increased by 2.33%. vec_GRU models that performed relatively well in the above
International Transactions on Electrical Energy Systems 9

Table 6: Table of embedding dimension experimental results.

embedding_dim CNN (%) LSTM (%) GRU (%) LSTM + CNN (%) GRU + CNN (%)
50 95.56 94.68 93.89 95.86 95.98
100 96.9 96.78 95.9 97.78 97.86
150 94.30 94.54 91.68 95.36 95.34
200 94.88 93.78 92.68 94.46 95.10
250 95.80 95.42 92.76 95.74 95.54
300 94.78 93.68 92 96.30 92.06

99
98
98
98
accuracy (%)

97
96
95
94

93

92
50 100 150 200 250 300

embedding_dim numbers

CNN LSTM+CNN
LSTM GRU+CNN
GRU
Figure 10: The accuracy curves of each model under different word vector dimensions.

99
97.78 97.86
98
96.92 96.78
97
95.92
96
95.18
accuracy (%)

95

94

93
92
92

91

90
LR NB CNN LSTM GRU LSTM+CNN GRU+CNN

model
Figure 11: Classification accuracy of each model.

experiments were selected for comparison. Experiment with From the experimental results in Table 8, it can be seen
the same parameter settings and the experimental results are that, compared with Word2vec word embedding followed
shown in Table 8. by only one layer of training network, the GRU_CNN model
10 International Transactions on Electrical Energy Systems

Table 7: The first group of ablation experiment results. Conflicts of Interest

Model Accuracy (%)
The authors declared no potential conflicts of interest with
CNN 95.69 respect to the research, author-ship, and/or publication of
LSTM 92.98
this paper.
GRU 93.59
Word2vec_CNN 96.92
Word2vec_LSTM 96.3 Acknowledgments
Word2vec_GRU 95.92
This work was supported by the National Natural Science
Foundation of China, under Grants 61702462 and
61906175, the Henan Province Scientific and Techno-
Table 8: The second group of ablation experiment results. logical Research Project, under Grants 222102210010 and
222102210064, the Research and Practice Project of
Model Accuracy (%)
Higher Education Teaching Reform in Henan Province,
Word2vec_CNN 96.22
under Grants 2019SJGLX320 and 2019SJGLX020, the
Word2vec_GRU 95.88
Word2vec_GRU_CNN 97.86
Undergraduate Universities Smart Teaching Special Re-
search Project of Henan Province, under Gran Jiao Gao
proposed in this paper effectively improves the accuracy of [2021] no. 489-29, and the Academic Degrees & Graduate
text classification. Compared with the Word2vec_CNN and Education Reform Project of Henan Province, under
Word2vec_GRU models, the accuracy of Word2vec_- Grant 2021SJGLX115Y.
GRU_CNN is increased by 1.64% and 1.98%, respectively.
References
5. Conclusions [1] J. Zhang, Q. Wang, Y. Li, D. Li, and Y. Hao, “A method for
Chinese text classification based on three-dimensional vector
This paper uses the GRU_CNN hybrid model to classify space model,” in Proceedings of the 2012 International Con-
news texts. The findings of the experiment reveal that the ference on Computer Science and Service System, pp. 1324–
GRU_CNN hybrid model possesses a strong classification 1327, Nanjing, China, August 2012.
performance on the Cnews dataset, with an accuracy of [2] X. Cheng, C. Zhang, and Q. Li, “Improved Chinese short text
97.86%, which is better than the single CNN, LSTM, and classification method based on ERNIE_BiGRU model,”
GRU models and the hybrid model LSTM_CNN. The Journal of Physics: Conference Series, vol. 1993, no. 1, Article
GRU_CNN hybrid model first preprocesses the dataset and ID 012038, 2021.
then uses the Word2vec model of the Gensim toolkit to train [3] L. Zhou and X. Bian, “Improved text sentiment classification
method based on BiGRU-Attention,” Journal of Physics:
word vectors. The word’s vector representation is fed into
Conference Series, vol. 1345, no. 3, Article ID 032097, 2019.
the GRU model for semantic information extraction to gain [4] N. Kalchbrenner, E. Grefenstette, and P. Blunsom, “A con-
a sea of contextual information about the text, and then the volutional neural network for modelling sentences. Multi-
output results are input into the CNN model for semantic media systems,” in Proceedings of the 52nd Annual Meeting of
feature extraction. Finally, text classification via Softmax the Association for Computational Linguistics, pp. 655–665,
layer. Baltimore, MD, USA, June 2014.
Although compared with the traditional machine [5] T. Joachims, “Text categorization with support vector ma-
learning classification models logistic regression and Naive chines: learning with many relevant features,” in Proceedings
Bayes, single CNN, LSTM, and GRU models as well as of the Machine Learning: ECML-98, pp. 137–142, Chemnitz,
hybrid model LSTM_CNN, this paper proposes that the Germany, April 1998.
hybrid model GRU_CNN has a better classification effect on [6] T. Cover and P. Hart, “Nearest neighbor pattern classifica-
tion,” IEEE Transactions on Information Theory, vol. 13, no. 1,
the Cnews dataset. However, there is still a lot of optimi-
pp. 21–27, 1967.
zation space in this experiment, such as further optimizing [7] S. B. Kim, K. S. Han, H. C. Rim, and S. Hyon Myaeng, “Some
the neural network model, optimizing the word vector effective techniques for naive Bayes text classification,” IEEE
method, and trying to introduce an attention mechanism Transactions on Knowledge and Data Engineering, vol. 18,
into the model. no. 11, pp. 1457–1466, 2006.
[8] S. J. Yen, Y. S. Lee, J. C. Ying, and Y. C. Wu, “A logistic
Data Availability regression-based smoothing method for Chinese text cate-
gorization,” Expert Systems with Applications, vol. 38, no. 9,
The data that support the findings of this study can be pp. 11581–11590, 2011.
obtained from the corresponding author upon reasonable [9] B. A. Wahiba and B. Ahmed, “New fuzzy decision tree model
for text classification,” in Proceedings of the 1st International
request.
Conference on Advanced Intelligent System and Informatics,
pp. 309–320, Beni Suef, Egypt, November 2015.
Ethical Approval [10] S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad,
M. Chenaghlu, and J. Gao, “Deep learning based text clas-
There are no animal studies conducted by any of the authors sification: a comprehensive review,” 2021, https://arxiv.org/
in this study. abs/2004.03705.
International Transactions on Electrical Energy Systems 11

[11] Y. Kim, “Convolutional neural networks for sentence clas-

sification,” in Proceedings of the 2014 Conference on Empirical
Methods in Natural Language Processing, pp. 1746–1751,
Doha, Qatar, October 2014.
[12] Y. Zhang and B. Wallace, “A sensitivity analysis of (and
practitioners’ guide to) convolutional neural networks for
sentence classification,” 2015, https://arxiv.org/abs/1510.
03820.
[13] R. Johnson and Z. Tong, “Deep Pyramid convolutional neural
networks for text categorization,” in Proceedings of the 55th
Annual Meeting of the Association for Computational Lin-
guistics, pp. 562–570, Vancouver, Canada, July 2017.
[14] L. Yao, C. Mao, and Y. Luo, “Graph convolutional networks
for text classification,” Proceedings of the AAAI Conference on
Artificial Intelligence, vol. 33, pp. 7370–7377, 2019.
[15] T. Mikolov, I. Sutskever, K. Chen, C. Kai, G. Corrado, and
J. Dean, “Distributed representations of words and phrases
and their compositionality,” in Proceedings of the Advances in
Neural Information Processing systems, pp. 3111–3119, Red
Hook, NY, USA, December 2013.
[16] S. Hochreiter and J. Schmidhuber, “Long short-term mem-
ory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[17] J. Chung, C. Gulcehre, K. H. Cho, and Y. Bengio, “Empirical
evaluation of gated recurrent neural networks on sequence
modeling,” 2014, https://arxiv.org/abs/1412.3555.
[18] A. Graves and J. Schmidhuber, “Framewise phoneme clas-
sification with bidirectional LSTM networks,” in Proceedings
of the IEEE International Joint Conference on Neural Net-
works, pp. 2047–2052, Montreal, QC, Canada, July 2005.
[19] Y. Cao, T. R. Li, Z. Jia, and C. F. Yin, “BGRU: a new method of
Emotion analysis based on Chinese text,” Computer Science
and Exploration, vol. 13, no. 6, pp. 973–981, 2019.
[20] Y. Li and H. B. Dong, “Text sentiment analysis based on
feature fusion of convolution neural network and bidirec-
tional long short-term memory network,” Journal of Com-
puter Applications, vol. 38, no. 11, pp. 3075–3080, 2018.
[21] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient
estimation of word representations in vector space,” in
Proceedings of the International Conference on Learning
Representations, pp. 1–12, Scottsdale, ARI, USA, May 2013.
[22] S. Lai, L. Xu, K. Liu, and J. Zhao, “Recurrent convolutional
neural networks for text classification,” in Proceedings of the
14th Python in Science Conference, pp. 2267–2273, Austin, TX,
USA, January 2015.