• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

Parallel Algorithm of Improved FunkSVD Based on Spark

Vol. 15, No. 5, May 31, 2021
10.3837/tiis.2021.05.004, Download Paper (Free):

Abstract

In view of the low accuracy of the traditional FunkSVD algorithm, and in order to improve the computational efficiency of the algorithm, this paper proposes a parallel algorithm of improved FunkSVD based on Spark (SP-FD). Using RMSProp algorithm to improve the traditional FunkSVD algorithm. The improved FunkSVD algorithm can not only solve the problem of decreased accuracy caused by iterative oscillations but also alleviate the impact of data sparseness on the accuracy of the algorithm, thereby achieving the effect of improving the accuracy of the algorithm. And using the Spark big data computing framework to realize the parallelization of the improved algorithm, to use RDD for iterative calculation, and to store calculation data in the iterative process in distributed memory to speed up the iteration. The Cartesian product operation in the improved FunkSVD algorithm is divided into blocks to realize parallel calculation, thereby improving the calculation speed of the algorithm. Experiments on three standard data sets in terms of accuracy, execution time, and speedup show that the SP-FD algorithm not only improves the recommendation accuracy, shortens the calculation interval compared to the traditional FunkSVD and several other algorithms but also shows good parallel performance in a cluster environment with multiple nodes. The analysis of experimental results shows that the SP-FD algorithm improves the accuracy and parallel computing capability of the algorithm, which is better than the traditional FunkSVD algorithm.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
X. Yue and Q. Liu, "Parallel Algorithm of Improved FunkSVD Based on Spark," KSII Transactions on Internet and Information Systems, vol. 15, no. 5, pp. 1649-1665, 2021. DOI: 10.3837/tiis.2021.05.004.

[ACM Style]
Xiaochen Yue and Qicheng Liu. 2021. Parallel Algorithm of Improved FunkSVD Based on Spark. KSII Transactions on Internet and Information Systems, 15, 5, (2021), 1649-1665. DOI: 10.3837/tiis.2021.05.004.

[BibTeX Style]
@article{tiis:24636, title="Parallel Algorithm of Improved FunkSVD Based on Spark", author="Xiaochen Yue and Qicheng Liu and ", journal="KSII Transactions on Internet and Information Systems", DOI={10.3837/tiis.2021.05.004}, volume={15}, number={5}, year="2021", month={May}, pages={1649-1665}}