Comparative study of the performance of the classification algorithms of the Apache Spark ML library

Título:

Comparative study of the performance of the classification algorithms of the Apache Spark ML library

Autor:

Camele, Genaro

Colaboradores:

; Hasperué, Waldo; Ronchetti, Franco; Quiroga, Facundo Manuel

Temas:

BIG DATAAPRENDIZAJE AUTOMÁTICOAPACHE

En:

Congreso Argentino de Ciencias de la Computación (27mo : 2021 : Salta, Argentina) 

Resumen:

Classification algorithms are widely used in several areas: finance, education, security, medicine, and more. Another use of these algorithms is to support feature extraction techniques. These techniques use classification algorithms to determine the best subset of attributes that support an acceptable prediction. Currently, a large amount of data is being collected and, as a result, databases are becoming increasingly larger and distributed processing becomes a necessity. In this sense, Spark, and in particular its Spark ML library, is one of the most widely used frameworks for performing classification tasks in large databases. Given that some feature extraction techniques need to execute a classification algorithm a significant number of times, with a different subset of attributes in each run, the performance of these algorithms should be known beforehand so that the overall feature extraction process is carried out in the shortest possible time. In this work, we carry out a comparative study of four Spark ML classification algorithms, measuring predictive power and execution times as a function of the number of attributes in the training dataset.

URL/DOI:

http://sedici.unlp.edu.ar/handle/10915/130348

Palabras clave:

Spark ML

Medio:

Impreso en papel

Tipo de documento:

Artículo

Descripción física:

1 archivo (243,9 kB)

Idioma:

Español

Publicación:

, 2021

Puede solicitar más fácilmente el ejemplar con: A1259

Ver estantes