bigdata

In case you missed it, last week was held the first Data+AI Summit (formerly Spark+AI Summit) and we had a chance to participate. The talks will be published online but if you don’t want to wait, take a shortcut and learn our key insights! TL;DR Are you wondering why that title? It summarizes the major announcements from a data engineering perspective. Apache Spark will become more Pythonic thanks to the Project Zen initiative, and very probably, it will work on top of a lakehouse architecture.…

ABSTRACT This article focuses on the features extraction from time series and signals using Fourier and Wavelet transforms. This task will be carried out on an electrocardiogram (ECG) dataset in order to classify three groups of people: those with cardiac arrhythmia (ARR), congestive heart failure (CHF) and normal sinus rhythm (NSR). Our approach consists of using scaleogram (i.e. 2D representation of 1D extracted features) as an input to train a Neural Network (NN). We conducted the different tasks using python as a programming language. The…

While working in different contexts with NiFi, we have faced recurring challenges of development, maintenance and deployment optimization of NiFi flows. Whereas the basic approach suggests to manually duplicate pipelines for similar patterns, we believe that an automated approach is relevant for production purpose when it comes to implementing a significant amount of ingestion flows relying on a limited set of patterns or, more simply, when it comes to deploying these flows on different environments of execution. The ability to reach the right level of…