Given a set of previously developed machine learning models that predicted whether a customer would purchase a financial product, the client also wanted to assess the potential sales amount in terms of currency. This would enable the marketing team to not only target individuals with a high propensity to buy but also to focus on specific audiences based on the estimated monetary value of the financial products they would purchase. This approach would result in more tailored and effective marketing campaigns. The objective was to quickly deploy several regression models for such predictions, as a Proof of Concept (PoC).

The dataset consisted of millions of rows and over 300 predictor variables, making the feature selection process particularly challenging. The data was highly sparse, with unusual and heavily skewed statistical distributions, which is common in the financial sector. Many variables had a high proportion of zero values for numerous customers. This created a difficult environment for data cleaning, transformation, and treatment, as well as for fitting traditional regression models. Moreover, the volume and complexity of the data would result in high computational time requirements, further complicating the analysis process.