Global Journal of Computer Science and Technology, C: Software & Data Engineering, Volume 22 Issue 2

Figure 7: Notebook with the Credit Card Propensity Model While we are working with the Scrum methodological framework, we have iterative deliveries, so each notebook which the Data Scientists release are productivized by the developers. The Productivization implies new developments very similar to ingests to the Data Lake, with the difference which have business logic which means the business logic is in each notebook of the different Data Scientists. Finally, finishing all the developments put in Live and meshed with the Control M programmer, the automatic process of Credit Card Campaigns is launched, the result from all the processes are flat files with the identifiers of the potential clients for acquiring a card credit, this list of customers has been sending to the different distribution channels from the financial institution (Web, ATM, Email, Social Networks). e) Pilot Validation In the validation from the pilot there are 2 stages: 1. Stage in Sandbox: The different users proceed to balance the results of their analytical models and procedures with the results obtained from the performing from the campaigns in a traditional way (35-day old process). An exact match is not expected since in many cases the models were optimized with more reliable data and expanded the universe of possible clients to obtain credit cards. Another characteristic from this stage is that validated notebooks are built, which will serve us in the second stage for matching the production processes. These notebooks contain automatic tables which are reading the results and matching them with the expected targets, as well as the business rules which are contemplated by the Data Scientist; Compared with the traditional software development projects these would be acceptance tests. 2. White March Stage: Started when the productivization of the processes performed by the developers was completed; here all Data Scientist notebooks have been implemented in jobs that will performed under Spark on a mesh orchestrated by Control M. In this way, the data resulting from the scheduled executions is going to monitoring with the validated notebooks from the previous stage. If there is some observation founded, the development is corrected and reprocessing is launched, allowing the data partition being validated to be overwritten. Integration of the Big Data Environment in a Financial Sector Entity to Optimize Products, Services and Decision-Making Global Journal of Computer Science and Technology Volume XXII Issue II Version I 46 Year 2022 ( ) C © 2022 Global Journals