Operational systems data warehousing, data mining and visualization solution
Project Overview
Background
The project’s objective was to develop a solution that integrates shared data from ERP and e-commerce systems. The ERP system used was Odoo, and the e-commerce platform was WooCommerce. The solution covers the following areas:
- Data Compilation (Data Warehousing)
- Data Analysis (Data Mining)
- Data Visualization (Business Intelligence Dashboards)
There was no data warehousing solution in place, and the ERP system (Odoo) had not yet been implemented. Consequently, the data from these systems were not centralized. The ERP and e-commerce platforms contain shared information on customers and products, as well as unique order data. The goal was to compile common basic information and order data from both systems into a unified data warehouse.
Objectives
- Creating a Common Data Warehouse:
- Combine essential data from Odoo and WooCommerce into a unified data warehouse.
- Ensure the ETL (Extract, Transform, Load) process collects all relevant and up-to-date information from both systems.
- Clean, integrate, and transform the data, addressing missing values and errors, to ensure a comprehensive and accurate data set.
- Data Mining and Analysis:
- Analyze the compiled data to understand customer purchasing behavior.
- Use clustering to group customers and decision trees to identify reasons leading to purchasing decisions.
- Discover new insights within the customer data, enhancing customer service and targeted marketing efforts.
- Data Visualization:
- Summarize the analyzed data into clear and comprehensible formats.
- Present data using straightforward charts and visual elements to make the information easily interpretable, even for those without technical expertise.
- Ensure real-time data availability, providing timely insights for business decisions.
This project aimed to enhance the client’s ability to understand and respond to customer needs, optimize marketing strategies, and support strategic decisions with accurate, up-to-date data insights. My role was to work as a Scrum Master and lead the project together with a Product Owner.
The tools we used on this project were SSIS (SQL Server Integration Services), Rapid Miner data mining tool and MS PowerBI. Microsoft Azure DevOps was used to plan and monitor the development work.