Day 04
Data Warehouse Modernization
Ensuring the success of your analytics strategy by compiling best practices
Watch the Recorded Session on Data Warehouse Modernization
Customer case
Effective Data Handling at Beneva
Breaking down the silos to unify the customer vision
With over three million customers and CA$13 billion in assets under management, Beneva is one of Canada’s largest financial institutions. The group offers auto, home, life, travel, group, health, and credit insurance, as well as investment products.

Beneva has been offering a comprehensive range of insurance products for over 75 years, during which time its information systems, databases, and customer portals have been increasing in number. In 2016, the company decided to break down these silos in order to unify its customer vision and personalize its relationship with its customers.
Beneva had two goals for its data project: to improve insight into its customers and to modernize its data foundations with an analytic-ready cloud infrastructure — all without compromising on data confidentiality standards.
The company quickly opted for Talend. "Talend offers a complete solution, from data integration to data enhancement to API-based applications," explains Robert Beauregard, BI Architect at Beneva. "Talend can be used to assemble these components to create reusable automation. These frameworks radically increase the speed of the BI development teams."
Although the insurance sector is rather conservative, we are compared to Amazon, Google, and Netflix, which offer highly personalized experiences. It was difficult to make relevant offers to customers without having a complete picture of their insurance coverage.
— Annie Pelletier,
Marketing and E-Business Director, Beneva
In order to ensure optimum results and performance, Beneva has developed pairing algorithms in different phases that are based on the components available in Talend. The company has also improved the process thanks to personalized code, which is a major strength of Talend.

Using this method, Beneva has put in place approximately 30 decision trees and around 60 matching rules, which are now fully automated using Talend, to establish their golden record. Talend has also helped resolve one of the most difficult issues in a master data project: data stewardship, when a human has to take back control from a machine. Once the data has been cleaned, it is loaded into the Snowflake Data Warehouse, on Microsoft Azure. With all their customer data consolidated in a single location, Beneva is able to send the right offer to the right customer, via the right channel and with the right message.
Historically, our data projects used to take between nine and 12 months. Now, with Talend, combined with the Data Vault 2.0 methodology, we enter production in agile mode every three weeks.
— Simon Latouche,
Director of Data Engineering, Beneva
Beneva now has a unified Customer Center portal, with a single login ID/password. The display of contracts is consolidated in a single location. Customers’ operations are automatically registered on the portal and call centers have access to more comprehensive data. Marketing can now customize its campaigns by using consolidated data to run predictive models. Beneva has tripled its conversion rates in terms of customer win-back actions.

“We now have the necessary foundations to achieve our marketing strategy based on the next best action,” says Annie Pelletier. “We can use Talend to build up a 360-degree view of the customer so that we are able to send the right offer to the right customer, via the right channel and with the right message.”

As Beneva’s data is highly sensitive, the company has to have a thorough command of the data to depersonalize or compile it depending on the nature of its targeted use. With the confidentiality of the data respected, it is the starting point for most of the analytical work. All of Beneva will be using the Master Golden Record, including the teams from actuarial services, inquiry and fraud teams, operational intelligence, customer experience, and call centers.
  • 30% improvement

    in efficiency of development teams

  • 3 weeks

    to deliver new data project, compared with 12 months previously

  • 3x increase

    in the conversion rate of email campaigns
Additional theory
Data Warehouse Modernization
Ensure the success of your analytics strategy by compiling best practices
The word "modernisation" may confuse some of you. In fact, "modernisation" more accurately captures the expanded universe of warehouse problems that we can address with Talend, which has been added to the Qlik Data Integration family.

Data warehouse modernisation describes a category of problems that typically arise when an organisation implements a data warehouse, whether in a traditional data center or the cloud. It is the culmination of best practices that feed, transform and enforce data quality across the enterprise to ensure the success of your data and analytics strategy.

Data Ingest

It may seem obvious, but you can only gain insight from the data warehouse if the data is there in the first place. That is why the first warehouse problem we solve is data ingestion. To ensure that your warehouse contains the right data, we offer the most flexible delivery options and the broadest connectivity. Other vendors offer limited delivery and data availability options in comparison. Qlik delivers:
Data Loading: Some situations require only the loading and periodically refreshing of data sets. That's where Stitch excels, especially when data needs to be sourced from cloud/SaaS applications.

E-L-T (Real-time Change Data Capture): Extract, load and transform (ELT) has become best practice for cloud data warehouses, where raw data is ingested in real-time and refined later. Our award-winning change data capture solutions can quickly deliver enterprise data from multiple sources including mainframes, SAP applications and relational databases.

E-T-L: The final data ingestion scenario is the traditional extract, transform and load (ETL) method. Contrary to what you may think, this approach is preferable in many business situations. For example, when large volumes of source data need to be parsed and formatted for multiple delivery targets.

Data Transformation, Data Mart Creation, and Lifecycle Automation

The second problem data warehouse users face is that they spend many hours manually writing SQL scripts to restructure the ingested data. This is especially true if they want to follow a dimensional modelling or data vault design methodology.

Qlik’s secret sauce is its intelligent data pipelines that automatically generate and maintain the push-down SQL required for data mart tables. Users can also use their own custom SQL transformations. The intelligent pipelines also have run-time optimisation features that users can customise to control SQL execution costs. Finally, cost-conscious organisations can choose to delegate transformation processing entirely to other engines. They can choose a native engine or Spark runtime in addition to SQL with a combined Qlik and Talend solution.

Data Quality and Governance

Data quality, which naturally follows on from ingest and transformation, is the final data warehouse problem we solve. Why is that? Well, ingest loads the data, which is transformed into structures such as data marts. Data quality ensures that the values are accurate and valid. But how does invalid data get into the data warehouse ingest and transformation pipeline in the first place?

The classic example of data quality is address validation. For example, the user mistypes their street address into a web application, which is ingested into the warehouse and transformed into a fact table. The error is only discovered when a downstream process audit is performed, or when processes that consume the address data fail. Failures can range from simple reporting errors that may have no cost, to non-delivery of physical mail that could cost thousands.
Implementing a data warehouse is genuinely transformational for many organizations, but the mere existence of the warehouse is not enough. To ensure the success of your data strategy, the techniques described above for modernising the Data Warehouse need to be applied.
Creating a simple pipeline in Talend Cloud
Sign up for a Talend Cloud free trial account, if don't have one. Step-by-step guide on accessing Talend Cloud free trial is published in the Day 1 of Data Integration Week.
Related Links
In this e-book you can find out how a cloud data warehouse in Azure has advantages in cost, time to value, and the ability to work with real-time data.
In this e-book you will find the top technical and business benefits of data warehouse modernization and how to choose the best solution for it.
Staying in touch with Qlik Data Integration and Analytics
subscribe to biweekly LinkedIn Newsletter "Data Matters" with news, events and cases from CEE market