De-dup on Hadoop

178 views · Published 11 September 2014 · 21:49 · Indexed 26 October 2026

Channel: Hasgeek TV · 2014 · Science & Technology

Watch on YouTube

In many enterprises, it is commonly seen that business data has a lot of client, customer, vendor or product lists in different formats and systems, many of which are near duplicates. MDM solutions on RDBMS have been prominent for many years in almost every enterprise to support master data management by removing duplicates, standardizing data and incorporating rules to eliminate incorrect data from entering the system in order to create an authoritative source of master data. MDM on Big data platforms like Hadoop have benefits as well as it's own set of challenges when compared with the RDBMS counterparts. The talk will cover them in detail, primarily focusing on building this solution on Hadoop.

More from this channel