Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 SSIS and Import/Export (2008)
 Fuzzy Grouping empty fields

Author  Topic 

vmanrao
Starting Member

2 Posts

Posted - 2010-05-27 : 13:48:28
I'm currently running a fuzzy grouping on some basic data for my work. We are trying to de-dup some of our records based on a few fields (First Name, Last Name, Address1, Address2, Email).

Some records have values in the Address2 field but most of them are empty. Fuzzy Grouping seems to give a score of 1 to an Address2 field even though there is no value in the Address2 field. I have tried setting all the empty fields to Null but Fuzzy Grouping is still returning the score as 1 for the Address2 field.

This problem is effecting our overall scores and is causing a lot of issues.

Is there anyway for Fuzzy Grouping to disregard all null values whether they are in Address2 or Email?

Any suggestions would be greatly appreciated. Thanks guys!

blindman
Master Smack Fu Yak Hacker

2365 Posts

Posted - 2010-06-01 : 16:30:05
What are you using to do your fuzzy grouping?

________________________________________________
If it is not practically useful, then it is practically useless.
________________________________________________
Go to Top of Page

vmanrao
Starting Member

2 Posts

Posted - 2010-06-02 : 17:40:46
SQL Server 2008
Go to Top of Page

blindman
Master Smack Fu Yak Hacker

2365 Posts

Posted - 2010-06-03 : 11:13:49
What, in SSIS?

In my experience, nobody embeds that kind of logic into ETL tools without inevitably regretting it. I've got clients that are still tied to 2000 because of all the logic-heavy DTS packages they rely upon that cannot be migrated to SSIS.

Use SSIS to load your data into staging tables, and perform you fuzzy matching in the database.

________________________________________________
If it is not practically useful, then it is practically useless.
________________________________________________
Go to Top of Page
   

- Advertisement -