Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
SlinginParts
Starting Member
18 Posts |
 Posted - 2012-06-25 : 22:20:33
|
| So I'm working with this huge bunch of files which we have provided from a data licensing company. The descriptions for instance are varchar200 for a column. However, running a suggestion scan shows some in excess of this. Which is a problem.I would just have it scan the whole file and figure it out, but I don't want to be so obtuse to just max that value out.So now I'm left with importing 128 files for 5 types by hand. Is there any other more sane way to manage this? |
|
|
SlinginParts
Starting Member
18 Posts |
Posted - 2012-06-25 : 22:22:14
|
| I want to clarify when I say "the descriptions" are provided to us by the data vendor in their documentation. So using a string200 failed, then string270 got me an extra couple million rows. Past that I can't scan the file in its entirety to find out where the widest part is in the column. I can't even open the file in any sane editor. |
 |
|
|
SlinginParts
Starting Member
18 Posts |
Posted - 2012-06-25 : 23:59:48
|
| OK so I'm cheating kind of. I'm assuming there are some real scragglers in all this data so I've got a string size of 4000. Once its all in I'll run a query to determine the total maximum string size and reduce it after the fact.So far I'm at 37M rows, crossing my fingers! |
|
|
|
SlinginParts
Starting Member
18 Posts |
Posted - 2012-06-26 : 01:00:35
|
| whoohoo went in! |
|
|
|
DonAtWork
Master Smack Fu Yak Hacker
2167 Posts |
Posted - 2012-06-26 : 05:58:42
|
now that it is all in, you can see the max value, and let the vendor know that they gave you bad info.  How to ask: http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx |
|
|
|
SlinginParts
Starting Member
18 Posts |
Posted - 2012-06-26 : 08:20:10
|
| Could have been an issue with the download too, or my computer corrupting it. Honestly is it unreasonable for me to request this already done in an SQL Server file? I mean I understand some people want start from ground zero, but this is just way too much work for anyone that isn't a data warehouse master.I think however it may be an issue on there end too, because on 3 separate tables the same day are corrupt. |
|
|
|
|
|
|
5gb file, 77M rows and column types don't match =(