Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 New to SQL Server Programming
 Is it possible?

Author  Topic 

satchmo
Starting Member

4 Posts

Posted - 2011-01-10 : 15:10:43
Hi

Am new to SQL and was curious if the following was at all possible to automate, somehow.

Currently, we get data from a vendor with a number of columns IF those columns have values in them. Problem is, I want to stack newer data on top of an older table, however the columns won't necessarily line up. The data is time series, where first column is date, with some metrics following.

Is this even possible in SQL? Can I use UNION to do this, or would the number of columns need to be identical and in the same order?

Thanks
Satchmo

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2011-01-10 : 15:26:03
Yes. We use SPARSE columns.
We convert the input file to XML with SSIS and then

1) Use the file column list to add newer column names
2) Insert the XML data into the columnset column.




N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

satchmo
Starting Member

4 Posts

Posted - 2011-01-10 : 15:44:39
Hi Peso

Thanks for the quick response. So if I have a table with columns A through J, and I need to update it with an XML file that has columns A through E, and H through J (with missing columns F and G)applying SPARS column attribute to F and G in the original table would automatically give me null values for those columns where I updated the table with the new XML data?

Sorry, still learning, so I hope my example makes sense.

Thanks
Satchmo
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2011-01-10 : 15:59:54
Here is a simple script to mimic your environment as I have understood it.
CREATE TABLE	#Sample
(
theDate DATE PRIMARY KEY CLUSTERED,
totalDATA XML COLUMN_SET FOR ALL_SPARSE_COLUMNS NULL,
ColumnName1 VARCHAR(40) SPARSE NULL
)

-- Read the first file
DECLARE @Data XML = '<ColumnName1>Peso</ColumnName1>'

-- Insert into target table
INSERT #Sample
(
theDate,
totalDATA
)
VALUES ('20110101', @Data)

-- Show the content of current table
SELECT thedate,
totalData,
ColumnName1
FROM #Sample

-- Now add new columns present in another file
ALTER TABLE #Sample
ADD ColumnName2 VARCHAR(40) SPARSE NULL,
ColumnName3 VARCHAR(40) SPARSE NULL

-- Add content from the new file with new columns
MERGE #Sample AS tgt
USING (
SELECT '20110101' AS theDate, '<ColumnName1>Peso</ColumnName1><ColumnName3>MVP</ColumnName3>' AS totalDATA UNION ALL
SELECT '20110201', '<ColumnName2>SQLTeam</ColumnName2>'
) AS src ON src.theDate = tgt.theDate
WHEN MATCHED
THEN UPDATE
SET tgt.totalData = src.totalData
WHEN NOT MATCHED BY TARGET
THEN INSERT (
theDate,
totalDATA
)
VALUES (
src.theDate,
src.totalDATA
);

-- Display the content of current table
SELECT theDate,
totalDATA,
ColumnName1,
ColumnName2,
ColumnName3
FROM #Sample

DROP TABLE #Sample
Please note, that when referencing the SPARSE columns, you cannot use "*".
You have to explicit name all columns you want to retrieve later.
SPARSE columns let you have about 32767 columns in a table.



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page

satchmo
Starting Member

4 Posts

Posted - 2011-01-11 : 14:42:52
Wow Peso! That's great. Too bad SQL Server won't work on my new work machine. I'll have to wait to test this out...

Satchmo
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2011-01-11 : 18:46:52
SPARSE columns is a feature of SQL Server 2008.



N 56°04'39.26"
E 12°55'05.63"
Go to Top of Page
   

- Advertisement -