Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-16 : 17:44:49
|
| I'm trying to parse header/detail records into two tables, and I can't seem to figure outA) How to pass a value from the select statement to the variableB) How to insert my variable into the appropriate table.My data looks like this:(table) DATAFILE_TEST(field) GENERIC_RECORDH002395000001011D00012010112010D00022010102010D00032010102010...So I have:BEGINDECLARE TEMP_PREMISE NVARCHAR(10);SET TEMP_PREMISE = "TEMPORARY";SELECTTEMP_PREMISE AS PREM_VALSUBSTRING(GENERIC_RECORD,1,1) AS HEADER_TYPE,SUBSTRING(GENERIC_RECORD,2,4) AS DETAIL REC COUNT,SUBSTRING(GENERIC_RCORD,6,10) AS PREMISE_NUMFROM DATFILE_TESTWHERE GENERIC_RECORD LIKE 'H%'ENDThis will pull all of the header records, and assign the correct values to the fields, and so I know I can create the insert into Header_Table.The problem is that I need to read the Header record, store the actual PREMISE_NUM in TEMP_PREMISE, and then read the Detail records and insert TEMP_PREMISE into Detail_Table along with the parsed fields found there.Any help would be appreciated. |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-03-16 : 22:11:54
|
| I am afraid I did not follow your requirements completely. But, from what i understood, you want to:a) read the data from the DATAFILE_TEST table,b) parse the rows that start with "H" and insert them into a header table.b) parse the rows that start with "D" and insert them into a detail table.You have given sample input data; that's good. Now can you also post the output data - the data that you would like to see in the two destination tables - given this input? That will make it very easy for someone to understand what you are trying to achieve. |
 |
|
|
khtan
In (Som, Ni, Yak)
17689 Posts |
Posted - 2011-03-16 : 22:21:18
|
does your DATAFILE_TEST table only contain one header record ? the line "H002395000001011" If you have more than 1 header record, how do you associate which detail line is corresponding to which header line ? KH[spoiler]Time is always against us[/spoiler] |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-16 : 22:28:16
|
| Can do:So I have several million records of concetenated data. For ease of understanding, I'll simplify the data format. The Header records lead with a "H" character, then a 4-character string for the CustID and a text field for the CustName. The Detail records lead with a "D" character, then a 4-character TransactionID and a 3-character Transaction Description:H0001BOBJOHNSONDXXXABananasDXXXBApplesDXXXCPearsH0002SUEJONESH0003STEVESMITHDXXXAMelonsDXXXBOrangesIf everything ends up correctly, I should have two tables:A "Header Table" with CustID___CustName0001_____BOBJOHNSON0002_____SUEJONES0003_____STEVESMITHAnd "Detail Table" withTrxnID_____TrxnName_____CustID00A________Bananas______000100B________Apples_______000100C________Pears________000100A________Melons_______000300B________Oranges______0003(hope that makes sense?)So I assume I'll have to create a statement that 'loops' and stores the CustID so that on the "H" records, it writes the appropriate fields to "Header" and then stores the CustID while it reads the "D" records and writes those fields to the "Detail" table in addition to writing the stored CustID value from the preceding record. |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-16 : 22:29:53
|
| @KHTAN - The records in the import are sequential, so I've loaded them into a table, and assigned them an autonumber id, so I know the sequence. There are approximately 1-million Headers with anywhere from 0-24 details/header. |
 |
|
|
khtan
In (Som, Ni, Yak)
17689 Posts |
Posted - 2011-03-16 : 23:41:34
|
what is the version of SQL server you are using ? KH[spoiler]Time is always against us[/spoiler] |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-16 : 23:44:23
|
| I'm actually stuck using an old version of Sybase with an Interactive SQL portal. I'm pretty comfortable with Microsoft SQL, so I'm fairly certain I can convert anything I might be able to get from these forums into the appropriate syntax and apply it. |
 |
|
|
khtan
In (Som, Ni, Yak)
17689 Posts |
Posted - 2011-03-16 : 23:55:13
|
the old version of Sybase is equivalent to which version of SQL Server ?Does it support CROSS APPLY, row_number() etc ? KH[spoiler]Time is always against us[/spoiler] |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-18 : 16:53:08
|
| Not sure on either count.Maybe if I ask differently:I have a list of Master/Detail records.Each is a single field, made up of concatenated info.For each record, I need to do several things:a) If the first character of the record is "H", parse the record into the appropriate fieldsb) Insert the resulting parsed/records into TABLE_Ac) Save the ACCOUNT_ID from the Header recordd) If the first character of the record is "D", parse the record the appropriate fieldse) Insert the resulting parsed/records into TABLE_B along with the saved ACCOUNT_ID from the Header record.I'm just not sure how to write the logic to store the ACCOUNT_ID until the next Header record, and how to best write the IF or CASE or whatever statement let's the code know which table to write the record to. |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-18 : 18:05:13
|
| Here's my statement, so far. If I don't try to add the @TEMP_PREMISE part, it creates the table I want, except for the PREMISE_ID being populated on the "D" records. Anything?As a reminder, it's Master/Detail records, set up like:H##############D##########And any 'H'eader record can have 0-24 'D'etail records.I need to pass the PREMISE_ID to the Detail records.BEGINDECLARE @TEMP_PREMISE varchar(10);SELECTSET @TEMP_PREMISE = (SELECT SUBSTRING(generic_record,6,10));SUBSTRING(generic_record,1,1)AS DETAIL_TYPE,CASE WHEN SUBSTRING(generic_record,1,1) ='H' THEN SUBSTRING(generic_record,2,4) END AS DETAIL_REC_COUNT,CASE WHEN SUBSTRING(generic_record,1,1) ='H' THEN SUBSTRING(generic_record,6,10) END AS PREMISE_NUM,CASE WHEN SUBSTRING(generic_record,1,1) ='D' THEN SUBSTRING(generic_record,2,4) END AS DETAIL_NUM,CASE WHEN SUBSTRING(generic_record,1,1) ='D' THEN SUBSTRING(generic_record,6,6) END AS REVENUE_MONTH,FROM "DBA"."DATAFILE_TEST"END |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-03-19 : 11:53:51
|
Since I don't know the capabilities of your version of SQL, I am going to try to do this using basic functionality that is hopefully in any version of SQL. I tested the code posted below and I have added comments to indicate what I am trying to do. Hope it helps you; if not, as they say in legal documents, "there are no implied warranties of merchantability or fitness for any purpose" offered with this code.(Yes, in case you are wondering, I am seriously considering ditching technology for a career in law) -- raw data table for testing. I added a column called autoId which you indicated you have-- in your table.create table #rawdata (data varchar(8000), autoId int not null identity(1,1));insert into #rawdata values ('H0001BOBJOHNSON');insert into #rawdata values ('DXXXABananas');insert into #rawdata values ('DXXXBApples');insert into #rawdata values ('DXXXCPears');insert into #rawdata values ('H0002SUEJONES');insert into #rawdata values ('H0003STEVESMITH');insert into #rawdata values ('DXXXAMelons');insert into #rawdata values ('DXXXBOranges');-- see what is in the raw data table.select * from #rawdata;-- header data table. The headerId is the same id in the raw data table. -- detailedRowcount is the number of detail rows for this customer.-- headerAutoId is an autoId for this table.create table #header (custId varchar(32), custName varchar(255), headerId int, detaliedRowCount int, headerAutoId int not null identity(1,1)); -- Parse and insert the header rows.insert into #headerselect substring(data,2,4), stuff(data,1,5,''), autoId, nullfrom #rawdatawhere data like 'H%';-- autoId of the last record in the raw data table. I need this to -- figure out how many details rows are there for the last header row.declare @rawRecordCount int;select @rawRecordCount = max(autoId) from #rawdata;-- update with number of detail rows that belong to each header row.update a set detaliedRowCount = coalesce(b.headerId,@rawRecordCount) - a.headerIdfrom #header a left join #header b on a.headerAutoId + 1 = b.headerAutoId; select * from #header;-- the detail table.create table #detail (trxnId varchar(32), trxnName varchar(255), custId varchar(32));-- parse and insert the detail rows into detail tableinsert into #detailselect substring(a.data,2,4), stuff(a.data,1,5,''), b.custIdfrom #rawdata a inner join #header b on a.autoId > b.headerId and a.autoId <= b.headerId + b.detaliedRowCountwhere a.data like 'D%' select * from #detail;-- and that should be it.drop table #rawdata;drop table #detail;drop table #header; |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-19 : 13:28:26
|
| Thank you thank you thank you. I'm in the midst of fine-tuning your code to work with my crappy DBMS, but so far so good. I like your ideas here. Updates to come. |
 |
|
|
HomerJ
Starting Member
21 Posts |
Posted - 2011-03-20 : 10:44:41
|
| This is working perfectly, with one exception. My data originates from a text file, and when I import it, the Autonumber/ID field doesn't update. So when the import is complete, all of the autonumbers are "0" and then the table order gets messed up.Is there a way to use and Import statement AND trigger the autonumber field? |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-03-20 : 14:56:29
|
Since I am completely unfamiliar with your version of SQL, I don't know how to make it work. If you were using SQL 2005, 2008 etc., you could use SSIS, bcp etc. to import the data while simultaneously populating an auto Id column.The only thing that comes to mind is ito add a line number to the end of each line in your file BEFORE importing it into the database. You can do this easily, for example, using a perl script. If you have a perl installation, from a command line window, try this:perl -pi.bak -e "s/\n/,$.\n/g" YourFile.txt That will add a comma and the current line number to the end of each line.Perl installations are available freely. My favorite is Active Perl here: http://www.activestate.com/activeperlGood luck.Sunita.PS: I realize SQL purists would not like these "hacks" that I am proposing, so my apologies to you if you count yourself among them. And, I am not a fan of Perl myself, but I was forced to hang around a few quants for a while, and they corrupted me - some people like the ones who star in here http://www.amazon.com/Quants-Whizzes-Conquered-Street-Destroyed/dp/0307453375 |
 |
|
|
|
|
|
|
|