Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-02 : 10:18:24
|
| Hi There,I am struggling to come with solution, here is my sample data Table A:SalesID LastPayDate Amt Rate2331739 2011-01-01 300 2 4131556 2010-12-01 800 5I need to 1. calculate ending balance by EndingBalance= Amt * Rate2. then ending balance will become Amt for next month, until it reachs to the most current month - 1, so result like this:SalesID LastPayDate Amt Rate EndingBalance2331739 2011-01-01 300 2 300x2 2331739 2011-02-01 600 2 600x2 2331739 2011-03-01 1200 2 1200x2 4131556 2010-12-01 800 5 800x5 4131556 2011-01-01 4000 5 4000x5 4131556 2011-02-01 20000 5 20000x5 4131556 2011-03-01 100000 5 100000x5Thanks in advance |
|
|
slimt_slimt
Aged Yak Warrior
746 Posts |
Posted - 2011-04-02 : 11:17:40
|
| hi,i don't understand what do you need under 2. line; the ending balance of current month will become the Amt (amount) for next month and so on? is this correct? |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-02 : 13:34:11
|
Your rate is constant which means the ending balance is a geometric progression. So perhaps the most efficient way is to calculate the ending balance for any given month as Amount*power(Rate,datediff(month,initial_date,desired_date)). But what I am giving below is an alternate method which uses a recursive CTE.-- THIS IS MY TEST DATAcreate table #tmp (SalesId varchar(32), LastPayDate datetime, Amt int, rate int, EndingBalance int);insert into #tmp values ('2331739','2011-01-01','300','2',null);insert into #tmp values ('4131556','2010-12-01','800','5',null);-- THIS IS THE QUERY USING RECURSIVE CTE;with CTE1 as( select -- anchor part of the cte calculates endingbalance for rows in the table SalesId, LastPayDate, Amt, Rate, cast(Amt*Rate as int) as EndingBalance from #tmp union all select -- recursive part calculates it for the subsequent month c1.SalesId, dateadd(month,1,c1.LastPayDate) as LastPayDate, c1.EndingBalance as Amt, c1.Rate as Rate, c1.EndingBalance * c1.Rate as EndingBalance from CTE1 c1 where dateadd(month,2,c1.LastPayDate) < getdate())select * from CTE1 order by SalesId, LastPayDate;-- AND THEN CLEANING UPif (object_id('tempdb..#tmp') is not null) drop table #tmp; |
 |
|
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-02 : 17:59:40
|
| Sunita,Once again thank you so much for providing such an elegant solution. Again it is working beautifully!Cross apply and CTE are very cool and I need to catch up on using them. My initial thought is to use cursor (even that is a drag for me).I am going to try on production data table which currently has about 110,000 record, any concern I need to pay attention on using CTE?Thanks again |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-02 : 21:44:39
|
| Oooh! You said a bad word! You said "cursor"!!Just kidding! it's a drag for me as well, but I am not religious about not using cursors. There are cases where cursor-based queries may be the only option. Even if not, I am sure if you look hard enough you could find examples where cursor-based queries would be more efficient.As for this query, if you have only 110,000 rows in your source table, and assuming an average of a year's worth of payment dates for each of those, you would be inserting about 1.3 million records. I think performance wouldn't be a problem if you are doing it infrequently, the query is simple enough. But I haven't tested, I am only speculating.You can of course, test the theory using a smaller test table. If performance is a problem and if you are doing it as a one-time update, you could do the operation in multiple chunks based on the Sales ID. You would probably need an index on that column if you are forced to do that. |
 |
|
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-05 : 17:26:33
|
| Sunita,I test your solution on production table with around 100 salesID, but I got the following error:Msg 530, Level 16, State 1, Line 1The statement terminated. The maximum recursion 100 has been exhausted before statement completion. Then I add query hint to allow 365, still return error. I think I may have infinite loop going on here.Not sure how I can solve this, or wondering if you can post another solution. I look through the record, for example, one sales ID has 02/01/2009 due date which is very, very late, so we have to populate from 02/01/2009 to 03/01/2011, I guess CTE is hit the limit?Also one more information is that actual ending balance calculation is not only relied on Rate, but also on another calculation, for example,SalesID LastPayDate Amt Rate InterestAmt EndingBalanceInterestAmt initially equals Amt*Int, then EndingBalance = Amt*Rate+InterestAmt.So when it becomes beginning balance for next month, then InterestAmt will change too.Thanks again. |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-05 : 18:47:18
|
The recursion should go only as deep as the maximum number of months. So if your earliest data is from 2009, it should go no more than 30 to 40 deep. Do the following:1. Check the data type of the LastPayDate column usingselect data_type from INFORMATION_SCHEMA.columns where table_name = 'YourTableName' andcolumn_name = 'YourLastPayDateColumnName' This should show datetime or smalldatetime.2. See what the earliest date that you have is using:select min(cast(LastPayDate as datetime)) from YourTable By default maxrecursion is 100, so you will need to increase the maxrecursion to the number of months from the earliest date to today. Since you tried 365, I can't imagine that that is what it is.3. Check if you have more than one row for a given SalesID usingselect SalesID, count(*) from YourTable group by SalesID having count(*) > 1 If this returns any rows at all, that means there is more than one row with the same sales id. This should not cause the maxrecursion problem, but it is worth checking.If none of these yield any clue, we have to think harder to see what might be causing the infinite recursion. |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-06 : 09:05:45
|
quote: Originally posted by dz0001Also one more information is that actual ending balance calculation is not only relied on Rate, but also on another calculation, for example,SalesID LastPayDate Amt Rate InterestAmt EndingBalanceInterestAmt initially equals Amt*Int, then EndingBalance = Amt*Rate+InterestAmt.So when it becomes beginning balance for next month, then InterestAmt will change too.
I had missed this part in my previous post, saw it only after I clicked "Post New Reply". I meant to come back and edit it, but could get to it only now.I was sort of surprised by your initial formula - this makes more sense. Can you post sample data for one or two SalesID's EXACTLY as it exists in your production table? I want to see how many rows you have for a given Sales ID and its patttern. |
 |
|
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-06 : 16:42:03
|
| Sunita,Thanks for so thorough. I have been looking at this for the past hours. Here is two rows of sample data:SalesID DueDate Amt Int IntAmt EndingBalance0021111133 2010-12-01 00:00:00.000 212229.57 7.26 128398.88985 83830.68020021112032 2011-01-01 00:00:00.000 557921.87 2 92986.9783333333 464934.8917Inital Amt and Int are constant, IntAmt is Amt*Int/12, then EndingBalance is (Amt - Amt*Int/12)Essentially I am building amorization schedule.Also thank you for list of three things to check: There is no duplicate on salesID, also DueDate type is datetime. However I do find out min(duedate) has "2000-03-18 00:00:00.000", is this too much record to return for CTE? Most of 110,000 records are from 2006 forward. But I only generate test data set from 2010-12-01. |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-06 : 20:14:58
|
I changed a few things:1. Increased MAXRECURSION to 150. If your earliest date is in 2000, that is less than 12 years, so certainly less than 150 months.2. I assume the interest rate is in percentage, so I divided by 1200 rather than 12. If it is not, please change back to 12.3. If it is an amortization schedule, there are easier ways to do it; any book on fixed income products should be able to tell us.If it still doesn't work, I am not ready to give up; but I am sure that it is not the code, it is something in the data that is causing it.create table #tmp (SalesId varchar(32), DueDate datetime, Amt float, InterestRate float,InterestAmount float, EndingBalance float);insert into #tmp values ('0021111133','2010-12-01','212229.57','7.26',null,null);insert into #tmp values ('0021112032','2011-01-01',' 557921.87','2',null,null);insert into #tmp values ('0021112031','2000-01-01',' 557921.87','2',null,null);-- THIS IS THE QUERY USING RECURSIVE CTE;with CTE1 as( select -- anchor part of the cte calculates endingbalance for rows in the table SalesId, DueDate, Amt, InterestRate, cast(Amt*InterestRate/1200.0 as float) as InterestAmount, cast(Amt-Amt*InterestRate/1200.0 as float) as EndingBalance from #tmp union all select -- recursive part calculates it for the subsequent month c1.SalesId, dateadd(month,1,c1.DueDate) as DueDate, c1.EndingBalance as Amt, c1.InterestRate, cast(c1.EndingBalance*InterestRate/1200.0 as float) as InterestAmount, cast(c1.EndingBalance - c1.EndingBalance*InterestRate/1200.0 as float) as EndingBalance from CTE1 c1 where dateadd(month,2,c1.DueDate) < getdate())select * from CTE1 order by SalesId, DueDate OPTION (MAXRECURSION 150);-- AND THEN CLEANING UPif (object_id('tempdb..#tmp') is not null) drop table #tmp; |
 |
|
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-06 : 21:21:13
|
| Thanks again. I don't think it is the issue with the code. And I am dertermined to make this work. I will run it again and find out what the issue is.Thanks for your guidance on this. |
 |
|
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-07 : 22:41:39
|
| Hi Sunita,I was running today with block of record of 20 from production table with due date from 2010-12-01, and it is working fine! I will try more data again. I think you are right, it is something to do with the data. Will update tomorrow.Thanks again. |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-08 : 07:49:16
|
If you run this query, you should be able to find which SalesIds are causing high recursions.-- THIS IS THE QUERY USING RECURSIVE CTE;with CTE1 as( select -- anchor part of the cte calculates endingbalance for rows in the table SalesId, DueDate, Amt, InterestRate, cast(Amt*InterestRate/1200.0 as float) as InterestAmount, cast(Amt-Amt*InterestRate/1200.0 as float) as EndingBalance, 1 as RecursionLevel from #tmp union all select -- recursive part calculates it for the subsequent month c1.SalesId, dateadd(month,1,c1.DueDate) as DueDate, c1.EndingBalance as Amt, c1.InterestRate, cast(c1.EndingBalance*InterestRate/1200.0 as float) as InterestAmount, cast(c1.EndingBalance - c1.EndingBalance*InterestRate/1200.0 as float) as EndingBalance, c1.RecursionLevel+1 from CTE1 c1 where dateadd(month,2,c1.DueDate) < getdate() and RecursionLevel < 149)select * from CTE1 where RecursionLevel > 148 order by SalesId, DueDate OPTION (MAXRECURSION 150); |
 |
|
|
mmarovic
Aged Yak Warrior
518 Posts |
Posted - 2011-04-08 : 15:33:42
|
| That type of processing is better to do on the client side. Query should return exactly data from table A and the client should do the rest.MirkoMy blog: http://mirko-marovic-eng.blogspot.com/ |
 |
|
|
dz0001
Starting Member
35 Posts |
Posted - 2011-04-11 : 15:37:29
|
| Sunita,I just run it against all data with 110,000 records, it works perfectly!!! It takes 1.18 min to generate around 1.7 million records. I think the recursion is my fault which doesn't follow your instruction carefully.I will incorporate with other calculation with full columns.This is very cool, thanks again. |
 |
|
|
sunitabeck
Master Smack Fu Yak Hacker
5155 Posts |
Posted - 2011-04-11 : 20:11:06
|
| Woohooo!!! Awesome!I will deny ever having said this, but I was almost ready to give up, even though in my last posting I said I was not going to give up! |
 |
|
|
|
|
|
|
|