Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 New to SQL Server Programming
 Another issue on calculating balance

Author  Topic 

dz0001
Starting Member

35 Posts

Posted - 2011-04-02 : 10:18:24
Hi There,

I am struggling to come with solution, here is my sample data Table A:

SalesID LastPayDate Amt Rate

2331739 2011-01-01 300 2

4131556 2010-12-01 800 5

I need to

1. calculate ending balance by EndingBalance= Amt * Rate

2. then ending balance will become Amt for next month, until it reachs to the most current month - 1, so result like this:

SalesID LastPayDate Amt Rate EndingBalance

2331739 2011-01-01 300 2 300x2
2331739 2011-02-01 600 2 600x2
2331739 2011-03-01 1200 2 1200x2
4131556 2010-12-01 800 5 800x5
4131556 2011-01-01 4000 5 4000x5
4131556 2011-02-01 20000 5 20000x5
4131556 2011-03-01 100000 5 100000x5

Thanks in advance

slimt_slimt
Aged Yak Warrior

746 Posts

Posted - 2011-04-02 : 11:17:40
hi,

i don't understand what do you need under 2. line; the ending balance of current month will become the Amt (amount) for next month and so on? is this correct?
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-02 : 13:34:11
Your rate is constant which means the ending balance is a geometric progression. So perhaps the most efficient way is to calculate the ending balance for any given month as Amount*power(Rate,datediff(month,initial_date,desired_date)). But what I am giving below is an alternate method which uses a recursive CTE.

-- THIS IS MY TEST DATA
create table #tmp (SalesId varchar(32), LastPayDate datetime, Amt int,
rate int, EndingBalance int);

insert into #tmp values ('2331739','2011-01-01','300','2',null);
insert into #tmp values ('4131556','2010-12-01','800','5',null);

-- THIS IS THE QUERY USING RECURSIVE CTE
;with CTE1 as
(
select -- anchor part of the cte calculates endingbalance for rows in the table
SalesId,
LastPayDate,
Amt,
Rate,
cast(Amt*Rate as int) as EndingBalance
from #tmp

union all

select -- recursive part calculates it for the subsequent month
c1.SalesId,
dateadd(month,1,c1.LastPayDate) as LastPayDate,
c1.EndingBalance as Amt,
c1.Rate as Rate,
c1.EndingBalance * c1.Rate as EndingBalance
from
CTE1 c1
where
dateadd(month,2,c1.LastPayDate) < getdate()
)
select * from CTE1 order by SalesId, LastPayDate;

-- AND THEN CLEANING UP
if (object_id('tempdb..#tmp') is not null) drop table #tmp;
Go to Top of Page

dz0001
Starting Member

35 Posts

Posted - 2011-04-02 : 17:59:40
Sunita,

Once again thank you so much for providing such an elegant solution. Again it is working beautifully!

Cross apply and CTE are very cool and I need to catch up on using them. My initial thought is to use cursor (even that is a drag for me).

I am going to try on production data table which currently has about 110,000 record, any concern I need to pay attention on using CTE?

Thanks again
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-02 : 21:44:39
Oooh! You said a bad word! You said "cursor"!!

Just kidding! it's a drag for me as well, but I am not religious about not using cursors. There are cases where cursor-based queries may be the only option. Even if not, I am sure if you look hard enough you could find examples where cursor-based queries would be more efficient.

As for this query, if you have only 110,000 rows in your source table, and assuming an average of a year's worth of payment dates for each of those, you would be inserting about 1.3 million records. I think performance wouldn't be a problem if you are doing it infrequently, the query is simple enough. But I haven't tested, I am only speculating.

You can of course, test the theory using a smaller test table. If performance is a problem and if you are doing it as a one-time update, you could do the operation in multiple chunks based on the Sales ID. You would probably need an index on that column if you are forced to do that.
Go to Top of Page

dz0001
Starting Member

35 Posts

Posted - 2011-04-05 : 17:26:33
Sunita,

I test your solution on production table with around 100 salesID, but I got the following error:

Msg 530, Level 16, State 1, Line 1
The statement terminated. The maximum recursion 100 has been exhausted before statement completion. Then I add query hint to allow 365, still return error. I think I may have infinite loop going on here.

Not sure how I can solve this, or wondering if you can post another solution. I look through the record, for example, one sales ID has 02/01/2009 due date which is very, very late, so we have to populate from 02/01/2009 to 03/01/2011, I guess CTE is hit the limit?

Also one more information is that actual ending balance calculation is not only relied on Rate, but also on another calculation, for example,

SalesID LastPayDate Amt Rate InterestAmt EndingBalance

InterestAmt initially equals Amt*Int, then EndingBalance = Amt*Rate+InterestAmt.

So when it becomes beginning balance for next month, then InterestAmt will change too.

Thanks again.
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-05 : 18:47:18
The recursion should go only as deep as the maximum number of months. So if your earliest data is from 2009, it should go no more than 30 to 40 deep. Do the following:

1. Check the data type of the LastPayDate column using

select data_type from INFORMATION_SCHEMA.columns where table_name = 'YourTableName' and
column_name = 'YourLastPayDateColumnName'
This should show datetime or smalldatetime.

2. See what the earliest date that you have is using:

select min(cast(LastPayDate as datetime)) from YourTable
By default maxrecursion is 100, so you will need to increase the maxrecursion to the number of months from the earliest date to today. Since you tried 365, I can't imagine that that is what it is.

3. Check if you have more than one row for a given SalesID using

select SalesID, count(*) from YourTable group by SalesID having count(*) > 1
If this returns any rows at all, that means there is more than one row with the same sales id. This should not cause the maxrecursion problem, but it is worth checking.

If none of these yield any clue, we have to think harder to see what might be causing the infinite recursion.
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-06 : 09:05:45
quote:
Originally posted by dz0001
Also one more information is that actual ending balance calculation is not only relied on Rate, but also on another calculation, for example,

SalesID LastPayDate Amt Rate InterestAmt EndingBalance

InterestAmt initially equals Amt*Int, then EndingBalance = Amt*Rate+InterestAmt.

So when it becomes beginning balance for next month, then InterestAmt will change too.


I had missed this part in my previous post, saw it only after I clicked "Post New Reply". I meant to come back and edit it, but could get to it only now.

I was sort of surprised by your initial formula - this makes more sense. Can you post sample data for one or two SalesID's EXACTLY as it exists in your production table? I want to see how many rows you have for a given Sales ID and its patttern.
Go to Top of Page

dz0001
Starting Member

35 Posts

Posted - 2011-04-06 : 16:42:03
Sunita,

Thanks for so thorough. I have been looking at this for the past hours. Here is two rows of sample data:

SalesID DueDate Amt Int IntAmt EndingBalance
0021111133 2010-12-01 00:00:00.000 212229.57 7.26 128398.88985 83830.6802
0021112032 2011-01-01 00:00:00.000 557921.87 2 92986.9783333333 464934.8917

Inital Amt and Int are constant, IntAmt is Amt*Int/12, then EndingBalance is (Amt - Amt*Int/12)

Essentially I am building amorization schedule.

Also thank you for list of three things to check: There is no duplicate on salesID, also DueDate type is datetime.

However I do find out min(duedate) has "2000-03-18 00:00:00.000", is this too much record to return for CTE? Most of 110,000 records are from 2006 forward. But I only generate test data set from 2010-12-01.
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-06 : 20:14:58
I changed a few things:

1. Increased MAXRECURSION to 150. If your earliest date is in 2000, that is less than 12 years, so certainly less than 150 months.

2. I assume the interest rate is in percentage, so I divided by 1200 rather than 12. If it is not, please change back to 12.

3. If it is an amortization schedule, there are easier ways to do it; any book on fixed income products should be able to tell us.

If it still doesn't work, I am not ready to give up; but I am sure that it is not the code, it is something in the data that is causing it.

create  table #tmp (SalesId varchar(32), DueDate datetime, Amt float, InterestRate float,
InterestAmount float, EndingBalance float);

insert into #tmp values ('0021111133','2010-12-01','212229.57','7.26',null,null);
insert into #tmp values ('0021112032','2011-01-01',' 557921.87','2',null,null);
insert into #tmp values ('0021112031','2000-01-01',' 557921.87','2',null,null);

-- THIS IS THE QUERY USING RECURSIVE CTE
;with CTE1 as
(
select -- anchor part of the cte calculates endingbalance for rows in the table
SalesId,
DueDate,
Amt,
InterestRate,
cast(Amt*InterestRate/1200.0 as float) as InterestAmount,
cast(Amt-Amt*InterestRate/1200.0 as float) as EndingBalance
from #tmp

union all

select -- recursive part calculates it for the subsequent month
c1.SalesId,
dateadd(month,1,c1.DueDate) as DueDate,
c1.EndingBalance as Amt,
c1.InterestRate,
cast(c1.EndingBalance*InterestRate/1200.0 as float) as InterestAmount,
cast(c1.EndingBalance - c1.EndingBalance*InterestRate/1200.0 as float) as EndingBalance
from
CTE1 c1
where
dateadd(month,2,c1.DueDate) < getdate()
)
select * from CTE1 order by SalesId, DueDate OPTION (MAXRECURSION 150);

-- AND THEN CLEANING UP
if (object_id('tempdb..#tmp') is not null) drop table #tmp;
Go to Top of Page

dz0001
Starting Member

35 Posts

Posted - 2011-04-06 : 21:21:13
Thanks again. I don't think it is the issue with the code. And I am dertermined to make this work. I will run it again and find out what the issue is.

Thanks for your guidance on this.
Go to Top of Page

dz0001
Starting Member

35 Posts

Posted - 2011-04-07 : 22:41:39
Hi Sunita,

I was running today with block of record of 20 from production table with due date from 2010-12-01, and it is working fine! I will try more data again. I think you are right, it is something to do with the data. Will update tomorrow.

Thanks again.
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-08 : 07:49:16
If you run this query, you should be able to find which SalesIds are causing high recursions.
-- THIS IS THE QUERY USING RECURSIVE CTE
;with CTE1 as
(
select -- anchor part of the cte calculates endingbalance for rows in the table
SalesId,
DueDate,
Amt,
InterestRate,
cast(Amt*InterestRate/1200.0 as float) as InterestAmount,
cast(Amt-Amt*InterestRate/1200.0 as float) as EndingBalance,
1 as RecursionLevel

from #tmp

union all

select -- recursive part calculates it for the subsequent month
c1.SalesId,
dateadd(month,1,c1.DueDate) as DueDate,
c1.EndingBalance as Amt,
c1.InterestRate,
cast(c1.EndingBalance*InterestRate/1200.0 as float) as InterestAmount,
cast(c1.EndingBalance - c1.EndingBalance*InterestRate/1200.0 as float) as EndingBalance,
c1.RecursionLevel+1

from
CTE1 c1
where
dateadd(month,2,c1.DueDate) < getdate()
and RecursionLevel < 149
)
select * from CTE1 where RecursionLevel > 148 order by SalesId, DueDate OPTION (MAXRECURSION 150);
Go to Top of Page

mmarovic
Aged Yak Warrior

518 Posts

Posted - 2011-04-08 : 15:33:42
That type of processing is better to do on the client side. Query should return exactly data from table A and the client should do the rest.

Mirko

My blog: http://mirko-marovic-eng.blogspot.com/
Go to Top of Page

dz0001
Starting Member

35 Posts

Posted - 2011-04-11 : 15:37:29
Sunita,

I just run it against all data with 110,000 records, it works perfectly!!! It takes 1.18 min to generate around 1.7 million records. I think the recursion is my fault which doesn't follow your instruction carefully.

I will incorporate with other calculation with full columns.

This is very cool, thanks again.
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-04-11 : 20:11:06
Woohooo!!! Awesome!

I will deny ever having said this, but I was almost ready to give up, even though in my last posting I said I was not going to give up!
Go to Top of Page
   

- Advertisement -