Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 New to SQL Server Programming
 Cross Join/Outer Apply - Dates to another attribut

Author  Topic 

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-13 : 14:49:05
I have a Calendar table for reporting. Typically, I need to be able to see ridership on bus routes over 2 year periods to see percent increase/decrease. However, some months will not have the routes active but we still want those routes to show up for reporting even though they didn't run.

What I've been doing is using an outer apply between the calendar table and the route and route status tables, and scanning all routes that at least existed for the two year period as such:


CREATE TABLE [dbo].[Calendar](
[sdate] [date] NOT NULL,
[SYear] [int] NOT NULL,
[SMonth] [int] NOT NULL
) ON [PRIMARY]

declare @counter int
set @counter=0

while @counter<731
begin
insert into calendar (sdate, syear, smonth)
(
select DATEADD(dd, @counter, '2011-01-01'), YEAR(DATEADD(dd, @counter, '2011-01-01')), MONTH(DATEADD(dd, @counter, '2011-01-01'))
)
set @counter=@counter+1
end

declare @Year int, @Month int, @Start date, @End date
select @Year=2012, @Month=3,
@Start=convert(date, convert(char(8), ((@Year-1)*10000)+101)),
@End =convert(date, convert(char(8), ((@Year )*10000)+1231))

declare @Routes table (RouteId int primary key identity not null, RouteNum int not null)
declare @RouteStatus table (RouteStatusId int primary key identity not null, RouteId int not null, FromDate date not null, ToDate date null)

insert into @Routes (RouteNum)
values
(101),
(102),
(103),
(104)

insert into @RouteStatus (RouteId, FromDate, ToDate)
values
(1, '2010-01-01', '2011-02-10'),
(1, '2011-02-11', null),
(2, '2011-01-01', '2012-01-01'),
(3, '2012-03-01', null),
(4, '2012-03-01', '2012-04-01')

select C.SMonth, RR.RouteNum
from Calendar C
outer apply (select R.RouteNum
from @Routes R
join @RouteStatus RS on RS.RouteId=R.RouteId
where RS.FromDate<=@End and (RS.ToDate>=@Start or RS.ToDate is null)

) RR

where C.SYear>=@Year-1 and C.SYear<=@Year and C.SMonth<=@Month

group by C.SMonth, RR.RouteNum

My question is, this gives me the desired results (each route matched to each day for the whole period if the routenum at least existed from the @Start to the @End) but it seems like it's not very efficient from reading the execution plan. When I start joining ridership tables to this query, it begins to produce the Nested Loop No Join Predicate error message and runs very slow.

Any suggestions on how to tune this so that it's running more efficiently?

Thanks much!

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-05-13 : 15:00:36
for calendar table you can try using number table or cte over a loop and see if it performs better

see logic used here for example

http://visakhm.blogspot.com/2010/02/generating-calendar-table.html

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-13 : 15:05:06
Hi Vis,

I would think that having an actual calendar table rather than a table variable or CTE that you run at the time of the query would be more efficient. My actual calendar table has multiple date attributes because it is for an archive or data warehouse (not a fully fledged data warehouse, but it's not a transactional database)
The full table looks something like this:

create table ServiceCalendar
(
Sdate date primary key not null,
SMonthNum int not null,
SMonth char(3) not null,
SQuarter tinyint not null,
SYear smallint not null,
SDayofWeek tinyint not null
)

Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-05-13 : 15:24:07
quote:
Originally posted by flamblaster

Hi Vis,

I would think that having an actual calendar table rather than a table variable or CTE that you run at the time of the query would be more efficient. My actual calendar table has multiple date attributes because it is for an archive or data warehouse (not a fully fledged data warehouse, but it's not a transactional database)
The full table looks something like this:

create table ServiceCalendar
(
Sdate date primary key not null,
SMonthNum int not null,
SMonth char(3) not null,
SQuarter tinyint not null,
SYear smallint not null,
SDayofWeek tinyint not null
)




sorry you missed my point
i was suggesting alternate logic for populating it. You can still maintain it as permanent table

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-13 : 15:41:02
oh, ok...thanks. I just used the insert as an example. My main concern is once the date table is created, trying to match the routenumbers through the outer apply.
Go to Top of Page

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-13 : 16:20:46
To clarify, is there a better way about cross joining/applying the routenumbers to the dates?
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-05-13 : 16:43:12
quote:
Originally posted by flamblaster

To clarify, is there a better way about cross joining/applying the routenumbers to the dates?


you want the routenum to be repeated for all dates that come between range right?

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-13 : 17:03:24
Yes, exactly. Like I was saying, this query produces the expected results. It just seems clunky. There isn't really going to be a join predicate because there's no correlation between the dates and routes other than the date range. When I join a ridership table to this (I was trying to set this up as a stored procedure so I can pull it into Crystal Reports), matching the ridership data to the query above becomes very slow and most of the execution takes place on "left side" of the execution plan rather than the right.
Go to Top of Page

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-14 : 16:47:12
quote:
Originally posted by flamblaster

I have a Calendar table for reporting. Typically, I need to be able to see ridership on bus routes over 2 year periods to see percent increase/decrease. However, some months will not have the routes active but we still want those routes to show up for reporting even though they didn't run.

What I've been doing is using an outer apply between the calendar table and the route and route status tables, and scanning all routes that at least existed for the two year period as such:


CREATE TABLE [dbo].[Calendar](
[sdate] [date] NOT NULL,
[SYear] [int] NOT NULL,
[SMonth] [int] NOT NULL
) ON [PRIMARY]

declare @counter int
set @counter=0

while @counter<731
begin
insert into calendar (sdate, syear, smonth)
(
select DATEADD(dd, @counter, '2011-01-01'), YEAR(DATEADD(dd, @counter, '2011-01-01')), MONTH(DATEADD(dd, @counter, '2011-01-01'))
)
set @counter=@counter+1
end

declare @Year int, @Month int, @Start date, @End date
select @Year=2012, @Month=3,
@Start=convert(date, convert(char(8), ((@Year-1)*10000)+101)),
@End =convert(date, convert(char(8), ((@Year )*10000)+1231))

declare @Routes table (RouteId int primary key identity not null, RouteNum int not null)
declare @RouteStatus table (RouteStatusId int primary key identity not null, RouteId int not null, FromDate date not null, ToDate date null)

insert into @Routes (RouteNum)
values
(101),
(102),
(103),
(104)

insert into @RouteStatus (RouteId, FromDate, ToDate)
values
(1, '2010-01-01', '2011-02-10'),
(1, '2011-02-11', null),
(2, '2011-01-01', '2012-01-01'),
(3, '2012-03-01', null),
(4, '2012-03-01', '2012-04-01')

select C.SMonth, RR.RouteNum
from Calendar C
outer apply (select R.RouteNum
from @Routes R
join @RouteStatus RS on RS.RouteId=R.RouteId
where RS.FromDate<=@End and (RS.ToDate>=@Start or RS.ToDate is null)

) RR

where C.SYear>=@Year-1 and C.SYear<=@Year and C.SMonth<=@Month

group by C.SMonth, RR.RouteNum

My question is, this gives me the desired results (each route matched to each day for the whole period if the routenum at least existed from the @Start to the @End) but it seems like it's not very efficient from reading the execution plan. When I start joining ridership tables to this query, it begins to produce the Nested Loop No Join Predicate error message and runs very slow.

Any suggestions on how to tune this so that it's running more efficiently?

Thanks much!




Anyone else have any suggestions on a better way to go about getting the RouteNum's repeated for each day of the calendar that would be more effiicient than the logic in red above?

Thanks!
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2012-05-14 : 22:15:45
what are indexes you've on the tables?

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

flamblaster
Constraint Violating Yak Guru

384 Posts

Posted - 2012-05-15 : 00:00:38
Hey Vis, thanks for replying...Not sure how much you want to see, so here are all the create table/indexes statements:


SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[ServiceCalendar](
[Sdate] [date] NOT NULL,
[Sldate] [int] NOT NULL,
[SMonth] [varchar](9) NOT NULL,
[SMonthAbbr] [char](3) NOT NULL,
[SMonthNum] [smallint] NOT NULL,
[SYear] [smallint] NOT NULL,
CONSTRAINT [PK_ServiceCalendar] PRIMARY KEY CLUSTERED
(
[Sdate] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [IX_ServiceCalendar_DateMonthYear] ON [dbo].[ServiceCalendar]
(
[Sdate] ASC,
[SMonthNum] ASC,
[SYear] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [IX_ServiceCalendar_Month] ON [dbo].[ServiceCalendar]
(
[SMonth] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [IX_ServiceCalendar_Year] ON [dbo].[ServiceCalendar]
(
[SYear] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Rte](
[RouteId] [int] IDENTITY(1,1) NOT NULL,
[RouteNum] [int] NOT NULL,
[RouteName] [varchar](10) NOT NULL,
[RouteGroupName] [varchar](10) NOT NULL,
[RouteRankGroupName] [nchar](20) NOT NULL,
CONSTRAINT [PK_Rte] PRIMARY KEY CLUSTERED
(
[RouteId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE UNIQUE NONCLUSTERED INDEX [UIX_Rte] ON [dbo].[Rte]
(
[RouteNum] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[RouteStatus](
[RouteStatusId] [int] IDENTITY(1,1) NOT NULL,
[RouteId] [int] NOT NULL,
[FromDate] [date] NOT NULL,
[ToDate] [date] NULL,
[IsActive] [tinyint] NOT NULL,
CONSTRAINT [PK_RouteStatus] PRIMARY KEY CLUSTERED
(
[RouteStatusId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE UNIQUE NONCLUSTERED INDEX [UIX_RouteStatus] ON [dbo].[RouteStatus]
(
[RouteId] ASC,
[FromDate] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
ALTER TABLE [dbo].[RouteStatus] WITH CHECK ADD CONSTRAINT [FK_RouteStatus_Rte] FOREIGN KEY([RouteId])
REFERENCES [dbo].[Rte] ([RouteId])
GO
ALTER TABLE [dbo].[RouteStatus] CHECK CONSTRAINT [FK_RouteStatus_Rte]
GO
Go to Top of Page
   

- Advertisement -