Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 New to SQL Server Programming
 Slow Join Query: Indexing Technique I'm Missing?

Author  Topic 

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 11:39:14
So there is a View that I need that involves joining 4 tables. The execution time is (CPU Time = 109ms , elapsed time = 148 ms). I looked at the execution plan and the biggest costs involves Hash Match and even a Index Spool.

So I'm no expert at SQL (but I'm learning) and have only started reading about indexes a few days ago. For the tables involved in this query, I put clustered indexes and non-clustered indexes on fields that will be commonly used for joins. I tried putting some other clustered indexes last night and was only able to bring the elapsed time down to 125 ms. To be honest, I have no idea how to improve the performance.

Here are DDLs. I can't make major changes to these tables right now but I can adjust the indexes. I assume that the [Group_Name] is causing the query to be so slow but adding non-clustered indexes to that column for the tables doesn't significantly improve performance.

Tables


CREATE TABLE tbl_Clauses
(
ClauseID smallint identity (1,1),
Standard varchar(100),
Clause varchar(100),
Description varchar(100),
IsAS9100 bit not null,
IsActive bit not null,
Primary key clustered (ClauseID),
Unique (Standard, Clause)
)

CREATE TABLE jntbl_RefDocToGroup
(
Group_Name varchar(100)not null,
RefDoc_Title varchar (100)not null,
Primary Key Clustered (Group_Name, RefDoc_Title)
)

CREATE TABLE tbl_GroupClauses
(
ID smallint primary key identity (1,1),
Group_Name varchar (100)not null,
Standard varchar (100) not null,
Clause varchar (100)not null,
IsPrimary bit not null,
Unique (Group_Name, Clause,Standard)
)

CREATE TABLE tbl_Groups
(
Group_ID smallint identity,
Dept_Name varchar(100),
Group_Name varchar(100),
ManagerEmpID smallint,
Key_Processes varchar(500),tb
ManagerEmail varchar (100),
Primary Key clustered (Group_ID),
Unique(Group_Name)
)


The Slowest Query Ever

CREATE VIEW v_GroupInfo
AS
SELECT Dept_Name,v_ConcatGroupClauses.Group_Name,tbl_Groups.Group_ID,Supervisor as Supervisors,ManagerName, PrimClauses as PrimaryClauses, SecClause as SecondaryClauses, RefDoc as RefDocs,Key_Processes, v_ManagerInfo.ManagerEmail, Email as Supervisor_Email
from dbo.v_ConcatGroupClauses
inner join dbo.v_ConcatGroupRefDocs
on v_ConcatGroupClauses.Group_Name = v_ConcatGroupRefDocs.Group_Name
inner join dbo.v_ConcatGroupSecClauses
on v_ConcatGroupClauses.Group_Name = v_ConcatGroupSecClauses.Group_Name
inner join dbo.v_ConcatGroupSupervisors
on v_ConcatGroupClauses.Group_Name = v_ConcatGroupSupervisors.Group_Name
inner join dbo.v_ManagerInfo
on v_ConcatGroupClauses.Group_Name = v_ManagerInfo.Group_Name
inner join tbl_Groups
on v_ConcatGroupClauses.Group_Name = tbl_Groups.Group_Name
GO



VIEWS
CREATE VIEW v_ConcatGroupRefDocs
AS
SELECT DISTINCT Group_Name, (SELECT SUBSTRING((SELECT ', ' + RefDoc_Title
FROM jntbl_RefDocToGroup
WHERE Group_Name = t.Group_Name
FOR XML PATH('')),2,200000)) AS RefDoc
FROM jntbl_RefDocToGroup t
GO

CREATE VIEW v_ConcatGroupClauses
AS
SELECT DISTINCT Group_Name, (SELECT SUBSTRING((SELECT ', ' + Clause
FROM tbl_GroupClauses
WHERE Group_Name = t.Group_Name
AND IsPrimary = '1'
FOR XML PATH('')),2,200000)) AS PrimClauses
FROM tbl_GroupClauses t
GO

CREATE VIEW v_ConcatGroupSecClauses
AS
SELECT DISTINCT Group_Name, (SELECT SUBSTRING((SELECT ', ' + Clause
FROM tbl_GroupClauses
WHERE Group_Name = t.Group_Name
AND IsPrimary = '0'
FOR XML PATH('')),2,200000)) AS SecClause
FROM tbl_GroupClauses t
GO


CREATE VIEW v_GroupSupervisor
as
select group_name, Full_name as 'Supervisor',Email
from tbl_groups as g inner join (jntbl_GroupSupervisor inner join tbl_employee on jntbl_GroupSupervisor.emp_id = tbl_employee.emp_id)
on g.Group_ID = jntbl_GroupSupervisor.Group_ID
GO

CREATE VIEW v_ConcatGroupSupervisors
AS
SELECT DISTINCT Group_Name, (SELECT SUBSTRING((SELECT ', ' + Supervisor
FROM v_GroupSupervisor
WHERE Group_Name = t.Group_Name
FOR XML PATH('')),2,200000)) AS Supervisor,(SELECT SUBSTRING((SELECT '; ' + Email
FROM v_GroupSupervisor
WHERE Group_Name = t.Group_Name
FOR XML PATH('')),2,200000)) AS Email
FROM v_GroupSupervisor t
GO


Any feedback on how I should approach this problem? I've tried using an index view on v_GroupInfo but SQL Server wouldn't allow it.

theboyholty
Posting Yak Master

226 Posts

Posted - 2011-10-07 : 11:56:40
Without going through this with a fine toothcomb (Its 10 to 5 on a Friday), I immediately noticed a couple of things with your code.

'The slowest query ever' has a number of joins, all of which join on Group_Name which being a VARCHAR(100) isn't going to be very quick. Its alwasy best to use an ID field if possible.
Also you've got some inline queries on your other views which is going to be a bit slow. There may be no way round these in your case but I would generally only use inline queries as a last resort.

So no mention of indexes there but often tweaking the queries can make a much bigger difference.


---------------------------------------------------------------------------------
http://www.mannyroadend.co.uk A Bury FC supporters website and forum
Go to Top of Page

nigelrivett
Master Smack Fu Yak Hacker

3385 Posts

Posted - 2011-10-07 : 12:02:27
Can you materialise the views and index them. If you do that then you can try substituting the tables for the views and find out where the issue is.
Otherwise try omitting the views in turn to see which is causing issues.

==========================================
Cursors are useful if you don't know sql.
SSIS can be used in a similar way.
Beer is not cold and it isn't fizzy.
Go to Top of Page

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 12:05:06
quote:
Originally posted by theboyholty

Without going through this with a fine toothcomb (Its 10 to 5 on a Friday), I immediately noticed a couple of things with your code.

'The slowest query ever' has a number of joins, all of which join on Group_Name which being a VARCHAR(100) isn't going to be very quick. Its alwasy best to use an ID field if possible.
Also you've got some inline queries on your other views which is going to be a bit slow. There may be no way round these in your case but I would generally only use inline queries as a last resort.

So no mention of indexes there but often tweaking the queries can make a much bigger difference.


---------------------------------------------------------------------------------
http://www.mannyroadend.co.uk A Bury FC supporters website and forum



I've tried using identity fields but the original tables itself doesn't have the Group_ID fields but instead have the Group_Name.

I also need to have the inline queries because it is used to concatenate multiple rows into one.

How's the indexing though? Awful?
Go to Top of Page

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 12:06:44
quote:
Originally posted by nigelrivett

Can you materialise the views and index them. If you do that then you can try substituting the tables for the views and find out where the issue is.
Otherwise try omitting the views in turn to see which is causing issues.

==========================================
Cursors are useful if you don't know sql.
SSIS can be used in a similar way.
Beer is not cold and it isn't fizzy.



What do you mean by materializing the views?
Go to Top of Page

nigelrivett
Master Smack Fu Yak Hacker

3385 Posts

Posted - 2011-10-07 : 12:23:19
Create tables from them.

==========================================
Cursors are useful if you don't know sql.
SSIS can be used in a similar way.
Beer is not cold and it isn't fizzy.
Go to Top of Page

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 13:01:15
quote:
Originally posted by nigelrivett

Create tables from them.

==========================================
Cursors are useful if you don't know sql.
SSIS can be used in a similar way.
Beer is not cold and it isn't fizzy.



I've read that the SQL Server equivalent to a materialized view is an indexed view. I tried this but I get the following error:

dbo.v_ConcatGroupClauses is not schema bound


When I try to schema bound v_ConcatGroupClauses, I get the following error:
Cannot schema bind view 'v_ConcatGroupClauses' because name 'tbl_GroupClauses' is invalid for schema binding. Names must be in two-part format and an object cannot reference itself.
Go to Top of Page

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 14:35:45
Turns out the reason why I couldn't create the indexed views is because I didn't prefix the table name with the schema name. The query went down from roughly 145 ms to 45 ms!

Does anyone have any other suggestions on improving the performance? Looks like the tables below is the bottleneck.

CREATE TABLE tbl_GroupClauses
(
Group_Name varchar (100)not null,
Standard varchar (100) not null,
Clause varchar (100)not null,
IsPrimary bit not null,
Primary Key Clustered (Group_Name, Clause,Standard)
)

CREATE TABLE tbl_Groups
(
Group_ID smallint identity,
Dept_Name varchar(100),
Group_Name varchar(100),
ManagerEmpID smallint,
Key_Processes varchar(500),
ManagerEmail varchar (100),
Primary Key clustered (Group_ID),
Unique(Group_Name)
)

Hash Match 1 (Cost 31%)
Keys Build: [DevCIDB].[dbo].[tbl_GroupClauses].Group_Name
Keys Probe: [DevCIDB].[dbo].[tbl_Groups].Group_Name

Hash Match 2 (Cost 36%)
Keys Build:[DevCIDB].[dbo].[tbl_Groups].Group_Name
Keys Probe:[DevCIDB].[dbo].[tbl_GroupClauses].Group_Name
Go to Top of Page

namman
Constraint Violating Yak Guru

285 Posts

Posted - 2011-10-07 : 15:02:08
How many rows returned if your query runs successfully?

How many rows are in each table?

Better if you can provide us some sample data (make it up) of those tables

Go to Top of Page

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 16:04:45
quote:
Originally posted by namman

How many rows returned if your query runs successfully?

How many rows are in each table?

Better if you can provide us some sample data (make it up) of those tables



This is from the GroupClauses table. It's a junction table and has 938 rows.

Group_Name Standard Clause IsPrimary
Aircraft Repair Station ISO9001 4.1 0
Aircraft Repair Station ISO9001 4.2.1 0
Aircraft Repair Station ISO9001 4.2.2 0
Aircraft Repair Station ISO9001 4.2.3 0
Aircraft Repair Station ISO9001 4.2.4 0
Aircraft Repair Station ISO9001 5.2 0


This is from the Groups table and only has 39 records.

8 Engineering Aircraft Repair Station 104 Aircraft Certification/Partnership for Safety
35 Salem Call Center Call Center NULL Customer Support and Service; Handling Customer Complaints. Orders, returns, complaints, for consumer product.
Go to Top of Page

tkepongo
Yak Posting Veteran

62 Posts

Posted - 2011-10-07 : 18:00:57
Okay, so I was really wrong about the execution time. I tested it and it went up to about 3000 ms so I decided to create a physical table that will act as a lookup table for all the concatenated values and everything since it's all static. This brought it down to 16 ms.
Go to Top of Page
   

- Advertisement -