| Author |
Topic |
|
carsten29
Starting Member
5 Posts |
 Posted - 2011-08-04 : 04:13:12
|
| In our System we have 2 Database servers and round about 70 clients. Each Database server has an SQL-Server-Instance with 4 databases (A, B, C, D). These databases are published for Replication. Each database server subscribes all 4 publications of the other database server. The Clients subscribe only 2 databases (A, B) from one of the database server (Push-Subscription).Everything went fine. But now we have more clienst and suddenly we get Errormessages on the database servers:Distrib.exe "Application Error : The application failed to initialize properly (0xc0000142)."I found out, that two distrib.exe-processes are startet on the server for each client. So 40 clients = 80 processes.If I add another client, the error appears and I have 81 processes. If I add one more client then I have 3 errormessages and so on. 2 errors per client.It seems that I reached a limitation. When I check the Taskmanager CPU amd Memory are ok. Memory usage is at 50%, CPU around 10%.If I click the errormessage away they come back after 1 minute.In the SQL-Server-Management-Studio I found out, that the subscription for a client is shown below the publication on the server but it is missing on the client-side (in subscriptions).My previous searches brought me the following “solutions”.1. Desktop Heap is to smallhttp://support.microsoft.com/kb/949296/en-us (Some SQL Server replication agents cannot run when you configure many replication agents to run on a server)http://support.microsoft.com/kb/824422/en-us (Unexpected behavior occurs when you run many processes on a computer that is running SQL Server)http://support.microsoft.com/kb/126962/en-us ("Out of Memory" error message appears when you have a large number of programs running)I checked the Heap with the Desktop Heap Monitor but everything was fine. I raised the Heap to test but the error messages still appear. 2. CreateProcess with another Account http://support.microsoft.com/kb/960266/en-us (Error message when you call the CreateProcess function to start a process of a console application by using an account other than the current logon account in Windows Server 2003: "The application failed to initialize properly (0xc0000142)")The distrib.exe-prcesses are using another account so I tried the SP2 for SQL-Server and the hotfix from Microsoft.The errormessages still appear. Can anyone help me to solve the problem and where does it come from? Is there a limitation for push-subscritions?Thank you.Carsten My environment:Database server: Windows Server 2008 Standard SP2 SQL Server 2008 Enterprise SP2 10.0.4000.0 Client Windows XP Professional 2002 SP3 SQL Server 2008 Express 10.0.1600.0 |
|
|
russell
Pyro-ma-ni-yak
5072 Posts |
Posted - 2011-08-04 : 11:18:31
|
quote:
Originally posted by carsten29
[font=Arial]In our System we have 2 Database servers and round about 70 clients. Each Database server has an SQL-Server-Instance with 4 databases (A, B, C, D). These databases are published for Replication. Each database server subscribes all 4 publications of the other database server. The Clients subscribe only 2 databases (A, B) from one of the database server (Push-Subscription).
I don't understand this part.Is this transactional replication? Peer-to Peer? Merge?What do you mean when you say "clients"?And where is the dsitributor? |
 |
|
|
carsten29
Starting Member
5 Posts |
Posted - 2011-08-05 : 02:13:09
|
| Good Morning,Database server using Peer to Peer replication.Clients are our "small" database server which subscribe to the Database Servers with transactional replication (data is going only from database server to client-server)The distributor is also on the Database servers. |
|
|
|
russell
Pyro-ma-ni-yak
5072 Posts |
Posted - 2011-08-05 : 08:27:52
|
| While I've read that there is no theoretical limit to the number of subscribers, or peers, the practical limit is often said to be 10.I THINK you're saying that Server A and Server B are 2 nodes in a peer to peer topology, and they're Publishers for seperate Transactional publications with 40 Subscribers each. Right?Anyway check all of the usual memory counters. Make sure you've configured memory properly. How much RAM is distrib.exe using? This is outside of sqlserver.exe process, so you may have run out of memory to spawn more processes.If I'm right about your setup (2 publishers, many subscribers) then it's time for a dedicated server for the distributor. In fact, depending on I/O and CPU load, it might already be time for TWO dedicated distributors.Are the subscribers on a cluster, or standalone? |
|
|
|
carsten29
Starting Member
5 Posts |
Posted - 2011-08-08 : 08:20:54
|
quote:
Originally posted by russell
I THINK you're saying that Server A and Server B are 2 nodes in a peer to peer topology, and they're Publishers for seperate Transactional publications with 40 Subscribers each. Right?
Right.quote:
Originally posted by russell
Anyway check all of the usual memory counters. Make sure you've configured memory properly. How much RAM is distrib.exe using? This is outside of sqlserver.exe process, so you may have run out of memory to spawn more processes.
The distrib.exe is using about 8.200k of Ram each. 8200k times 80 = 640,6MB.Taskmanager says 1.79GB of 4GB Memory used. So there is some space left.The processor is about 10% average usage.I checked also some other other memory counters with the performancemonitor.Available Bytes are about 483MB.Pages/s are about 0.Page Faults/s are about 50 when I have no error and 200 when I got the errors. But I think these are only soft page faults because distrib.exe is already running/in memory.Pagefile usage is about 4-5%The SQL-Server has also 2 Counter.Total Server Memory and Target Server Memory. Target Server Memory is about twice as big as the Total Server Memory. So no problem here.Also the HEAP is still around 13% of usage.What else can pretend to spawn new processes?quote:
Originally posted by russell
If I'm right about your setup (2 publishers, many subscribers) then it's time for a dedicated server for the distributor. In fact, depending on I/O and CPU load, it might already be time for TWO dedicated distributors.
I hope not.quote:
Originally posted by russell
Are the subscribers on a cluster, or standalone?
All standalone.Today I also tried to stop some distribution agents in the SQL-Server Agent and then add new Subscriber.The same result, when I reach the magical limit of 80 processes I get my errors. |
|
|
|
russell
Pyro-ma-ni-yak
5072 Posts |
Posted - 2011-08-08 : 08:39:55
|
| Well, I'm not aware of, nor can I find any reference to a hard limit, unless using developer edition (in which case the limit is 10, if I remember correctly).Might be worth opening a case with MS. But I'd still use a dedicated distributor. Or you might build out a 3rd node in the peer-to-peer.So there's 4 GB RAM on the boxes? What is the Max memory set to for SQL Server?Not sure if there's a connection limit in Win Server 2008 SE...is this 64 bit or 32?Might also have a look at this |
|
|
|
carsten29
Starting Member
5 Posts |
Posted - 2011-08-09 : 04:43:01
|
| The Max memory isnt' changed so it is default value of 2147483647MB.32bit System.If it would be the connections, then another events should raise. "TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts".But no such messages anywhere.Yes, I opened a case at MS already. Their first guess was also the HEAP. No I wait for answer from MS.Is there nothing more I can do?Do you have another idea? |
|
|
|
russell
Pyro-ma-ni-yak
5072 Posts |
Posted - 2011-08-09 : 10:49:02
|
| Well, you need to set the max memory setting to allow enough memory for the OS and other processes.With 4 GB, I'd probably set max memory to 3GB, but with all the distribution threads, I'd leave more. Maybe set it to around 2.25 or 2.5 GB. |
|
|
|
carsten29
Starting Member
5 Posts |
Posted - 2011-08-10 : 07:00:21
|
| I got it.It was the Desktop-HEAP but not the Limit of the non-interactive-Desktops as I thought but the total size of Desktop-HEAP which is 48MB and stored in the registry.HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\SessionViewSizeI checked with the DesktopHeapMonitor again and found out that when I reach 44MB of total HEAP and 84 Desktops/Processes I got my first errormessages.When I raised the Total size to 64MB I could launch 117 desktops/processes with about 61MBSo there are still 3-4MB difference.I think this is because of the SessionPoolSize which is in the same Registry-Folder.48MB-4MB = 44MBDo you have any experience with SessionViewSize and SessionPoolSize?I'm not sure if i will get some other problems when i raise the ViewSize.I found out that ViewSize + PoolSize should not be more then 450MB and I should raise both in 16MB-steps.Nevertheless I am happy to found it. Unfortunately after I opened a Microsoft Support Call which costs. |
|
|
|
russell
Pyro-ma-ni-yak
5072 Posts |
Posted - 2011-08-10 : 09:46:43
|
| Glad you got it. Thanks for the update. |
|
|
|
|
distrib.exe throws Error 0xc0000142