Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
cheowkwen
Starting Member
4 Posts |
 Posted - 2011-11-03 : 23:10:24
|
| Hi all, currently I'm working on a project which are generating some data from my software and I wish to save those data into sql server. But my concerns is the speed of saving and the large amount of data. For every 0.5s, one set of data will be generated and the process will keep running for about 8 hours, so at the end of 8 hours, roughly there will be 60,000 set of data generated. May i know is there anyway i can tweak around so that the server will not hang and also keep pace with the speed of 0.5s for each generated data?Please advise and thanks in advance. |
|
|
visakh16
Very Important crosS Applying yaK Herder
52326 Posts |
Posted - 2011-11-04 : 01:40:56
|
| 60,000 in 8 hours is not such a big data volume. Not sure what performance issue you're facing currently------------------------------------------------------------------------------------------------------SQL Server MVPhttp://visakhm.blogspot.com/ |
 |
|
|
cheowkwen
Starting Member
4 Posts |
Posted - 2011-11-04 : 11:55:02
|
| Hi visakh16, sorry to say that I'm not familiar with the SQL server. Based on your reply, it seems the volume of 60,000 is not an issue. So what about the speed of saving? As i mentioned in above, each data will be generated after 0.5s, I think it is faster than what the time it takes to write to the SQL server for each data. My only concern is what if the time it takes to write the data to SQL is more than 0.5s and the first data haven't finish writing to SQL, the second set of the new data has been generated. So in this case, it might cause my software or the server hang up. |
|
|
|
visakh16
Very Important crosS Applying yaK Herder
52326 Posts |
Posted - 2011-11-04 : 12:42:06
|
| are you doing this server saving in seperate transactions? also does it happen only row by row? is there an option of doing batch addition instead?------------------------------------------------------------------------------------------------------SQL Server MVPhttp://visakhm.blogspot.com/ |
|
|
|
cheowkwen
Starting Member
4 Posts |
Posted - 2011-11-06 : 11:44:23
|
| Hi visakh16, my software will directly write it to the SQL server after the data is generated and it is row by row addition. What do you meant by batch addition? Even it is batch addition, it is consider row by row right? Sorry to say that I'm not a superuser for the SQL. |
|
|
|
visakh16
Very Important crosS Applying yaK Herder
52326 Posts |
Posted - 2011-11-07 : 04:00:55
|
quote:
Originally posted by cheowkwen
Hi visakh16, my software will directly write it to the SQL server after the data is generated and it is row by row addition. What do you meant by batch addition? Even it is batch addition, it is consider row by row right? Sorry to say that I'm not a superuser for the SQL.
nope batch addition is different. SQL Server RDBMS is designed to be optimised for batch processing and hence set based processing is what you should be targeting if at all possibleIs software doing data insertion by means of procedure or is it inline sql query------------------------------------------------------------------------------------------------------SQL Server MVPhttp://visakhm.blogspot.com/ |
|
|
|
cheowkwen
Starting Member
4 Posts |
Posted - 2011-11-07 : 10:58:13
|
| Hi visakh16, I have no idea about the batch addition at all, can you please send me some links which explaining what is and how to do the batch addition? In my software, I have a INSERT string where I save all the values by specifying each column name. I do not know is this by means of procedure or inline sql query? |
|
|
|
|
|
|
|
|
60K of data saving to SQL server?