At the moment I create a database connection when my web page is first loaded. I then process the page and run any queries against that conection. Is this the best way to do it or should I be creating a database connection each time I run a query?
p.s It makes more sense to me to create 1 connection and use it but I don't know if this can cause any other issues.
I am using C# (ASP.NET) with MSSQL.
If you create one per query / transaction, it is much easier to manage "closing" the connections.
I can see why common sense dictates that you should open one and use it throughout, but you will run into problems with dropped connections and multithreading. So your next step will be to open a pool, say of 50, connections and keep them all open, doling them out to different processes. And then you'll find out that this is exactly what the .NET framework does for you already.
If you open a connection when you need it and dispose of it when you've finished, that will not actually close the connection, it'll just return it to the connection pool to be used again.
Best practice it to create one connection per query - and in the case of displaying data, best practice is to have the query bring in all the needed data in one go.
Background information:
In .NET, calling SqlConnection.Open() will by default always transparently use connection pooling (see "Using Connection Pooling with SQL Server" on MSDN). So you can just grab a new connection using Open(), and call Close() when you're done, and .NET will do the right thing.
Note that without connection pooling, one connection per query would be a very bad idea because creating real database connections can be very costly (authentication, network overhead etc.), and the number of simultaneous open connections is usually very limited.
SqlConnection.Open() will always transparently use connection pooling. So the difference between "open a connection" and "retriebve a connection from a pool" does not exist. My misunderstanding. I took the liberty of editing a little explanation into the question, and took back the vote.
Developers sometimes want to optimize or simplify their code to re-use their database connection objects. Given the context of this question puts us in the .Net ecosystem, this is almost always a mistake. (Note: many other platforms handle this in similar ways, as well).
ADO.Net has a feature called Connection Pooling. When you create and open a new connection object, what you're really doing is requesting a connection from a pool. When you close a connection, you return it to the pool.
It's important to understand the objects we use directly in code — SqlConnection, MySqlConnection, OleDbConnection, etc — are merely wrappers around a real underlying connection managed by ADO.Net. The ADO.Net real connections are much "heavier" and more expensive from a performance standpoint. It's these underlying objects that have worries like authentication, network transit, encryption, etc, and those things far outweigh the small amount of memory in the object you actually see in your own code.
When you try to re-use your connection object, you gain efficiency in the small thing at the expense of the much bigger thing. It's a net loss, because you break the ability for the ADO.Net connection pool to effectively manage the important underlying connections.
Re-using a connection across an application or http request can also force you to unintentionally serialize operations that might otherwise be able to run asynchronously, and become a performance bottleneck. I've seen this happen in real applications.
In the case of the web page example here, where you at least only keep the small connection for the duration of a single http request/response, you could gain even more efficiency by evaluating what queries you run in your request pipeline, and try getting them down to as few separate requests to the database as possible (hint: you can submit more than one query in a single SQL string, and use DataReader.NextResult() or check different tables in a DataSet to move between them).
In other words, rather than thinking in terms of minimizing the number of connection objects you create (by re-using one connection for an application or http request), think in terms of one connection for each time you call out to the database... each round trip. Then try to minimize the number of connections by minimizing the number of those trips. In this way you can satisfy both goals.
But that's just one kind of optimization. There's also optimizing programmer time, and gaining effective code re-use. Developers don't want to write the same boilerplate code over and over again just to get a connection object that's open and ready-to-use. It's not only tedious, it's a way to introduce bugs into a program.
Even here, though, it's still generally better to have one connection per query (or round trip). There are patterns you can use to help avoid re-writing the same boilerplate code, while still keeping the goal of one connection per call to the database. Here is one example I like, but there are many others.
Personally, I usually find that software acquires a connection (often, "behind the scenes"), then re-uses that connection until the routine which acquired it exits. I anticipate finding little reason to "improve upon" this.
