3

I have an ASP.NET web site on a shared server. The site is mainly used to house some WCF services that are used by a desktop application.

We had a problem in that the server stopped responding, and the hosting company said that the application pool logs showed that we had hit our memory limit (500Mb), which caused the application pool to recycle.

I need to find out what is causing the memory problem. The code is all C#, no unmanaged calls at all. It's a tiered application, with the WCF layer being very thin, basically passing calls through to the service layer. The service layer uses a repository layer to do the database access (via Entity Framework), and all dependencies are injected. Don't know if there are any more technical details that are relevant. If so, please comment and I'll add them in.

There are only about three users for the application, so it's very unlikely to be a sheer size of usage.

As this is a shared server, I do not have any direct access to it, therefore cannot use any profiling tools that require attaching to the process etc. Anything I do has to be written in the C# itself.

I thought about setting up a test environment locally, and hitting it with loads of service calls, but as I don't really know exactly what the users are doing, it's hard to know if I would be reproducing the production scenario.

Is it possible to see what memory the web site is using from C# code, so I can see what's happening on the production server?

  • 5
    I'm tempted to read between the lines here but it's probably better for you to elaborate here. You seem to imply that because C# manages memory, you would never exceed 500MB (or any other arbitrary limit). GC will only collect things that your application is not referencing. It cannot figure out what parts of memory you are holding onto unnecessarily. – JimmyJames Sep 11 '17 at 18:00
  • 1
    @AvrohomYisroel it's a bit tricky when you have multiple applications but you may find this useful blogs.msdn.microsoft.com/cclayton/2013/05/21/… – Aluan Haddad Sep 11 '17 at 20:22
  • 4
    "My point about managed memory was that I expected the GC to collect my objects". My experience with GC is almost entirely with the JVM but I think it's safe to say that the GC will collect objects but only in the case that nothing is referencing them from the GC roots (basically the stack and static variables). It's crucial to understand this. The other thing that it's important is that GC won't run when you hit this size unless the CLR knows it needs to. I'm not sure of the platform here but it's possible that the CLR thinks it is allowed to grow bigger than 500MB. – JimmyJames Sep 11 '17 at 20:58
  • 4
    You mentioned that you use dependency injection. Most dependency injection libraries use an application global singleton life cycle by default. That means they will persist between web requests. Regardless I suggest that you profile the application locally. You'll probably get some very useful information. For example, I have an application that receives warnings from the dot net profiler regarding long-lived objects. While in my case that behavior was desired, the long-lived objects were caches that were very expensive to rebuild, it still notified me. – Aluan Haddad Sep 11 '17 at 22:02
  • 3
    You should add logging so you know what service calls were made before hitting the memory limit. It is important to be able to tell what is going on in you server and if you can't get memory dumps then logs are even more important. – Mike Sep 12 '17 at 0:46
2

If it is a traditional ASP.NET web app (WebForms or MVC), the most likely culprit is session storage. Particularly if you are using in-memory session storage (which is the default), and then using session to keep user state between pages or WCF calls. This data never goes away until the server is shut down, or there is an explicit logout, or you explicitly delete it.

You can mitigate the problem somewhat by using a separate state service or storing state in the database. See this MSDN article for session storage options. Then the session data will be loaded from the external storage when the user makes a request instead of keeping it around in memory. Significantly, storing session data externally requires that data to be serializable.

The second culprit is likely to be static properties or fields. These are lazy initialized on first reference, but afterwards are never garbage collected. If you are using static properties to cache data, for instance, it will stay in memory until the program terminates.

Lastly, in recent versions of Visual Studio you can see usage while debugging locally or explicitly start a performance profiling session. Start the profiler on your local machine with memory sampling turned on, interact with the application, and then look at the analysis it generates on memory and CPU usage.

Edit: Adding more general data usage issues

There are data practices that can lead you to use too much memory. For instance, if you are loading data from a database into a DataSet or converting to a list of objects (including using an ORM). This enumerates the entire set of returned rows and places them in memory. In order to limit memory usage, you should use paging to return only XX results at a time. Or use a DataReader instead filling a Dataset (or ORM). In order to stay memory efficient with DataReader, you will need to: read a small batch of records from the DataReader, write that batch directly to the response stream on the web service, repeat. If you need to convert rows to objects before writing to response stream, then I recommend the Dapper micro-ORM. It uses DataReader internally and you can turn off buffering, and deal with an IEnumerable of objects which are not loaded in memory until iterated.

Reading and returning files has the same issue. If you read a file into a MemoryStream, the entire file will be allocated in memory. If it's a file on disk, to conserve memory, you will need to loop through the file stream in chunks and write it to the response stream. Newer versions of .NET have Stream.CopyTo for this.

If you are using ORMs (e.g. Entity Framework) or DI libraries, those frequently cache objects in memory. So you will have to dig into the configuration to see if you can limit it or turn it off. ORMs can additionally load large object graphs, depending on how they are setup.

  • Thanks for the tips. Unfortunately, neither or the suggestions is the culprit here. The web site is really only there to house the WCF services, there is only one actual "page" on the site, and that is static. I don't use session at all, nor do I use static variables. As I mentioned in my question, I do intend to profile it locally, but I'm dubious as to how accurate the results will be, without knowing what the users are doing. – Avrohom Yisroel Sep 13 '17 at 13:13
  • @AvrohomYisroel Glad to see you didn't fall into the same traps I did. I'll update my answer with a couple of more I thought of. – Kasey Speakman Sep 13 '17 at 14:44
  • Well, I am using Entity Framework, and one of the service calls does return a pretty large collection of objects. That's my current prime candidate for the problem. However, I didn't think about EF caching the objects. That's definitely worth checking. Thanks – Avrohom Yisroel Sep 13 '17 at 15:52
1

I suggest building a similar machine in your development or testing environment and reproduce the memory leak on it. Then you can attach a profiler to your .net processes in order to catch the leak. I suggest dotMemory profiler for this purpose. Here an article on how to identify memory leaks im .NET applications:

http://www.borismod.net/2015/02/how-investigate-memory-leak-in-net.html

  • Thanks for the suggestion, however it doesn't actually answer the question, which was about adding code to see what's going on on the real server. As for your suggestion, I mentioned a test environment in my original question, and it was further discussed at length in the comments to my question. I fully intend to do this, but without knowing exactly what the users are doing, it will be hard to reproduce the problem exactly. Thanks anyway. – Avrohom Yisroel Sep 13 '17 at 13:11
  • @AvrohomYisroel Is it possible to record users' call to the production server and then replay them on your tedting environment? – Boris Modylevsky Sep 13 '17 at 13:13
  • I intend to add more logging, which will give me some idea, but I'm not sure if it will give me enough to be confident I'm reproducing what's happening – Avrohom Yisroel Sep 13 '17 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.