What Is the Target Virtual Memory Usage Requirement For Able

[drollins]

I am running Able V7.1

Using the following code to pull the current process virtual memory usage into a variable I can log/display:

Code: Select all

Process p = Process.GetCurrentProcess();
MemUsage.Text = (p.VirtualMemorySize64 / 1024 / 1024).ToString();

I am consistently seeing 630MB or so?

My ISP gives me a virtual space of 195MB ... therefore my site recycles constantly.... my customer is not happy

Can someone tell me....

1. What is a "normal" virtual memory footprint range for my Able site (600 products / 20 categories)?
2. What are steps I can take to reduce this footprint?
3. What are things to avoid to minimize virtual memory consumption?

[Logan Rhodehamel]

If you are using multiple worker processes in your application pool, try using 1. See if your application recycles less often.

[drollins]

There is an issue with multiple processes? Or are you saying if you have 4 processes running able will load up 4 times or in my instance 630MB / 4 or 158MB per instance?

[drollins]

According to my ISP my application pool for this site is already set at 1 Worker Process.

Also - I would like to know what the rule of thumb virtual memory requirement for Able is so I know what I am shooting for.

My current memory usage of 630MB seems ridiculous - either I am doing something very BAD wrong or Able is a monster memory piggy.

Hopefully it is me.... and I just need to find out what I should do.

Does anyone know a way to get the memory usage of each DLL in your Able installation? This type of info would at least point you where to pay attention.

[Logan Rhodehamel]

Just keep in mind that virtual memory is quite different from private memory. The application is not actually using 630MB. I will reexamine memory usage and see if I can spot any trouble. One thing you might be able to do to reduce memory impact of the application is to remove DLLs for payment processors.

1) Backup the current contents of your "bin" folder to a safe location in case you need to restore files

2) Delete any payment processor DLL that you do not require, EXCEPT Google and PayPal. You cannot delete these two because they are referenced by the front end scripts.

Payment processor DLLs have the name CommerceBuilder.PaymentGatewayName.DLL. Example would be CommerceBuilder.LinkPoint.DLL.

[compunerdy]

Can we have a list of the DLL's we can remove with a short description?

[drollins]

Thanks for the payment gateway dll tip - I will see what type of impact that has.

IIS is measuring Virtual Memory - that is what they, my ISP, sets my limit by for app pool recycling purposes so that is what I care about.

I am still interested to know what Able says is your normal footprint for the stock application. Obviously if I tweak the app at all this may change... I really just want a base line to measure my memory usage reduction success by.

Also have you seen this:
http://blog.kristofrennen.be/index.php/ ... mory-leak/

[drollins]

OK... so no answers huh?

I have put together a little memory viewer you can all play around with if you want.

But first... to CMA....

USE AT YOUR OWN RISK - REVIEW THE CODE BEFORE YOU EXECUTE IT!
I TAKE NO RESPONSIBILITY FOR ANY DAMAGE DONE BY RUNNING THIS MEMORY USAGE VIEWER!

Ok.. that's out of the way.

Attached is a zip - just put the files where they are found in the zip in the corresponding folders on your Able installation.

MemoryViewer.zip

Then fire up http://yoursiteurl/memoryusage.aspx to see what's cookin.

Mine looks like this:


Application Module Load Memory - total memory needed to load the DLLs on your site
Microsoft Module Load Memory - total memory needed to load Microsoft's framework stuff - I hid the actual names of the modules loaded since it is not what we are after.

So... why is my Virtual memory pegged at 620MB???????????

What is consuming all this virtual memory???

You see my site is only allocating 67MB of phsical memory and 10 MB of loaded dlls...

This means somewhere is loading (620MB - 67MB - 10MB) or 543MB of extra stuff!!!!!

Is this cache, raw files what?

Hey... if anyone has an idea how to embellish this script further to get more memory detail... love to see it!

[afm]

Is your database on a different machine?

Tess Ferrandez (a Microsoft employee) has a fantastic web site about diagnosing ASP.NET issues. Many of her posts are about memory issues:

http://blogs.msdn.com/tess/archive/tags ... fault.aspx

Her process is usually the same:

1. Take a snapshot of the ASP.NET process at the time of the problem.
2. Analyze the dump using various tools.

I don't think anyone has answered your posts, because there is no "typical". Really. Every site is different...even out of the box...because each server is configured differently. If you want to know more about your site, you will have to dig in. Hopefully Tess' site will help.

[Logan Rhodehamel]

So... why is my Virtual memory pegged at 620MB???????????

What is consuming all this virtual memory???

[...]

Is this cache, raw files what?

It's not memory in use, so there is nothing there. The application has said to the process, "hey, I might wish to use this memory so allocate me a virtual address space". It's an allocation, not a consumption.

The working set does represent phyiscal memory currently in use. But your memory listing should also include PrivateMemorySize64. The value in this property represents memory that the process is consuming that cannot be shared. You don't know what the 67MB in use within the working set represents - something perhaps that could be paged out of physical memory.

It is somewhat hard for me to troubleshoot the virtual memory issue on x64. The ASPNET process automatically allocates 3GB virtual memory, even for an empty application. I will have to do some testing on a 32bit machine.

[Logan Rhodehamel]

Do this for me if you can. I converted the memory control to a page. Create a new ASPNET application and make sure it's in it's own application pool. The application does not need any files besides this memory usage page. Then browse to the page and post your results.

[drollins]

I did mention my problem is on a HOSTED platform - I don't have alot of control on the server.

And unless I am going insane (which there is a high probability of), or I don't understand something (again, something that happens frequently) the version of the memory viewer you posted is what I posted - same files etc.... at least that is what downloaded for me.

So far I have been playing with this memory problem on the hosted server. I am going to pull the site to my local machine so I have more sticks to poke the thing with the post more info here.

Also ... I removed the following DLLs from the Bin and my Able site still "rendered" OK (not sure how anyone would get anything shipped etc ) but it made NO difference to the virtual memory consumption.

CommerceBuilder.CanadaPost.dll
CommerceBuilder.CCH.dll
CommerceBuilder.CCH.dll.config
CommerceBuilder.CyberSource.dll
CommerceBuilder.DataClient.Api.dll
CommerceBuilder.DataClient.Csv.dll
CommerceBuilder.DHL.dll
CommerceBuilder.ECB.dll
CommerceBuilder.FedEx.dll
CommerceBuilder.FRBNY.dll
CommerceBuilder.FRBNY.dll.config
CommerceBuilder.LinkPoint.dll
CommerceBuilder.Paradata.dll
CommerceBuilder.Paradata.dll.config
CommerceBuilder.PayFlowPro.dll
CommerceBuilder.PayJunction.dll
CommerceBuilder.SkipJack.dll
CommerceBuilder.UPS.dll
CommerceBuilder.USPS.dll
CyberSource.Base.dll
CyberSource.Clients.dll
CyberSource.Clients.XmlSerializers.dll
CyberSource.WSSecurity.dll
CybsWSSecurityIOP.dll
edtFTPnet.dll
LinkPointTransaction.dll
PaygatewayNET.dll
PFProCOMLib.dll
PFProdotNET.dll
wwHoverPanel.dll

[drollins]

Andy, in this hosted scenario SQL Server IS on a different box.

[drollins]

Added PrivateMemory64 to the viewer

[Logan Rhodehamel]

And unless I am going insane (which there is a high probability of), or I don't understand something (again, something that happens frequently) the version of the memory viewer you posted is what I posted - same files etc.... at least that is what downloaded for me.

My mistake. I reattached the file you uploaded.

The reason I asked this test, on my 32 bit server with a blank aspnet application, virtual memory starts at something over 300mb.

[Logan Rhodehamel]

The results from my new application, .NET 2.0 IIS7 with application running in integrated mode.

[drollins]

Logan - I have VERY similar results to yours. The virtual memory value on XP is around 320MB to start, on Win2003 it is around 520MB to start... just running this simple memory page.

This is very interesting. Something to stump my ISP on for sure

[Logan Rhodehamel]

Logan - I have VERY similar results to yours. The virtual memory value on XP is around 320MB to start, on Win2003 it is around 520MB to start... just running this simple memory page.

This is very interesting. Something to stump my ISP on for sure

And like I mentioned, on a 64bit machine it's a completely pointless measure because the blank app on my test server starts at 3GB. Clearly the application is not using that much memory.

Everything I read while researching this issue said to pay more attention to the private bytes - this is the true measure of how much memory an application has consumed that is no longer available to other processes. I am not sure if that will sway your ISP to reconsider using that as the limiting value over the virtual memory counter.

[drollins]

OK so using this memoryviewer app I successfully convinced my ISP to eliminate the virtual memory limit.

A more accurate limitation would be actual memory allocation by the application which is also an app pool option.

Using that yard stick it would seem that, at under 100 MB of allocated memory, the AbleCommerce app is a relatively well behaved application.

The bottom line is the following:

If your ISP is limiting you to some arbitrary virtual memory threshhold then they need to set that limit at X over the default 520MB (32bit OS) pre-allocated (and which you have absolutely no control over) block IIS gives by default on Win2003. I am assuming here that Win2008 (32 Bit) is similar. So if they wanted to set your limit to say... 200MB, the fair virtual memory thresh-hold would be around 720MB.

If they choose allocated memory as their yard stick then a flat 200MB would also (at least in my scenario) be a fair limitation.

[Logan Rhodehamel]

Using that yard stick it would seem that, at under 100 MB of allocated memory, the AbleCommerce app is a relatively well behaved application.