NullReferenceException

[jnethery]

We are using a modified version of Able 7.0.4, and occasionally (about every 1-2 weeks) we encounter some error that causes the site to go down. It is resolved by causing the site to re-build (I usually just open the web.config and change compilation debug=true, save and open the site. It rebuilds, and is fine, then I change it back to debug=false and it's still fine). It's obviously some kind of glitch somewhere since it only happens sometimes and is easily resolved, but having our site go down hurts business so we are trying to find out why this is happening. This last time, the site was down for 12 hours because of this error (it occurred around 9pm when no one was monitoring the site, and was not noticed until 9am when we started the work day). The time before that (9 days prior) it was down an entire day - a Saturday when no one was working. It's always the same error in the stack trace. There is nothing in the app.log from around the time of the error. It seems like it always follows a pattern, however, beginning with a Forms Authentication failure (A) followed by a Sql Timeout (B) and then repeated NullReferenceException errors (C) until the site is rebuilt or the server is re-started. Any help that you can offer would be much appreciated. The error details are as follows:

(A)
Event Type: Information
Event Source: ASP.NET 2.0.50727.0
Event Category: Web Event
Event ID: 1315
Event code: 4005
Event message: Forms authentication failed for the request. Reason: The ticket supplied has expired.
Event sequence: 2315
Event occurrence: 1
Event detail code: 50202
Application information:
Trust level: Full
Application Virtual Path: /
Application Path: E:\onlinescuba-ac7\
Machine name: CORAZON
Process information:
Process ID: 3940
Process name: w3wp.exe
Account name: NT AUTHORITY\NETWORK SERVICE
Request information:
Request URL: https://www.onlinescuba.com:443/Members ... rderId=699
Request path: /Members/MyOrder.aspx
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\NETWORK SERVICE

(B)
Event Type: Warning
Event Source: ASP.NET 2.0.50727.0
Event Category: Web Event
Event ID: 1309
Event code: 3005
Event message: An unhandled exception has occurred.
Event sequence: 2509
Event occurrence: 12
Event detail code: 0
Process information:
Process ID: 3940
Process name: w3wp.exe
Account name: NT AUTHORITY\NETWORK SERVICE
Exception information:
Exception type: SqlException
Exception message: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Request information:
Request URL: http://www.onlinescuba.com/CategoryGrid ... egoryId=25
Request path: /CategoryGrid4.aspx
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\NETWORK SERVICE
Thread information:
Thread ID: 9
Thread account name: NT AUTHORITY\NETWORK SERVICE
Is impersonating: False
Stack trace: at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at CommerceBuilder.Data.Database.OpenConnection()
at CommerceBuilder.Data.Database.ExecuteReader(DbCommand command)
at CommerceBuilder.Users.RoleDataSource.IsUserInRole(Int32 userId, Int32 roleId)
at CommerceBuilder.Users.User.IsInRole(ICollection`1 roleCollection)
at CommerceBuilder.Users.User.get_IsAdmin()
at CommerceBuilder.Services.c.a(Object A_0, EventArgs A_1)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

(C)
Event Type: Warning
Event Source: ASP.NET 2.0.50727.0
Event Category: Web Event
Event ID: 1309
Event code: 3005
Event message: An unhandled exception has occurred.
Event sequence: 2683
Event occurrence: 99
Event detail code: 0
Process information:
Process ID: 3940
Process name: w3wp.exe
Account name: NT AUTHORITY\NETWORK SERVICE
Exception information:
Exception type: NullReferenceException
Exception message: Object reference not set to an instance of an object.
Request information:
Request URL: http://www.onlinescuba.com/Product.aspx?ProductId=7530
Request path: /Product.aspx
Is authenticated: False
Authentication Type:
Thread account name: NT AUTHORITY\NETWORK SERVICE
Thread information:
Thread ID: 7
Thread account name: NT AUTHORITY\NETWORK SERVICE
Is impersonating: False
Stack trace: at Layouts_AdminPanel.Page_Init(Object sender, EventArgs e)
at System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr fp, Object o, Object t, EventArgs e)
at System.Web.Util.CalliEventHandlerDelegateProxy.Callback(Object sender, EventArgs e)
at System.Web.UI.Control.OnInit(EventArgs e)
at System.Web.UI.UserControl.OnInit(EventArgs e)
at System.Web.UI.Control.InitRecursive(Control namingContainer)
at System.Web.UI.Control.InitRecursive(Control namingContainer)
at System.Web.UI.Control.InitRecursive(Control namingContainer)
at System.Web.UI.Control.InitRecursive(Control namingContainer)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

[plugables]

I think the problem starts from your database server not being able to respond timely. Once the database server becomes unavailable it breaks the application.

There was a bug fixed in version 7.0.5 for exactly the same symptoms.

See Issue 8646 in 7.0.5 change log http://help.ablecommerce.com/upgrades/a ... _7.0.5.htm
"IIS Application pool stops when database connection is unavailable"

[jnethery]

Thank you for your reply! That makes perfect sense. The issue I am facing now is that we have heavily customized our 7.0.4 version due to there being certain requirements for our business that were not included in the vanilla version. Does anyone know how to manually fix this specific problem? I have all the source code, and I'm pretty sure I could figure out a fix, if I knew where the problem was located in the code. It would be much less difficult than trying to merge my modified code with the new version's code.

[mikek]

Hello,

The database fix is in the CommerceBuilder code. If you need to manually fix the issue for 7.0.4 you have to have access to the CommerceBuilder source code which
is available as a separate product.

http://www.ablecommerce.com/CommerceBui ... 95C51.aspx

Once you have the CommerceBuilder code you can CheckIn fresh AC 7.0.4 version under version control like SVN or Mercurial and then apply the latest 7.0.5 code and see which CommerceBuilder.Data
files require merge.

Another option is to fix your hosting environment and have stable web and mssql servers.