Page 1 of 1

Error on Every page Last Night

Posted: Thu Feb 18, 2016 6:12 am
by MaryP
Between around 8:00 and 12:#0 AM last night, we received the following error for every page of our website in the error log. Can someone help us with this?

An error has occured at http://www.hotsaucedepot.com/Blairs-Pur ... Sauce.aspx
Exception: While preparing SELECT this_.StoreId as StoreId109_0_, this_.Name as Name109_0_, this_.ApplicationName as Applicat3_109_0_, this_.LoweredApplicationName as LoweredA4_109_0_, this_.LicenseKey as LicenseKey109_0_, this_.DefaultWarehouseId as DefaultW6_109_0_, this_.NextOrderId as NextOrde7_109_0_, this_.OrderIdIncrement as OrderIdI8_109_0_, this_.WeightUnitId as WeightUn9_109_0_, this_.MeasurementUnitId as Measure10_109_0_ FROM ac_Stores this_ an error occurred Stack Trace: at NHibernate.AdoNet.AbstractBatcher.Prepare(IDbCommand cmd) at NHibernate.AdoNet.AbstractBatcher.ExecuteReader(IDbCommand cmd) at NHibernate.Loader.Loader.GetResultSet(IDbCommand st, Boolean autoDiscoverTypes, Boolean callable, RowSelection selection, ISessionImplementor session) at NHibernate.Loader.Loader.DoQuery(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies) at NHibernate.Loader.Loader.DoQueryAndInitializeNonLazyCollections(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies) at NHibernate.Loader.Loader.DoList(ISessionImplementor session, QueryParameters queryParameters) at NHibernate.Loader.Loader.ListIgnoreQueryCache(ISessionImplementor session, QueryParameters queryParameters) at NHibernate.Loader.Criteria.CriteriaLoader.List(ISessionImplementor session) at NHibernate.Impl.SessionImpl.List(CriteriaImpl criteria, IList results) at NHibernate.Impl.CriteriaImpl.List(IList results) at NHibernate.Impl.CriteriaImpl.List[T]() at CommerceBuilder.DomainModel.RepositoryWithTypedId`2.LoadAll(String sortExpression, Int32 maximumRows, Int32 startRowIndex) at CommerceBuilder.Services.StoreLocator.LocateCurrent() at CommerceBuilder.Common.AbleContext.get_Store() at CommerceBuilder.Services.HttpModule.a(HttpContext A_0) at CommerceBuilder.Services.HttpModule.b(Object A_0, EventArgs A_1) at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) Inner Exception: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. Inner Exception Stack Trace: at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) at System.Data.SqlClient.SqlConnection.Open() at NHibernate.Connection.DriverConnectionProvider.GetConnection() at NHibernate.AdoNet.ConnectionManager.GetConnection() at NHibernate.AdoNet.AbstractBatcher.Prepare(IDbCommand cmd)

Re: Error on Every page Last Night

Posted: Thu Feb 18, 2016 6:37 am
by mazhar
Inner Exception: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Above details from the error provide hint about database connection problem. I wonder if you database server was having some problem last night?

Re: Error on Every page Last Night

Posted: Fri Feb 19, 2016 7:52 am
by AbleMods
We've dealt with this a lot on one of our large sites.

As Mazhar suggested, it's quite possible there was an issue with the database server. Sometimes when the database server is a 'shared' server, some other user might have executed a large query and slowed everyone else down. Or the provider was performing a backup of the database server during this time and caused the server to slow down responding to SQL data requests.

Last but not least, SQL does support scheduled maintenance routines. So a scheduled maintenance plan could have fired and bogged down the database server.

We've also seen this happen when customized programming does not properly close a database connection to release it. The web server defaults to 100 concurrent db connections. As the custom code keeps consuming connections without releasing them, eventually the app hits the 100 limit and crashes.

App pool limit can be increased, and it will probably make the error go away. But the real cause has never been addressed, so it's not necessarily the best solution the problem.

Tough one to track down, for sure.

Re: Error on Every page Last Night

Posted: Fri Feb 19, 2016 8:34 am
by MaryP
Thank you.