Page 1 of 2

Encryption - Unable to validate data error

Posted: Mon Nov 02, 2015 10:23 am
by jguengerich
I am using Gold R5 (with patch) on Windows Server 2008 R2 (IIS 7.5).

I wanted to make use of a .Net 4.5 feature in my mods, so I installed .Net 4.5 on our test site server to test compatibility. I also changed EncryptionKeyManager.cs (in source code, so I won't post it unless requested by someone from AC) to use the newer PCI-compliant code in the ParseConfigFile() method. At the time, I did not have an encryption key set in Able Commerce. After installing .Net 4.5, everything worked, but I would get the following error when the site would get loaded:

Code: Select all

Unable to validate data.
Exception: Unable to validate data.
Stack Trace:    at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, Boolean useValidationSymAlgo, Boolean useLegacyMode, IVType ivType, Boolean signData)
   at System.Web.Security.MachineKey.Decode(String encodedData, MachineKeyProtection protectionOption)
   at CommerceBuilder.Configuration.EncryptionKeyManager.ParseConfigFile()
Once I set an encryption key, the error no longer occurred. Now I can remove encryption (using "DECRPYT" as the Random Text) or enable encryption at will and I do not get the error. I can restart the server, and I do not get the error.

However, this weekend I tried installing .Net 4.5 on our production site server, which already had an encryption key set. I also installed the updated CommercBuilder.dll with the new ParseConfigFile() code. When I restarted and went to the site, I got the above error and the encrypted data (UPS settings for example) did not load. I reverted the .Net 4.5 install (Shapshots on VM servers are a wonderful thing!) and removed encryption, then installed .Net 4.5 and the updated dll again. Like the test site, I got the error, but everything worked. So I added an encryption key again and everything worked. However, when I rebooted the server (which i subsequently tried several times), I kept getting the above error, and (as expected) the encrypted data was not loaded. So reverted the production server to the "before I did anything" snapshot again until I figure this out.

So in summary, my test server works fine with .Net 4.5 even after restarts, but the production server lost the ability to read the encrypted data when the server was restarted.

Can anyone from AbleCommerce comment on whether you ran into this when making R8 compatible with .Net 4.5, or when working on the new PCI-compliant ParseConfigFile, and if so, how you solved it? Anyone else seen this, or know how to work around it?

Re: Encryption - Unable to validate data error

Posted: Mon Nov 02, 2015 2:12 pm
by Katie
Hi Jay,

We ran into a similar issue yesterday. I reported an issue and am waiting to hear back from the dev team.

I updated the bug report to reference this post.

Thanks for letting us know.

Katie

Re: Encryption - Unable to validate data error

Posted: Tue Nov 03, 2015 5:01 am
by jguengerich
After a little more research and testing, I think a static machineKey element needs to be set in the web.config file. It is easy to do in IIS Manager*. I removed encryption on our test server, set the machineKey, and restarted it. I got the error after reboot, but after encrypting again, I no longer get the error, even after restarts with or without encryption. However, since the test server was already working OK, I'm not ready to claim that this is the fix. I won't be able to test setting a static machineKey on our production server until tomorrow late afternoon.

* Select site on left, double-click Machine Key icon in the ASP.NET section, uncheck all 4 check boxes, click Generate Keys on right, click Apply on right. I left Encryption Method and Decryption Method at their defaults of SHA1 and Auto. I rebooted afterwards, I don't now if just restarting IIS would be OK.

Re: Encryption - Unable to validate data error

Posted: Wed Nov 04, 2015 5:07 am
by jguengerich
Update - The live server with a machineKey in web.config is OK with .Net 4.5 and the original ParseConfigFile code. I removed encryption and added the machineKey. After restarting, I got the error (as I expected), but after setting a new encryption key, I can now restart multiple times without error, whether I have encryption set or not. I will try it with the new ParseConfigFile code this weekend.

Re: Encryption - Unable to validate data error

Posted: Wed Nov 04, 2015 6:07 am
by mazhar
It seems like this happens when you have Load User Profile disabled on application pool. This setting seems related to IIS6 behavior and it makes sense since windows manged keys will be per user profile. Somehow when Load User Profile is disabled something seems changed which takes part in encryption/decryption process leading to invalid data error.

Re: Encryption - Unable to validate data error

Posted: Wed Nov 04, 2015 7:18 am
by jguengerich
Mazhar,

So are you recommending changing "Load User Profile" to True, and not using a machineKey in web.config? Have you tested this?

Both our test server and production server are using an application pool with "Load User Profile" set to False. The difference is the test server is using the DefaultAppPool that is set up by IIS, and the production server is using an application pool that I created. I suppose there may be some file permissions set for DefaultAppPool somewhere that are not set for the application pool I created.

Re: Encryption - Unable to validate data error

Posted: Thu Nov 05, 2015 4:38 am
by mazhar
jguengerich wrote:Mazhar,

So are you recommending changing "Load User Profile" to True, and not using a machineKey in web.config? Have you tested this?

Both our test server and production server are using an application pool with "Load User Profile" set to False. The difference is the test server is using the DefaultAppPool that is set up by IIS, and the production server is using an application pool that I created. I suppose there may be some file permissions set for DefaultAppPool somewhere that are not set for the application pool I created.
Hi Jay,

While trying to add extra level of protection for encryption key we tried to keep it simple by allowing windows to manage machine keys. Specifying machine key yourself is a fine solution and in some cases its the way to go for example in web farms/load balancing. If you want access encrypted data across multiple computers like using same encrypted file on both server and testing machine then you need a shared machine key in web.config.

Yes I have tried using custom machine key in web.config and it works. You have an interesting point that production has the profiles disabled but still working since you yourself created application pool. Somehow IIS maybe using your windows profile to store the generated keys data. You may check the process model identity value if its set to use ApplicationPoolIdentity or some other account.

Re: Encryption - Unable to validate data error

Posted: Thu Nov 05, 2015 12:48 pm
by jguengerich
Hi Mazhar,
You have an interesting point that production has the profiles disabled but still working since you yourself created application pool.
Actually it is the other way - the test server has the profiles disabled but it WAS still working, and I DID NOT create the application pool on the test server.
The production server has the profiles disabled and WAS NOT working, and I DID create the application pool on the production server.
Yes I have tried using custom machine key in web.config and it works.
Have you tried it with the Load User Profile set to True and no custom machine key?
You may check the process model identity value if its set to use ApplicationPoolIdentity or some other account.
Both servers have it set to ApplicationPoolIdentity.

Re: Encryption - Unable to validate data error

Posted: Wed Dec 16, 2015 3:23 am
by vsammons
Kate - Have you verified the appropriate fix for this? I ran into this in Gold R11 and encryption. We encrypted and used 3 days and then had issues where all of the setting were wiped out assuming the encryption failed at some point (or changed) not able to pull the encrypted setting from the database. Uncertain why it was fine and then on the 3rd day failed?
Unable to validate data.
Exception: Unable to validate data. Stack Trace: at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, Boolean useValidationSymAlgo, Boolean useLegacyMode, IVType ivType, Boolean signData) at System.Web.Security.MachineKey.Decode(String encodedData, MachineKeyProtection protectionOption) at CommerceBuilder.Configuration.EncryptionKeyManager.ParseConfigFile

Re: Encryption - Unable to validate data error

Posted: Wed Dec 16, 2015 5:00 am
by Katie
Kate - Have you verified the appropriate fix for this? I ran into this in Gold R11 and encryption. We encrypted and used 3 days and then had issues where all of the setting were wiped out assuming the encryption failed at some point (or changed) not able to pull the encrypted setting from the database. Uncertain why it was fine and then on the 3rd day failed?
Hi Vincent,

I just saw your ticket come in and I forwarded this to Mike and our lead developer. We are almost certain that this issue is only happening with our own hosted servers. We have spent many days (weeks?) trying to figure out why it's only being reported by a few clients on our own servers on no one else (using Gold R11) is having the problem. We have clients running R11 and hosted with us who are not having this problem, so it has been quite frustrating trying to figure out the cause.

We found the only way to fix the issue is to set a machine key in the web.config.

The application pool restarted on the 3rd day, and this is what caused the gateway info to be wiped out.

I am sorry for the trouble you had. It is something that has really got us all stumped here.

Thanks
Katie

Re: Encryption - Unable to validate data error

Posted: Mon May 02, 2016 9:42 am
by jmestep
Katie,
We just had the error happen on a site on Saturday. They are hosted at Rackspace and we upgraded them from R8 to R11 on Friday. At first the upgrade failed because they didn't have .net 4.5 on the server so that had to be installed. Then on Saturday, Rackspace ran windows updates on the server and there were quite a few .net updates and from looking in the logs, it appears that the problem happened when the server was restarted. (This is all per my conversation with our network guy doing the upgrade and looking into the server logs for me.) He was able to re-do the authorize net settings because on Friday he had set up a dev copy of the old R8 site so I could do some testing and he got the information from that site, so there were only a few problem orders. I think he said the authorize.net settings had been wiped out. Here was the error- I can't download the app.log to see the rest of the message.
He said the the site was running under NetworkService, the LoadUserProfile was set to false.
There was no machineKey in the web.config folder.

Code: Select all

Unable to validate data. 
Exception: Unable to validate data. Stack Trace: at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, Boolean useValidationSymAlgo, Boolean useLegacyMode, IVType ivType, Boolean signData) at System.Web.Security.MachineKey.Decode(String encodedData, MachineKeyProtection protectionOption) at CommerceBuilder.Configuration.EncryptionKeyManager.a()

Re: Encryption - Unable to validate data error

Posted: Mon May 02, 2016 1:36 pm
by Katie
Hello Judy,

It happened to me once when my OS was upgraded. That was the only time though. We found the only way around it is to set a machine key in the web.config.

We've spent weeks trying to determine what causes the problem, but no luck.

Thanks
Katie

Re: Encryption - Unable to validate data error

Posted: Mon May 02, 2016 11:57 pm
by jmestep
Thanks, Katie. I just wanted to report another instance of the problem in case you needed to know. We are setting a machineKey.

Re: Encryption - Unable to validate data error

Posted: Tue May 03, 2016 2:43 am
by Katie
Thanks Judy. The more information we can accumulate will help us figure out the cause.

Re: Encryption - Unable to validate data error

Posted: Tue May 03, 2016 9:41 am
by Katie
Judy,

I was reviewing the upgrade instructions and noticed that we require the LoadUserProfile to be true instead of false. Maybe this was the reason that the gateway settings disappeared...
7) Post-Upgrade Instructions and Information

IMPORTANT INFORMATION REGARDING NEW ENCRYPTION

If you experience a problem with the (payment, tax, or shipping) gateway configuration
data disappearing after the server is restarted or application pool recycled, then
follow one of the two remedies below.

The extra protection code for encryption key requires the Load User Profile setting
to be enabled for the Application Pool. In order to solve above problem either this
setting must be enabled or one must specify a machine key in web.config
I'll let you know that we had one site where this setting didn't seem to make a difference. So, there may be multiple issues at play here. Anyway, I just wanted to pass along this info in case it happened again.

Thanks
Katie

Re: Encryption - Unable to validate data error

Posted: Tue May 03, 2016 11:54 pm
by jmestep
Thanks, Katie. I will point this out to the network person who ran the upgrade.

Re: Encryption - Unable to validate data error

Posted: Wed Dec 14, 2016 11:44 am
by nethrelm
I've been trying to figure this issue out myself for quite some time, and I found a tangentially related bug when attempting to remove or change the encryption key. The algorithm used in the RecryptionHelper class to perform the actual data recryption operation (InternalRecryptDatabase method) is flawed. Here is that method's code:

Code: Select all

        private static void InternalRecryptDatabase(int storeId, byte[] oldKey, byte[] newKey)
        {
            // WE MUST INITIALIZE THE STORE CONTEXT AS THIS THREAD HAS NO HTTPCONTEXT
            Store store = AbleContext.Resolve<IStoreRepository>().Load(storeId);
            if (store != null)
            {
                // INITIALIZE THE TOKEN WITH THE STORE CONTEXT
                AbleContext.Current.Store = store;

                // PROCESS RECORDS IN BATCHES OF 100
                int lastCount = 0;
                int count = RecryptionHelper.GetRecryptionWorkload();
                while ((count > 0) && (count != lastCount))
                {
                    List<RecryptRecord> records = RecryptionHelper.LoadForRecrypt(100);
                    foreach (RecryptRecord record in records)
                    {
                        record.DoRecrypt(oldKey, newKey);
                    }

                    // if lastCount and count ever match, it means nothing changed in the last iteration
                    // keep track of this to prevent endless looping in case a recrypt operation fails
                    lastCount = count;
                    count = RecryptionHelper.GetRecryptionWorkload();
                }

                // REMOVE RECRYPT FLAG
                RecryptionHelper.SetRecryptionFlag(false);
            }
        }
It only does a maximum of 100 records from each of the four tables (ac_Payments, ac_PaymentGateways, ac_ShipGateways, and ac_TaxGateways) and then "completes" because the while loop evaluates to false after one loop. This will generally result in all the gateway config data being properly recrypted, but only 100 payment records since they will make up the bulk of the encrypted data.

Now, the reason the while loop fails after one loop is that the GetRecryptionWorkload method returns the sum total of records from the four tables where the ReCrypt field is true. This field is set to true for all the records in those tables by an earlier call to RecryptionHelper.SetEncryptionFlag(true) -- this occurs in the EncryptionKeyManager.SetEncryptionKey method, but that is not important to this bug.

The problem is that as the while loop executes and it calls DoRecrypt on each RecryptRecord, the ReCrypt field remains true after returning from DoRecrypt, so the count returned by GetRecryptionWorkload at the end of the loop does not change from the previous call, therefore count will have the same value as lastCount, and the while statement evaluates to false. It is at this point that the ReCrypt field is set back to false for all the records, whether they were actually recrypted or not.

There is also an inconsistency in the criteria for querying the records. RecryptionHelper.LoadForRecrypt only loads the records for the current Store, whereas all the other methods that query or modify these tables (EstimateRecryptionWorkload, GetRecryptionWorkload, and SetRecryptionFlag) do not specify the Store.Id as part of the criteria. Hopefully I have provided enough information for you to locate and resolve these issues. It is important to me that they be fixed promptly. Please let me know when a patch becomes available. Thanks!

Re: Encryption - Unable to validate data error

Posted: Sat Dec 17, 2016 8:11 am
by Katie
Which version and build number are you using?

Thanks for this information.

Katie

Re: Encryption - Unable to validate data error

Posted: Mon Dec 19, 2016 6:04 am
by nethrelm
GoldR12SR1 (build 9266)

Re: Encryption - Unable to validate data error

Posted: Tue Dec 20, 2016 2:07 am
by Katie
Thank you. We are looking into this and will report back soon.

Katie

Re: Encryption - Unable to validate data error

Posted: Tue Dec 20, 2016 9:29 am
by Katie
Hello,

The issue has been reported and the developer provided a fix that you can implement. Be sure to take a backup before making changes.

Code: Select all

/// <summary>
            /// Updates the encrypted data for a record being re-encrypted
            /// </summary>
            /// <param name="tableName">Name of the table to update</param>
            /// <param name="primaryKey">Primary key ID for the record to be updated</param>
            /// <param name="encryptedData">Re-encrypted data to save for the specfied record</param>
            /// <returns>True if the update is successful</returns>
            private static bool SaveRecryptedData(string tableName, int primaryKey, string encryptedData)
            {
                // DETERMINE APPROPRIATE FIELD NAMES
                int affected = 0;
                switch (tableName)
                {
                    case "ac_Payments":
                        affected = NHibernateHelper.CreateSQLQuery("update ac_Payments set EncryptedAccountData = :encryptedData, ReCrypt = :reCrypt where PaymentId = :primaryKey")
                            .SetString("encryptedData", encryptedData)
                            .SetBoolean("reCrypt", false)
                            .SetInt32("primaryKey", primaryKey)
                            .ExecuteUpdate();
                        break;
                    case "ac_PaymentGateways":
                        affected = NHibernateHelper.CreateSQLQuery("update ac_PaymentGateways set ConfigData = :encryptedData, ReCrypt = :reCrypt where PaymentGatewayId = :primaryKey")
                            .SetString("encryptedData", encryptedData)
                            .SetBoolean("reCrypt", false)
                            .SetInt32("primaryKey", primaryKey)
                            .ExecuteUpdate();
                        break;
                    case "ac_ShipGateways":
                        affected = NHibernateHelper.CreateSQLQuery("update ac_ShipGateways set ConfigData = :encryptedData, ReCrypt = :reCrypt  where ShipGatewayId = :primaryKey")
                            .SetString("encryptedData", encryptedData)
                            .SetBoolean("reCrypt", false)
                            .SetInt32("primaryKey", primaryKey)
                            .ExecuteUpdate();
                        break;
                    case "ac_TaxGateways":
                        affected = NHibernateHelper.CreateSQLQuery("update ac_TaxGateways set ConfigData = :encryptedData, ReCrypt = :reCrypt  where TaxGatewayId = :primaryKey")
                            .SetString("encryptedData", encryptedData)
                            .SetBoolean("reCrypt", false)
                            .SetInt32("primaryKey", primaryKey)
                            .ExecuteUpdate();
                        break;
                    default:
                        throw new ArgumentException("tableName must be one of these values: ac_Payments, ac_PaymentGateways, ac_ShipGateways, ac_TaxGateways", "tableName");
                }

                return affected == 1;
            }
We believe that this will only be a problem if 1), a store gets more than 100 orders within 30-60 days and 2), happens to change the encryption key during this time period, and 3), the merchant has a need to view the cc account information.

If you have any questions or concerns, please let us know.

Katie

Re: Encryption - Unable to validate data error

Posted: Mon Dec 26, 2016 6:22 am
by nethrelm
While I certainly do appreciate the fix, I only keep the source code for reference (exactly for circumstances like this). I never make changes to it. I used to in the pre-Gold version and found myself changing way too many things to the point that there were irreconcilable differences between my version of the code and yours. To make sure this would not happen again, when we upgraded to Gold I stopped making changes to the CommerceBuilder source code. I need to get an official patch from you when available. Also, in reference to your points:

1) Yes, did you not expect people using your software to do that amount of business? We process significantly more than 100 orders a month.
2) Given #1, we are always within this period, so we can't ever change the encryption key until the issue is resolved.
3) It's not just about viewing it. It's also a problem for capturing cc payments that were only authorized during checkout (which is the way we do it; we capture when orders are ready to ship).

Re: Encryption - Unable to validate data error

Posted: Wed Dec 28, 2016 6:22 am
by Katie
Hello,

I've requested a patch. Our development team takes this week off, so the soonest I can expect anything after the first of the year.

I was posting the conditions to reproduce in case anyone else has seen this issue before. I didn't mean to imply that no one gets more than 100 orders each month. We have customers who get that many each day! It's just an unusual circumstance for all conditions to happen. We suspect this bug has been there for years.

Thanks,
Katie

Re: Encryption - Unable to validate data error

Posted: Tue Jan 03, 2017 11:41 am
by egormsen
We are also seeing this error.

We are on Gold R12 sr1.

Re: Encryption - Unable to validate data error

Posted: Tue Jan 03, 2017 7:39 pm
by gdelorey@mitcs.com
Hi Katie -

We are seeing this issue as well as of the past few days. We are on VERSION: 7.0.91.8643.

Thanks,
Greg