ShipStation Sync notifying wrong customers when sync run

For general questions and discussions specific to the AbleCommerce GOLD ASP.Net shopping cart software.
Post Reply
User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Thu Sep 28, 2017 11:48 pm

Gold R12

A large client is implementing ShipStation and immediately encountered a pretty big problem. Pushing orders/shipments into ShipStation is working great. But when they initiate the sync to pull shipment info back into Able, the sync routine creates a major problem. The sync posted the same, single Able user as the author of the final order note showing the order status has changed to 'Completed'.

It also triggered the 'Order Shipped' email template to the same, single Able user. So one shopper got all 32 'Order Shipped' emails even though they had only placed 1 order. The other 31 shoppers got no email at all. That's really bad.

ShipStation setup is pretty basic on the Able side of things, so we don't think we've got the set up wrong. And it's not doing it on every single order, hundreds of other orders all updated property when Sync was run. We have not yet found a pattern to narrow the issue down.

Any other reports of this? Any ideas where to look?

We're getting a Sandbox account set up for us so we can do more testing. Until then we just can't trust it enough to use it.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
Katie
AbleCommerce Admin
AbleCommerce Admin
Posts: 2651
Joined: Tue Dec 02, 2003 1:54 am
Contact:

Re: ShipStation Sync notifying wrong customers when sync run

Post by Katie » Fri Sep 29, 2017 1:09 am

Hi Joe,

Well, you are right - this is a major problem, but I've never heard of anyone else reporting it. ShipStation is one of our more popular features, so I would think that something like this would have been found. However, one thing that I know to be true is that merchants use AbleCommerce in ways that I would never imagine. That can lead to things we never thought about testing. What is most concerning is that it's not doing it every time. That makes it super hard to track down, unfortunately...as you know.

Maybe check to see if anything looks strange with custom order statuses?

Let me know if you need any help,

Katie
Thank you for choosing AbleCommerce!

http://help.ablecommerce.com - product support
http://wiki.ablecommerce.com - developer support

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Fri Sep 29, 2017 1:32 am

Katie wrote:ShipStation is one of our more popular features, so I would think that something like this would have been found
I agree. Since it's not every single order, it's either something incredibly obscure or it's being used in an unanticipated way.

One pattern I noticed this morning is vast majority of orders involved were placed via admin, not front-end. We're digging more into it today, maybe I'll have more information for you later on.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Tue Oct 03, 2017 3:35 am

So I managed to set up a semi-sandbox account for ShipStation testing. I did a single test order transfer, shipped it, and then sync'd back to Able.

Here's what's happening: The 'Author' (UserId) value did not post when the transfer to ShipStation updated the order status for me. UserId is null for that specific order note.

On the live site, we're seeing this happen on the order note "Status changed from <blah><blah> to Completed" that is generated when the SS sync routine marks the order as shipped.

Bizarre.
Capture.JPG
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Tue Oct 03, 2017 7:54 am

Think I found the problem:

The call that updates the order status (and subsequently triggers the order note to be created) is made from within ShipStationExportManager.cs:

Code: Select all

if (importMap.Count > 0)
                                {
                                    imported = true;
                                    if ((exportOptions.NewOrderStatusId ?? 0) > 0)
                                    {
                                        order.UpdateOrderStatus(exportOptions.NewOrderStatusId.Value);
                                    }

                                    this.ExportInfo.ItemsExported += 1;
                                }
Eventually the calls reach the OrderStatusService.cs which generates the actual order note record and saves it to the order:

Code: Select all

            // CHECK IF ORDER STATUS IS CHANGING FROM ONE TO ANOTHER
            if (!string.IsNullOrEmpty(originalStatusName) && originalStatusName != newStatus.Name)
            {
                // UPDATE ORDER NOTES
                order.Notes.Add(new OrderNote(order, AbleContext.Current.User, LocaleHelper.LocalNow, string.Format(LocalizedStrings.OrderStatusUpdated, originalStatusName, newStatus.Name), NoteType.SystemPrivate));
                order.Notes.Save();

                // TRIGGER ORDER STATUS EVENT
                StoreEventEngine.OrderStatusUpdated(order, originalStatusName);
            }
This code assumes AbleContext.Current.User has been set, because that's what it's using as the author of the order note being created.

The problem is the context of this particular call is from a separate ASP.Net thread invoked from the Export() method of the main ShipstationExportManager class. User context does not exist in that separate thread. So AbleContext.Current.User is always null.

This only happens if you have set the Update Order Status dropdown on the Publish page for ShipStation. Obviously no order note would get generated if order status is not required to be updated.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
Katie
AbleCommerce Admin
AbleCommerce Admin
Posts: 2651
Joined: Tue Dec 02, 2003 1:54 am
Contact:

Re: ShipStation Sync notifying wrong customers when sync run

Post by Katie » Thu Oct 05, 2017 6:03 am

This only happens if you have set the Update Order Status dropdown on the Publish page for ShipStation. Obviously no order note would get generated if order status is not required to be updated.
Hi Joe. Are you saying that the "Update Order Status" option had a blank value which caused the order note to have a null value for author?

Thanks,
Katie
Thank you for choosing AbleCommerce!

http://help.ablecommerce.com - product support
http://wiki.ablecommerce.com - developer support

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Thu Oct 05, 2017 7:55 am

No.

We believe we have three issues and one concern. I'll detail them here.

Issues:
1) Order-shipped email templates are being triggered to the wrong shopper. Say we run 50 orders through ShipStation in a single batch. When we fire off the SYNC, Able will update each shipment as shipped.
That fires the order-shipped trigger, which in turn fires off the associated email template. The TO address of each email template fired off should be the email address of the shopper who placed the respective order. Intermittently this does not happen: ALL of the order-shipped emails go out to the exact same, single shopper email. None of the other shopper emails in that batch receive the order-shipped email. Needless to say the one shipper is quite pissed and the rest are still wondering when their order will ship...

2) When the PUBLISH is told to update the order status after it pushes the order to SS, an order note is generated for each order transferred to SS. The Author of that order note is blank. When looking at those order notes in the SQL table, the UserId value is null for those notes.

3) Same as #2, except we're seeing the Author as a GUID. No email address. Not blank. Not null. It's a GUID. Like somehow the Author of the note got set to a Guest user record.

Concern:
1) The ShipStation API documentation states that it throttles API calls to the rate of 40 per minute. There is no error-handling within the ShipStation communication code that accommodates for the throttle limit being hit. SS includes the remaining requests available in the current throttle count as an HTTP header value with every response they send back. This needs to be parsed to ensure proper error handling if the throttle limit is actually hit.
Clearly we are pushing much more than 40 requests per minute. So whatever their throttle limit is, it's not 40. We've got a call into the rep to find out what the real limit is so we know what to expect. Right now, there's no way to know IF we hit the limit or how close we are to hitting that limit.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
Katie
AbleCommerce Admin
AbleCommerce Admin
Posts: 2651
Joined: Tue Dec 02, 2003 1:54 am
Contact:

Re: ShipStation Sync notifying wrong customers when sync run

Post by Katie » Thu Oct 05, 2017 11:32 pm

Hi Joe,

I asked a developer to look at the email and order note issues. Please let us know if Shipstation is truly throttling the API calls. That doesn't sound good either.

Thanks
Katie
Thank you for choosing AbleCommerce!

http://help.ablecommerce.com - product support
http://wiki.ablecommerce.com - developer support

nadeem
Captain (CAPT)
Captain (CAPT)
Posts: 258
Joined: Tue Jul 31, 2012 7:23 pm

Re: ShipStation Sync notifying wrong customers when sync run

Post by nadeem » Fri Oct 06, 2017 3:56 am

Hi Joe,

Yes, order note author is null and this is because the export and sync statuses operation is asynchronous which is initiated by a separate thread. To fix the issues, we can set the current user from the context in async calls for export and sync statues.

For making the required updates, please open CommerceBuilder/DataExchange/ShipStationExportManager.cs, locate the following code:

Code: Select all

/// <summary>
        /// Provides the ability to call export in asynchronusly
        /// </summary>
        /// <param name="exportOptions">Export options</param>
        private delegate void AsyncExport(ShipStationExportOptions exportOptions);

        /// <summary>
        /// Provides the ability to call status sync asynchronusly
        /// </summary>
        private delegate void AsyncStatusSync();
and replace with

Code: Select all

/// <summary>
        /// Provides the ability to call export in asynchronusly
        /// </summary>
        /// <param name="exportOptions">Export options</param>
        /// <param name="userId">Current User Id</param>
        private delegate void AsyncExport(ShipStationExportOptions exportOptions, int userId);

        /// <summary>
        /// Provides the ability to call status sync asynchronusly
        /// </summary>
        /// <param name="userId">Current User Id</param>
        private delegate void AsyncStatusSync(int userId);
Now locate the following line of code inside Export function,

Code: Select all

IAsyncResult result = caller.BeginInvoke(exportOptions, new AsyncCallback(OnExportComplete), null);
and replace with

Code: Select all

IAsyncResult result = caller.BeginInvoke(exportOptions, AbleContext.Current.UserId, new AsyncCallback(OnExportComplete), null);
Similarly, locate the following line of code inside SyncStatuses() function

Code: Select all

IAsyncResult result = caller.BeginInvoke(new AsyncCallback(OnSyncComplete), null);
and replace with

Code: Select all

IAsyncResult result = caller.BeginInvoke(AbleContext.Current.UserId, new AsyncCallback(OnSyncComplete), null);
After setting the above parameters, update the DoExport function

From

Code: Select all

private void DoExport(ShipStationExportOptions exportOptions) 
        {
            try
            {
To

Code: Select all

private void DoExport(ShipStationExportOptions exportOptions, int userId) 
        {
            try
            {
                if (AbleContext.Current.User == null)
                    AbleContext.Current.User = Users.UserDataSource.Load(userId);
Also update DoStatusSync function

From

Code: Select all

private void DoStatusSync()
        {
            this.IsSyncing = true;
            try
            {
To

Code: Select all

private void DoStatusSync(int userId)
        {
            this.IsSyncing = true;
            try
            {
                if (AbleContext.Current.User == null)
                    AbleContext.Current.User = Users.UserDataSource.Load(userId);
Now open CommerceBuilder/Eventing/DefaultEventsHandler.cs locate the following line inside OrderShipped event

Code: Select all

OrderNote note = new OrderNote(order, order.User, LocaleHelper.LocalNow, LocalizedStrings.OrderShipped, NoteType.SystemPrivate);
and update with

Code: Select all

var user = AbleContext.Current.User;
if (user == null) user = order.User;
OrderNote note = new OrderNote(order, user, LocaleHelper.LocalNow, LocalizedStrings.OrderShipped, NoteType.SystemPrivate);

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Tue Oct 10, 2017 6:45 am

Thanks Nadeem. We essentially did the exact same change (which makes me feel really good!). We did not update the DeafultEventsHandler though, I'll take a look at that later this week.

We did find another bug in ShipStation:
If a product is deleted from the catalog before Shipstation can push an order item associated to that product, a null exception is thrown during the ShipStation push. This is because there is no check on oi.Product != null before attempting to identify the warehouse of the product or the product dimensions (height, length, width).

Line 105

Code: Select all

                var warehouse = oi.Product.Warehouse != null ? oi.Product.Warehouse : shipment.Warehouse;
                item.warehouseLocation = warehouse.Name;

                request.dimensions = new Dimensions();
                request.dimensions.units = DimensionUnits.inches.ToString();
                if (oi.Product.Length > 0)
                    request.dimensions.length = oi.Product.Length;
                if (oi.Product.Width > 0)
                    request.dimensions.width = oi.Product.Width;
                if (oi.Product.Height > 0)
                    request.dimensions.height = oi.Product.Height;

Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

nadeem
Captain (CAPT)
Captain (CAPT)
Posts: 258
Joined: Tue Jul 31, 2012 7:23 pm

Re: ShipStation Sync notifying wrong customers when sync run

Post by nadeem » Wed Oct 11, 2017 12:57 am

You are right about the null exception. Thanks for pointing that out. You can wrap the order item dimensions inside oi.Product != null check. Also apply the same null check before setting the product warehouse.

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Fri Oct 13, 2017 2:11 am

So it's been nearly a week now and we haven't seen the original issue of one shopper getting everyone's shipment notification. So the fix for that seems to be the solution.

The other issues are gone as well thanks to the fixes posted here.

We are seeing one additional issue when a variant is deleted (but the parent product remains) prior to pushing to ShipStation. It's throwing an exception. It's rare for that to happen, so we'll probably leave that one for next week.

Overall the client is super pleased with the ShipStation integration. Still haven't gotten a definitive answer from SS regarding the API throttle limit. I can tell you we're pushing 100-200 orders per transfer, so if there's a limit then it must be a whole lot higher than 40 calls per minute.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Wed Nov 01, 2017 2:43 am

We have confirmed there is also an issue with variants. If the variant is deleted from the product variant matrix before the order is pushed to ShipStation, an exception is thrown during the push and the order will not push.

The code is trying to locate each option choice using the original optionChoiceId found in the order lineitem OptionList field. But there's no null check if the OptionChoice fails to load.

Code: Select all

                            foreach (int choiceId in choiceIds)
                            {
                                var choice = OptionChoiceDataSource.Load(choiceId);
                                ItemOption io = new ItemOption();
                                io.name = choice.Option.Name;
                                io.value = choice.Name;
                                item.options.Add(io);

                                // SS ONLY SUPPORTS MAXIMUM OF 10 OPTIONS
                                if (item.options.Count == 10)
                                    break;
                            }
I get what it's trying to do. You need to load up the original option name and the specific choice made by the shopper because this is displayed in ShipStation with each line item.

What I don't know is a feasible solution. My only idea is to add a null check on the choice variable, and if true then populate io.name with something generic like "Variant" and io.value with the orderItem.VariantName value. Not ideal, but it would work. Technically, the original option is lost. It's deleted. So there's not much that can be done at this point in the process.
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
Katie
AbleCommerce Admin
AbleCommerce Admin
Posts: 2651
Joined: Tue Dec 02, 2003 1:54 am
Contact:

Re: ShipStation Sync notifying wrong customers when sync run

Post by Katie » Wed Nov 01, 2017 9:07 am

Hi Joe,

If you have time, I wanted to pick your brain on the email issue below:
1) Order-shipped email templates are being triggered to the wrong shopper. Say we run 50 orders through ShipStation in a single batch. When we fire off the SYNC, Able will update each shipment as shipped.
That fires the order-shipped trigger, which in turn fires off the associated email template. The TO address of each email template fired off should be the email address of the shopper who placed the respective order. Intermittently this does not happen: ALL of the order-shipped emails go out to the exact same, single shopper email. None of the other shopper emails in that batch receive the order-shipped email.
I have been trying to reproduce this issue all afternoon with no luck. I am working in batches of 15-20 orders and no matter what I try, the shipped emails are always sent to the correct user.

Other than that, I have no idea how to reproduce the issue. What I can't figure out is why, after the code changes, it is working now. Why did fixing the order note author also fix the email issue....or did it?

So...my theory is that the store suffering this issue might have custom order statuses. And perhaps one of those statuses is triggering an email that isn't configured properly. Maybe this isn't 'intermittent' at all, and the person who exports the orders had selected the wrong order status with the 're-sync order status' option, instead of using the default setting, which automatically uses the Completed order status, which in turn triggers the (default) Shipped email. Do you think that might be a possibility?

Anyway, I can ask the dev team tomorrow if you are busy. It is just odd to run across something that is fixed but never reproduced in the first place.

Thanks
Katie
Thank you for choosing AbleCommerce!

http://help.ablecommerce.com - product support
http://wiki.ablecommerce.com - developer support

User avatar
Katie
AbleCommerce Admin
AbleCommerce Admin
Posts: 2651
Joined: Tue Dec 02, 2003 1:54 am
Contact:

Re: ShipStation Sync notifying wrong customers when sync run

Post by Katie » Thu Nov 02, 2017 12:16 am

So the code fixes shown above having nothing to do with the sending of email. I just confirmed with Nadeem. He also tried to reproduce the email issue and couldn't. We are both thinking that the order statuses still might be the culprit.

If you can, please let me know if this is possible. We are wrapping up the final testing on Gold R12 SR2, and this is one of the last remaining issues that I need to confirm.

At the very least, I will rename the bug report title and confirm only the changes above. We'll have to assume the email issue as invalid.

Thanks Joe. I know we're all super busy this time of year, so if you haven't heard more about the email problem, then maybe we should just let it go.

Katie
Thank you for choosing AbleCommerce!

http://help.ablecommerce.com - product support
http://wiki.ablecommerce.com - developer support

User avatar
AbleMods
Master Yoda
Master Yoda
Posts: 5170
Joined: Wed Sep 26, 2007 5:47 am
Location: Fort Myers, Florida USA

Re: ShipStation Sync notifying wrong customers when sync run

Post by AbleMods » Thu Nov 02, 2017 2:47 am

Katie wrote:I have been trying to reproduce this issue all afternoon with no luck. I am working in batches of 15-20 orders and no matter what I try, the shipped emails are always sent to the correct user.
This was the one issue I couldn't reproduce locally either. It only happened on the live website. I can tell you the client stopped doing large batches of orders ( > 100 each push) and switched to doing only 40 orders per push. This change in their procedure occurred within a few days of when I implemented the other fixes, thus making it difficult to tell what exactly resolved this particular issue. But, that shouldn't have impacted the sync back into Able triggering the shipment emails....they're two entirely different routines. So logically I don't see any sort of connection between the procedure change and email issue.

I checked triggers and didn't find anything suspect. There's only one email template assigned to the Order-Shipped trigger. They hardly use triggers at all, only 5 are configured.

I think at this point there's no effective means to troubleshoot the issue further. It's been weeks since the issue was last seen. If you're at least setting the Context.User when the push or sync thread starts, I'd leave it at that for now and wait for more information.

Sometimes, there just isn't enough detail available to prove or disprove a specific bug. I think this is one of those cases.

Me personally, my gut tells me it was setting the Context.User at the start of the threads. But I can't back that up with much more than "Well, it hasn't happened since I did that" and that's not quite good enough for me to make my stand :D
Joe Payne
AbleCommerce Custom Programming and Modules http://www.AbleMods.com/
AbleCommerce Hosting http://www.AbleModsHosting.com/
Precise Fishing and Hunting Time Tables http://www.Solunar.com

User avatar
Katie
AbleCommerce Admin
AbleCommerce Admin
Posts: 2651
Joined: Tue Dec 02, 2003 1:54 am
Contact:

Re: ShipStation Sync notifying wrong customers when sync run

Post by Katie » Thu Nov 02, 2017 5:30 am

I did enough testing that I feel pretty good there is no email problem.

Anyway, thanks for getting back to me. I appreciate it.

Katie
Thank you for choosing AbleCommerce!

http://help.ablecommerce.com - product support
http://wiki.ablecommerce.com - developer support

Post Reply