12/1 - AbleCommerce Data Client Utility 7.0

[mkeith1]

In the next dataport revision, would it be possible to add a new field to the import? This new "Product warning" field which i added as a true/false value to a product is not a field included in the current dataport.

Also, while on the subject of adding fields to the dataport, lets say I wanted to add a new text field to a product, which may be available to use on all products, but isn't required (Like the "name" field is), this field would be more like the "Model Number" field, but would be called something such as "ISBN", or "UPC". or whatever.

Making these new fields available to records would be done I assume by using The same process of adding the Product Warning field (making conlib controls for the new fields, then editing the product details page. Entering information to these fields make them viewable on this page to the customer. (I'm not talking about creating a product template which could be be associated to the product). These new fields would not be searchable, because the information in them could be also put in the "kewords" field, which is searchable. They just need to be viewable by the customer.

Is it easy to integrate those new fields in the dataport?

Obviously not everyone out there would have a use for additional fields, but is it possible to easily modify the dataport to accomodate this?

[Naveed]

It is not possible to add custom fields in DataPort xml/csv import & export it is because each customer requirements are different. The fields you want to be included will never be used by anyone else, because they may have their own unique requirements.

I hope you understand...

[mfreeze]

I downloaded 7.0.34 DataPort and attempted to upload a ColdFusion 5.5 store export file. I repeatedly received the following message and nothing was uploaded.

Authentication Failed. An unknow error occured while processing request. Exception Message:Index and length must refer to a location within the string.
Parameter name: length
at
CommerceBuilder.DataClient.piACRequestHandler.ProcessRequest(String requestXml)
at ClientApiService.ProcessRequest(HttpContext.context) in d:\ac7_audiotech\ClientApi\ClientApiService.ashx:line39

As was reported in an earlier post for dataport 31, I have uploaded an export from the same store with dataport 19 but it fails in 34.

I attempted to upload the XML file but the upload timed out. I can send via email if you need it.

[Naveed]

I downloaded 7.0.34 DataPort and attempted to upload a ColdFusion 5.5 store export file. I repeatedly received the following message and nothing was uploaded.

Strange!
Same error was reported earlier for .31. And also Judy sent me some sample data. However later on Judy reported that it is working fine with DataPort 7.0.34.

Anyhow I will try it again and if reproduced will be fixed for next release.

[jmestep]

I had tested the Dataport .34 with a CF file export on a site that had Able 7.1 and it didn't work there. When I tried it on the real site on Able 7.0, it did work.

[Naveed]

I downloaded 7.0.34 DataPort and attempted to upload a ColdFusion 5.5 store export file. I repeatedly received the following message and nothing was uploaded.

I tested your provided CFMX exported xml file using DataPort vs. 7.0.34 and the import was successful without any such warnings.

Please keep in mind that DataPort 7.0.34 works only for AbleCommerce 7.0 stores. It will not work for 7.1 stores....

[Naveed]

Hello Mary,

I have found from your store "/Admin/Help/About.aspx" page that your store is not properly upgraded. Build number of your store is (10125), which is quite old and DataPort 7.0.34 is not compatible with that.
Here you can find more about 7.0 release history:
http://wiki.ablecommerce.com/index.php/Release_History

To use latest versions of DataPort like v. 7.0.34 you have to upgrade your 7.0 store to SR2. However older versions of DataPort can work with your non-updated stores, but it is strongly recommended to use latest version of DataPort and to update your stores properly. Please have a look at the following link to find how to upgrade your stores to 7.0 SR2.
http://help.ablecommerce.com/upgrades/A ... ce_7.0.htm

Also please keep in mind DataPort 7.0.34 and all other older versions of DataPort will not work for 7.1 stores. DataPort version for 7.1 stores is under QA currently and will be released very soon.

[mfreeze]

I upgraded to SR2 and the import worked.

When a new dataport is released, would it be possible to post what releases/service levels it will work with. I now have 24 AC7 sites and not all have been upgraded to SR2 due to customizations. It would be helpful to know what level of AC7 each version of the dataport will work with.

[Naveed]

Hello Mary,

Yes, you are right... I will try to post these details.

However it is recommended to upgrade your websites to 7.0 SR2, and use latest version of DataPort available for 7.0 stores. It is because the older versions were beta versions and may have serious bugs and other issues like limited features..

[mfreeze]

I posted the following in the forum post Upgrade from 5.5 to 7.0 where I found a question about the dataport and custom fields but am moving it to here since it seems to be the proper place for dataport questions.

I also need the capability to import 5.5 custom fields into 7.0 template fields. I have modified the Books template to contain all the custom fields I need for my products but they do not seem to be handled by the import. Does the dataport (7.0.34) have this capability? If not, when might we expect to see this?

I can't move forward on this conversion project until I have a resolution to the custom field issue. Is there any intent to include this capability in the dataport or do I need to write a script myself to transfer the information?

[mkeith1]

is dataport 7.0.34.0 compatible with version 7 build 10125?

I've been using this version of the dataport since it was released, and have recently found that a number of orders have been received with no products. all of these orders are for products which have image alt fields for .jpg, _t.jpg and _i.jpg which are longer than 100 characters.

Once i've determined the products which should have been in these orders, I've attempted to edit the order manually and add these products. If any of these products have more than 100 characters in the image alt fields, i get an error when adding them until i manually edit the product and shorten these fields. (If there are multiple products in the order, they are all dropped from the order upon checkout because of the bad item, as well.) Once the item's altimage field's are shortened to less than 100 characters, then you can add it back to the order.

I've done an export of the store and searched for products which have this problem, and cut the image alt fields to less than 100 characters, then reimported them, but still see errors after the import. After the import I've manually edited many of the products which incurred this error, but am now finding that when i manually edit the products, the Name field can not be edited to more than 55 characters long. I've added products manually, and don't recall the name field being this short before.

So, for clarification, of the name and image alt fields, which fields can be 255 characters, and which must be only 100, and what's with the name field being only 55 characters?

[Naveed]

So, for clarification, of the name and image alt fields, which fields can be 255 characters, and which must be only 100, and what's with the name field being only 55 characters?

DataPort 7.0.34 is compatible with 7.0 stores. You can find more information about database fields at following links:
http://wiki.ablecommerce.com/index.php/Database

Here is the Database schema for 7.0 stores:
http://wiki.ablecommerce.com/images/3/3 ... Schema.pdf

[jmestep]

Mike- there was a bug in older versions of Able where some of the field sizes were wrong- they were 100 instead of 255. You might need to change those manually- I can't remember if an upgraded did it automatically. I know the image alt tags was one of the fields and the product name. The alt tags pick up the product name automatically so if the product name is set at 255 and you go over 100 characters, that would cause a problem with the alt tags.

[mkeith1]

ok, but why when i'm manually entering products does the name field only accept 55 characters?

and also

I have found from your store "/Admin/Help/About.aspx" page that your store is not properly upgraded. Build number of your store is (10125), which is quite old and DataPort 7.0.34 is not compatible with that.

this is per Naveed in an earlier post.

So the current dataport will not work correctly with 10125 right?

[jmestep]

I don't remember back to that build. I think at one time the Name took 255 characters, but the alt tags field took only 100. I just did a search and found this post relating to the field size:
viewtopic.php?f=42&t=8556&hilit=field+size

[mfreeze]

Naveed,
Back in December, you helped me with an import of an Able Commerce 5.5 store when the import was failing. It turned out that I needed to upgrade to SR2 and I did that. However, even though the import now functions, I am getting an excessive number of errors including truncations in the product name and the digital goods name. I have changed all field lengths I can find but the errors still occur. Is there any documentation regarding what fields have shorter lengths in 7.0 than in 5.5?

I really need to get this import to work. This customer has been waiting for almost a year to convert to 7.0 and get off unsupported CFMX 5.5. He has many ESD files so we had to wait for the import to support them but now that it does, I still can't get a clean import. I have all these errors plus the fact that the Dataport doesn't seem to handle custom fields.

I am attaching the complete log with a list of all the errors at the bottom. Please help. I can send you the export file and give you access to the site if you need it; I'll do whatever needs done to get this imported successfully.

Complete Import Log.txt

[Naveed]

Hello Mary E Freeze,

I will examine the log file you attached. However can you please let me know that your store is properly upgraded to 7.0 SR2?

Meanwhile, You can email me the AC55 xml file.

[mfreeze]

I just sent you a zipped version of the file. I have deleted and reinstalled the store to ensure a clean version at SR2. Below is the data from the 'about able commerce' in the admin.

PLATFORM: ASP.NET
VERSION: 7.0
BUILD: 10863

AjaxControlToolkit: 1.0.10301.0
CommerceBuilder: 7.0.10752.0
CommerceBuilder.AuthorizeNet: 7.0.9764.0
CommerceBuilder.CanadaPost: 7.0.0.0
CommerceBuilder.CCH: 7.0.0.0
CommerceBuilder.Configuration: 7.0.10752.0
CommerceBuilder.CyberSource: 7.0.9764.0
CommerceBuilder.DataClient.Api: 7.0.34.0
CommerceBuilder.DataClient.Csv: 7.0.34.0
CommerceBuilder.DHL: 7.0.8825.0
CommerceBuilder.ECB: 7.0.0.0
CommerceBuilder.FedEx: 7.0.9725.0
CommerceBuilder.FRBNY: 7.0.0.0
CommerceBuilder.GoogleCheckout: 7.0.10052.0
CommerceBuilder.InternetSecure: 7.0.9764.0
CommerceBuilder.LinkPoint: 7.0.9764.0
CommerceBuilder.Paradata: 7.0.9764.0
CommerceBuilder.PayFlowPro: 7.0.10061.0
CommerceBuilder.PayJunction: 7.0.9764.0
CommerceBuilder.PayPal: 7.0.10840.0
CommerceBuilder.Services: 7.0.10752.0
CommerceBuilder.SkipJack: 7.0.9764.0
CommerceBuilder.UPS: 7.0.10041.0
CommerceBuilder.USPS: 7.0.9728.0
CommerceBuilder.Web: 7.0.10752.0
ComponentArt.Web.UI: 2007.1.1566.3
CyberSource.Base: 2.0.0.0
CyberSource.Clients: 5.0.2.0
CyberSource.Clients.XmlSerializers: 5.0.2.0
CyberSource.WSSecurity: 2.0.0.0
CybsWSSecurityIOP: 1.0.0.0
edtFTPnet: 1.2.4.0
FredCK.FCKeditorV2: 2.5.2912.21007
ICSharpCode.SharpZipLib: 0.85.5.452
LinkPointTransaction: 1.0.1797.29766
log4net: 1.2.10.0
Microsoft.Practices.EnterpriseLibrary.Common: 2.0.0.0
Microsoft.Practices.EnterpriseLibrary.Data: 2.0.0.0
Microsoft.Practices.ObjectBuilder: 1.0.51205.0
Microsoft.Web.Preview: 1.2.61025.0
NVelocity: 7.0.8530.0
PaygatewayNET: 3.0.0.0
paypal_base: 4.2.1.0
PFProCOMLib: 1.0.0.0
PFProdotNET: 0.0.0.0
Validators: 1.0.0.0
WebChart: 1.1.1.6
wwHoverPanel: 1.85.0.0

[Naveed]

Hello Mary,

I have tested your data, and reported the bugs. Regarding products import the issue is same as discussed by Judy in above post. Alt Text fields causing the issue when product name exceeds 100 characters. current workaround is to exceed the length of following fields to nvarchar(255) under ac_Products table.

ImageAltText
ThumbnailAltText
IconAltText

I tested after this update and all of products imported fine.

[mfreeze]

Could I ask what bugs you found besides the field lengths in ac_Products?

[Naveed]

2 more error messages which were logged while importing your data.

----------------------------------------------------
Error while user import
----------------------------------------------------

Error: User Import, UserID:1 , Name: Admin, Object reference not set to an
instance of an object.
at CommerceBuilder.DataClient.Api.Ac55Importer.ImportUsers(XmlNode
nodSourceStore) in E:\AbleCommerce\DataPort\7.1 Stable\Api\Ac55Importer.cs:line
3674


----------------------------------------------------
Some error messages while importing Order Files
----------------------------------------------------
DEBUG 2009-02-09 23:30:03,796 1571218ms Object a
- An error occurred while importing AC5x data: Order File Import, Old
OrderFileID:2057, File Name:9022.pdf , OrderID: 156780 , OrderNumber: 2687, The
INSERT statement conflicted with the FOREIGN KEY constraint
"ac_OrderItems_ac_OrderItemDigitalGoods_FK1". The conflict occurred in database
"AC71S", table "dbo.ac_OrderItems", column 'OrderItemId'.
The statement has been terminated.
.....

[Naveed]

So, for clarification, of the name and image alt fields, which fields can be 255 characters, and which must be only 100, and what's with the name field being only 55 characters?

Image AltText fields should be of 255 characters. As when not specified the product name is automatically picked up for these, which may be up to 255 characters in size. However unfortunately we missed this when fixing the issue.

However if you have access to your database, and increase the size of following 3 three fields for Products table, it will solve the issue for you.

---------
ImageAltText
IconAltText
ThumbnailAltText
----------

We are also going to release a cumulative patch in next couple of days, which will also fix this database issue.

[mfreeze]

There were some other errors I am concerned about.

Error: Order File Import, Old OrderFileID:1925, File Name:2005\10051.pdf , OrderID: 142798 , OrderNumber: 2553, INSERT statement conflicted with COLUMN FOREIGN KEY constraint 'ac_OrderItems_ac_OrderItemDigitalGoods_FK1'. The conflict occurred in database 'ac7_au(for more details check server log)...

Error: Product File Associations (ESD) Import: Product options not imported correctly, skipping the Product -> ESD association import. Data:<ProductFile Product_ID="1336" OSD_ID="5" File_ID="756"

Warning: ESD file import, DigitalGood automatic fullfilment setting may not be imported correctly, fullfilment mode is set to: UseAutomaticFulfillment. Digital Good Name: World Out of Balance, Digital Good Id: 1, Old ESD file Id:2.Warning: ESD file import, DigitalGood activation/validation mode may not be imported correctly, setting it to Manual mode. Digital Good Name: World Out of Balance, Digital Good Id: 1, (The setting should be consistent, preferrablly always UseAtormaticFulfillment).

Warning: A User with the same ID already exists, saving it with new ID.
Old User ID=831980
User Name=mlombardo@pointb.com
(How can the user ID already exist when I am importing into an empty database? This could also cause errors associating an order with a user ID if the ID changes.)

Warning: OrderItem import, Name: 'Overcoming the Five Dysfunctions of a Team: A Field Guide for Leaders, Managers, and Facilitators (e-Transcript) <img border=0 src="/images/pdf.jpg">' is longer then 100 characters, truncating it to 100 characters. Old OrderItem Id:35070, Old Order Id:348
(This is another field where the length changed and I am still getting the error after changing the Name in ac_OrderItems to 255)

Could you take a look at these, please?

[Naveed]

Error: Order File Import, Old OrderFileID:1925, File Name:2005\10051.pdf , OrderID: 142798 , OrderNumber: 2553, INSERT statement conflicted with COLUMN FOREIGN KEY constraint 'ac_OrderItems_ac_OrderItemDigitalGoods_FK1'. The conflict occurred in database 'ac7_au(for more details check server log)...

In AC55 we were able to directly associate ESD's to orders from order administration pages, in such case no order item is created or associated with. Please have a look for following order file data.

<OrderFile ID="83">
<Order_ID>444</Order_ID>
<OrderItem_ID>0</OrderItem_ID>
<File_ID>59</File_ID>
<DisplayName>Blindsided</DisplayName>
<FileName>9021.pdf</FileName>
<SaveAsName>9021.pdf</SaveAsName>
<MaxDownloads>2</MaxDownloads>
<RelativeTimeout>0%2c0%2c0</RelativeTimeout>
<AbsoluteTimeout>0%2c0%2c0</AbsoluteTimeout>
<ValidateDT>5%2f6%2f2005+9%3a30%3a01+AM</ValidateDT>
</OrderFile>

OrderItem_ID have a zero (0) value, means no order item associated with it. So, you might have manually associated it to order from AC55 order admin page. This is however not possible to associate an ESD file to Order without order item. So, this error was happening. I have updated the implementation to show a proper error message instead of exception details.

Error: Product File Associations (ESD) Import: Product options not imported correctly, skipping the Product -> ESD association import. Data:<ProductFile Product_ID="1336" OSD_ID="5" File_ID="756"

Error message is registered because from data it seems that the ESD is attached to some option / variant of a product (OSD_ID="5"). However the product have no options/variants defined. In this case the ESD should be associated to respective product instead. And this is what we already doing and if you look at the respective imported product in AC7 the ESD is properly attached to the respective product.
Updated to register it as a more proper warning instead of an error message, so that the merchant can review and confirm it.

Warning: ESD file import, DigitalGood automatic fullfilment setting may not be imported correctly, fullfilment mode is set to: UseAutomaticFulfillment. Digital Good Name: World Out of Balance, Digital Good Id: 1, Old ESD file Id:2.Warning: ESD file import, DigitalGood activation/validation mode may not be imported correctly, setting it to Manual mode. Digital Good Name: World Out of Balance, Digital Good Id: 1, (The setting should be consistent, preferrablly always UseAtormaticFulfillment).

In AC55 we were having more automatic fulfillment options then in AC7. However in AC7 for 7.0 and 7.0.1 we only have to options:
UseAutomaticFulfillment
UseManualFulfillment
And we try to import is as close as possible. For AC 7.0.2 release we however added more fullfillment and activation modes.

Warning: A User with the same ID already exists, saving it with new ID.
Old User ID=831980
User Name=mlombardo@pointb.com
(How can the user ID already exist when I am importing into an empty database? This could also cause errors associating an order with a user ID if the ID changes.)

This is one of the known issues and already fixed. But fix will be available for future versions.

Warning: OrderItem import, Name: 'Overcoming the Five Dysfunctions of a Team: A Field Guide for Leaders, Managers, and Facilitators (e-Transcript) <img border=0 src="/images/pdf.jpg">' is longer then 100 characters, truncating it to 100 characters. Old OrderItem Id:35070, Old Order Id:348
(This is another field where the length changed and I am still getting the error after changing the Name in ac_OrderItems to 255)

Thanks for pointing this out. As the field lengths were 100 chars so, when importing we have to truncate it to avoid SQL errors. Now as field lengths are already updated for 7.0.1 and 7.0.2 releases. So, we have removed the truncation and updated.

All the fixes will be however available for next release of DataPort.

[mfreeze]

Is there an ETA for the next release?

Also, will the next release handle custom fields?