How does AC7 decide to assign user = "Search_Engine"?

[sweeperq]

How does AC7 decide to assign user = "Search_Engine" and is there a text or XML file somewhere where I can add additional bots?

[plugables]

AbleCommerce does not do such a thing. Robots are supposed to respect robots.txt file which defines paths that are allowed or disallowed for robots.

[sweeperq]

Sorry, not following what the robots.txt has to do with the "Search_Engine" user?

Reference Post: viewtopic.php?f=36&t=5349

How does AbleCommerce decide if it is a bot or not?

[sweeperq]

Just noticed the post referenced is from 2007, so it probably pertains to AC5.

[*]If I hadn't ported over from 5.5 would the "Search_Engine" user exist in AC7?
[*]If there isn't something that identifies the bots/crawlers, how do you prevent new Anonymous users every time they hit a page?

[plugables]

bots and crawlers respect robots.txt

Anonymous and non-anonymous users are easily identified. If a user is not logged in he/she is anonymous. Anonymous users can't access pages that are for logged in members. For example MyAccount page. Anonymous user, be it a normal browser or a bot/crawler, is treated the same way.

[Logan Rhodehamel]

You are testing my memory. In AC5x we did this by looking at user agent strings. There was a list of known user agents that corresponded to spiders. But then we had to maintain this list and release periodic updates... I can't remember where the list was stored.

In AC7 and upward we took a different approach. We opted to require cookies and move away from cookieless sessions. We don't create user records for any browsing without cookies. This eliminated the situation where a spider could create thousands of abandoned baskets or get a user session included in a spidered link.

[sweeperq]

Logan, thanks for the info. I was aware of the spider agents file in AC5x, but wasn't sure how you handled it in AC7. I am curious how you are checking for cookies though. .Net offers some quick checks to see if the browser supports cookies, but to my knowledge, cannot tell if the user has them turned on/off. To test for cookie acceptance requires you to set a value on one request, and to test for its existence on another request. Is this happening somewhere and I am just missing it?

[Logan Rhodehamel]

To test for cookie acceptance requires you to set a value on one request, and to test for its existence on another request. Is this happening somewhere and I am just missing it?

Which is what is happening. If you come into the site and we can determine there is a value set in HTTP_COOKIE, we assume you have cookies enabled and save the user record. If there is no value in this field we would not save the user record but we will still attempt to set a cookie. On the next request, we'll pick up the value in HTTP_COOKIE.

Having a user record is not required for viewing the site - the user record is primarily required at checkout or for my account type features. There is no loss of functionality by not always having the user record created instantly.

The search engine spiders have always caused havoc with our user session tracking but I think in AC7 we finally found a reasonable solution. If spiders are well behaved (no cookies, respect robots.txt) then they won't create runaway user records.

[sweeperq]

Thank you for the clarification. Makes perfect sense.