SUPER URLS EXPOSED

Amazon Super URLs Exposed

With our recommendation that sellers stop using Amazon Super URLs last week, we’ve had a lot of questions from people about what they are, and our reasoning for believing that they are a manipulation of Amazon’s system.

Specifically, we believe that Super URLs are an attempt to manipulate Amazon’s search algorithm. We explain some of the importance of keywords in our Amazon Search Secrets white paper. Since “Search is the primary way that customers use to locate products on Amazon” (login required), being ranked well in Amazon’s search algorithm can mean a significant boost in sales and profitability of a product. So much so, that sellers do a lot of things to fight for the top positions. Amazon Super URLs have become a tool that gives some sellers an unintended benefit in Amazon’s search algorithm.

Amazon’s Search Algorithm

To understand how the hack works, you need to understand a bit about Amazon’s search algorithm. Amazon is obsessed with customer satisfaction. When a customer searches for a product on Amazon.com, it wants to provide the customer with the search results of the products they are most likely to buy. It has an entirely separate business entity called A9, which is headquartered in Silicon Valley and employs ridiculously smart people who work on engineering and tweaking the product search algorithm.

Experience, logic, and even Amazon says that “In general, better-selling products tend to be towards the beginning of the results list. As your sales of a product increase, so does your placement.” So the number of sales of a given product definitely help it to rank better within the algorithm. Taking that logic one step further, it makes sense that a product would rank better in search for a search term that the customer used to buy it. So if Amazon shoppers consistently search for “water bottle” and end up buying product A, then product A should be moved up in the search results for “water bottle.”

So how does it work

You can see some evidence of this happening if you walk through the process yourself. Go toAmazon.com, search for something in the main search box, then click on the link for one of the products. Then look at the URL for that page. You’ll see something like this:

http://www.amazon.com/Hacking-Art-Exploitation-Jon-Erickson/dp/1593271441/ref=sr_1_2?ie=UTF8&qid=1440904995&sr=8-2&keywords=hacking

Notice the keywords I searched for are at the end of the URL. There are some other parts of the URL that I’ll get to shortly.

Let’s pretend you bought this item. I believe Amazon would then save that information into a database that may look something like this:

Sales By Keyword

Order ID ASIN Time stamp Keyword qid
103-1039485-19238405 B000111222 2015-08-30 12:34:56  “water bottle” 22244466688
103-9182737-85736284 B999888777 2015-08-30 13:45:54 “water bottle” 11133355577

With this data, saved across millions of sales, the A9 search team has some fantastic data to use in its search algorithm. They can figure out what products were purchased most often for a given keyword.

One more detail

With the keywords being passed in the URL, there is one other consideration. What would happen if somebody linked to one of these URLs from another website? They wouldn’t want sales from such links to be counted over and over again. This is where we need to dig into the URL to get a little more detail.

http://www.amazon.com/Hacking-Art-Exploitation-Jon-Erickson/dp/1593271441/ref=sr_1_2?ie=UTF8&qid=1440904995&sr=8-2&keywords=hacking

The other important part of the URL is the part that contains qid=1440904995.  If you perform the same search twice in a row–just a few seconds apart from each other–then look at both of the URLs for the products on the page. You’ll notice the qid parameter has changed by a little bit. The value of that parameter is actually a Unix Timestamp, or the number of seconds that have elapsed since January 1, 1970.

So, the effect of that qid= parameter in the URL looks like this to make it appear like the search counted once for the sale of the product. One of those URLs could be published and generate hundreds of sales, and since all of them use the same qid parameter, it would only count towards Amazon’s search algorithm once.

Search position

There are two other parameters in the Amazon search URL. Those are the ref=sr_1_2, and sr=8-2from the example above. The number 2 in both of these examples is because the product that was clicked was the second item in the search results. I don’t see an obvious technical reason as to why those are included but in order to make the URL look as natural as possible, most Super URL generators will periodically check the search position of the product to make sure that it is correct. That’s why Amazon Super URL services are always paired with Search Rank tracking services–they need the data anyway.

So what exactly is an Amazon Super URL then?

The Super URL is like a redirection service, similar to bit.ly, tinyurl.com (and hundreds of others), in that it generates a “short” URL that can be published and used throughout the Internet. This short URL then redirects the user to a dynamic URL generator that goes to Amazon, which creates the qid parameter and the search position parameters to make it look like an Amazon search occurred, and that the buyer selected the desired product to buy.

This is a Super URL that I generated just for this blog post so that you can try it out: http://iazrs.com/ntwkak7htb

A seller would use this link wherever they posted links to their products throughout the internet: social media, emails, etc. When a user clicks on the link, the Amazon Super URL service dynamically generates the qid parameter and the search rank parameters, then it performs an HTTP 302 redirect so it looks like the user just performed a search on Amazon.com. If the user then buys the product, it will get counted as a sale for the desired keyword–even though the buyer didn’t actually perform the search and select that product.

Making it effective

When using Amazon Super URLs in combination with a seller that is likely to convert many sales (as with providing the user with a coupon code for the product), the result is that the seller has been effectively able to manipulate Amazon’s search algorithm into placing their product higher in search results.

Is it a manipulation?

In the paragraphs above, I have made quite a few assumptions about how certain portions of the Amazon system works. It should be noted that all of these are assumptions based on logic and observations. We have no visibility into how Amazon’s system actually works. However, with the popularity of these services and number of people who claim success from using them, I believe it is evident this trick has worked for at least some period of time. There seems to have been some chatter in the past month or so that Super URLs aren’t as effective as they used to be.

Regardless of whether they are effective or not, the use of a Amazon Super URL is clearly an attempt to manipulate Amazon’s search algorithm. This is why Seller Labs no longer uses these dynamic URLs and is publicly encouraging our customers, and others in the industry to stop using them as well. With several of the phrases in the recent policy change having to do with “intent”, we believe that it is prudent for sellers to avoid tools and tactics that are a clear attempt to manipulate in order to avoid policy violations and account suspensions. Identifying products that have been sold using a Super URL is probably one of the easiest and most conclusive technical ways in which Amazon can identify potential violators of the new policy.

Common Questions

If I don’t use a Super URL, then what URL should I use?

We recommend you copy and paste any URL on the Amazon.com website that gets you to the page you want. The manipulative part is using a service that changes those parameters each time a user clicks on them. If you want to use the most basic URL possible, then you can generate one in this format: http://www.amazon.com/dp/{$ASIN}

How can Amazon really identify products sold using Super URLs?

One of the reasons we are taking such a bold stance on this is that identifying products sold using Super URLs is probably one of the easiest technical ways for Amazon to identify sellers who are trying to manipulate the system. To get a bit technical, the &qid parameter in the URL is a time stamp showing that a search for the given keyword was performed. Certainly Amazon maintains a database of each time stamp and search performed on the site. All they need to do is find sales that occurred for searches that never took place.

But that is a huge set of data, and they will never be able to find the very small percentage of sales resulting from a Super URL in such a large data set.

You may have heard the term “Big Data” being used a lot lately by IT professionals. The Amazon AWS division is at the forefront of making Big Data tools available to IT professionals around the world. Amazon AWS provides several Big Data tools available, including Redshift and Elastic MapReduce that are made to perform queries on huge data sets like this. Since the Amazon website runs on the same infrastructure, you can be sure that these tools are readily available for such queries.

But the policy doesn’t say anything about Amazon Super URLs or Search Position

It’s true that there was no language to specifically say that linking to Amazon in an unnatural way is against its policy. We see it as a larger issue of trying to manipulate its system, and the new policy does make several mentions about intentional manipulation.

But if a seller receives a policy violation or account suspension, we don’t think an argument like “I was trying to manipulate my products’ search position, not its sales rank” would go very far to bring your account back into good standing.

Brandon Checketts

Brandon is one of the co-founders and main data geek for Seller Labs. He started Seller Labs after finding no other tools that could provide the flexibility needed for his used-book business. He no longer sells online, but now guides Seller Labs as the lead innovator to make sure that our products remain on the cutting edge.