W-Shadow.com

(This one was hard to find a post title for (sound like Yoda I do, hmm?))

Google added some new features to Google Trends a few weeks ago – namely Google Trends for Websites. This new tool shows – among other things – what other websites were commonly visited by people who visited a given site. That information is probably most useful to SEO people and internet marketers but you can also do it just for fun and to discover new sites.

To this end I have created a PHP/JS script that will let you enter a list of sites and use the Google tool to analyze the sites and generate a list of related websites – a kind of recommendation engine. For example, if you enter a webcomic site A you will probably get a list of other webcomics B, C and D that are also enjoyed by people who read webcomic A.

Here’s a Screenshot

Where’s The Online Demo?

Now you probably (hopefully?) want to see the script in action. Unfortunately there is no online demo to be had here. If you want to try out this script you’ll have to download it and run it on your own PC/server.

The reason why I can’t host it on my own site is that the script needs to send a lot of requests to Google Trends (up to 11 queries for each entered site). If I had the working script on public access here that would amount to a lot of queries. While my server could probably handle the traffic, Google wouldn’t be amused and the server would be banned from Google faster than you can say “TOS violation”. On the other hand, you should be safe if you run it on your own server for “private use”.

How It Works

The script is composed of two main parts – the PHP scripts and the JavaScript scripts. The PHP end is fairly simple – it loads the Trends page for a given domain (using CURL), extracts the “Also visited” info using some regexps and outputs the results in the JSON format suitable for AJAX. The script also uses optional file-based caching to speed up repeated requests.

Most of the business logic is handled by JavaScript (see suggestWebsites() in index.php). In short, the algorithm works like this –

1. For each website entered by the user find the “related” sites via Google Trends.
2. Assign “points” to each related site based on how long the bar graph is on the Trends page. If a site is encountered more than once add the points to the total.
3. Repeat the above for each “related” site. These second-level recommendations get less points (configured by the second_level_factor variable in index.php).

The script uses AJAX to fetch “also visited” site info via the aforementioned PHP script and updates results in real time. There’s also a “Sort” button that will sort results by points, in descending order.

You can configure how many related websites per site you want to get, how much importance to assign to second-level suggestions and whether favicons shoould be shown by tweaking the appropriate JS variables in index.php. I’ve included (a few) explanatory comments in the script so editing it shouldn’t be too much of a problem.

Download The Script

related_sites.rar (113 KB)

Requirements :

• PHP 5.2.1 or later
• cURL
• Write access to the cache subdirectory if caching is enabled.

License : Creative Commons Attribution-Share Alike

Installation : If your server meets the requirements you only need to extract & upload the script to get it working; no additional configuration is required. You can tweak some settings in index.php if you wish.

Credits And Related Links

• JavaScript progress bar by Bramus
• Table design and random bits of other CSS from table sort script by frequency-decoder.
• json_encode() documentation.
• cURL wrapper class by 5ubliminal. Note that the file included in the download (above) is a modified version.