Today’s goal is a basic php Google Suggest scraper because I wanted traffic data and keywords for free.
Before we start :
google scraping is bad !
Good People use the Google Adwords API : 25 cents for 1000 units, 15++ units for keyword suggestion so they pay 4 or 5 dollar for 1000 keyword suggestions (if they can find a good programmer which also costs a few dollars). Or they opt for SemRush (also my preference), KeywordSpy, Spyfu, and other services like 7Search PPC programs to get keyword and traffic data and data on their competitors but these also charge about 80 dollars per month for a limited account up to a few hundred per month for seo companies. Good people pay plenty.
We tiny grey webmice of marketing however just want a few estimates, at low or better no cost : like this :
| data | num queries |
| google suggest | 57800000 |
| google suggestion box | 5390000 |
| google suggest api | 5030000 |
| google suggestion tool | 3670000 |
| google suggest a site | 72700000 |
| google suggested users | 57000000 |
| google suggestions funny | 37400000 |
| google suggest scraper | 62800 |
| google suggestions not working | 87100000 |
| google suggested user list | 254000000 |
Suggestion autocomplete is AJAX, it outputs XML :
< ?xml version="1.0"? > <toplevel> <CompleteSuggestion> <suggestion data="senior quotes"/> <num_queries int="30000000"/> </CompleteSuggestion> <CompleteSuggestion> <suggestion data="senior skip day lyrics"/> <num_queries int="441000"/> </CompleteSuggestion> </toplevel>
Using SimpleXML, the PHP routine is as simple as querying g00gle.c0m/complete/search?, grabbing the autocomplete xml, and extracting the attribute data :
-
if ($_SERVER['QUERY_STRING']=='') die('enter a query like http://host/filename.php?query');
-
$contentstring = @file_get_contents("http://g00gle.c0m/complete/search?output=toolbar&q=".urlencode($kw));
-
$content = simplexml_load_string($contentstring );
-
-
foreach($content->CompleteSuggestion as $c) {
-
$term = (string) $c->suggestion->attributes()->data;
-
//note : traffic data is sometimes missing
-
$traffic = (string) $c->num_queries->attributes()->int;
-
echo $term. " ".$traffic . "
-
" ;
-
}
I made a quick php script that outputs the terms as a list of new queries so you can walk through the suggestions :
The source is as text file up for download overhere (rename it to suggestit.php and it should run on any server with php5.* and simplexml).
Dear Web Master,
We are in process of link building of our site to increase its relevancy and traffic.
Can you give me information on how I can place my link on your page?
Our link details are following:
Title: Canadian Pharmacy
URL: http://www.canadadrugcenter.com/
Description:
CanadaDrugCenter.com is America’s choice for safe and affordable prescription and non-prescription medications. Our licensed Canadian mail order pharmacy will provide you with substantial savings on all your medication needs.
Seems you found a way ;)
it seems that it is not working anymore :(
hi peter, could you give me some more feedback ? I checked just to be sure, on my localhost and a remote host, the listed code runs fine. It does require simplexml but that is almost always installed with php.
seems that the code still works I was able to make a very useful code… thanks
Not working. Admin, maybe you should try curl.
It works well for me! At first a little bit confused but I figure it out how to use it.
it still works, but it is limited only for 10 keywords. My question is, how to make it unlimited results? Thanks
The ajax api is limited to 10 results. You can also use the Adwords API http://code.google.com/apis/adwords/docs/reference/latest/TargetingIdeaService.html#getBulkKeywordIdeas
the getBulkKeywordIdeas-function returns pages of 500 keyword suggestions, it has a rate of usage price, about 1 cent per page. Nice tool for a membership site.
Hi,
It’s a great tool. How can I use it to get query of more than one word.
Thanks
Pratik
thanks, simple code but work perfect