serp and pagerank tools

php serp and pagerank tools
  • rss
  • begin
  • about juust
  • php scripts
    • flickr wp widget
    • Crappy the pagerank spider v.1.00
    • google multi key serp tool, php script
  • php classes
    • php pagerank class
    • fibonacci class
    • robots.txt parser php class
  • serp
  • Blacknorati
    • php content class
  • web franchises

this, that, which

what is my website worth ? (php fetching)

juust | November 19, 2008

What is the worth of my puny blog ? Here’s two snippets (download) to grab estimates from two sites. The first one is business-opportunities.biz, the second webvaluer.org.

According to www.business-opportunities.biz my blog is worth 1700 dollar. Their estimate is derived from the Technorati blog cosmos so it only works for blogs that are registered there.

  1. function what_my_site_is_worth_according_to_business_opportunities($url)
  2. {
  3.     $html=file_get_contents('http://www.business-opportunities.biz/projects/how-much-is-your-blog-worth/submit/?url='.$url);
  4.     $cut1=explode('is worth',$html);
  5.     $cut2=explode('< /p>',$cut1[1]);
  6.     $result=trim($cut2[0]);
  7.     if(strlen($result)==0) return(0);
  8.     return($result);
  9. }

use

  1. echo what_my_site_is_worth_according_to_business_opportunities('www.juust.org');

That returns in my case $1,693.62

According to webvaluer.org my website is worth $7,549 US dollar. Their site returns a value for every type of site. In the page source there is a span format tag around the text

  1. < h1>juust.org Estimated Worth < span style="color:green; font-weight:bold;">$7,549 USD< /span>< /h1>

I cut it before and after the span tag, and use the php function strip_tags() to remove the span tag, and only the text string remains.

  1. function what_my_blog_is_worth_according_to_webvaluer($url)
  2. {
  3.     $html=file_get_contents('http://www.webvaluer.org/us/'.$url);
  4.     $cut1=explode('Estimated Worth', $html);
  5.     $cut2=explode('< /h1>',$cut1[1]);
  6.     $result=(trim($cut2[0]));
  7.     if(strlen($result)==0) return(0);
  8.     return(strip_tags($result));
  9. }

use :

  1. echo what_my_blog_is_worth_according_to_webvaluer('www.juust.org');

How to make your own php fetch function :

  • check the result page html source
  • see what tag or phrase comes before the text you want (’Estimated Worth’)
  • use explode() or split() to cut the source up in pieces on that phrase
  • take the piece after the phrase (arrays are 0-based,use cut1[1] to get the second piece)
  • cut that up at the first tag (’< /h1>‘) after the text you want
  • use the first piece (cut2[0]) that holds the text you want

It’s easier to use source splitting for this sort of fetching than regular expressions, which is sooner meant to retrieve generic patterns (email addresses and stuff) from lots of pages (preg_match()), or an array of similar string from one page (preg_match_all(), for serp pages or report tables).

note : my php code highlighter refuses to display html tags in functions proper so in the code I added a blank space : $cut2=explode(’< /p>‘,$cut1[1]);
Remove the blanks from the html tag in the function for it to work proper or use the text source
.

Comments
No Comments »
Categories
php
Tags
php
Comments rss Comments rss
Trackback Trackback

some thoughts on search engine marketing

juust | November 17, 2008

I want a quick way to check the competitors in a niche, and estimate what can I spend on a search engine marketing campaign to make money in the segment, using the search engines (read: Google).

For a quick test I dive into the money segment, got the top 70 search phrases of October off of 7Search and for each get the top 100 results, resulting in a mysql database with 7.000 url’s on 3500 domains.

Then I try three cross-sections to extract the top 25 domains in the segment :

  • (100 - position) / 100 per result
  • (100 - position) / 100 per result, +1 for first 10, +1 for first 6, +1 for first three
  • AOL experience based percentage of search volume

rating by rank

If I rate the sites’ urls based on (100 - position) / 100 per result.

place 1 = 0.99, place 100 = 0, that yields this table :

domain score
www.amazon.com 51.05
www.youtube.com 34.82
en.wikipedia.org 34.04
wordpress.com 30.93
www.streetdirectory.com 29.62
money.cnn.com 24.5
www.wikihow.com 22.84
www.experienced-people.co.uk 22.29
ezinearticles.com 22.26
books.google.com 19.22
www.problogger.net 17.52
entrepreneurs.about.com 16.53
www.43things.com 14.75
moneycentral.msn.com 14.74
answers.yahoo.com 14.41

That makes Amazon top dog for all phrases containing ‘money’. I know the first pages get most traffic, and that first method doesn’t express that, so I change the routine and do my old trick.

rating by rank, bonus for front page

(100 - pos) / 100, and
for place 1-3 3 points
for place 4-6 2 points
for place 7-10 1 points

That gives this table :

domain score
en.wikipedia.org 91.04
www.wikihow.com 67.84
www.amazon.com 65.05
www.youtube.com 56.82
money.cnn.com 50.5
entrepreneurs.about.com 45.53
ezinearticles.com 45.26
www.experienced-people.co.uk 42.29
www.freemoneyfinance.com 31.9
www.streetdirectory.com 31.62
wordpress.com 30.93
www.43things.com 30.75
abcnews.go.com 30.15
moneycentral.msn.com 29.74
moneymakerinfo.blogspot.com 29.55
www.thisismoney.co.uk 27.06
www.moneymakingmommy.com

That still does not show what domains actually get the traffic I need for conversion.

It is easy to score in a low traffic niche and anyone can be a winner on long tails, “alabama ski resort”, but long tails only get you a few hits (and wonder if they are serious?). And these results doesn’t give me a clue what I can spend (or what my competition would be willing to spend) on the actual traffic spots.

So I am going to estimate what traffic every site gets. I need search volumes and percentages for the serp ranks. I grabbed the search volumes per phrase from 7Search. For percentages, Aaron Wall quotes an old AOL source on the average click through rate of search engine result pages per spot on the first page :

Overall Percent of Clicks

Relative Click Volume

  1. 42.13%, 2,075,765 clicks
  2. 11.90%, 586,100 clicks
  3. 8.50%, 418,643 clicks
  4. 6.06%, 298,532 clicks
  5. 4.92%, 242,169 clicks
  6. 4.05%, 199,541 clicks
  7. 3.41%, 168,080 clicks
  8. 3.01%, 148,489 clicks
  9. 2.85%, 140,356 clicks
  10. 2.99%, 147,551 clicks
  1. 3.5x less
  2. 4.9x less
  3. 6.9x less
  4. 8.5x less
  5. 10.4x less
  6. 12.3x less
  7. 14.0x less
  8. 14.8x less
  9. 14.1x less

1st page totals: 89.82%, 4,425,226 clicks
2nd page totals: 10.18%, 501,397 clicks

That was what i was looking for. Given these percentages I can estimate the traffic any spot on the search engine result front page in Google would generate, and that yields a more realistic table of the money segment :

domain traffic result result+(1,2,3)
www.bidvertiser.com 737478 6.65 17.65
money.cnn.com 294316 24.5 50.5
en.wikipedia.org 160370 34.04 91.04
www.wikihow.com 132332 22.84 67.84
www.wealthsuccess.usana.com 123503 9.4 24.4
entrepreneurs.about.com 117069 16.53 45.53
www.moneymakingmommy.com 112507 11.65 26.65
www.freemoneyfinance.com 102122 12.9 31.9
www.experienced-people.co.uk 86498 22.29 42.29
www.forbes.com 84393 7.22 19.22
www.netjobs4all.com 75868 4.61 12.61
makemoneyforbeginners.blogspot.com 75399 6.53 15.53
www.missingmoney.com 65326 3.63 12.63
www.youtube.com 60586 34.82 56.82
technology.timesonline.co.uk 55009 3.53 8.53
moneymakerinfo.blogspot.com 54431 13.55 29.55
www.moneyclaim.gov.uk 50544 1.9 5.9
moneycentral.msn.com 39399 14.74 29.74

That shows which sites actively target and get the traffic in the segment, a site like amazon doesn’t show in the top of the last table.

One way of testing the validity of the estimates is comparing with alexa rank, doing a quick SeoQuake toolbar check :

domain est. search traffic alexa rank
www.wealthsuccess.usana.com 200000 20000
moneymakingmommy 110000 70000
freemoneyfinance 100000 60000
makemoneyforbeginners.blogspot.com 75000 140000
problogger.com 6500 43000

That’s roughly correct based on old AOL percentages (well done, Aaron) and Oktober’s search volumes. It does not work for youtube, msn and others that have an alexa rank based on the entire domain. Being a search engine traffic estimate it doesn’t cover your ‘audience’ (returning visitors on bookmarks and referrals off of other sites (which are included in the alexa ranking) or other segments.

Problogger for example, according to that calculation would get about 6K hits through Google but the Alexa rank indicates it has ~~160K hits a month. That indicates high return visits and direct traffic, a high usability and I think they get traffic from other segments (’blog’ ?).

The use of it

Why did I start doing this ? To estimate a budget for niche penetration.

If I were interested in a niche and have a 1% conversion at $100,- per year per conversion, what would I be able to spend to get Bidvertisers’ traffic in the search engines ? With their traffic and my conversion, 730.000/100 = 7300 conversions per month =y 88.000 a year @100/conversion = 8.8 million dollar gross revenue on search engine traffic conversions.

If my marketing budget is 20% of gross sales, for a year project that amounts to 1.7 million dollar. If I already had 300K hits my budget would be a million.

That (roughly) answers my question.

I have my doubts about the use of a tool like this, it could function as quick way to scan a segment, make a ‘heatmap’ and pinpoint the soft spots that are easy to penetrate and get a foothold, but that would require some stiff programming and a lot of switches, as well as processing a lot more segments. Nice project for the winter.

Comments
1 Comment »
Categories
google, sem, tool
Tags
sem, tool
Comments rss Comments rss
Trackback Trackback

how to keep a traffic log with mysql and php

juust | November 16, 2008

A rather obsolete post. I hacked a quick referrer tracking script together because I wanted to see how Googlebot indexed my serp tool pages. I can use Google analytics or phpMyVisites, but that is not as rewarding as keeping my own logs.

I start with a mysql table LOG to store the traffic data

  1. CREATE TABLE IF NOT EXISTS `log` (
  2.   `id` bigint(11) NOT NULL auto_increment,
  3.   `ip` varchar(20) NOT NULL,
  4.   `page` varchar(120) NOT NULL,
  5.   `visitpage` varchar(120) NOT NULL,
  6.   `visitdate` varchar(25) NOT NULL,
  7.   `agent` varchar(200) NOT NULL,
  8.   `protocol` varchar(10) NOT NULL,
  9.   `url` varchar(120) NOT NULL,
  10.   `visitpath` varchar(80) NOT NULL,
  11.   PRIMARY KEY  (`id`)
  12. ) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

on every page I track, I start with

  1. session_start();
  2. log_hit();

The function log_hit() relies on the php $_SERVER array that holds all data the web server collects:

  1. function log_hit();
  2.         $ip   = $_SERVER['REMOTE_ADDR'];
  3.  
  4. //if it is my own ip, exit the routine
  5.  if($ip =="my.own.ip.here") return;
  6.  
  7. //what page is my visitor at ?
  8.  $page  = "http://".$_SERVER['HTTP_HOST'].$_SERVER['PHP_SELF'];
  9.  
  10. //what page did he come from ?
  11.  $visitpage  = $_SERVER['HTTP_REFERER'];
  12.  
  13.  $visitdate  = date("YmdHi");
  14.  
  15. //what user-string is it (iow: is it googlebot and pals or an actual visitor ?)
  16.  $agent      = $_SERVER["HTTP_USER_AGENT"];        
  17.  
  18. //the rest i dont use but i'll throw them in just for fun :
  19.  $protocol   = $_SERVER['SERVER_PROTOCOL'];
  20.  $method    = $_SERVER['REQUEST_METHOD'];
  21.  $url           = $_SERVER['REQUEST_URI'];
  22.  $visitpath   = parse_url($page, PHP_URL_PATH);
  23.  
  24.  if(!$ip) {} else {
  25. //if I have an Ip, register the hit :
  26. //connect to mysql table
  27.   $m=connect_data();
  28.  
  29. //and store it in the LOG table
  30.   $myqry="INSERT INTO `serp_log` (
  31.  `ip` ,`page` ,`visitpage` ,`visitdate` ,`agent` ,`protocol` ,`url` ,`visitpath` ) VALUES (
  32.  '".$ip."', '".$page."', '".$visitpage."', '".$visitdate."', '".$agent."', '".$protocol."', '".$url."', '".$visitpath."')";
  33.   mysql_query($myqry, $m) or die('store log :'.mysql_error());
  34.   mysql_close($m);
  35. }

the connect_data() function is standard mysql connection

  1. function connect_data() {
  2.   $DB_USER =  "user";
  3.   $DB_PASSWORD = "password";
  4.   $DB_HOST = "server";
  5.   $DB_DATA = "database";
  6.   $link =  mysql_connect($DB_HOST, $DB_USER, $DB_PASSWORD) or $error = mysql_error();
  7.   if (!$link) {
  8.       return $error;
  9.   }  
  10.      mysql_select_db($DB_DATA, $link) or $error = mysql_error();
  11.   return $link;
  12.  }

It is far from perfect, but it works. It helps check the crawl-rate of pages. You can filter out the useragent ‘Googlebot’ and run a query grouping useragent by date and see what times Googlebot comes by.

  1. //66.249.72.143 is GoogleBot
  2. check_traffic_by_IP('66.249.72.143' );
  3.  
  4. function check_traffic_by_IP($thisIP) {
  5.  $m = connect_data();
  6.  $qry = "SELECT * from `log` WHERE `ip` = '".$thisIP."' GROUP BY `visitdate`";
  7.  $lst = mysql_query($qry, $m);
  8.  echo '<table><tbody>';
  9.  while($row=mysql_fetch_assoc($lst)) {
  10.   echo '<tr><td>'.$row['visitdate'].'</td></tr>';
  11.  }
  12.  echo '</tbody></table>';
  13. }

…or exclude all records if the UserAgent indicates it was a Bot, and get a listing of organic beings visiting the pages :

  1. function check_real_traffic() {
  2.  $m = connect_data();
  3.  
  4. //the bots
  5.  $bots = array('googlebot', 'anonymouse');
  6.  
  7.  $qry = "SELECT * from `log`";
  8.  $lst = mysql_query($qry, $m);
  9.  
  10.  echo '<table><tbody>';
  11.  
  12.  while($row=mysql_fetch_assoc($lst)) {
  13.  
  14. //check if the entries in the bots-array
  15. //occur in the stored useragent string
  16.   $traffic=-1;
  17.   for($i=0;$i<count ($bots);$i++) {
  18.    if(strpos(strtolower($row['agent']), $bots[$i])>0) $traffic+=1;
  19.   }
  20.  
  21. //if it's not a bot (then the $traffic var is still -1), output the info
  22.   if($traffic==-1) {
  23.                echo '<tr><td>'.$row['visitdate'].'</td><td>'.$row['ip'].'</td><td>'.$row['page'].'</td></tr>';
  24.                 }
  25.  }
  26.  echo '</count></tbody></table>';
  27. }

…that shows ‘organic’ visitors.

other resources
tek-tips.com/ Limit Requests per day by IP

morrowland.com/ HOW TO SHOW CURRENT VIEWERS IN PHP
phpMyVisites open source free referrer and traffic analysis

Comments
No Comments »
Categories
mysql, php
Tags
mysql, php, robots
Comments rss Comments rss
Trackback Trackback

« Previous Entries

Categories

  • google (3)
  • juust (2)
  • linkdir (1)
  • links (8)
  • mysql (1)
  • optimisation (2)
  • optimization (2)
  • pagerank (8)
  • php (20)
  • sem (2)
  • seo (10)
  • seo tips and tricks (3)
  • serp (6)
  • tool (8)
  • Uncategorized (1)
  • widget (1)
  • wordpress (8)
  • xml-rpc (1)

optimization google php xml-rpc wordpress tool pressmark Uncategorized serp optimisation juust mysql scrape links openid pagerank class robots seo seo tips and tricks sem linkdir widget

nooked

joopita.com free web directory and search engine

interesting sites

  • digitalpoint forums
  • webmaster world
  • bluehatseo.com
  • joost
  • hamlet batista
  • scripture4all.org


Want to rip DVD? go here: easy DVD ripper software

site links

  • marketing blog
  • static pages
  • backlink spider

rss
Comments rss
Blog Directory
Web Developement Blogs - BlogCatalog Blog Directory
Listed in LS Blogs the Blog Directory and Blog Search Engine
Blog Flux Directory
design by jide
sitemap