what is the forum doing there

A forum out of nowhere, I noticed that no one posts on an empty forum. So I made a new plugin that gets Yahoo Answers and stuffs them in Simplepress. I grabbed a php yahoo answers class from HungryCoder’s blog and hacked a backend together, mapping yahoo categories with a keyword to a forum, retrieving topics and either use cron or direct publishing. It takes 2 minutes to have a forum full of free content. I am going to debug it. Maybe in spring I’ll put it on wordpress.org.

RpSequel : rfc sql crud with wordpress

I set out to use MsAccess with xhr/ajax to maintain tables I added to my wordpress database, from my desktop. (Because I suck at html forms backends and consider them a waste of time.)

I used a similar technique ten years ago, setting up msaccess as reporting tool for SAP R/3 with RFC dll’s and ActiveX. That remained stable for eight years without maintenance. Hey, I might get lucky with WordPress xml-rpc and xhr/ajax.

I called the example RpSequel.

adding sql rpc functions

To sync the data sets, I will duplicate a list with sequel operations from my desktop database as rfc call to my blog’s xmlrpc-endpoint. To handle that list, I plug a sql crud method into the xml-rpc method array in WordPress :

add_filter( 'xmlrpc_methods', 'rpsequel_methods' );

function rpsequel_methods( $methods ) {
    $methods['rpsequel.rpsequelInsert'] = 'rpsequelInsert';
    return $methods;
}

function rpsequelInsert($args) {
}

The basic INSERT method itself can be simple:

function rpsequelInsert($args) {
        global $wpdb;

//the first parameters
        $blog_id	= (int) $args[0];
        $username	= $args[1];
        $password	= $args[2];
        $tablename      = $args[3];

//get the structs
        $structs        = $args[4];

//pick the first 
        $struct = $structs[0];

//compose the mysql insert statement     
        foreach($struct as $key => $value){
            $SqlFields .= " `".$key . "`,";
            $SqlValues .= " '".$value . "',";
        }

        $SqlFields=substr($SqlFields, 0, strlen($SqlFields)-1);
        $SqlValues=substr($SqlValues, 0, strlen($SqlValues)-1);
    
        $SqlStatement .= "INSERT INTO ".$wpdb->prefix.$tablename. " (". $SqlFields .  ") VALUES (".$SqlValues . ")";
        
//execute the query        
        $wpdb->query($wpdb->prepare($SqlStatement));

//return the record id
       return $wpdb->insert_id; 
}

That in itself is not very exciting, it stuffs records in the host’s database. Which is fine, however, I want the blog to respond to individual operations.

I can write rpc-functions for every single function I want the blog to perform, that means truckloads of rpc-functions, on both client and server end. I am incredibly lazy, so I ain’t gonna go there.

adding hooks to sql rpc functions

Lucky for me, WordPress has hooks, hooks are cool.

Adding action hooks (before_insert and after_insert) to the crud method makes it more powerful. Two hooks are enough to separate the business logic of the desktop database from the blogs reporting logic.

function rpsequelInsert($args) {
        global $wpdb;

//the first parameters
        $blog_id	= (int) $args[0];
        $username	= $args[1];
        $password	= $args[2];
        $tablename      = $args[3];

//get the structs
        $structs        = $args[4];

//here is the first action hook,
//it hands the tablename and the records over to any
//function that 'listens' on the action hook

do_action('rpsequel_before_insert',  $tablename, $structs);

//the rest of the rather dull method

        $struct = $structs[0];
//.... 
//execute the query        
        $wpdb->query($wpdb->prepare($SqlStatement));

//here is the second action hook,
//it hands the table name with the new record id
//to any function that 'listens' on the action hook

do_action('rpsequel_after_insert',  $tablename, $wpdb->insert_id);

//return the record id
       return $wpdb->insert_id; 
}

Now it is more exciting.

  • I can send a list of records and a rfc insert-method to the blogs xmlrpc endpoint
  • the rfc crud-plugin can process the records one by one
  • Before and after each insert operation, the method triggers an action.
  • Before inserting, it exposes the record data.
  • After inserting, it exposes the record id.

And the last two, was exactly what I wanted.

adding functionality with plugins

Now I can add tiny plugins, that ‘listen’ on the action hooks in the rfc-methods. If there is an INSERT into the database, my plugins read which table it affects. They can perform actions, either before the insert, with the new record data, or after the insert, with the new record id.

add_action ( 'rpsequel_before_insert', 'before_insert_logic', 10, 2);
add_action ( 'rpsequel_after_insert', 'after_insert_logic', 10, 2);

function before_insert_logic($rpc_tablename, $rpc_array) {
   if($rpc_tablename=="ships") {
//do some stuff before inserting incoming records
   }
}

function after_insert_logic($rpc_tablename, $rpc_insert_id) {
   if($rpc_tablename=="ships") {
//do some stuff after inserting incoming records
   }
}

That’s basically all it takes. As technique, it has it’s limitations, but it can come in handy sometimes.

bing api with php and simplexml

About scraping results off of Bing : Bing use a set of about eight cookies. You can grab 200 results with php curl, as 20 pages of 10, but after the first 200 the Bing server checks for the cookie and for lack of one returns a blank page. I can fidget with the curl cookiejar, but Bing also offer a straighforward API.

Using the Bing API to list search results is easier.

Bing TOS : not for seo rank checks

In the last paragraph of the api guide, Bing give a quick recap of their TOS, you can do max 7 queries per second, and using the results for SEO rank checks is explicitly prohibited.

These following snippets (text source) are hence explicitly not to be used for bing search engine result page (‘serp’) rank checks.

bing api with simplexml

So here is one for web results using php simplexml. The web api (which uses namespaces) allows for retrieving max 1000 results per term at max 50 results per query, you can specify the number of results and the offset, where to start grabbing results.

$Appid="A_VERY_LONG_STRING";
$Query = "seo rank check";
$Numres = 50; //max 50
$Offset = 1;    //up to 1000

$url = 'http://api.search.live.net/xml.aspx?
Appid='.$Appid.'
&query='.$Query.'
&sources=web
&web.count='.$Numres.'
&web.offset='.$Offset;

$feed = simplexml_load_file($url);
//use the web: namespace
 $children =  $feed->children('http://schemas.microsoft.com/LiveSearch/2008/04/XML/web');
      foreach ($children->Web->Results->WebResult as $d) {
                echo $d->Title.'
'; echo $d->Description.'
'; echo $d->Url.'
'; echo $d->DisplayUrl.'
'; }

..and one for the pictures using php simplexml :

$Appid="A_VERY_LONG_STRING";
$Query = "alkmaar";
$Numres = 10;
$Offset = 1;

$url = 'http://api.search.live.net/xml.aspx?';
$url .= 'Appid='.$Appid;
$url .= '&query='.$Query;
$url .= '&sources=image';
$url .= '&image.count='.$Numres;
$url .= '&image.offset='.$Offset;

$feed = simplexml_load_file($url);

//use the mms: namespace       
  $children =  $feed->children('http://schemas.microsoft.com/LiveSearch/2008/04/XML/multimedia');

    echo('
    '); foreach ($children->Image->Results->ImageResult as $d) { echo('
  • ' . $d->Title . '
    '); echo('
    '.$d->Thumbnail->ContentType.'
    '.$d->Thumbnail->Height.'
    '.$d->Thumbnail->Width.'
    '.$d->Thumbnail->FileSize.'
  • '); } echo("
");

I actually like that api, I am going to use that.

bing api with json

Bing seem to prefer you use json, less bandwidth usage. After their example in the api basics guide :


$Numres = 10;
$Offset = 1;
$Query='alkmaar';

$url = 'http://api.search.live.net/json.aspx?';
$url .= 'Appid='.$Appid;
$url .= '&query='.$Query;
$url .= '&sources=image';
$url .= '&image.count='.$Numres;
$url .= '&image.offset='.$Offset;


$response = file_get_contents($url);
$jsonobj = json_decode($response);
echo('
    '); foreach($jsonobj->SearchResponse->Image->Results as $value) { echo('
  • '); echo('
  • '); } echo("
");

Of course there is the old RSS-option, which doesnt require an appid but also falls under the api 2.0 tos, and a soap option.

other sources :
There is a bing api php class made over at routecafe, and a jquery bing plugin using json over at Einar Otto Stangvik’s blog.