juust ~ php oddities » xml-rpc http://www.juust.org Link theory and search engine optimization Thu, 19 Jan 2012 09:39:32 +0000 en hourly 1 http://wordpress.org/?v=3.2.1 RpSequel : rfc sql crud with wordpress http://www.juust.org/index.php/rpsequel-rfc-sql-crud-with-wordpress/2009/10/ http://www.juust.org/index.php/rpsequel-rfc-sql-crud-with-wordpress/2009/10/#comments Fri, 23 Oct 2009 09:50:07 +0000 juust http://www.juust.org/?p=1011 Continue reading →]]> I set out to use MsAccess with xhr/ajax to maintain tables I added to my wordpress database, from my desktop. (Because I suck at html forms backends and consider them a waste of time.)

I used a similar technique ten years ago, setting up msaccess as reporting tool for SAP R/3 with RFC dll’s and ActiveX. That remained stable for eight years without maintenance. Hey, I might get lucky with WordPress xml-rpc and xhr/ajax.

I called the example RpSequel.

adding sql rpc functions

To sync the data sets, I will duplicate a list with sequel operations from my desktop database as rfc call to my blog’s xmlrpc-endpoint. To handle that list, I plug a sql crud method into the xml-rpc method array in WordPress :

  1. add_filter( 'xmlrpc_methods', 'rpsequel_methods' );
  2.  
  3. function rpsequel_methods( $methods ) {
  4.     $methods['rpsequel.rpsequelInsert'] = 'rpsequelInsert';
  5.     return $methods;
  6. }
  7.  
  8. function rpsequelInsert($args) {
  9. }

The basic INSERT method itself can be simple:

  1. function rpsequelInsert($args) {
  2.         global $wpdb;
  3.  
  4. //the first parameters
  5.         $blog_id = (int) $args[0];
  6.         $username = $args[1];
  7.         $password = $args[2];
  8.         $tablename      = $args[3];
  9.  
  10. //get the structs
  11.         $structs        = $args[4];
  12.  
  13. //pick the first
  14.         $struct = $structs[0];
  15.  
  16. //compose the mysql insert statement    
  17.         foreach($struct as $key => $value){
  18.             $SqlFields .= " `".$key . "`,";
  19.             $SqlValues .= " '".$value . "',";
  20.         }
  21.  
  22.         $SqlFields=substr($SqlFields, 0, strlen($SqlFields)-1);
  23.         $SqlValues=substr($SqlValues, 0, strlen($SqlValues)-1);
  24.    
  25.         $SqlStatement .= "INSERT INTO ".$wpdb->prefix.$tablename. " (". $SqlFields .  ") VALUES (".$SqlValues . ")";
  26.        
  27. //execute the query        
  28.         $wpdb->query($wpdb->prepare($SqlStatement));
  29.  
  30. //return the record id
  31.        return $wpdb->insert_id;
  32. }

That in itself is not very exciting, it stuffs records in the host’s database. Which is fine, however, I want the blog to respond to individual operations.

I can write rpc-functions for every single function I want the blog to perform, that means truckloads of rpc-functions, on both client and server end. I am incredibly lazy, so I ain’t gonna go there.

adding hooks to sql rpc functions

Lucky for me, WordPress has hooks, hooks are cool.

Adding action hooks (before_insert and after_insert) to the crud method makes it more powerful. Two hooks are enough to separate the business logic of the desktop database from the blogs reporting logic.

  1. function rpsequelInsert($args) {
  2.         global $wpdb;
  3.  
  4. //the first parameters
  5.         $blog_id = (int) $args[0];
  6.         $username = $args[1];
  7.         $password = $args[2];
  8.         $tablename      = $args[3];
  9.  
  10. //get the structs
  11.         $structs        = $args[4];
  12.  
  13. //here is the first action hook,
  14. //it hands the tablename and the records over to any
  15. //function that 'listens' on the action hook
  16.  
  17. do_action('rpsequel_before_insert',  $tablename, $structs);
  18.  
  19. //the rest of the rather dull method
  20.  
  21.         $struct = $structs[0];
  22. //….
  23. //execute the query        
  24.         $wpdb->query($wpdb->prepare($SqlStatement));
  25.  
  26. //here is the second action hook,
  27. //it hands the table name with the new record id
  28. //to any function that 'listens' on the action hook
  29.  
  30. do_action('rpsequel_after_insert',  $tablename, $wpdb->insert_id);
  31.  
  32. //return the record id
  33.        return $wpdb->insert_id;
  34. }

Now it is more exciting.

  • I can send a list of records and a rfc insert-method to the blogs xmlrpc endpoint
  • the rfc crud-plugin can process the records one by one
  • Before and after each insert operation, the method triggers an action.
  • Before inserting, it exposes the record data.
  • After inserting, it exposes the record id.

And the last two, was exactly what I wanted.

adding functionality with plugins

Now I can add tiny plugins, that ‘listen’ on the action hooks in the rfc-methods. If there is an INSERT into the database, my plugins read which table it affects. They can perform actions, either before the insert, with the new record data, or after the insert, with the new record id.

  1. add_action ( 'rpsequel_before_insert', 'before_insert_logic', 10, 2);
  2. add_action ( 'rpsequel_after_insert', 'after_insert_logic', 10, 2);
  3.  
  4. function before_insert_logic($rpc_tablename, $rpc_array) {
  5.    if($rpc_tablename=="ships") {
  6. //do some stuff before inserting incoming records
  7.    }
  8. }
  9.  
  10. function after_insert_logic($rpc_tablename, $rpc_insert_id) {
  11.    if($rpc_tablename=="ships") {
  12. //do some stuff after inserting incoming records
  13.    }
  14. }

That’s basically all it takes. As technique, it has it’s limitations, but it can come in handy sometimes.

]]> http://www.juust.org/index.php/rpsequel-rfc-sql-crud-with-wordpress/2009/10/feed/ 0 integrating ms office and wordpress with vba and xml-rpc http://www.juust.org/index.php/integrating-ms-office-and-wordpress-with-vba-and-xml-rpc/2009/10/ http://www.juust.org/index.php/integrating-ms-office-and-wordpress-with-vba-and-xml-rpc/2009/10/#comments Tue, 20 Oct 2009 11:20:53 +0000 juust http://www.juust.org/?p=979 Continue reading →]]> Okay.

Yesterday I made some basic stuff to grab my WordPress blog data with visual basic for applicationsn cos I don’t feel like programming php admin pages for tables.

So let’s make a basic xml-rpc crud plugin and put my agenda on vba remote control.

The basic xml-rpc plugin is simple, plug extra methods into the xml-rpc method array, and write a function per crud-method.

  1.  
  2. add_filter( 'xmlrpc_methods', 'add_agenda_xmlrpc_methods' );
  3.  
  4. function add_agenda_xmlrpc_methods( $methods ) {
  5.     $methods['agenda.addAgendaItem'] = 'addAgendaItem';
  6.     $methods['agenda.updateAgendaItem'] = 'updateAgendaItem';
  7.     $methods['agenda.deleteAgendaItem'] = 'deleteAgendaItem';
  8.     $methods['agenda.reportAgendaItem'] = 'reportAgendaItem';
  9.     return $methods;
  10. }
  11.  
  12. function addAgendaItem($args) { }
  13.  
  14. function updateAgendaItem($args) { }
  15.  
  16. function deleteAgendaItem($args) { }
  17.  
  18. function reportAgendaItem($args) { }
  19.  
  20. //basic login helper function
  21. function CheckLogin($user, $pwd) {}

There now, if I call on the xmlrpc.php file, the extra methods are added to the callback array and I can use the table CRUD functions from my vba desktop.

  1. function addAgendaItem($args) {
  2.  
  3.         $blog_id = (int) $args[0];
  4.         $username = $args[1];
  5.         $password = $args[2];
  6.         $AgendaItem     = $args[3];
  7.  
  8. //remember : add a login check
  9. //(for the example it is irrelevant)
  10.  
  11.         global $wpdb;
  12.         $sql = "INSERT INTO ".$wpdb->prefix."Agenda (
  13.                `userid`, `tags`, `description`, `firstdate`, `enddate`, `link`, `price`, `location`
  14.                ) VALUES (
  15.                '".$AgendaItem[0]['userid']."',
  16.                '".$AgendaItem[0]['tags']."',
  17.                '".$AgendaItem[0]['description']."',
  18.                '".$AgendaItem[0]['firstdate']."',
  19.                '".$AgendaItem[0]['enddate']."',
  20.                '".$AgendaItem[0]['link']."',
  21.                '".$AgendaItem[0]['price']."',
  22.                '".$AgendaItem[0]['location']."'                
  23.                )";
  24.         $wpdb->query($wpdb->prepare($sql));
  25.  
  26.         return $wpdb->insert_id;
  27. }

note : the agendaitem is a struct in an array (see below), I use [0] to get the first struct (which is the actual array with field-value pairs, my record with agenda info).

Activate the plugin, and write a simple test

  1.  
  2. 'Type to hold an agenda info record
  3. Type AgendaItem
  4.     userid As String
  5.     tags As String
  6.     Description As String
  7.     firstdate As String
  8.     enddate As String
  9.     link As String
  10.     price As String
  11.     location As String
  12. End Type
  13.  
  14.  
  15. Function AddAgendaItem()
  16.  
  17. txtURL = "http://wwwblog.com/xmlrpc.php"
  18. txtUserName = "MyUsername"
  19. txtPassword = "MyPassword"
  20.  
  21.   Dim objSvrHTTP As ServerXMLHTTP
  22.   Dim strT As String
  23.   Set objSvrHTTP = New ServerXMLHTTP
  24.  
  25.   objSvrHTTP.Open "POST", txtURL, False, CStr(txtUserName), _
  26.    CStr(txtPassword)
  27.  
  28.   objSvrHTTP.setRequestHeader "Accept", "application/xml"
  29.   objSvrHTTP.setRequestHeader "Content-Type", "application/xml"
  30.  
  31.     strT = ""
  32.     strT = strT & "<methodcall>"
  33.     strT = strT & "<methodname>agenda.addAgendaItem</methodname>"
  34.    
  35.     strT = strT & "<params>"
  36.     strT = strT & "<param><value><string>" & txtBlogId & "</string></value></param>"
  37.     strT = strT & "<param><value><string>" & txtUserName & "</string></value></param>"
  38.     strT = strT & "<param><value><string>" & txtPassword & "</string></value></param>"
  39.  
  40. 'now we go make the struct, I use a Type (stdobject), normally
  41. 'you'd use a recordset
  42.  
  43. Dim a As AgendaItem
  44.  
  45. With a
  46.     .userid = 1
  47.     .Description = "rpc testing"
  48.     .firstdate = "2009/10/14"
  49.     .enddate = "2009/10/14"
  50.     .link = "http://www.juust.org/"
  51.     .price = "rpc testing"
  52.     .location = "limmen"
  53.     .tags = "php, xml-rpc"
  54. End With
  55.  
  56.     strT = strT & "<param><value><array>"
  57.     strT = strT & "<data>"
  58.    
  59.     strT = strT & "<value><struct>"
  60.  
  61.     strT = strT & "<member><name>userid</name><value><string>" & a.userid & "</string></value></member>"
  62.     strT = strT & "<member><name>tags</name><value><string>" & a.tags & "</string></value></member>"
  63.     strT = strT & "<member><name>description</name><value><string>" & a.Description & "</string></value></member>"
  64.     strT = strT & "<member><name>firstdate</name><value><string>" & a.firstdate & "</string></value></member>"
  65.     strT = strT & "<member><name>enddate</name><value><string>" & a.enddate & "</string></value></member>"
  66.     strT = strT & "<member><name>link</name><value><string>" & a.link & "</string></value></member>"
  67.     strT = strT & "<member><name>price</name><value><string>" & a.price & "</string></value></member>"
  68.     strT = strT & "<member><name>location</name><value><string>" & a.location & "</string></value></member>"
  69.  
  70. 'close the struct    
  71.     strT = strT & "</struct></value>"
  72.     strT = strT & "</data>"
  73.  
  74. 'close the struct array
  75.     strT = strT & "</array></value></param>"
  76.  
  77. 'end parameters
  78.     strT = strT & "</params>"
  79.  
  80. 'end method
  81.     strT = strT & "</methodcall>"
  82.  
  83. 'send the lot to the blog  
  84.   objSvrHTTP.send strT
  85.  
  86. 'print the response to debug
  87.   Debug.Print  objSvrHTTP.responseText
  88.  
  89. End function

Et voila :

agenda rpc

That’s yer basic Office-Wordpress XML-RPC integration.

]]> http://www.juust.org/index.php/integrating-ms-office-and-wordpress-with-vba-and-xml-rpc/2009/10/feed/ 0 a social spider http://www.juust.org/index.php/a-social-spider/2009/01/ http://www.juust.org/index.php/a-social-spider/2009/01/#comments Sat, 24 Jan 2009 04:10:22 +0000 juust http://www.juust.org/?p=310 Continue reading →]]> I was reading about the BloGee project and some other stuff and then I thought “how much trouble would it be writing a WordPress plugin to do some basic ‘social spidering’”. BloGee is about a micro-content format and that’s a bit out of my scope.

I want a simple ‘social’ spider for WordPress so I am going to take some functions of simpleTags and add them to the WordPress xml-rpc server methods, to get some basic functionality I can call upon through the xml-rpc endpoint.

I don’t know if I mentioned that, adding methods to WordPress XML-RPC differs from the straight forward use of Incutio because WordPress uses its own filter/hook system, the actual IXR_Server instance is made and managed by WordPress itself.

I hence don’t use the

  1. class PeekAtYouServer extends IXR_Server {

style class instancing, in stead I make a class where I hook the function I would normally hand to the IXR_Server as callback into wordpress by adding method/callback to the ‘xmlrpc_methods’ filter array.

  1.  class PeekAtYouServer {
  2.  
  3.   function PeekAtYouServer() {
  4.   //  add callbacks as methods to the array (filter) xmlrpc_methods
  5.    add_filter('xmlrpc_methods', array(&$this, 'filterXmlrpcMethods'));
  6.   }
  7.  
  8.   /**
  9.    * Here I connect the methodName pay.PeekAtYou to a custom function
  10.    *
  11.    * This is the array that is added (as pair) to the xmlrpc_methods filter
  12.    **/
  13.  
  14.   function filterXmlrpcMethods(&$methods) {
  15.    $methods['pay.PeekAtYou'] = array(&$this, 'onXmlRpcpayPeekAtYou');
  16.    return $methods;
  17.   }
  18.  
  19.   /**
  20.    * the custom function used as callback for pay.PeekAtYou
  21.    **/
  22.  
  23.   function onXmlRpcpayPeekAtYou($args) {
  24.     global $wpdb;
  25. //grab posts
  26.     $sql = "SELECT ID, post_title FROM " . $wpdb->posts . " WHERE post_status = 'publish'";
  27.     $posts = $wpdb->get_results($sql);
  28. //cycle through all posts and grab the IDs
  29.  
  30.     $result = array();
  31.     if (!empty($posts)) {
  32.      foreach ($posts as $post) $this->postids[] = $post->ID;
  33. //I got all the ids in an array,
  34. //now I grab the tags (query is from the simpleTags plugin)
  35.      $this->getTagsFromCurrentPosts();
  36.     } else {
  37.      return new IXR_Error(404, 'no posts for the selected criterium.');
  38.     }
  39.     return $this->tags_currentposts;
  40.   }
  41.  
  42.  /**
  43.   * Get tags from current post views
  44.   * (SimpleTags plugin)
  45.   * @return boolean
  46.   */
  47.  function getTagsFromCurrentPosts() {
  48.   if ( is_array($this->postids) && count($this->postids) > 0 ) {
  49.  
  50.    // Generate SQL from post id
  51.    $postlist = implode( "', '", $this->postids);
  52.  
  53.    global $wpdb;
  54.    $results = $wpdb->get_results("
  55.    SELECT t.name AS name, t.term_id AS term_id, tt.count AS count
  56.    FROM {$wpdb->term_relationships} AS tr
  57.    INNER JOIN {$wpdb->term_taxonomy} AS tt ON (tr.term_taxonomy_id = tt.term_taxonomy_id)
  58.    INNER JOIN {$wpdb->terms} AS t ON (tt.term_id = t.term_id)
  59.    WHERE tt.taxonomy = 'post_tag'
  60.    AND ( tr.object_id IN ('{$postlist}') )
  61.    GROUP BY t.term_id
  62.    ORDER BY tt.count DESC");
  63.  
  64.    $this->tags_currentposts = $results;
  65.    unset($results, $key);
  66.   }
  67.   return true;
  68.  }
  69.  }

I saved the file as PeekAtYouServer.class.php.

Now I need a simple file for WordPress to ‘spot the plugin’, so I can activate it and make the class instance, that adds the custom method and callback function.

  1.  /*
  2.  Plugin Name: PeekAtYou XMLRPC Server
  3.  Plugin URI: http://www.juust.org/
  4.  Description: Adds Social Spidering to your blog
  5.  Author: juust
  6.  Author URI: http://www.juust.org/
  7.  License: GPL
  8.  Version: 1.1
  9.  */
  10.  
  11.  require_once 'PeekAtYouServer.class.php';
  12.  $PeekAtYouServer = new PeekAtYouServer();

I save that as PeekAtYouServer.php and upload the lot to a directory /wp-plugins/pay-xmlrpc-server.

In the Plugin screen (wp 2.5) I can activate the plugin, and then make a call to the xmlrpc-endpoint of the blog using pay.PeekAtYou as methodName.

  1.  include('wp-includes/class-IXR.php');
  2.         $client = new IXR_Client('http://www.juust.org/xmlrpc.php');
  3.  $client->query('pay.PeekAtYou');
  4.  $response = &$client->getResponse();
  5.  print_r( $response);

That returns all tags the blog uses.

Next week : adding some basic social blog-spider functions.

]]> http://www.juust.org/index.php/a-social-spider/2009/01/feed/ 1 hands on xml-rpc : copying msql tables http://www.juust.org/index.php/xml-rpc-copy-mysql-table/2009/01/ http://www.juust.org/index.php/xml-rpc-copy-mysql-table/2009/01/#comments Thu, 08 Jan 2009 17:13:41 +0000 juust http://www.juust.org/?p=286 Continue reading →]]> I don’t have anything to blog on, so I will bore you all with a quick generic function to copy mysql tables from one host to another, using xml-rpc.

I use the Incutio xml-rpc library on both hosts, to handle the tedious stuff (xml formatting and parsing). That leaves only some snippets to send and receive table data and store it on a mysql database.

First : how to handle the table data on the sending end:

  • I take an associative array from a mysql query
  • I make an array to hold the records
  • I add each row as array
  • I make an IXR-client.
  • I add some general parameters
  • I hand these and the entire table array to my IXR-client.
  • send…
  1. //the snippet with the client is at the bottom of the post
  2. $ThisClient = New SerpClient('http://serp.trismegistos.net/db/xmlrpc.php', 'user', 'pass', 'sender');
  3.  
  4. $tablename = "serp_tags_keys";
  5. $tableid = "id";
  6. $result = $serpdb->query("SELECT * FROM ".$tablename);
  7. $recordcount = mysql_num_rows($result);
  8.  
  9. while($row=mysql_fetch_assoc($result)) {
  10.  $record=array();
  11.  foreach($row as $key => $value) $record[$key]=$value;
  12.  $records[]=$record;
  13. }
  14.  
  15. $ThisClient->putTable($tablename, $recordcount, $tableid, $records);

I consider some additional fields necessary for basic integrity checks : I add “ID” as key field, so on the receiving end the server knows which field is my table’s auto-increment field. Other fields are a username, password, tablename and the batch recordcount.

The IXR_Client then generates a tangled mess of xml-tags holding the entire prodecure call and data. (you can put the client on ‘debug’, then it dumps the generated xml to the screen).

The first part of the xml file contains the single parameters :

  • username
  • password
  • tablename
  • recordcount
  • id-field

<methodCall>
<methodName>serp.putTable</methodName>
<params>
<param><value><string>user</string></value></param>
<param><value><string>pass</string></value></param>
<param><value><string>serp_tags_keys</string></value></param>
<param><value><int>91</int></value></param>
<param><value><string>id</string></value></param>

Then the entire table is sent as one parameter in the procedure call.

That parameter is built from an array containing the table rows as ‘struct’. If I want to use the routine for any table, I need the fieldname-value pairs to compose a standard mysql insert statement. A struct type allows me to use key-value pairs in the xml-file that can be parsed back into an array.

<param><value><array>

<data>

<value><struct>
<member><name>id</name><value><string>4</string></value></member>
<member><name>tag</name><value><string>ranking</string></value></member>
<member><name>cat</name><value><string>alexa ranking seo internet ranking internet positi</string></value></member>
<member><name>date</name><value><string>200901</string></value></member>
</struct></value>

<value><struct>
<member><name>id</name><value><string>94</string></value></member>
<member><name>tag</name><value><string>firm</string></value></member>
<member><name>cat</name><value><string>firm seo</string></value></member>
<member><name>date</name><value><string>200901</string></value></member>
</struct></value>

</data>

</array></value></param>

That was the last of the param holding the table, so the entire tag-mess is closed :

</params&gt</methodCall&gt

Then the second part : on the receiving end the Incutio class parses the whole tag-mess, and hands an array of the param sections as input to my function putTable.

  1.  function putTable($args)
  2.  {
  3.   $user   = $args[0];
  4.   $pass   = $args[1];
  5.   $tname   = $args[2];
  6.   $tcount  = $args[3];
  7.   $id           = $args[4];
  8.   $table   = $args[5];

$table is a straightforward array holding as items an array ($t) created from the struct with the pairs of fieldname-value. I turn the recordsets key-value struct into a mysql INSERT query :
$query = “INSERT INTO `”.$tname.”` (” field, field… “) VALUES (” fieldvalue, fieldvalue “)”;

All I have to do is add the fieldnames and fieldvalues to the mysql insert query.

  1.   foreach($table as $t) {
  2.  
  3. //the fixed parts
  4.     $query0 = 'INSERT INTO `'.$tname.'` (';
  5.     $query2 .=") VALUES (";
  6.  
  7. //make the (`fieldname`, `fieldname`, `fieldname`) query-bit
  8. //and the ('fieldvalue', 'fieldvalue', 'fieldvalue') query-bit :
  9.  
  10.     foreach($t as $key=>$value) {
  11.      if($key!=$id) {
  12.       $query1 .="`".$key."`, ";
  13.       $query3 .="'".$value."', ";
  14.      }
  15.     }
  16.  
  17. //remove the trailing ", "
  18.     $query1=substr($query1, 0, strlen($query1)-2);
  19.     $query3=substr($query3, 0, strlen($query3)-2);
  20.  
  21. //glue em up and add the final ")"
  22.     $query0 .= $query1.$query2.$query3.")";
  23.  
  24. //query…
  25.     $this->connection->query($query0);
  26.  
  27. //reset the strings
  28.     $query0='';
  29.     $query1='';
  30.     $query2='';
  31.     $query3='';
  32.    }
  33.  }

that generates mysql queries like
INSERT INTO `serp_tags_keys` (`tag`, `cat`, `date`) VALUES (‘ranking’, ‘alexa ranking’, ’200901′) and copies the entire table.

That is how I handle the table data.

Of course I have to define two custom classes to process the serp.putTable procedure itself, using the Incutio class.

First the class for the sending script, which is pretty straight forward :

  • make an IXR_Client instance
  • hand the record set to it
  • have it formatted and sent
  1. //include the library
  2. include('class-IXR.php');
  3.  
  4. //make a custom class that uses the IXR_client
  5. Class SerpClient
  6. {
  7.  var $rpcurl;         //endpoint
  8.  var $username;   //you go figure
  9.  var $password;
  10.  var $bClient;      //incutio ixr-client instance
  11.  var $myclient;  //machine/host-id
  12.  
  13.     function SerpClient($rpcurl, $username, $password, $myclient)
  14.     {
  15.  $this->rpcurl = $rpcurl;
  16.     if (!$this->connect()) return false;
  17.  
  18.      //Standard variables to send in the message
  19.  $this->rpcurl = (string) $rpcurl;
  20.      $this->username = (string) $username;
  21.      $this->password = (string) $password;
  22.  $this->myclient = (string) $myclient;
  23.      return $this;
  24.     }
  25.  
  26.      function connect()
  27.    {
  28. //basic client, it takes the endpoint url, tests and returns true if it exists
  29.      if($this->bClient = new IXR_Client($this->rpcurl)) return true;
  30.     }
  31.  
  32. //the function I use to send the data
  33.   function putTable($tablename, $recordcount, $tableid, $array)
  34.  {
  35. //first parameter is always the methodname, then the parameters, which are
  36. //added sequential to the xml-file (with the appropriate tags for datatypes.
  37. //the script figures that out. note : it uses htmlentities on strings.
  38.   $this->bClient->query('serp.putTable', $this->username, $this->password, $tablename, $recordcount, $tableid, $array);
  39.  }
  40.  
  41. }

I use it in the snippets above with :

  1. $ThisClient = New SerpClient('http://serp.trismegistos.net/db/xmlrpc.php', 'user', 'pass', 'sender');
  2. //…
  3. $ThisClient->putTable($tname, $tcount, $tableid, $records);

Then, on the receiving end, my program has to know how to handle the xml containing the remote procedure call.

I define an extension on IXR_server and pass serp.putTable as new ‘method’ (callback function).

  1. //go away cookie…
  2. $_COOKIE = array();
  3.  
  4. //make sure you get the posted crap, the ixr instances grabs it input from it
  5. if ( !isset( $HTTP_RAW_POST_DATA ) ) $HTTP_RAW_POST_DATA = file_get_contents( 'php://input' );
  6. if ( isset($HTTP_RAW_POST_DATA) ) $HTTP_RAW_POST_DATA = trim($HTTP_RAW_POST_DATA);
  7.  
  8. //include the library
  9. include('class-IXR.php');
  10.  
  11. //make an extended class
  12. class serp_xmlrpc_server extends IXR_Server {
  13.  
  14. //use the same function name…
  15.  
  16.  function serp_xmlrpc_server() {
  17.  
  18. //build an array of methods :
  19. //first the procedurename you use in the xml-text,
  20. //then which function in the extended class (this one) it maps to
  21. //to be used as $this->method
  22.  
  23.   $this->methods = array('serp.putTable'  => 'this:putTable');
  24.  
  25. //hand em to the IXR server instance that will map it as callback
  26.   $this->IXR_Server($this->methods);
  27.  }
  28.  
  29. //now IXR_Server instance uses ($this->)putTable
  30. //to process incoming xml-text
  31. //containing serp.putTable as methodname
  32.  
  33.   function putTable($args)
  34.  {
  35. //(for routine : see the snippet above to store the xml data in mysql)
  36.  }
  37. }
  38.  
  39. //make the class instance like any regular get-post php program,
  40. //the only actual program line, that instantiates the extended class,
  41. //which handles the posted xml
  42.  
  43. $serp_xmlrpc_server = new serp_xmlrpc_server();

That’s all. I am not going to list a cut-and-paste version. You have to build some stuff with it, then you will come up with lots of stuff you can do with it.

WordPress and iPhone built a plugin that receives pictures from iPhone. WordPress uses Incutio so you can ‘piggyback’ on that and have an iPhone plugin for your own website in two days flat using an ajax lightbox gallery script. Or go monetize small websites with some seo oriented ‘optimisation’ functions like ChangeFooterLinks(array($paidurl, $anchortext)) :) or whatever… boring, isn’t it ?

]]> http://www.juust.org/index.php/xml-rpc-copy-mysql-table/2009/01/feed/ 0 RedHat Seo : scraper auto-blogging http://www.juust.org/index.php/redhat-seo-christmas-edition/2008/12/ http://www.juust.org/index.php/redhat-seo-christmas-edition/2008/12/#comments Fri, 26 Dec 2008 18:07:01 +0000 juust http://www.juust.org/?p=270 Continue reading →]]>

Just give us your endpoint and we’ll take it from there, sparky!

I was going to make one of these tools to scrape google and conjur a full blog out of nowhere, as Christmas special, RedHat Seo. The rough sketch has arrived , far from perfect, but it does produce a blog and don’t even look too shabby. I scraped a small batch of posts off of blogs, keeping the links intact and adding a tribute links. I hope they will pardon me for it.

structure

I use three main classes,

BlogMaker the application
Target the blogs you aim for
WPContent the scraped goodies

…and two support classes

SerpResult scraped urls
Custom_RPC a simple rpc-poster

Target blogs have three texts,

file contents maintenance
blog categories category you post under manual
blog tags tags you list on the blog manual
blog urls urls already used for the blog system

routine

The BlogMaker class grabs a result list (up to 1000 urls per phrase) from Google, extracts the urls and stores them in SerpResult, scrapes the urls and extracts the entry divs, stores div-entries in the WPContent class (that has some basic functions to sanitize the text), and uses the BlogTarget-definitions to post it up blogs with xml-rpc.

usage

  1.  
  2. //make main instance
  3. $Blog = new BlogMaker("keyword");
  4.  
  5. //define a target blog, you can define multiple blogs and refer with code
  6. //then add rpc-url, password and user
  7. //and for every target blog three text-files
  8.  
  9. $T=$Blog->AddTarget(
  10.  'blogcode',
  11.  'http://my.blog.com/xmlrpc.php',
  12.  'password',
  13.  'user',
  14.  'keyword.categories.txt',
  15.  'keyword.tags.txt',
  16.  'keyword.urls.txt'
  17.  );
  18.  
  19. //read the tags, cats and url text files stored on the server
  20. //all retrieved urls are tested, if the target blog already has that
  21. //scraped url, it is discarded.
  22. $T->CSV_GetTags();
  23. $T->List_GetCats();
  24. $T->ReadURL();
  25.  
  26. //grab the google result list
  27. //use params (pages, keywords) to specify search
  28. $Blog->GoogleResults();
  29.  
  30. $a=0;
  31. foreach($Blog->Results as $BlogUrl) {
  32.   $a++;
  33.   echo $BlogUrl->url;
  34. //see if the url isnt used yet
  35.  
  36.  if($T->checkURL(trim($BlogUrl->url))!=true) {
  37.    echo '…checking ';
  38.    flush();
  39. //if not used, get the source
  40.    $BlogUrl->scrape();
  41. //check for divs marked "entry", if they arent there, check "post"
  42. //some blogs use other indications for the content
  43. //but entry and post cover 40%
  44.  
  45.    $entries = $BlogUrl->get_entries();
  46.    if(count($entries)&lt;1) {
  47.     echo 'no entries…';
  48.     flush();
  49.     $entries = $BlogUrl->get_posts();
  50.      if(count($entries)&lt;1) {
  51.       echo 'no posts either…';
  52. //if no entry-post div, mark url as done
  53.  
  54.       $T->RegisterURL($BlogUrl->url);
  55.      }
  56.    }
  57.  
  58.    $ct=0;
  59.    foreach($BlogUrl->WpContentPieces as $WpContent) {
  60. //in the get_entries/get_post function the fragments are stored
  61. //as wpcontent
  62.     $ct++;
  63.  
  64.     if($WpContent->judge(2000, 200, 5)) {
  65.      $WpContent->tribute();  //add tribute link
  66.      $T->settags($WpContent->divcontent); //add tags
  67.      $T->postCustomRPC($WpContent->title, $WpContent->divcontent, 1); //1=publish, 0=draft
  68.      $T->RegisterURL($WpContent->url);  //register use of url
  69. usleep(20000000);  //20 seconds break, for sitemapping
  70.     }
  71.    }
  72.   }
  73.  }

notes

  • xml-rpc needs to be activated explicitly on the wordpress dashboard under settings/writing.
  • categories must be present in the blog
  • url file must be writeable by the server (777)

It seems wordpress builds the sitemap as background process, the standard google xml sitemap plugin wil attempt to build in the cache (takes anywhere between 2 and 10 seconds), and apart from building a sitemap the posts also get pinged around. Giving the install 10 to 20 seconds between posts allows for all the hooked in functions to be completed.

period

That’s about all,
consider it gpl, I added some comments in the source but I will not develop this any further. A mysql backed blogfarm tool (euphemistically called ‘publishing tool’) is more interesting, besides, I am off to the wharves to do some painting.

if you use it, send some feedback,
merry christmas dogheads

]]> http://www.juust.org/index.php/redhat-seo-christmas-edition/2008/12/feed/ 1 an xml rpc endpoint http://www.juust.org/index.php/how-to-program-an-xml-rpc-endpoint/2008/12/ http://www.juust.org/index.php/how-to-program-an-xml-rpc-endpoint/2008/12/#comments Sun, 07 Dec 2008 02:21:15 +0000 juust http://www.juust.org/?p=254 Continue reading →]]> (geek content:) Integrating the Incutio xml rpc class into a phpLinkDirectory install opens a lot of possibilities for running remote control automated networks. For a basic example I took the submit routine of phpLD and the xml-rpc routine from wordpress, deleted all nonsense, and ended up with a simple xml-rpc endpoint for my link directory.

On the sender side, I make an xml file that holds the methodName (which is the function I want to execute remotely : phpld.SubmitLink), and the array values I want to pass to the function as parameters. I attach the xml-string as post to a curl call and fire it at the xmlrpc endpoint.

  1. function getmyxml() {
  2.  
  3. //make the $data array
  4. //normally phpLd makes it when someone submits a site
  5.  $data['LINK_TYPE']=1;
  6.  $data['DESCRIPTION']='DESCRIPTION';
  7.  $data['TITLE']='TITLE';
  8.  $data['OWNER_NAME']='OWNER_NAME';
  9.  $data['URL']='http://www.domain.com/xmlrpc.php';
  10.  $data['ID']='';
  11.  
  12. //put the data array in an xml string to post to the xmlrpc endpoint
  13.  
  14. //make the xml header
  15.  $myxml='< ?xml version="1.0" encoding="UTF-8"?>';
  16.  $myxml.='< methodCall>';
  17.  $myxml.= '< methodName>phpld.SubmitLink< /methodName>';
  18.  $myxml.='< params>';
  19.  
  20. //loop to add the $data elements as param-tags
  21.  foreach($data as $d) {
  22.   $myxml.='< param>';
  23.   $myxml.='< value>< string>'.trim($d).'< /string>< /value>';
  24.   $myxml.='< /param>';
  25.  }
  26.  
  27. //finish the xml file :
  28.  $myxml.='< /params>';
  29.  $myxml.='< /methodCall>';
  30.  
  31. //return it
  32.  return $myxml;
  33. }
  34.  
  35. //make the call to the endpoint
  36.     $ch = curl_init('http://www.domain.com/xmlrpc.php');
  37. //use content-type text/xml as extra header
  38.     curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
  39. //get the xml-string and use it as post var
  40.     curl_setopt($ch, CURLOPT_POSTFIELDS, getmyxml());
  41.     curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  42.     curl_setopt($ch, CURLOPT_TIMEOUT, 1);
  43.     $return = curl_exec($ch);
  44.     unset($ch);
  45. }

So much for the sending end, now the receiving end : the xmlrpc.php endpoint file. I need the Incutio xml handler and the phpLinkdirectory database class, with the proper settings, and a function SubmitLink to add the data in the posted xml-file as a record to the database.

Some general settings :

  1. define('XMLRPC_REQUEST', true);
  2. $_COOKIE = array();
  3. if ( !isset( $HTTP_RAW_POST_DATA ) ) $HTTP_RAW_POST_DATA = file_get_contents( 'php://input' );
  4. if ( isset($HTTP_RAW_POST_DATA) )  $HTTP_RAW_POST_DATA = trim($HTTP_RAW_POST_DATA);

I include the Incutio class file, which is a general xml client/server class, and I include init.php from the phpLD script. In the init.php script the AdoDb database instance is declared, that handles the mysql connection, as well as the basic phpLd environment.

  1. include_once('include/class-IXR.php');
  2. include_once('init.php');
  3. $xmlrpc_logging = 0;

Let’s put them both to work for me : the IXR_Server has to know it has to run the SubmitLink function if that function is set as methodName in the xml file. In php you can add methods dynamically to a class, and in the function call($methodname, $args) the IXR_Server handles callbacks.

In my phpLD_xmlrpc_server class, which extends IXR_Server, I add a function SubmitLink that stuffs the data in the phpld database, and I make an array with xml.methodName=>class:function as key=>value pair and pass that to the IXR_Server, that’s enough.

  1. class phpLD_xmlrpc_server extends IXR_Server {
  2.  
  3.  var $methods=array();
  4.  
  5.  function phpLD_xmlrpc_server() {
  6.  
  7. //mapping the custom methods
  8.     $this->methods['phpld.SubmitLink'] = 'this:SubmitLink';
  9.  
  10. //handing the custom methods to the base class
  11.     $this->IXR_Server($this->methods);
  12.  }

When the IXR class parses the posted xml and finds the phpld.SubmitLink methodName, it executes the custom method ($this->)SubmitLink with the data posted in the xml param tags as input :

  1.  
  2. function SubmitLink($params) {
  3.  
  4. //use db and tables from include(init.php)
  5.  global $db;
  6.  global $tables;
  7.  
  8. //map the param-values passed to the $data array
  9.  $data['LINK_TYPE']=$params[0];
  10.  $data['DESCRIPTION']=$params[1];
  11.  $data['TITLE']=$params[2];
  12.  $data['OWNER_NAME']=$params[3];
  13.  $data['URL']=$params[4];
  14.  $data['ID']='';
  15.  
  16. //pass the data array to the adodb $db instance's Replace function :
  17.   if ($db->Replace($tables['link']['name'], $data, 'ID', true) > 0) {
  18.    return $data['DESCRIPTION']." entered";
  19.   } else {
  20.    return " refused";
  21.   }
  22.  }

The AdoDb function Replace maps key-value to tablefield-value, so I use the database field names as keys for the $data array, and assign the $param-values to them. I pass the $data array to the adoDb function and that takes care of the rest.

After adding the includes, settings, and classes,
all I have to do is add a final call at the end to start a new instance of the extended class to handle the posted xml-data :

  1. $phpLD_xmlrpc_server = new phpLD_xmlrpc_server();

…and I have an xml-rpc endpoint :

  1. define('XMLRPC_REQUEST', true);
  2. $_COOKIE = array();
  3. if ( !isset( $HTTP_RAW_POST_DATA ) ) $HTTP_RAW_POST_DATA = file_get_contents( 'php://input' );
  4. if ( isset($HTTP_RAW_POST_DATA) ) $HTTP_RAW_POST_DATA = trim($HTTP_RAW_POST_DATA);
  5.  
  6. include_once('include/class-IXR.php');
  7. include_once('init.php');
  8.  
  9. $xmlrpc_logging = 0;
  10.  
  11. class phpLD_xmlrpc_server extends IXR_Server {
  12.  
  13.  var $methods=array();
  14.  
  15.  function phpLD_xmlrpc_server() {
  16.     $this->methods['phpld.SubmitLink'] = 'this:SubmitLink';
  17.   $this->IXR_Server($this->methods);
  18.  }
  19.  
  20.  function SubmitLink($args) {
  21.  
  22.  global $db;
  23.  global $tables;
  24.  
  25.  $data['STATUS']         = $args[0];
  26.  $data['IPADDRESS']      = $args[1];
  27.  $data['VALID']          = $args[2];
  28.  $data['LINK_TYPE']      = $args[3];
  29.  
  30.   if ($db->Replace($tables['link']['name'], $data, 'ID', true) > 0) {
  31.    return $data['DESCRIPTION']." entered";
  32.   } else {
  33.    return " refused";
  34.   }
  35.  }
  36. }
  37. $phpLD_xmlrpc_server = new phpLD_xmlrpc_server();

Short and sweet.

The actual strength of xml-rpc is in defining a set of standard methodNames and parameters passed with the methodCall (for instance for small website maintenance tasks and statistics reporting, an rpc.sms protocol as successor to Twitter, or standard socialgraph functions for ajax/javascript/widgets) for developing standard API’s.

]]> http://www.juust.org/index.php/how-to-program-an-xml-rpc-endpoint/2008/12/feed/ 0 xml rpc – remote posting with incutio wordpress ixr class http://www.juust.org/index.php/xml-rpc-remote-posting-with-wordpress-incutio-ixr-class/2008/11/ http://www.juust.org/index.php/xml-rpc-remote-posting-with-wordpress-incutio-ixr-class/2008/11/#comments Sat, 01 Nov 2008 23:42:22 +0000 juust http://www.juust.org/?p=205 Continue reading →]]> In order to build a successful blog network I need xml-rpc. I read up on xml-rpc, incutio ixr and tried some basic flat file xml rpc. It’s either that or a custom php url to post data to and then use the wordpress functions to process it.

Xml-rpc is a lot easier especially with the WordPress Incutio-IXR class:

  1. include_once('wp-includes/class-IXR.php');
  2.  
  3. //just give us your endpoint and we'll take it from there Sparky…
  4. $client = new IXR_Client('http://tryout.blacknorati.com/xmlrpc.php');
  5.  
  6. // xml rpc post function form :
  7. // metaWeblog.newPost( blog_id, username, password, struct, publish )
  8.  
  9. //struct = structure for post data
  10.  $post['title'] = 'test titel';
  11.  $post['description'] = 'test';
  12.  $post['categories'] = array("frontpage");
  13.  
  14. //use 'query' to interact with the server, set publish to 0 (draft) for now
  15.  if (!$client->query('metaWeblog.newPost', 6, 'user', 'password', $post, 0)) {
  16.  die('Something went wrong – '.$client->getErrorCode().' : '.$client->getErrorMessage());
  17.  }
  18.         echo $client->getResponse();

Normally getResponse will return a PostID for this function.

Is it really that simple ? Yes. I skipped to the IXR_Client because my host does not have xmlrpc running and the IXR_class doesn’t use it, and solutions with email don’t easily allow access to tags and categories.

  1. $post['categories'] = array("frontpage");
  2. $post['mt_keywords']='key1, key2';
  3. $post["mt_allow_comments"] ='closed';
  4. $post['mt_allow_pings']='open';
  5. //$post['mt_tb_ping_urls'] ='url url url';
  6. //$post['enclosure']=array('url', 'length', 'type')';

mt_keywords holds my tags, that’s a bit more flexible than categories. For my blogs the comments are closed (mt_allow_comments), mt_allow_pings is open, and the real tasty bit is the trackbacks (mt_tb_ping_urls : simply urls separated with a blank space, just like in the wordpress form). I played with the Arrousi trackback class, that one has an autodiscover routine, so I can try and fetch a trackback url from the content page I am ‘using’.

I don’t work with enclosures, but to be complete I added it here.

That covers most data I want to send around a blog network. Very nice class, Incutio IXR.

Next week : more nonsense.

]]> http://www.juust.org/index.php/xml-rpc-remote-posting-with-wordpress-incutio-ixr-class/2008/11/feed/ 2