« October 2005 | Main | December 2005 »

November 07, 2005

Simplifying XML navigation in Ruby

So I have been developing a bit of Ruby on Rails code that queries web services and map the results to rhtml.  The REXML library is very nice and results in clean code to get the resulting content and provides a nice navigation API.  However, I really wanted an even simpler api to navigate across XML elements in a similar way that you can navigate through a network of database objects using ActiveRecord. 

Since REXML provided most of what I wanted, it was pretty straight forward to get what I wanted.  All I really needed was a class that basically wraps the REXML::Element and  allows navigation through basic properties. 

Here is a quick example of its use from the weather.com web service


  response = Net::HTTP.get_response("xoap.weather.com",
                                    "/weather/local/#{@zip}?cc=*&link=xoap
&prod=xoap" +
                                    "&par=#{@@par_id}&key=#{@@weather_key}";    
  @weather = XMLElementWrapper.new(REXML::Document.new(response.body).root)

Then you can simply access the network of objects as:


@weather.cc.wind.s   # city wind speed
@weather.cc.hmid     # city humidity
@weather.loc.sunr    # time of sunrise

This allows me to think of the XML graph as a network of objects instead of xml contents.  The weather and Yahoo demos on openrico.org both use this approach.

Notice that each property is either an attribute or another XML wrapper that we can continue navigating through.  Now, if the XML tags are too cryptic, we could add a mechanism to define mappings ('cc' and 'loc' are a bit cryptic).  This would be trivial in Ruby.  However, I have not implemented user defined mapping in this example.

Also the wrapper has an 'each' method that would support iteration through children of a specific tag.

Now the code for the wrapper is very simple due to the nice features provided in REXML:


class XMLElementWrapper
  def initialize( element);
    @element = element;
    @cache = {}
  end

   def method_missing(method_id)
       elem = @element.elements[method_id.to_s];

      if  elem == nil && @element.attributes != nil then
         return @element.attributes[method_id.to_s] 
      end

      if elem == nil || (elem.attributes.empty? && elem.elements.empty? && elem.text == nil)then
         return nil;
      end
      
      wrap(elem);
   end
   
   def name() @element.name end
   
   def each
      @element.elements.each{|e| yield DynamicXMLElement.new(e);}
   end
   
   def each(name)
      name = name.to_s;
      @element.elements.each{|e| yield wrap(e) if e.name == name}
   end
   
   def wrap(element) 
      @cache[element] = create(element) if @cache[element] == nil;
      @cache[element]
   end
   
   def create(element) XMLElementWrapper.new(element) end
   
   def to_xml() @element end
   
   def to_s() @element.text end
      
   def to_f() @element.text.to_f end
      
   def to_i() @element.text.to_i end
end

The class uses the 'method_missing' method to handle all the accessors that are getting attributes or navigating to other objects.  It also caches each navigation and uses a factory method so that it can be easily extended to customize the creation of subnodes.

The following example might be used to provide an xml wrapper that sanitizes or textifies all contents for displaying something like yahoo search results in your page.

class SanitizedXMLObject
  def create(element)
     SanitizedXMLObject.new(element)
  end

  def to_s() sanitize super end
end

Like I said, this is very simplistic and I am sure it does not serve all needs for XML navigation (like XPath).  However, you can retrieve the REXML::Element when you need to break out of the object navigation approach.

The cool thing about Ruby and the current libraries, is that I can do this with such simple code.

November 06, 2005

Openrico Yahoo Example in Rails

When I converted the yahoo demo to Ruby on Rails, the resulting ruby code was quite simple - actually downright trivial. Part of this simplification is due to the way Ruby allows the developer to think about the solution in its simplest most direct form. 

The original java code used beans that were built from the xml results.  I really liked the way you could navigate from bean to bean through simple properties.  However, I did not really want to define all the beans and mappings.  Call me lazy.  So, I decided to do a simple wrapper around the XML root from REXML so that I could access the nested xml elements/objects as simple named properties.  This allowed me to do the following implementation for the yahoo demo of the LiveGrid.  It has 3 ajax calls for the web, image, and video search.

@@YAHOO_HOST =  "api.search.yahoo.com";  

after_filter :mark_ajax, :only => [:ajax_yahoo_web_search,
                                   :ajax_yahoo_image_search,
                                   :ajax_yahoo_video_search];   

def mark_ajax
   @response.headers["content-type"] = 'text/xml'; 
end


def ajax_yahoo_web_search
@yahoo = _get_object_response(@@YAHOO_HOST,      
                               _yahoo_path("/WebSearchService/V1/webSearch");
render :action => :ajax_yahoo_web_mock unless !@yahoo.nil?;   
end
def ajax_yahoo_image_search
   @yahoo = _get_object_response(@@YAHOO_HOST,
                                 _yahoo_path("/ImageSearchService/V1/imageSearch"));
   render :action => :ajax_yahoo_image_mock unless !@yahoo.nil?;      
end
def ajax_yahoo_video_search      
   @yahoo = _get_object_response(@@YAHOO_HOST,
                                 _yahoo_path("/VideoSearchService/V1/videoSearch"));      
   render :action => :ajax_yahoo_video_mock unless !@yahoo.nil?; 
end

private

def _yahoo_path(base)   
"#{base}?query=#{@params[:query]}&start=#{@params[:offset].to_i+1}&results=#{@params[:page_size]}&appid=#{@@yahoo_app_id}"
end

def _get_object_response(host, path, sanitize)
   response = Net::HTTP.get_response(host, path);

   if (response.message != "OK") then
      flash.now[:notice] = "Could not access server";
      return nil;  #let the caller generate mock data
   end;
   SanitizedXMLObject.new(REXML::Document.new(response.body).root)
end

We use a The SanitizedXMLObject that extends the XML wrapper class and automatically textifies (so we can nest it into our page cleanly) all content.  This takes the root of the REXML::Document object and makes it simpler to work with.

This is all the code needed to provide the 3 yahoo search actions that query the Yahoo search API and prepares the results ready for the rhtml templates. 

Now, the template is very simple and clean and uses the @yahoo as if it is a network of objects that can be traversed as either a property or through an "each" method.

 <ajax-response>
  <response type="object" id="webSearchResultsGrid_updater">
    <rows update_ui="true" >
      <% i = 0;
        @yahoo.each(:Result) do |result| %>
        <tr>
            <td>
               <span class="webSearchIndex"><%=@params[:offset].to_i + (i+=1)%>.</span>
            </td>
            <td>
               <a class="webSearchTitle" href="<%=result.ClickUrl%>">
                  <%=result.Title%>
               </a>
                <div class="yahooSearchItemSummary">
                   <%=result.Summary == nil ? "" : result.Summary.to_s[0..200]%>
                </div>
                <div class="webSearchUrl"><%=result.Url%> -
                   <span class="webSearchFileFormat">
                      <%=result.MimeType%>
                   </span>
                </div>
            </td>
        </tr>
    <% end %>
    </rows>
  </response>
    <response type="element" id="webResultStats">
      <span>&#160;of about <%= @yahoo.totalResultsAvailable%> for <%=@params[:query]%></span>
    </response>
    <response type="object" id="configureWebSearchRows">
      <numResults><%=[@yahoo.totalResultsAvailable.to_i, 1000].min%></numResults>
    </response>

</ajax-response>

This seems to be pretty straight forward (as it should be).

The initial page has a bit more content and javascript code to initiate the LiveGrid with different types of search results.  I will not go over it at this point, but it is available here for download.

I will talk later about the wrapper class for the xml.

November 02, 2005

Safari now supports LiveGrid & AjaxEngine

I updated with the latest version of Mac OSX 10.4.3 and now the Rico LiveGrid and AjaxEngine work fine with Safari.  We still have a few issues with some of the other Rico components on Safari, but this is a big help.

Thanks Wayne for bringing this to my attention.