JSON and AJAX and XML, Oh My!

Published on February 22, 2011 by in Development

11

A question that has been coming up with some regularity lately has to do with doing dynamic, AJAX based work with the content engine in dotCMS. For some time, dotCMS has offered DWR as a means of handling those needs, but ultimately, DWR is a cumbersome, limited tool. In the end, there are two other ways of getting similar results which are much easier for most people to implement.

Since we’re going to ignore the existence of DWR, you’re going to need to be familiar with something like jQuery for doing AJAX work. You’re not going to get away from using JavaScript (it’s… you know… sort of integral to the whole concept of AJAX), but we can at least use something that’s more common and familiar. It doesn’t have to be jQuery, but that’s what I’ll be using here.

Calling a Proxy Page

This is the simplest way of handling AJAX needs. First, we’ll assume that you already know the context that the AJAX response is going to be used in. In that case, we’ll return some actual HTML ready to be inserted into a page. Set up a purely blank template if you don’t already have one (e.g. it has no other HTML on the page besides that supplied from its container), and include a single container. Now, make a page with that template, add a Simple Widget, and code it out to pull some content.

## CHECK TO SEE IF A CATEGORY FILTER WAS SUPPLIED
#if($UtilMethods.isSet($request.getParameter('cat')) && $request.getParameter('cat').length() > 0)
	#set($queryParams = " +categories:${request.getParameter('cat')}")
#end
## PULL THE LIST OF PRODUCT CONTENTLETS
#set($productList = $dotcontent.pull("+structureName:Product$!{queryParams}",0,"modDate desc"))
#if($productList.size() > 0)
	#set($numResults = "$!{productList.size()} product")
	## CHECK TO PLURALIZE 'product'
	#if($productList.size() > 1)
		#set($numResults = "$!{numResults}s")
	#end
#else
	#set($numResults = 'Sorry, no products were found in that category')
#end
<p>Found: $!{numResults}</p>
<ol>
## SHOW THE LIST OF PRODUCTS
#foreach($con in $productList)
	<li><a href="/products/$!{con.urlTitle}">$!{con.title}</a></li>
#end
</ol>

This is a very simple example of an HTML response you could build to have the page return. Realistically, you can include as much or as little information here as you want, it just depends on what you plan to do with the data on the page where it ends up. In this case, assume it’s a basic search filter to show products on a page, allowing the visitor to filter based on the available categories. The result is a basic ordered list of links to the products that matched the search.

Next it’s all about setting up your front end page where visitors will view the content. Create a second page (using a normal template), and we’ll add in a search dropdown box and a little JavaScript to run things.

<form id="productSearch" method="get" action="">
	<label for="productCats">Filter by:</label>
	<select id="productCats">
		<option value="">ALL</option>
#foreach($cat in $categories.getChildrenCategoriesByKey('productTypes'))
		<option value="${cat.key}">${cat.categoryName}</option>
#end
	</select>
</form>  <!-- ${esc.h}productSearch -->

<div id="productSearchResults"></div>

<script type="text/javascript">
	${esc.d}(document).ready(function() {
		var showProducts = function(cat) {
			${esc.d}.ajax({
				type: 'GET',
				url: 'product-response.dot',
				data: 'cat=' + cat,
				dataType: 'html',
				success: function(data) {
					${esc.d}('${esc.h}productSearchResults').html(data);
				}
			});
		};
		$('${esc.h}productCats').change(function() {
			var cat = $(this).val();
			showProducts(cat);
		});
		showProducts(null);
	});
</script>

So, very simple – search happens, jQuery requests info, response is returned, HTML is inserted into div. It’s simple to do, and works great when you know how the HTML is going to be handled.

The JSONContent Servlet

In 1.9, there is a new servlet tool you can use now that will return content as either JSON or XML data on demand. This is perfect when you don’t necessarily know what the HTML markup is that will be needed. And better, simply calling http://www.yoursite.com/JSONContent will give you the basic instructions to using the servlet. It takes six different parameters:

  • q: URLEncoded lucene query (required, try the $UtilMethods.encodeURL($query) viewtool)
  • type: json or xml (default: xml)
  • limit: number of results (default: 10, max of 1000)
  • offset: start at specific result number (default: 0)
  • orderBy: field to order by (default: modDate)
  • debug: true/false; if set, will return JSON as text/plain (defaults: false)

By making an AJAX request to this servlet we can basically do the same job as above. For instance, instead of hitting the product-response.dot page in the JavaScript, we could do http://demo.dotcms.org/JSONContent/?type=xml&q=%2BstructureName:product. This returns all the info on the contentlets in JSON format (remember, you could change the type parameter if you want to stick with XML) that we could then parse into the page. By working this way, we can drop the need for the proxy page, and we only need our main page to send this query. For the same kind of page, the changes would look something like:

<form id="productSearch" method="get" action="">
    <label for="productCats">Filter by:</label>
    <select id="productCats">
        <option value="">ALL</option>
#foreach($cat in $categories.getChildrenCategoriesByKey('productTypes'))
        <option value="${cat.categoryVelocityVarName}">${cat.categoryName}</option>
#end
    </select>
</form>  <!-- ${esc.h}productSearch -->
 
<div id="productSearchResults"></div>
 
<script type="text/javascript">
${esc.d}(document).ready(function() {
	var showProducts = function(cat) {
		var catQ = '';
		if(cat && cat.length > 0) {
			catQ = '%20%2Bcategories:' + cat;
		}
		${esc.d}.ajax({
			url: '/JSONContent/',
			data: 'type=json&q=%2BstructureName:product' + catQ,
			dataType: 'json',
			success: function(data) {
				${esc.d}('${esc.h}productSearchResults').html('');
				${esc.d}('${esc.h}productSearchResults').append('<p>Found: ' + data.contentlets.length + ' products</p><ol></ol>');
				${esc.d}.each(data.contentlets, function(i, contentlet) {
					${esc.d}('${esc.h}productSearchResults ol').append('<li>' + contentlet.title + '</li>');
				});
			}
		});
        };
    
	${esc.d}('${esc.h}productCats').change(function() {
		var cat = ${esc.d}(this).val();
		showProducts(cat);
	});
	showProducts(null);
});
</script>

This effectively produces the exact same results as above. The only change we made to the form itself was the dropdown values now use the category’s Velocity variable name (in the previous example, we used the category key). Otherwise we get a JSON data response back from the /JSONContent servlet, and use the jQuery to loop the results into the page.

Conclusions

Overall, either way works just as well as another, and it is up to you which works best for your style or application. Additionally, you could write your own custom tools to output XML or JSON from dotCMS for other applications. The first method works well when you know the context you’ll use the response in and when you want to limit the amount of JavaScript you’re using. The second way is good when you may not know how the response is used or if you need to manipulate the base data before displaying it.


Photo Credit: AttributionNoncommercialShare Alike Some rights reserved by ptufts

11 Responses to “JSON and AJAX and XML, Oh My!”

  1. Brad Rice says:

    It was a great article, except I would disagree with this statement – “DWR is a cumbersome, limited tool.” I think DWR would be a great tool to use for this if more methods were exposed in dotcms such as a way to query and pull content.

  2. Great post – and using JSON means we can write pages (called by AJAX) that can act as services for more than just returning HTML.

    Rob
    🙂

  3. Moon says:

    Great post. I have a question though.

    How do I get full url of an image field? It looks the json serverlet returns image id instead of its URL.

  4. Finny says:

    I have same question as Moon – how do you get the full url of an image field? If it’s not possible, please post the answer anyway so we’re not wondering.

  5. Finny says:

    insert tumbleweed here

  6. Sorry guys, my email alerts weren’t working apparently on this. If you have the ID, you can always predictively construct the URL using the SpeedyAssetServlet
    .
    1.7
    /dotAsset/${imageIdentifier}

    1.9
    /contentAsset/image/${imageIdentifier}

  7. Henry Versemann says:

    Michael,
    So far I’ve put the first two pieces of code above into widgets and dropped them on to two different pages. I’m stuck in trying to get the search script to work. The form correctly displays my products category options, but when I select one nothing happens. I have copied the code into a dreamweaver file and dreamweaver is flagging line 18 as an error. Line 18 starts with the first line show below:

    ${esc.d}(document).ready(function()
    {
    var showProducts = function(cat)
    {
    ${esc.d}.ajax(
    {
    type: ‘GET’,
    url: ‘/training/hversemann/test-ajax-products-page.html’,
    data: ‘cat=’ + cat,
    dataType: ‘html’,
    success: function(data)
    {
    ${esc.d}(‘${esc.h}productSearchResults’).html(data);
    }
    });
    };

    $(‘${esc.h}productCats’).change(function()
    {
    var cat = $(this).val();

    showProducts(cat);
    });

    showProducts(null);
    });

    Some of the syntax shown is foreign to me. More specifically the following code snippets found throughout the code above ${esc.d}(document).ready, ${esc.d}.ajax, ${esc.d}(‘${esc.h}productSearchResults’).html, and ${esc.h}productCats. I’m guessing that they might be referencing jquery/ajax/asynchronous request objects, but that’s just a guess, at this point. The “url” coded
    in the ajax request map is actually my targeted test page. Can you please explain the foreign (to me) syntax if possible, or can you point me to one or more links to explain that syntax? I’m assuming that the .ajax command is formatting the various format parameters for an ajax request. Thanks for the help.

  8. Michael Fienen says:

    Well, problem number one might be related to developing in something like Dreamweaver, assuming you’re trying to test from there. It has no idea what stuff like ${esc.d} and ${esc.h} mean (they are a way of encoding the $ and # symbols respectively in Velocity code so they don’t collide with any named elements). So for instance, if you look at the line with ${esc.d}(document).ready in it on the actual, rendered HTML page from dotCMS, it actually comes out $(document).ready.

  9. Henry Versemann says:

    I just got around to reading your previous reply to me. I understand the reason for the encoding then. Is JQuery always already available to any page throughout dotcms then, or does a “” tag with a “src” attribute need to be coded to pull in jquery and make it available to be called by the javascript on a page? I ask because the code I have does not seem to do anything when I click any of the offered options in the filter select box. Thanks for clarifying the encoding characters shown in the original code. Here’s the current code I have:

    Filter by: 

    ALL
    #foreach($cat in $categories.getChildrenCategoriesByKey(‘tst-prod-typ’))
    ${cat.categoryName}
    #end

    ${esc.d}(document).ready(
    function()
    {
    var showProducts =
    function(cat)
    {
    var url = “test-ajax-products-page.html”;
    ${esc.d}.ajax( { url: url, type:”GET”, data:”cat=”+cat, dataType:”html”, success:function(data){ ${esc.d}(‘${esc.h}productSearchResults’).html(data);}});
    };

    ${esc.d}(‘${esc.h}productCats’).change(
    function()
    {
    var cat = $(this).val();

    showProducts(cat);
    }
    );

    showProducts(null);
    }
    );

    Is there anything in particular I should look for or can do to help debug the javascript/jquery code using either IE or FireFox, that might point to the problem(s) with the code?

  10. Eric Mcdowell says:

    This is probably a shot in the dark, but the:

    #foreach($cat in $categories.getChildrenCategoriesByKey(‘productTypes’))
    ${cat.categoryName}
    #end

    Snippet has me tripped up. Is categories a field on the product content, or is it some dotCMS structure? Or is “productTypes” the field?

    • Michael Fienen says:

      Eric, in this case, $categories is a reference to the categories viewtool, and productTypes is the key of a category is the system that has X number of children categories. Category keys are a way of uniquely identifying a specific category. So, semantically, $categories.getChildrenCategoriesByKey(‘productTypes’) means “Get me every child category of the parent ‘productTypes’.” The Categories portlet can typically be found under the Structures tab in the dotCMS backend.

Leave a Reply