Leveraging XML in Automatic Conversion of Your CLient/Server Applications to Internet

Build XML-producing Java Servlets, JSP Framework,

XSL stylesheets, formatting, validation and calculation JavaScript with the tools you are familiar with.

Written by

ÜVictor Rasputnis

Anatole TartakovskyÞ

Produce Intranet and Web application easily with integrated customizable XML/XSL-driven approach

Leveraging XML in Automatic Conversion of Your

Client/Server Applications to Internet.

Introduction.

IT projects closely follow the technology path: we see more and more Java/XML/HTML projects today. They replace PowerBuilder or VisualBasic systems developed just a few years ago. And developers ask themselves the very same question: Do I have to rewrite the very same app from scratch again?

Integrating existing systems developed in the “previous millennium” environments with “net” is a costly and difficult task. The other approach would be to “convert” existing applications to native Internet technologies.

Sounds complex? But in reality, it’s not. The source of “legacy” systems contains rich metadata, although in a proprietary format, such as PowerBuilder’s PBL or VisualBasic’s FRM files. All graphic controls – listboxes, buttons, labels, etc. – show up in that metadata with all their positions, sizes, colors and other attributes. Database queries allow reconstruction of the original SQL select statements or stored procedure calls. Code scripting of the events is available also.

Suppose we learn how to read that metadata and get it into XML format. What can we do with it? We can generate systems for the “net” by automatic conversion of the existing legacy code.

Magic Wand

In this article we will outline the design of the “magic wand” converting Client/Server programs into Java/XML/XSL solution. In particular, we will demonstrate how you can leverage investment in all of your PowerBuilder’s datawindows migrating them into sites residing on J2EE Application Servers. Authors maintain free online implementation of the conversion software at www.xmlsp.com.

The following diagram presents the logical flow of the conversion process:

Figure 1. Conversion process flow

Conversion process starts with extraction of the XML metadata. Included in this step is translation of PowerScript formulas into corresponding Java and JavaScript code.

Below is abridged fragment of XML metadata for a simple report listing data from a “Customer” table:

…<COLUMNS>

<column name="id" javatype="Int" xsltype="number" dbtype="int" updatekey="yes" updatable="yes" colId="1"/>…

</COLUMNS>

<QUERY Method="allupdatable" Table="customer"> <SELECT>SELECT id, fname, lname, address, city, state, zip, phone, company_name FROM customer </SELECT></QUERY>

<PRESENTATION ><LAYOUT>Grid</LAYOUT>…

<STATICTEXT><NAME>fname_t</NAME><X>4px</X><Y>2px</Y><W>98px</W> <H>15px</H> <ALIGN>center</ALIGN><WEIGHT>bold</WEIGHT><SIZE>8pt</SIZE>

<FONT>"MS Sans Serif"</FONT><VALUE>First Name</VALUE> </STATICTEXT>

…<DATAFIELD><NAME>state</NAME><TABORDER>0</TABORDER><X>576px</X> <Y>2px</Y><W>75px</W><H>16px</H> <ALIGN>left</ALIGN><SIZE>8pt</SIZE>

<FONT>"MS Sans Serif"</FONT><VALUE>state</VALUE>

<LOOKUP><NAME>d_dddw_state</NAME><CASE>any</CASE><EDITABLE>no</EDITABLE>

<DATA>state</DATA><DISPLAY>state_id</DISPLAY></LOOKUP>

<CONTROLTYPE>lookup</CONTROLTYPE><LABEL>State</LABEL>

</DATAFIELD>…</LAYER></PRESENTATION>

Extracted XML metadata undergoes several independent XSL transformations in order to generate Java servlet, XSL stylesheet, XML Schema, JSP framework objects, etc. Finally, appropriate generated objects get deployed into a J2EE server as components of Web Application.

Code generation via metadata: The basic idea

Powered with XSLT, metadata in XML format simply lends itself into code generation. For, instance, we can write a generic XSL stylesheet to convert metadata into a Java code that executes query and provides data in XML format. We would not have to write and debug servlets anymore, but just generate them! The following fragment of servlet generation stylesheet produces query and data output portions of the code:

<xsl:template match="SELECT">

public PreparedStatement getPreparedStatement(Connection conn) throws Exception {

PreparedStatement st = null;

String sSQL = "<xsl:value-of select="." disable-output-escaping="yes"/>";

st=conn.prepareStatement(sSQL);

return st; }

public ResultSet doRetrieve(PreparedStatement st,ServletParameters parms) throws Exception{

java.sql.ResultSet rs=null;

rs = st.executeQuery();

return rs; }

</xsl:template>

<xsl:template match="COLUMNS" ">

public String getXML() throws Exception {

StringBuffer sb = new StringBuffer();

rowId++;

sb.append("<Detail id=\""+rowId+"\">");

<xsl:for-each select=”column”>

sb.append("<<xsl:value-of select="@name"/>>"+<xsl:value-of select="@name"/>+"</<xsl:value-of select="@name"/>>");

sb.append("</Detail">");

return sb.toString(); };

</xsl:template>

In our sample case, it results in the following section in Java code:

public PreparedStatement getPreparedStatement(Connection conn) throws Exception {

PreparedStatement st = null;

String sSQL = SELECT id, fname, lname, address, city, state, zip, phone, company_name FROM customer ";

st=conn.prepareStatement(sSQL);

return st; }

public ResultSet doRetrieve(PreparedStatement st,ServletParameters parms) throws Exception{

java.sql.ResultSet rs=null;

rs = st.executeQuery();

return rs; }

public String getXML() throws Exception {

StringBuffer sb = new StringBuffer();

rowId++;

sb.append("<Detail id=\""+rowId+"\">");

sb.append("<id>"+id+"</id>");…//other columns to follow

sb.append("</Detail">");

return sb.toString(); };

Similarly, XML metadata precisely prescribes how to layout all fields and records on the page with position, size, color, font, etc. It means, we have full information to build the XSL stylesheet transforming the output of the generated servlet into HTML. Better yet, we can generate such a stylesheet with … a stylesheet!

So, the basic idea is code generation via metadata. Automatically converting existing reports and forms into Java/XML code supporting data model, XSL stylesheet controlling view and JavaScript functioning as controller, we make them applicable for any J2EE server:

Sample in point: Migrating Reports To Web

The JavaServlet for “customers list” report, partially illustrated in previous section, will produce data in XML format with the layout presented below:

<Detail status="Primary" id="4"> <id>102</id> <city>Rutherford</city> <state>NJ</state> <zip>07070</zip> <phone>2015558966</phone>

<company_name>The Power Group</company_name>

<address>3114 Pioneer Avenue</address>

<fname>Michaels</fname> <lname>Devlin</lname>

</Detail>…</DocumentRoot>

This XML data gets transformed into HTML with the following XSL stylesheet (we will show how this XSL has been generated later in the article):

<xsl:template match="Detail">

<xsl:if test=" @status='Primary'">

And finally, merge of XML and XSL results in HTML page rendered in a browser:

Figure 2. “Customer List” report rendered upon XML/XSL transformation

Looking into details.

The following paragraphs contain selected topics of the “magic wand” design, such as XSL (with XSL) generation, handling lookups, reusing code between JavaScript and XSLT. We will show how XML metadata comes into play once again - at run-time, supporting built-in end-user dialogs. We will also introduce XMLControl as a manifestation of Model/View/Controller architecture . First, however, we would like to explain our design considerations.

Generated code vs. generic run-time.

The classical alternative approach to the proposed solution is to interpret metadata during the run-time. However, there are several major advantages of code generation that made us decide in its favor.

First and foremost, it is performance. Generated servlet code does not contain any logic related to metadata processing, since it was used for the code generation itself. Also, generated servlets produce reduced XML output optimized for the data layout. For instance, XML document corresponding to the population report grouped by country and city should not contain repeating country and city nodes within the appropriate sublevels.

Next, it is flexibility. Template based generation lends itself for end-user modifications and integration with client frameworks (as well as porting to other platforms). Users can freely incorporate their business rules, standard actions and validations. Also, with templates it is very easy to switch to produce servlets with custom data layout. The same SQL query can produce different XML data layout. In fact, automatic conversion uses different templates to generate different layouts for relational data, hierarchical collapsible “treeview” and OLAP crosstab. The same is true for the presentation stylesheets.

And last, maintenance and versioning are much simpler. With interpretive approach any change to the common run-time potentially affects your entire site, while in code generation case damage is always limited to the object being regenerated.

Server-side transform vs. client-side transform: where is my DOM?

Dynamic generation of HTML via XML/XSL always presents a question: should “XML transform” be performed on the server, or, rather, on the client?

In the first case, XML and XSL stay at the server. The entire HTML page - result of XML transformation is sent to the browser. In the second case, XML and stylesheet are sent to the browser for the client-side transform.

We can use either approach. Our generated XSL stylesheet stays the same whether to be applied on the server or within a browser. For server-side solution, row-level and aggregation formulas found within the metadata result in server-side Java code. For client-side XML transform we generate calculation and validation in JavaScript. That said, there is certain cost associated with either approach for users and developers.

Server-side transform usually leads to extra round-trips. Recalculating, sorting or filtering data within the same document results in a new HTML page generated by the server. Client-side transform eliminates this round trip. This becomes crucially important for advanced data entry systems: imagine Microsoft Excel or Lotus 1-2-3 reloading entire spreadsheet every time you changed the cell value! (If your users settle for a “refresh” button, we want them as our clients)

For programmers, persisting XML and XSL on the server(s) presents additional file, DOM and session management, while loading them into browser eliminates these problems. Besides, XML tends to be 3-4 times smaller then HTML, which means faster download. Also, XML on the client allows front-end scripting with strictly defined, strongly typed document model totally independent from the presentation.

To keep things simpler, in this article we assume XML is processed on the client.

Creating XSL with XSL.

Creating of the presentation stylesheet is a pretty straightforward task since all positioning is kept in the metadata. The trick is in mapping of original presentation objects into HTML ones.

Let’s take radiobutton as an example. If we have “gender” column in DataWindow styled as radiobutton, it amounts to two HTML <INPUT> controls of type “radiobutton”, two associated <LABEL>s indicating option name, two JavaScript actions that modify document values “onclick” and surrounding <SPAN> to define CSS style of the whole set.

Below is a fragment of the template for “radio button” control:

<xsl:namespace-alias stylesheet-prefix="alt" result-prefix="xsl"/>

<xsl:when test="CONTROL='radiobuttons'">

<xsl:for-each select="OPTIONS/OPTION">

<xsl:if test="TABORDER='0'"> <xsl:attribute name="disabled"></xsl:attribute></xsl:if>

<alt:if test="{../../NAME}='{@value}'"> <alt:attribute name="CHECKED"/></alt:if>

</INPUT>

<xsl:value-of select="."/></LABEL>

</xsl:for-each></SPAN></xsl:when>

Given the following XML metadata,

<NAME>gender</NAME> <TABORDER>10</TABORDER> <X>33px</X> <Y>1px</Y> <WIDTH>144px</WIDTH> <HEIGHT>19px</HEIGHT>

<OPTIONS> <OPTION value="M">Male:</OPTION> <OPTION value="F">Female:</OPTION> </OPTIONS>

<CONTROLTYPE>radiobuttons</CONTROLTYPE>

</DATAFIELD>….</DocumentRoot>

the above template results in generated presentation XSL which can be used to generate exactly two input fields, two labels and two JavaScript for “onclick” actions:

<INPUT name="gender_{@id}" type="radio" value="F" onclick="sample.itemChanged( {@id}, 'gender', 'F')" id="gender_{@id}_2"><xsl:if test="gender='F'"><xsl:attribute name="CHECKED" /></xsl:if> </INPUT> <LABEL for="gender_{@id}_2">Female:</LABEL></SPAN>

During run-time, for the following XML produced by our servlet:

<Root>…<Detail id=”1”><fname>Tim</fname><gender>M</gender>…</Detail>… </Root>,

such presentation XSL will, in turn, result in the following HTML:

<INPUT name="gender_1" type="radio" value="F" onclick="sample.itemChanged( 1, 'gender', 'F')" id="gender_1_2"><xsl:if test="gender='F'"><xsl:attribute name="CHECKED" /></xsl:if> </INPUT> <LABEL for="gender_1_2">Female:</LABEL></SPAN>

Handling lookups

The data we use often requires translation table functionality, or “lookups”. For instance, state codes - CA, MA, IN, etc. - have to be displayed as corresponding state names – California, Massachusetts, Indiana. We will demonstrate how XML DOM objects facilitate effective implementation of lookups.

Let’s assume that we declare a JavaScript array for all different lookup tables within a page:

var allLookups = new Array();

Suppose the state information is in XML file with the following layout.

<key id=”AK”> <selection name=”description”>Alaska</selection ></key>

<key id=”AL”><selection name=” description”>Alabama</selection></key>…

</LookupRoot>

We can write a following function to load a particular element of array allLookups with a reference to a lookup data retrieved from a given url:

function loadLookup(url) {

if ((allLookups[url]!=null) return ""; // Avoid reloading existing data

allLookups[url] = new ActiveXObject("MSXML.DOMDocument");

allLookups[url].async = false;

allLookups[url].load(url);

if (allLookups[url].xml != "") return "";

else return "Error loading \"" + url + "\"" ; }

Next, we can write showLookup() function, which returns a translation of the value contained the XML node if it is found:

function showLookup(nodeToTranslate, ddUrl, display_column) {

// Get key as text of the first item in nodeList

var key = nodeToTranslate.item(0).text;

err = loadLookup(ddUrl);

if (err!="") return key;

rootElement=allLookups[ddUrl].documentElement;

var lookupRow = rootElement.selectSingleNode(

"key[@id='" + key + "']");

if (lookupRow == null) s return key;

return lookupRow.selectSingleNode( "selection[

@name='" + display_column+"']").text;}

As a result, presuming we add these functions under the namespace “lookup”, we can write the following XSL statement <xsl:value-of select=”lookup:showLookup( state, ‘/cache/states.xml’, ’description’)”/>. This statement will perform conditional load of the “states” lookup data and will result in the description of code values contained in the “state” column. It is worth mentioning that with the browser caching turned on, we may not need to reload data from “/cache/states.xml”.

Sharing objects between JavaScript and XSLT

Besides serving as translation tables, lookups are often used to validate user input. And while translation table functionality is a part of an XSLT generation of HTML, JavaScript-based input validation is invoked interactively – later and in the different context. To avoid duplicate load of data and allow reuse of the same code in XSLT and JavaScript the following has to be done.

First, we should encapsulate allLookups array in a Lookup object and make showLookup() and loadLookup() into object methods

function Lookup () { var this.allLookups = new Array();}

Lookup.prototype.showLookup=showLookup; Lookup.prototype.loadLookup=loadLookup;

function Lookup () { var this.allLookups = new Array();}

Lookup.prototype.showLookup=showLookup; Lookup.prototype.loadLookup=loadLookup;

function loadLookup(url) {

if ((this.allLookups[url]!=null) return ""; // Avoid reloading existing data

this.allLookups[url] = new ActiveXObject("MSXML.DOMDocument");

this.allLookups[url].async = false; this.allLookups[url].load(url);

if (this.allLookups[url].xml != "") return ""; else return "Error loading \"" + url + "\"" ; }

function showLookup(column, ddUrl, display_column) {

var key = column.item(0).text; // Get key as text of the first item in nodeList

err = this.loadLookup(ddUrl); if (err!="") return key;

rootElement= this.allLookups[ddUrl].documentElement;

var lookupRow = rootElement.selectSingleNode( "key[@id='" + key + "']");

if (lookupRow == null) return key;

return lookupRow.selectSingleNode( "selection[@name='" + display_column+"']").text;}

Second, to use lookup functions (and data) in JavaScript we need to create an instance of Lookup object:

var lookupInstance = new Lookup();

Finally, to use the very same object in XSLT, we have to register this instance with the XLST processor via AddObject function. Below we associate it with the “lookup” namespace:

xslProcessor.AddObject( “lookup”, lookupInstance); xslProcessor.transform();

As a result, the following XSL statement <xsl:value-of select=”lookup:showLookup( state, ‘/cache/states.xml’, ’description’)”/> uses data and functions with browser’s JavaScript-hosted instance of Lookup at the transform time.

Is metadata for design-time only?

So far, we have seen how metadata is used in generation of J2EE objects. However, XML metadata can be just as valuable at run-time.

For example, it is possible to build generic ad-hoc sort dialog to accompany any Web report supported with XML metadata. Of course, we need to access business names of the columns to present them to the end user. And we need to know data types of the columns to add appropriate <xsl:sort> tags to presentation XSL. We can also, build dynamic ad-hoc filter and query-by-example dialogs, once we know appropriate presentation, i.e. whether particular column is a checkbox, edit field or selection list etc. All this information is contained in metadata, so metadata URL becomes an essential component for the run-time. Below is the example of the sort dialog running against “Customer List” metadata, with the records sorted by “Last Name, First Name” (Figure 3):

Figure 3. Generic sort dialog based on “Customer List” metadata

XMLControl – more then sum of parts.

By now we have mentioned four run-time components: data XML, presentation XSL stylesheet, client-side JavaScript and metadata XML. The first three are essential for rendering HTML of the original document, while metadata is required primarily for add-hoc dialogs like generic sort and filter.

It is time to introduce XMLControl object:myControl= new XMLControl(dataXML, presentationXSL, placeholderHTML, metaURL, "myControl");

where placeholderHTML is a <DIV> that is filled with the results of XML transformation. XMLControl puts Model/View/Controller paradigm to work. It “glues” presentation and underlying XML data by providing transparent synchronization of presentation HTML and XML data.

Front-end programmers can work purely with data model instead of pulling data out of HTML debris. For data entry subsystems, in particular, there is no guessing which data element has been updated and which not, since XMLControl maintains state information inside the DOM.

Concept of XMLControl provides for adding handlers like onItemChanged(), onRadioButtonClicked(), onAcceptChanges(), etc. emulating rich set of event notifications for databound controls found in PowerBuider and VisualBasic. By the same token, XMLControl lends itself into implementing high-level methods like paint(), update(), setItem(), scrollToRow(), selectRow(), etc. All these event handlers and methods correspond to the document model, not the view. For example, the task of updating presentation upon each setItem() called by a programmer is handled by XMLControl as a part of setItem() implementation.

As a result, 90% of legacy Client/Server code becomes portable with just minor cosmetic changes.

Conclusion

We demonstrated the working approach to migration Client/Server applications into J2EE environment. It allows “magic wand” automatic conversion of databound legacy control to cutting-edge Internet technologies. The cornerstone of the solution – “code generation from XML metadata” – extends it far beyond the conversion process. Indeed, it is totally irrelevant where the metadata is coming from. In combination with a proper graphic design tool this solution may become a full-scale IDE for creating Internet applications.

Proposed approach puts to work Model/View/Controller paradigm, enforcing strict separation of data model (XML) from presentation (XSL). That alone, in authors view, should bring developers productivity back to the level of RAD tools. In addition, XSL-based approach to code generation opens limitless possibilities for end-user customization.

Authors Bio

Victor Rasputnis and Anatole Tartakovsky are co-founders of CTI, a New-York consulting firm servicing financial institutions and Wall Street firms. They provide architectural design, implementation management and mentorship to the companies migrating to XML Internet technologies. Victor holds PhD in Computer Science from Moscow Institute of Robotics where he conducted research of advanced interactive systems. Anatole holds MS in Mathematics. He also attended the post-graduate program in Computer Science at Moscow Institute of Robotics with specialization in theory of expert systems. They can be contacted via e-mail: VictorRasputnis@teamcti.com and AnatoleTartakovsky@teamcti.com