I. Introduction

Notice: This document has not been revised for some months, and some content is out of date. In particular, DataRelationTables and DataSelectors have been removed from the API. Also, the project has moved and renamed to DataBuffer

The goal of the SwingLabs project is to significantly reduce the effort and expertise required to build rich data-centric Java desktop applications. The focus of the SwingLabs project is desktop front-ends for network services, notably relational databases. Even small businesses often deploy dozens of custom applications like this for data entry, visualization, and analysis using data provided by networked databases or other (often J2EE based) web services.

Applications of this kind must load data from network sources, bind that data to GUI elements and, in many cases, store new data and updates back to the original source. To facilitate "Data Binding", developers often wrap the raw data source with a generic container type. By employing a set of "Data Encapsulation" classes that unify access to raw data, the support for data binding and other data processing tasks can be developed independently of the data source. A generic data encapsulation API can also support features that the raw data source can't, e.g. offline operation.

II. Data Encapsulation and Data Binding in Practice

Any developer who has built a database front-end has employed data encapsulation and data binding in principle. Desktop Java developers who've built large scale or just numerous applications of this kind typically discover the utility of a data encapsulation API and sometimes devise an explicit data binding API as well.

A simple example will serve to remind the uninitiated about the motivation for both kinds of APIs. Assume that we've got a relational database that contains a list of customer records, with fields like first name and last name. The database is a network resource available via JDBC. Our desktop application will enable users to view and change customer records. We'll display the complete list of customers in a JList and the first/last name etc of the selected customer in JTextFields. When the user changes a text field, we'll validate the change and then update the database.

A small application like this can be readily coded by hand and most developers do just that the first time around. Doing so repeatedly quickly loses its appeal when it becomes obvious that some reusable classes can be factored out of the application. The classes that abstract the connection to the database as well as the the cached records themselves are what we've termed the data encapsulation. They download and cache a copy of the customers (database) table and provide for storing updates made to the local copy, back in the database using JDBC.

The SwingLabs DataBuffer API is a generic encapsulation of tabular data. In our example, we'd create a DataBuffer that contained a single "Customers" DataTable, with one DataRow per customer, and one DataColumn per customer field. The DataBuffer uses a DataProvider to load and store data to the database. The DataProvider handles the potentially intermittent connection to the database by synchronizing changes each time it's reconnected. The DataBuffer API can also be applied to other types of data stores, like JavaBeans. DataRows and DataColumns can encapsulate individual beans and their properties, DataTables can encapsulate lists of beans of the sample type.

The connection between a JList and the customers DataTable, or between a single customer's DataColumn and a JTextField is called a data binding. To bind the customers DataTable to a JList, the JList's ListModel is replaced with one that points to the DataTable. The binding can be configured to map one or more customer DataColumns to each ListModel element. Binding a DataColumn, like "Last Name" from the selected customer to a JTextField's DocumentModel is similar. In this case the binding adds a DocumentModel change listener that updates the DataColumn when the users press enter.

III. Relational Data Visualization: Master/Detail

A database for an application like the one we've described so far would contain several tables: customers, orders, products. A "data relation" defines a mapping from a record in a parent table to a list of records in a child table. The mapping is usually defined in terms of a single column that both tables have in common. For example each customer has a unique (string) customerID column, and each order has a customerID column that identifies the customer who placed the order. A data relation like "customersToOrders" is just the list of all orders placed by a list of customers. Similarly a relation called "ordersToProducts" is the list of products included by a list of orders.

A common way to visualize relational data is a show a single parent table and the value of a set of relations based on the row (or rows) the user has selected in the parent table. This is called a master/detail GUI, where the "master" is the parent table and detail is the value of one or more relations based on the rows selected in the parent table. Many applications display chains of data relations, e.g. all of the orders for the selected customer (customerToOrders relation) and all of the products for the selected orders (ordersToProducts relation). A master detail GUI also typically includes a view of all of the columns in each selected record.

IV. Details

The DataBuffer APIs are based on about half a dozen core classes. The names of these Java classes all begin with "Data". The data classes are intended to be used in desktop applications that download a subset of some persistent data store, present the cached data to the user and collect (and validate) updates and additions, and then save the changes back to the data store. Hence, DataBuffers are inherently disconnected.

Most of the data classes include a "name" property to simplify writing text expressions that refer to the objects by name. It is also useful in a GUI builder context, as well as providing a useful method for extracting a certain element (such as a DataTable or DataValue) from a DataBuffer using only the name. This allows the developer to maintain a single reference to the DataBuffer and yet have access to all of the constructs within the DataBuffer. All names in the DataBuffer API conform to the same naming conventions. A valid name is one that follows the Java naming rules for indentifiers, except that Java reserved words can be used, and the name may begin with a number.

DataBuffer is a container for DataTables, DataRelations, DataRelationTables and DataValues. DataTables contain a homogenious set of records. DataRelations represent the relationship between two DataTables. DataRelationTables are special "view" like DataTables that always display the result of some parent/child relationship. Finally, DataValues are single objects that contain a computed value, generally based on some expression involving DataTables in the DataBuffer.

A DataBuffer can be constructed manually by using the API or through a special xml schema file. Both the schema and the data in the DataBuffer can be serialized to XML.

A DataTable contains a list of DataRows and each DataRow contains a list of values defined by the DataTable's DataColumns. A DataProvider encapsulates interaction with the data store on some background thread. The DataProvider is reponsible for synchronizing a DataTable with the data store, and with loading a DataTable from a data store.

V. Data

There are three classes at the core of the API that encapsulate a homogenous list of database records or Java beans. These are DataTable, DataRow, and DataColumn. Logically a DataTable has a list of DataRows, one for each record or bean. It also has a list of DataColumns, one for each field in the record or bean property. The DataColumns represent the DataRow's schema, each DataColumn has name and type properties as well as various other data related meta data.

Properties of the DataColumn class include meta data for the column's semantics:

• DataTable table
• String name
• Class type
• boolean readOnly
• boolean required
• boolean keyfield
• Object defaultValue

The table property is a read-only property. It is a reference back to the DataTable that created the DataColumn. Most of the classes in the DataBuffer API contain a read-only reference back to the object that created them. In this way, given any object, we can easily traverse the DataBuffer.

DataColumns may also be "calculated" columns. That is, it is possible for a DataColumn to be the result of a computation. For instance, I might have 2 DataColumns, Price and Quantity. I might then define a DataColumn ExtPrice that was the result of Price * Quantity.

A DataRow is simpler, it simply represents a collection of values, one for each DataColumn in its associated DataTable. It also contains a flag indicating the DataRowStatus. This status could be one of the following states:

• DELETED: The row has been deleted, but not yet removed. it will be removed when synchronized with the data store.
• INSERTED: The row has been inserted. It may have been modified (UPDATED) since then, but is still flagged as INSERTED since it has not yet been saved. If deleted, the row status will be changed to DELETED. This flag is cleared after the DataTable is synchronized with the data store.
• UPDATED: Indicates that a field value in the row has been changed, and that the previous row status was UNCHANGED
• UNCHANGED: The row has been untouched since it was last synchronized

To access a DataTable's DataRow, one must provide a zero based index. Also, to access a DataRow value that corrosponds to a DataColumn one must provide either the DataColumn itself or the name of the DataColumn. Access to row/column DataTable "cells" would always pass through:

                    Object getValue(int index, String columnName);
                    void setValue(int index, String columnName, Object value);

In other words, the definition of a convenience method for accessing table values would be defined in terms of these two methods, e.g.:

        Object getValue(int index, DataColumn col) {
            // assert that col != null, col.getTable() == this
            return getValue(index, col.getName());
        }
                    

DataTable methods that return rows or columns will return DataRows and DataColumns. DataTable methods that require rows or columns as arguments will always be defined in terms of integer row indices and string column names.

The main properties of the DataTable include:

• DataBuffer dataBuffer;
• DataProvider dataProvider;
• String name;
• List<DataRow> rows;
• List<DataColumn> columns;
• boolean deleteRowSupported;
• boolean appendRowSupported;

Of these, three properties are of interest to us. First, it is possible for a DataTable to restrict whether rows can be added. Second, it can restrict whether rows can be deleted. The third property is the dataProvider. Each DataTable may be associated with a specific DataProvider. This association is for convenience. For example, consider this code in which DataProviders are NOT associated with DataTables:

        DataBuffer ds = new DataBuffer();
        //configure data set
        ...
        table1.clear();
        provider1.load(table1);

        table2.clear();
        provider2.load(table2);
                    

Now consider this code:

        DataBuffer ds = new DataBuffer();
        //configure data set
        ...
        table1.refresh();
        table2.refresh();
                    

Or, since the DataBuffer is a wrapper for all of its DataTables:

        DataBuffer ds = new DataBuffer();
        //configure data set
        ...
        ds.refresh();
                    

If a DataTable does not have a DataProvider, then calls to load() or refresh wil not cause any data to be added to the DataTable, and a warning to be logged.

The lsts returned by DataTable.getRows() and DataTable.getColumns() are read only. DataTables include the following methods for adding/deleting rows:

        public DataRow appendRow();
        public void deleteRow(int recordIndex);
                    

The following methods use the DataTable's DataProvider to load data into the DataTable:

        public void load();
        public void save();
        public void clear();
        public void refresh(); // as defined {clear(); load();}
                    

VI. Relationships

Once DataTables have been defined in a DataBuffer, it is possible to define relationships between these tables, or even from one table to itself, just like in relational database management systems. Once these definitions are made, it is convenient to set up special DataTables that always show the correct detail information in a master/detail relationship. The main classes used to define relationships in the SwingLabs DataBuffer are the DataRelation, DataSelector, and the DataRelationTable.

A DataRelation is a mapping between a single DataRow in a " parent" DataTable and a list of related DataRows in a "child" DataTable. The relationship is defined in terms of a DataColumn from the parent and a DataColumn of the same type from the child. The value of the DataRelation is the list of DataRows in the child for which the specified columns have the same value. Note: it is not necessary for either the parent DataColumn or the child DataColumn to be unique, or to be keyColumns.

This simple parent/child idea is most easily understood with an example. Consider two DataTables, customerDT and orderDT. The customerDT contains a list of customers and has columns named "id", "name" and "address" that contain respectively the customers ID number, name, and billing address. The orderDT contains a list of orders and has columns named "id" and "product" that contain the customer's ID and the product that they"ve ordered. A DataRelation that defines all of the orders for a particular customer can be defined as all of the DataRows in orderDT whose "id" column matches the "id" column in a customerDT DataRow.

The key properties of a DataRelation are:

• DataBuffer dataBuffer;
• String name;
• DataColumn parentColumn
• DataColumn childColumn

To use the DataRelation to produce a list of DataRows for which the value of the parentColumn and childColumn are equal, one specifies the index of a record in the parent DataTable like this:

        List<DataRow> getRows(int index);
                    

The implementation of DataRelation.getRows() could be:

        List<DataRow> getRows(DataRow parentRow) {
            DataColumn parentCol = getParentColumn();
            DataTable childTable = getChildColumn().getTable();
            Object parentKey = parentCol.getTable().getValue(parentRow, parentCol);
            List<DataRow>rv = new ArrayList<DataRow>();
            for (DataRow childRow : childTable.getRows()) {
                Object childKey = childTable.getValue(chlidRow, childCol);
                if (parentKey.equals(childKey)) {
                    rv.add(childRow);
                }
            }
            return rv;
        }
                    

A DataRelation can also construct results based on an array of parent DataRows. The resulting list will simply be the union of the child DataRows for each parent DataRow.

To resolve a relation, one or more DataRows in the parent DataTable have to be selected. A DataSelector is a named object that provides a list of DataRow indices (though often the list length will be one). DataSelectors are defined on DataTables. Each DataTable may have zero or more DataSelectors.

DataSelectors are usually bound to GUI component selection models. For example, a common idiom would be to bind a JList to the contents of the DataTable, and then to bind the JList's selection model to a DataSelector on the same DataTable. In this way, whenever a different row is selected in the JList, the DataSelector is also updated.

DataSelectors cannot be defined statically, but are created when necessary and maintained with a weak reference so that they are released when no other references to them are maintained. Since DataSelectors are not persisted there is no point in them being created statically. NOTE: is there a point to persisting DataSelectors? Most of the time there is not (the user doesn't want the selection to be maintained from one program invocation to another), though at times it could be useful.

A DataSelector has the following properties:

• DataTable dataTable;
• String name
;
• List<Integer> getRowIndices();

The third piece to establishing and using relationships between DataTables in a DataBuffer is the DataRelationTable. Conceptually, this object is a DataTable that filters its content based on a DataSelector and a DataRelation. Essentially, it is a detail table in a master/detail relationship. Using the previous example with customerDT and orderDT, suppose I created a customerOrdersDRT. This DataRelationTable will use a DataSelector on customerDT and the DataRelation between the customerDT and orderDT to populate itself with the rows in orderDT corrosponding to the currently selected row(s) in the customerDT.

VII. Example

The following code block is an example of how to create and initialize a DataBuffer in Java code. As you can see, this process can be quite verbose!

        ds = new DataBuffer();
        /*
                    packageid char(20) not null,
                    catid char(20) not null,
                    locale varchar(10) not null,
                    location varchar(30) not null,
                    price decimal(10,2) not null,
                    name varchar(80) null,
                    description varchar(255) null,
                    imageuri varchar(80) not null,
                    lodgingid char(20) not null,
         */
        DataTable packageTable = ds.addTable();
        packageTable.setName("package");
        DataColumn packagePackageId = packageTable.createColumn("packageId");
        DataColumn packageCatId = packageTable.createColumn("catid");
        packageTable.createColumn("location");
        packageTable.createColumn("price");
        packageTable.createColumn("name");
        packageTable.createColumn("description");
        packageTable.createColumn("imageuri").setType(String.class);
        SqlDataProvider packageProvider = new SqlDataProvider();
        packageProvider.setConnection(conn.getConnection());
        packageProvider.setSelectSql("select * from package");
        DataTable packageTable = ds.getTable("package");
        packageTable.setDataProvider(packageProvider);
        DataSelector currentPackage = packageTable.createSelector("current");

        /*
                    catid char(20) not null,
                    locale varchar(10) not null,
                    name varchar(80) null,
                    description varchar(255) null,
                    imageuri varchar(80) null,
         */
        DataTable categoryTable = ds.addTable();
        categoryTable.setName("category");
        DataColumn categoryCatId = categoryTable.createColumn("catid");
        categoryTable.createColumn("name");
        categoryTable.createColumn("description");
        categoryTable.createColumn("imageuri");
        SqlDataProvider categoryProvider = new SqlDataProvider();
        categoryProvider.setConnection(conn.getConnection());
        categoryProvider.setSelectSql("select * from category");
        categoryTable.setDataProvider(categoryProvider);

        /*
                    activityid char(20) not null,
                    locale varchar(10) not null,
                    location varchar(30) not null,
                    name varchar(80) not null,
                    description varchar(255) not null,    
                    price decimal(10,2) null,
                    imageuri varchar(80) not null,   
         */
        DataTable activityTable = ds.addTable();
        activityTable.setName("activity");
        DataColumn activityActivityId = activityTable.createColumn("activityid");
        DataColumn activityPackageId = activityTable.createColumn("packageid");
        activityTable.createColumn("location");
        activityTable.createColumn("name");
        activityTable.createColumn("description");
        activityTable.createColumn("price");
        activityTable.createColumn("imageuri");
        SqlDataProvider activityProvider = new SqlDataProvider();
        activityProvider.setConnection(conn.getConnection());
        activityProvider.setSelectSql("select a.*, al.activityid, al.packageid "
            + "from activity a, activitylist al where a.activityid = al.activityid");
        activityTable.setDataProvider(activityProvider);

        DataRelation packageCategoryRelation = ds.addRelation();
        packageCategoryRelation.setName("packageCategory");
        packageCategoryRelation.setParentColumn(packageCatId);
        packageCategoryRelation.setChildColumn(categoryCatId);

        DataRelation activitiesRelation = ds.addRelation();
        activitiesRelation.setName("activities");
        activitiesRelation.setParentColumn(packagePackageId);
        activitiesRelation.setChildColumn(activityPackageId);

        //set up relational tables
        DataRelationTable categoryDetail = ds.addRelationTable();
        categoryDetail.setName("categoryDetail");
        categoryDetail.setRelation(packageCategoryRelation);
        categoryDetail.setParentSelector(currentPackage);

        DataRelationTable activitiesDetail = ds.addRelationTable();
        activitiesDetail.setName("activitiesDetail");
        activitiesDetail.setRelation(activitiesRelation);
        activitiesDetail.setParentSelector(currentPackage);
                    

VIII. XML Support

The DataBuffer contains basic XML serialization support. Both the DataBuffer schema can be saved/loaded from XML, as well as the actual data. This process bypasses the normal DataProvider mechanism, and is intended for serialization of the DataBuffer, not populating of it. To help clarify the intent of the XML serialization, an example is described below. [TODO]