Clay: Synchronous Collaborative Interactive Environment

Building Application Service Provider Software

By: Michael E. Locasto & Michael Hulme
May 3, 2002

Table of Contents

  1. Abstract
  2. Introduction
  3. Background
  4. Design
  5. Methods
  6. Results
  7. References
  8. Appendices
Keywords: open-ended client design, non-textual collaborative communication, collaborative tools, synchronous collaboration, web services, application server, application service provider software

Abstract

Computing platforms become more powerful and capable of dealing with different types of media at a fantastic rate. At the same time, the powerful networked communications abilities of computers present a tantalizing reward: what if these new media could easily be edited by a group of collaborators in real time? Instant communication in the course of building this media has a number of advantages.

First, speed of communication promises to lubricate the entire process of building the media and reduce the turnaround time for project completion. Second, multiple inputs at the same time may open up fantastic areas of media development that have not been explored. Third, a design solution for collaborative architecture can take advantage of many modern techniques for improving performance, communication, and ease of use.

However, the concurrent collaborative process may not be right for many projects. Concurrent collaboration has a suitable place in many projects as part of one specific and well-designed phase. Trying to rely on the advantages of concurrent collaboration outside its scope will probably be detrimental to the project in general, because traditional measures for communication exist that perform the appropriate duty very well. This admonishment amounts to the saying "The right tool for the right job." Clearly, concurrent collaborative methods offer significant advantages, but only when applied to the correct problems.

In addition, concurrent collaboration through software faces a number of very difficult problems. Deadlock, dirty writes, and loss of ownership are typical of the technical problems involved. Besides the normal technical challenges faced when building a complex software system, collaborative systems have additional requirements to address.

In general, there are three primary areas to be addressed when developing concurrent collaborative systems. First, content management, the process of controlling versions and ownership of the media, must be examined, and a suitable system implemented or integrated. Second, workflow management, the process of controlling the path a job or piece of data takes through a team of people, must be researched to discover the protocols for communication between team members. Third, user management, the process of giving and controlling access rights and privacy, must be carefully incorporated into the whole solution.

Clay is a software environment that promotes a high level of concurrent collaborative interaction between content developers. The Clay architecture defines a robust set of backend services that many collaborative projects will need. The architecture also lets the project developers specify the protocol for working on their specialized media by defining the appropriate interfaces between their agents.

Introduction

Clay is an architecture that allows content developers simultaneous, real-time editing access to the same documents. It encourages developers to work together in a collaborative environment and provides guidelines for enforcing logical constraints on developer interaction. Clay is constructed of an extremely diverse set of technologies, with Java acting as a common platform and a transportation mechanism (RMI). This allows us to concentrate on defining the high-level interaction of the clients rather than low-level protocols.

The Clay software grew out of the need to provide a team of physically dispersed software developers with the ability to work cohesively on various coding projects. This initial goal provides the motivation for Clay's strong incorporation of traditional solutions (chat, bulletin boards, e-mail, and whiteboard) to problems facing current software collaboration efforts. However, the Clay model has always kept in mind the objective for any type of document to be the focus of collaboration. As a consequence, the Clay architecture is highly extensible and distinctly flexible.

One of Clay's central aims is to provide a uniform, stable, scaleable, and reliable environment that encompasses centralized security, data storage, authentication, transport, and message passing.

Background

To understand both the motivation for and goals of Clay, it is useful to gain an appreciation of these four core questions and their answers. Along the way, we will define terminology that is used throughout the rest of the paper.

What is Clay?

Clay is an application architecture that promotes synchronous collaborative interaction. Clay currently uses Java RMI for transport, but will soon be re implemented to use SOAP.

What is RMI?
RMI is Remote Method Invocation, the Java implementation of RPC (Remote Procedure Calls). It allows an object in one Java Virtual Machine (JVM) to call methods of an object in another Virtual Machine. The Virtual Machines may be separated by a network. 		What is SOAP?
SOAP stands for Simple Object Access Protocol, an open standard based on XML that provides transport for objects implemented in different languages. The major advantage is that clients no longer have to be implemented in the same language to be capable of understanding complex object types.

Clay is an environment in which multiple types of server software can exist. Clay provides the lookup, authentication, storage, and transport mechanisms for different specialized servers. Servers are specialized to the type(s) of documents or artifacts being edited.

Clay has grown into powerful ASP software and is maturing into a viable "web service." While it is true that the clients we have built center on replicating traditional collaborative technologies (whiteboard, chat, shared text area) as proofs of concept, we have designs for and are testing Clay's ability to edit other types of media such as music, drawings, voice, and curriculum maps.

Clay does not seek to replace traditional tools like e-mail or simple chat, but rather fill the rapidly growing void where application servers fail and traditional client-server is not scaleable or flexible enough. In the course of doing so, Clay works best when some traditional tools are incorporated into the Clay environment rather than spread over the user's desktop in five or ten separate applications.

Most of the other technologies for collaboration are of two flavors. Older architectures reflect the "discovery" of the benefits of networking and thus are peer to peer or strict heavy client-server. Newer architectures employ the n-tiered application server architecture.

What is an application server?

An application server is a piece of software that provides a service to clients by containing the main thread of execution for that software. Clients include little programmatic logic and often are only display containers, or the "presentation layer."

In this architecture, a client is extremely light, and a number of middle tier application servers and backend databases distribute the workload of the software server-side. The application server architecture attempts to address a number of the shortcomings of traditional client-server and peer to peer based applications. As can be expected, the application server architecture has shortcomings as well, especially as clients for this architecture attempt to become more independent and robust.

Like current application servers, the Clay environment separates client functionality from presentation and provides centralized services (storage, communication, message passing) to distributed clients (and middle-tier servers) of different types at the same time. An application server like Tomcat or Resin can support multiple uploaded web applications by compiling and running different sets of JSP (Java ServerPages) pages simultaneously. Each collection of JSP's probably serves entirely different purposes and deals with unrelated data. Similarly, Apache can support PHP or Perl server-side CGI scripting.

In both these schemes, execution is centralized on the server; the client (most often a web browser) is delivered normal HTML that has been dynamically generated on the server.

This off-loading of work to the server is the core of the application server architecture.

This architecture is powerful for several reasons. Most notably, applications have the ability to take advantage of the server's power for compute-intensive tasks and clients do not need to be updated - this feature removes any client incompatibility issues. The application server architecture takes a heavy burden off client developers because updates are restricted to one update of the software on the server. The software does not have to be distributed to the clients.

The client becomes a presentation container, and processing is moved to the server. HTML is the common language that most of these "light" clients understand, and most application servers aim at delivering HTML content to their clients. Since HTML has a number of attendant technologies (Javascript, Flash, Perlscript, ActiveX, Java applets) the HTML delivered to the client can invoke a number of local processes and supplement the functionality raw server-generated HTML provides to the client.

However, the application server architecture is not ideal in a growing number of situations for a variety of reasons.

First, the trend to supplement server-generated HTML with client-side technologies runs directly counter to the benefits derived from employing an application server architecture. Servers are not guaranteed that all clients will have the same supporting infrastructure; for example, Flash may not be installed, a Java Runtime Environment (JRE) may not be installed, or the client may have old Javascript capabilities. If server-side programming starts to rely on features that may not be present in the client, the application server architecture disintegrates into traditional heavy client-server interaction. The cost of making the client compatible is incurred once again.

Second, there exists significant and realizable benefits to an architecture that incorporates a robust client-side piece. A web browser is a very complicated and robust piece of software, but it is hardly a cure-all or substitute for every type of client application today or tomorrow.

If the client is a presentation container, the protocol it uses and the language it interprets must remain stable. In the case of HTTP and HTML, this stability is generally guaranteed. However, the HTTP/HTML combination may not the best solution for a wide variety of client-server applications, especially applications that demand high level coordination and communication between the client and server. This difficulty leads some companies to implement a proprietary protocol, depend on numerous supplemental technologies to enhance the HTTP client, or at worst, abandon the application server architecture altogether in favor of a heavy client.

Third, as long as client hardware remains cheap and powerful, a good portion of compute-intensive tasks can remain client-side. This move reduces the burden on the server, thereby preventing some server bottlenecks and increasing overall application performance.

In addition, much of the current presentation work done client-side (e.g. Flash, Java applets) is outside the scope of current application server communication protocols. For example, HTML delivered over HTTP provides only limited support for the range of activities that supplemental technologies can perform. A logical and seemingly simple solution would be to incorporate these client-side presentation technologies more tightly into the standard. However, there is a very significant problem with this approach.

As new client-side presentation technologies are invented, a decision to include or exclude the technology from the standard would have to be made. Incorporating specialized technologies in the standard would destabilize it and bloat it, making it worthless as a standard or potentially too heavy to use for simple communication.

What is needed is an application-level standard protocol that is dynamic and responsive to changes and updates of the client and server software that it connects, while remaining a stable standard.

It seems as if application developers are in a bind; they cannot address the real demand for more robust clients without giving up the benefits gained by using an application server. The current workaround, to use supplemental technologies, will eventually regress to the heavy-client model of interaction, as the browser loses its role of presentation layer in favor of a communications endpoint for the client implemented in supplemental technology.

The Clay architecture provides a middle-of-the-road solution for implementing application servers that have more robust (and therefore heavier) clients while maintaining the benefits gained from employing an application server architecture.

Whereas an application server like Tomcat, Resin, or Apache with mod_php4 or mod_perl supports centralized server-side application processing, Clay supports distributed desktop (client-side) computing with centralized control and processing. Clay does not undo the application server approach, but rather modifies it so that the significant benefits remain and clients become more robust, but easily updated and compatible at the same time.

How is Clay different from traditional and contemporary collaborative tools?

There are already a number of traditional tools in various environments that are aimed at facilitating collaborative work. E-mail is the most simple and famous example. A large part of collaborative efforts can be facilitated by traditional communication methods. These methods include e-mail, CVS (code versioning system) & centralized source code control, simple chat, common development tools (gcc, ant, libraries), content management systems, and the SLC (requirements specifications, design documents, testing documents, etc).

However, this traditional approach is heavily centered on managing textual content; multimedia or embedded images are not the primary focus, but rather window dressing. It is not clear if these tools and methodologies are best for supporting alternative types of media. Also, this approach is geared towards separation of tasks and asynchronous communication. Clay seeks to address an environment where task separation is not clearly defined, and the project is in need of synchronous collaborative efforts.

Asynchronous communication occurs when updates or messages occur in turn or at unpredictable and spontaneous instants in the flow of communication. E-mail and chat are clearly examples of asynchronous communication.

On the other hand, synchronous communication is characterized by concurrent update events or messages. Both synchronous and asynchronous software may run the risk of deadlock or blocked messages, especially if there is a common shared area in which communication takes place. The most famous example is the common Producer/Consumer problem, in which two processes write to and read from a shared area of memory, respectively. If the communication between the processes is not implemented carefully, deadlock may occur.

Typically, synchronized communication is meant to deal with deadlock or starvation issues that may arise from the use of asynchronous communication. However, synchronized communication, if implemented improperly, may also cause deadlock. In addition, synchronous communication sometimes presents a performance bottleneck because a mutually exclusive lock (or mutex) must be awarded to one process at a time, restricting other processes from using the locked resource.

Proper synchronous communication is vital to Clay because unlike traditional collaborative software, Clay is specifically designed to update or replicate events to many shared objects at the same time, essentially allowing many writers to the same content at the same time. A discussion of the readers-writers and deadlock issues is presented in the Methods section.

There are many more technologies that support collaborative efforts; e-mail and chat barely scratch the surface. Specifically, ICQ, IRC, AOL Instant Messenger, & UNIX 'talk', are all fairly robust chat and file transfer programs. ICQ and the UNIX 'talk' utility in particular provide an interface in which users may update a space that is shared visually. This interface allows all users to witness contributions as they happen. Note that these interfaces make it appear as if a shared space is being updated, the updates still occur in mutually exclusive areas. Each user has writing privileges to their own space and no other.

One of the core goals of Clay is to provide a space that is concurrently writeable by all users.

Microsoft Netmeeting is a technology that incorporates a shared whiteboard. The functionality present in this technology is a specific example of the interaction that the Clay environment provides.

Difficulties and limitations of the Clay architecture are discussed in the Methods section.

What is an ASP?

An ASP (Application Service Provider) is a company or organization that offers web services. According to Webopedia.com [3], " Application Service Providers are third-party entities that manage and distribute software-based services and solutions to customers across a wide area network from a central data center."

Clay can be offered as an application service for many organizations that have physically separated development teams. A major portion of Clay's development is aimed at providing services for a school district in need of storing, annotating, analyzing, and creating curriculum maps and other data.

Design

General Design Discussion

This paper will detail the current specific design of the Clay application architecture and provide an outline for the next-generation design. The first proof of concept architecture will not be described, having been detailed in the Fall 2001 paper [1].

Clay has three distinct designs that directly reflect each stage of development. Most of the previous design is contained in the paper we wrote last semester [1]. As implementation of the application proceeded, the developers noticed that certain requirements were either too burdensome or wholly unnecessary. A glaring example of this is the ClayCustomClassloader, which was removed from the development tree and current version. Therefore, Clay underwent a second design phase in tandem with the construction and implementation of the current version. Upon completion of the demonstration version (Clay-1.1.9), it was evident that some of the core technologies utilized in Clay were in fact limiting the future potential of Clay. At this point, the researchers determined to investigate other emerging technologies to address the current shortcomings and limitations of the application. It is hoped that this future design work will not only address the limitations but enable the application to be far more powerful.

It is very important to distinguish between these design phases because each phase reflects the growth of knowledge of the students as well as the gradual shaping of our vision. Understanding these changes leads to understanding the overall direction and motivation of the project. Moreover, these different design phases were the direct result of research into the core technologies of Clay, the type of user interaction (e.g. Hopewell) required, and the myriad other issues that arise when building a secure, stable, scalable, and reliable distributed multi-threaded application.

It must be noted that the Clay project owes both pieces of source code and some software methodologies to the Jakarta-Tomcat project [2]. The specific contributions will be discussed in detail in the Methods section.

In addition, the licensing of Clay has been an issue of primary importance. Having incorporated work that falls under both open-source style licensing and proprietary licensing, formulating our license was a matter of careful consideration, because of our desire to respect both styles of license in our academic atmosphere.

The key element that Clay utilizes to address the difficult issues facing the application server architecture (as mentioned in [Background]) is the dynamic update of both client and server implementations. As in regular application server architecture, one update of the server or client software is performed, and then the update is pushed out to the client, if required, automatically.

Clay has a four point approach to the current problems.

First, abstract out and supply common services in the main Clay backend. Clay provides common services to reduce the complexity of developing distributed applications from scratch.

Second, use only standard, well-known open protocols and technology to transport data. This group includes Java, RMI, HTTP, SOAP, and XML. Open protocols enhance communication and support interoperability. The platform independent nature of Java is a significant gain, especially because Clay's ability to dynamically load pieces of the server and automate client updates are greatly simplified by Java's capabilities in these areas.

Third, allow the developer to define both client and server interfaces for high-level communication. This tactic moves changes a level higher into the application rather than altering the existing protocol. Interfaces provide a layer of abstraction with data hiding and encapsulation for both the client and server, protecting them from minor updates and changes in the specific implementations. This buffer should address a number of incompatibility issues. However, for the times when the interfaces themselves must be updated, Clay takes the next approach.

Fourth, allow dynamic, automatic updates to the client (downloading JAR to the client). While the client GUI may contain as many "bells and whistles" as desired, the core client work is carefully managed from the server side. In the Clay environment, most updates to the server will proceed as they would in normal application server operation, with no update required on the client. However, should a server update require distribution to the client, this distribution would happen automatically, possibly when the client first connects after such an update occurs. The developer does not have to push the client out, phase out old clients, or otherwise deal with incompatibility issues. Likewise, the user does not have to pull the software update, keep track of updates and patches, or otherwise manage the software. It remains a tool ready to be used, thereby reducing the complexity of developing, distributing, and using the client software.

Sun Microsystems now distributes Java WebStart with the Java 2 SDK, v1.4.0, a client that is capable of downloading JAR (Java ARchive) files of desktop applications. Clay will employ similar delivery mechanisms to dynamically deliver updated client software to a user. See the Java WebStart site for more information [5].

This four point approach allows Clay to be a flexible system and address the current problems facing distributed collaborative systems and application servers.

Proof of Concept Design

The legacy design can be found in the paper we wrote last semester [1]. This design was a proof of concept that enabled the developers to learn just how to utilize RMI. It also assisted the developers by laying out the logical structure of the server and client interactions and the backend environment.

Current Design

The current design of Clay is implemented in the Clay version 1.2 release. There are significant differences between the 1.1.9 release and the 1.2 release, but both are based on the same design. The 1.2 version implements added functionality and carries the design to its logical conclusion. The specifics of the design discussed in this section will be based on the source code and application architecture in the 1.2 release.

As the 1.2 design reflects the implementation of enhancements and changes to the proof of concept, the design detailed here is not full of requirements specifications or "future tense" language.

Future Design

The future design is centered around a number of key updates to the backend infrastructure of Clay. In addition, stress and unit testing of the application will identify areas of the code that need to be refactored or improved. The driving force behind the future design is interoperability of clients and servers. We anticipate clients being implemented in languages different from their corresponding servers.

Storage Content is XML Tagged

Since the CStore saves documents as serialized Java objects to the file system, it will be difficult for clients implemented in a language other than Java to obtain and edit those documents. The server would have to translate the stored document into the appropriate language. Since the backend is Java, this would be difficult to achieve. Instead, documents should have some XML markup done, or be converted to an XML format and stored as a regular XML file. Then, either the client or server could do the conversion to the specified editable format.

Flexible Logging Elements

The current implementation does not allow Logger elements to be associated with any object in the Clay architecture. Instead, three "standard" logging streams are defined. This will be changed so that a Logger element can be associated with any other object in the clayserver.xml file and Clay will create the appropriate Logger.

The ClassFileServer

Currently, the RMI mechanism provides for the use of an HTTP server to make RMI stub files available for automatic download to the client JVM. ClassFileServer is an attempt to provide this utility within Clay itself. If Clay is integrated with Apache or another webserver (to support the PHP web-admin application) Clay may include an option to turn the ClassFileServer off. ClassFileServer currently serves the stub files out of the same directory that the PHP application would reside in.

Rename docServer XML Tag

ProjectServer entries are mislabeled in clayserver.xml as "docServer" nodes. This will be changed to reflect reality.

Content Management System (file,edit locking, transfer of control)

The CStore needs to become a full-fledged CVS and Content Management System. This process is a major undertaking and needs to support most of the functionality of a traditional CVS. This change also needs to peacefully co-exist with the planned move to an LDAP storage system for user authentication and project metadata information. The CStore should write into the LDAP directory.

SOAP-RPC

The RMI transport mechanism will be deprecated in favor of SOAP-RPC. The major advantage here is the switch to an open protocol will all clients to be developed in a language independent of the server. This advantage must be balanced against the ability to perform dynamic updates of the client. In addition, using XML to pass data through the system will complement the movement of storage of artifacts as XML documents. An XML schema rather than a specialized Java object would then represent an artifact or document.

OpenLDAP Backend

Using JNDI to access an OpenLDAP directory for authentication and document storage presents significant advantages and simplification of design. Clay will have a true centralized storage system rather than using both a database and the file system. Authentication should be done against a directory rather than a database for performance purposes.

Security Focus

Clay has a vast number of interesting security areas, and a design and rewrite of this magnitude should look to the security issues raised in a multi user, multi server, networked architecture. All traffic should be encrypted over the wire, and authentication information should also be encrypted and stored in encrypted form.

Methods

This section will examine some of the difficulties in collaborative interaction and the software methodologies employed in Clay to address those problems. Specifically, a discussion of the possible deadlocks, the transport of artifacts over RMI, limitations, and alternate approaches will be covered.

Methodology Inherited From Jakarta-Tomcat

The developers gratefully acknowledge the hard work of the Apache Software Foundation, and the Jakarta-Tomcat project [2] in particular. This project had a significant impact on the Clay project. Tomcat is an open-source, production-quality application server written in Java. Tomcat is the reference implementation of the JSP and servlet specification, which define the core of the server-side Java architecture. The Clay project developers observed general good software practices and specific software methodologies used in the Tomcat project.

The developers observed Tomcat's use of an XML configuration file and also imitated the form of the Windows .bat files to aid in compilation and execution of the Clay software. The Clay method of reading and utilizing the XML configuration file differs significantly from Tomcat's methodology; in fact, the basis of edu.tcnj.cs.clay.server.ClayXMLConfigReader and the supporting architecture was implemented even before the developers examined the Tomcat internals.

The developers also modified and reused two significant pieces of code from the Tomcat project.

The edu.tcnj.cs.clay.util.SessionManager is based on the org.apache.tomcat.session.StandardManager in Tomcat version 3.2.4. Three methods in edu.tcnj.cs.clay.server.Clay, showHelp(), execute(), and processArgs() are based on methods from org.apache.tomcat.startup.Tomcat.

Due to these contributions, the developers have reproduced and included the Apache License as required both in the source code of the appropriate Clay objects, and as part of the Clay Project License.

Transport of Artifacts

RMI utilizes Java serialization of objects to marshal data across the network. Therefore, each artifact should either implement the java.rmi.Remote interface and extend java.rmi.server.UnicastRemoteObject or implement java.io.Serializable.

Since ProjectServers in Clay should be designed specific to the operations needed to edit a certain artifact, abstracting and representing that artifact is of vital importance.

The Hopewell project CurriculumMap object is an outstanding example of a complex object type that must be abstracted to enable the RMI mechanism to transport it .

Limitations

Our approach to defining the interaction supported by Clay forms the basis of a significant limitation of the system. The exact implementation details of client-server communication are left to the developer of the ProjectServer. Clay does not impose an arbitrary set of coordination rules. Doing so would both slow performance and unnecessarily restrict the ProjectServer developer.

In doing so, it is possible for the user of the client to misuse the client controls in such a way as to overwrite another client's updates as they occur. Clay relies partly on human good sense; abuse of the technology to intentionally overwrite another's work as it is broadcasted is not addressed by the software. It is possible for a ProjectServer to assign levels of authority to different users, thereby establishing a "pecking order" to the users, or keeping a cache of locations that have been previously updated and only allow the original author to update that area. However, implementing these or any other protection schemes means significant and clear design work on the part of the ProjectServer developer.

Deadlock Issues

Clearly, supporting concurrent collaboration means dealing with and resolving deadlock issues. The Java Tutorial for the Java Swing components dealing with modeling textual data warns that

"Document listeners are notified of changes only if the change has already occurred. Sometimes, you might be tempted to change the document's text from within a document listener. For example, if you have a text field that should contain only integers and the user enters some other type of data, you might want to change the text to 0. However, you should never modify the contents of text component from within a document listener. In fact, if you do, your program will likely deadlock! Instead, provide a custom document and override the insertString and remove methods as needed." [4]

Three deadlock issues occur in the Clay architecture. First, dirty writes could propagate through the system due to race conditions. Second, an "echo" from one client to the server and back to the client with duplicated information (user-entered + server-updated) could flood the server and clients with useless information. Third, clients should avoid rebroadcasting information handed to them via the server. They should only broadcast data from local text events.

Diagrams of these situations can be found in the appendix.

These problems are handled by two simple conditions:

First, extend and overwrite the Document object so that the insert() and remove() methods do not "rebroadcast" information. In conjunction with this step, a ProxyDocumentListener is implemented. It will receive DocumentEvents only after a document on the client has been modified via the API. Remember we implement the API methods insert() and remove() in such a way as to prevent "rebroadcasting" of changes to the document by the server. In addition, the ProxyDocumentListener maintains a cache of events. If it has "seen" a particular uniquely identified event before, it will not retransmit the event.

Second, the server will not broadcast a change to a document to the client that originates that change. This measure prevents the specific case of "echo". The other two deadlock issues are addressed by the first measure.

These two conditions are expressed as so: 


if( docEvent comes from named client )
    do not broadcast;
else
    broadcast to clients;


if( docEvent comes from [our client | server] )
    update gui
    BUT DO NOT send via api to proxy to server;
else if( docEvent comes from [text input | user] )
    send via api to proxy to server;

The first condition is implemented on the server and will prevent the server from transmitting a change to the client that made that change. This prevents the client from receiving and transmitting "old" information. It has the effect of eliminating one specific case from consideration in the second condition.

The second condition is implemented on the client and prevents clients from retransmitting "new" information, or information they didn't get from the keyboard.

Other Approaches

There is at least one other approach to creating an interactive collaborative environment. This approach is based on a combination of biology and Java GUI abilities.

For example, imagine that the artifact to be collaboratively edited is a text document. Clearly, users cannot write to the same position at the same time. Updates happen in some order; this order is probably the order in which the update event was received. There are also a good number of readers/writers problems with multiple concurrent updates of a shared document.

The Java Foundation Classes (JFC), or Java Swing, offers the ability to make a component a varying degree of opacity. If we rely on the human eye to sort out and distinguish important conflicts or areas of distortion, and the user to identify areas of heavy use, we can implement an artifact that does not have to deal with readers/writers issues but still provides analogous support for concurrent document editing.

In fact, if at all possible, all Clay clients should take this approach in their implementation because it bypasses a significant amount of design and implementation server side.

The artifact is a three dimensional layered pane. The top layer belongs to the current user, and is the most opaque (and therefore the most visible). Only the current user has write permissions onto this layer.

Beneath this user owned layer is a series of layers at varying degrees of opacity that the user has read-only access to. In this way, a user has write access to all of their panes in both their client and their collaborator's clients. This write access in no way conflicts with any other users, and each client can still observe all other's work.

If the users notice an interesting event or conflict, a mechanism can be implemented to cycle through the layers to bring the conflicting layer to a sharper focus.

Layers can be uniquely identified in a client by the following addressing scheme. Specific positions of characters (or pixels) in the layer can be addressed by appending the (x,y) or (r,c) coordinates to the layer address. The address is a three part coordinate system; the first coordinate identifies the client, the second coordinate identifies the (unique) title of the document being edited, and the third coordinate identifies the specific layer (by owner name). 

String clientID = this.getID();
String docTitle = this.getCurrentDocumentTitle();
String layerID = "nameOfCollaborator";

LayerCollection layers = client.getLayerCollection();
Layer layer = layers.getLayer( clientID, docTitle, layerID );

This approach is but one of many possible approaches. As Clay develops and matures, new approaches will probably suggest themselves as a result of testing or refactoring the code.

Results

There were two major results of this research and software design. First, the researchers designed a stable and robust architecture for distributed concurrent collaborative computing. Second, everyone involved learned a significant amount of useful technology and the specific software methodologies used to put that knowledge into practice.

Other results include

References

1. http://www.tcnj.edu/~assistf/clay/ClayResPres.html

2. http://jakarta.apache.org/tomcat/

3. http://www.webopedia.com/TERM/A/Application_Service_Provider.html

4. http://java.sun.com/docs/books/tutorial/uiswing/components/generaltext.html#doclisteners

5. http://java.sun.com/products/javawebstart

Appendices

Deadlock Diagrams

Rebounding echo from one client to all others. As a new event comes from the keyboard or other input device, the client1 GUI makes the update to client1, and instructs client1 to pass the event to the server. The server is implemented to pass this new event to all clients except the client it originated from. When client2 receives this event, it updates its GUI through the GUI API. When this happens, the GUI fires a "new" event (the same one that has just propagated through the system). All other clients then receive this "new" event as before. In this way, the event is endlessly "echoed" to all clients. With more clients sending, the rate of echoing increases geometrically.
Race to update between clients. Two clients may "race" to update the same document. If they are editing close areas of the document, each update may be overwritten by the other client. If one client is locked out from writing to the document by the server, that client may starve, or update events may be lost when a buffer is filled. This race condition could lead to deadlock, because client1 is a listener for GUI events in client2, and client1 also updates the GUI in client2, and vice versa.
Another client joins the editing session, but does not get the latest document. If two clients are engaged in an editing session, and a third client wants to join, the third client may not receive the most recent changes to the document. This condition may partially be addressed by a caching scheme on the server, but updates during the time the document is transmitted to client3 will be lost. The server then needs to lock out the other clients from editing the document while the transfer is in process, introducing possible deadlock or long wait periods for a slow network connection from client3.

Database Schema and SQL scripts

Connection Information:

Driver org.postgresl.Driver
Connection String jdbc:postgresql://192.168.0.3:5432/claydb
Username postgres
Password postgres70

Schema:


	Table1: ClayUser
	- username    varchar(30)  Primary Key
	- firstname   varchar(20)
	- lastname    varchar(20)
	- password    varchar(20)
	- affiliation varchar(100)
	- isLoggedIn  boolean
	- chatFgColor varchar(7)
	- chatBgColor varchar(7)

	Table2: Projects
	- name        varchar(100)
	- id          autoincrement integer Primary key
	- summary     varchar(500)
	- status      varchar(20)
	- startdate   varchar(20)
	- enddate     varchar(20)

	Table3: UserProjects
	- username  varchar(30) foreign key
	- id        serial integer foreign key
	- isAdmin   boolean

SQL Scripts:


(for Mysql)
(for PostgreSQL)
 
CREATE TABLE ClayUser(
   username varchar(30) not null,
   firstname varchar(20),
   lastname varchar(20),
   password varchar(20),
   affiliation varchar(100),
   isLoggedIn char(1),
   chatFgColor varchar(7),
   chatBgColor varchar(7),
   primary key(username)
);
CREATE TABLE Projects(
   name varchar(100),
   id mediumint not null auto_increment,
   summary varchar(255),
   status varchar(20),
   startdate varchar(20),
   enddate varchar(20),
   primary key(id)
);
CREATE TABLE UserProjects(
   username varchar(30),
   id mediumint,
   isAdmin char(1),
   primary key(id,username)
);

CREATE TABLE ClayUser(
   username varchar(30) PRIMARY KEY,
   firstname varchar(20),
   lastname varchar(20),
   password varchar(20),
   affiliation varchar(100),
   isLoggedIn boolean,
   chatFgColor varchar(7),
   chatBgColor varchar(7)
);
CREATE TABLE Projects(
   name varchar(100),
   id SERIAL PRIMARY KEY,
   summary varchar(255),
   status varchar(20),
   startdate varchar(20),
   enddate varchar(20)
);
CREATE TABLE UserProjects(
   username varchar(30),
   id,
   isAdmin boolean,
   FOREIGN KEY(username) REFERENCES ClayUser,
   FOREIGN KEY(id) REFERENCES Projects,
   PRIMARY KEY(username,id)
);

Installation Guide

Due to the diverse specifications of the numerous operating systems in which Clay can be run on, configuring the install scripts must be tailored to each. This scripting does not prove to be a problem within version releases of the software; however, the software development phase indicates that radically different platform configurations must be accounted for.

Our development team, consisting of seven core members and a mentor, each has a different setup with different resources available to him or her. The diversity in resources proves very hazardous to the installation of compatible versions of the codebase on each computer for testing purposes. At the same time, everyone is eager to contribute more code, time, and energy to turn our dream into a reality. Therefore, it has become very necessary to have multiple entries in the configuration file, with remarked sections matching each member within the group and their specific setup. In addition, as our technology is being upgraded, this problem will not soon go away. As developers, we need a solution to this problem, which comes by the way of an easy-to-use installer.

To continue our commitment to supporting a variety of operating systems, this installer program will also be written in the Java language. However, it will be lightweight, in order to assure an easy installation of itself. In fact, the file available from our main download site will be in the .jar format. This will allow a user to simply download it and double click (on a Windows platform, if UNIX, run java -jar [jarfile.jar] from the command line).

The installer must be able to do multiple things. It should contact Clay's home website and automatically download the appropriate version of the software, according to machine and operating system. In addition, it must correctly install the correct software on the user's machine. As a third and final step, it should allow the user to configure their terminal to run the software. Each one of theses steps should be as automated as possible.

Currently, such an installer is being developed, and appropriate documentation being written concurrently. As it stands now, the installer just handles the third job described above. It can generate the appropriate XML file for the users machine, and install the correct .bat/script files to the user's computer. Currently, there is little automation in the setup. However, this will be added as the program becomes more streamlined and developer constraints and demands become more pressing.

Clay XML Configuration File

<?xml version='1.0' encoding='utf-8'?>

<!DOCTYPE ClayServerConfig
[
<!-- Begin DTD -->

<!ELEMENT clay (mainServer, classServer, UserProjectDatabase, cstore, documentServers, utilities)>


<!ELEMENT mainServer EMPTY>
	<!ATTLIST mainServer
		rmiservername CDATA      "String"
		port CDATA	            "String"
		serverInetAddress CDATA  "String"
		keyFile            CDATA "String"
		consolePort        CDATA "String"
		gracefulMessage       CDATA "graceful"
		stopMessage        CDATA "stopclay"
		debug CDATA	            "String"
	>

<!ELEMENT classServer EMPTY>
	<!ATTLIST classServer
		name CDATA               "String"
		port CDATA	            "String"
		classpath CDATA          "String"
		debug CDATA	            "String"
	>


<!ELEMENT userProjectDatabase EMPTY>
	<!ATTLIST userProjectDatabase
		driverName		 CDATA 		"String"
		url                      CDATA	        "String"
		username                 CDATA               "String"
		password                 CDATA	        "String"
		minConnections           CDATA               "String"
		maxConnections           CDATA               "String"
	>

<!ELEMENT cstore EMPTY>
	<!ATTLIST cvs
		rootdir         CDATA       "String"
		debug 		CDATA       "String"
	>

<!ELEMENT documentServers (docserver*) >


<!ELEMENT docserver EMPTY >
	<!ATTLIST docserver
		projectid      CDATA          "String"
		projectname    CDATA          "String"
		classname      CDATA          "String"
		minAgent  CDATA	         "String"
		maxAgent  CDATA          "String"
		authPoolInc    CDATA          "String"
		timeOutCheck   CDATA          "String"
		sessionTimeOut CDATA          "String"
		debug          CDATA          "String"
	>


<!ELEMENT utilities (Logger*)>


<!ELEMENT Logger EMPTY>
	<!ATTLIST Logger
		name     CDATA "String"
		fullDateFormat CDATA "String"
	>



]
>
<!-- it's clear that this file contains passwords, usernames, and commands in clear-text -->

<clay>

	<!-- While you can specify the name of the server and the port, clients must know about this
	     connection string. Just telling them to connect to 1099 will not get them the Clay object.
	     name = name of the server object itself, to uniquly id in rmi registry
	     port = port to connect on
	     serverInetName = the IP address or resolvable host name of the computer running Clay and the server
	                      RMI Registry
	-->
	<mainServer rmiservername="ClayServer"
		    port="1099"
		    serverInetAddress="192.168.0.2"
		    keyFile="claypid.key"
		    gracefulMessage="graceful"
		    stopMessage="stopclay"
		    consolePort="4033"
		    debug="6"
	/>

	<!--  this information must match that in the clay.bat script for startup, specifically, the
	   -Djava.rmi.server.codebase=http://serverInetAddress:port
	   see $CLAY_HOME/docs/ClayClassServer.readme.txt
	   note it defaults to this machine, the classfileserver is in the same jvm as clay.main()
	   classpath attribute should be url (file: or otherwise). it does not support multiple paths.
	-->
	<classServer name="ClayClassServer"
		     port="3780"
		     classpath="file:/c:/usr/projects/clay/clay-1.2/www/"
		     debug="6"
	/>

	<!-- This will enable Clay to start up the connection pool
	    Comment out the ones you don't need.
	-->

	<!--
	<userProjectDatabase driverName = "org.gjt.mm.mysql.Driver"
		    url="jdbc:mysql://www.freesql.org:3306/mikeylo"
		    username="mikeylo"
		    password="mikeylo"
		    minConnections="2"
		    maxConnections="6"

	/>
	-->

	<!--
	<userProjectDatabase driverName = "sun.jdbc.odbc.JdbcOdbcDriver"
		    url="jdbc:odbc:ClayAuth"
		    username="maciej"
		    password="nowacki"
		    minConnections="3"
		    maxConnections="5"

	/>
	-->

	<!--
	<userProjectDatabase driverName = "sun.jdbc.odbc.JdbcOdbcDriver"
			    url="jdbc:odbc:testclay"
			    username=""
			    password=""
			    minConnections="3"
			    maxConnections="10"

	/>
	-->

	<userProjectDatabase driverName = "org.postgresql.Driver"
		    url="jdbc:postgresql://192.168.0.3:5432/claydb"
		    username="postgres"
		    password="postgres70"
		    minConnections="4"
		    maxConnections="8"

	/>


	<!-- Configuration for the CVS goes here
	     the rootdir attribute should be a full path, not relative from $CLAY_HOME
	-->
	<cstore rootdir="c:\\usr\\projects\\clay\\clay-1.2\\cstorehome\\"
	     debug="6"
	/>

	<!-- We get the number of document servers from reading the database. Make sure it syncs up
	    with what is listed here. The default ProjectServer is edu.tcnj.cs.clay.server.ProjectServer.
	    You may specify a different class to handle the project id in the database. In the entry for
	    the projectServer, you may specify the type of client designed to work with that project server.
	-->
	<documentServers>

		<docserver projectid="1"
			   projectname="clay"
			   classname="edu.tcnj.cs.clay.server.CoopCoderProjectServer_Impl"
			   minAgent="4"
			   maxAgent="10"
			   authPoolInc="2"
			   timeOutCheck="60"
			   sessionTimeOut="20"
			   debug="6"
		/>

		<docserver projectid="3"
			   projectname="PPP Pretty Pictures Project"
			   classname="edu.tcnj.cs.clay.server.WBProjectServer_Impl"
			   minAgent="4"
			   maxAgent="10"
			   authPoolInc="2"
			   timeOutCheck="60"
			   sessionTimeOut="20"
			   debug="6"
		/>

		<docserver projectid="4"
			   projectname="Hopewell Curriculum Map Editing"
			   classname="edu.tcnj.cs.clay.server.HopewellProjectServer_Impl"
			   minAgent="4"
			   maxAgent="10"
			   authPoolInc="2"
			   timeOutCheck="60"
			   sessionTimeOut="20"
			   debug="6"
		/>

	</documentServers>




	<!-- Some general purpose utilities -->
	<utilities>

		<!--
		  Clay will create an object for each logger element below.
		  The three supported ones are
		  	'clay_log' : general clay exceptions, clay main server
		  	'ds_log'   : document server and client error logs
		  	'gen_log'  : class file server log, cstore log
		-->

		<Logger name="clay_log" fullDateFormat="true" />

		<Logger name="ds_log"  fullDateFormat="true" />

		<Logger name="gen_log" fullDateFormat="true" />


	</utilities>


</clay>

License

The full license is online on the Clay website.

THE LICENSE

License Summary:

The Clay Synchronous Interactive Environment is a product of TCNJ computer science undergraduate research. As explained in the Summary Statement, " The "Software" includes the code under the src/ tree in the application, the Documentation written by the authors of the Software, and the scripts in the bin/ directory. Specifically exempt from this license are the JAR files in the lib/ directory that are NOT the work of the authors and the license documents redistributed with the Software that are NOT authored by the Authors of the Software."

This work "the Software" is Copyright 2001-2002, Michael E. Locasto, Michael Hulme, Ryan Gladysiewicz, Justin Tracy, Maciej Nowacki, Michael Massimi, Bob Hutzel. All rights reserved.

This Software is not subject to any license of The College of New Jersey, or any employers of the above named people, "the Authors".

This Software is provided at http://www.tcnj.edu/~assistf/clay with full source code, documentation, and supporting libraries.

Redistribution and use in source and binary forms are permitted without restriction or fee of any kind as long as this notice is preserved and redistributed.

Modification is allowed as long as these licences remain intact, original credit is reflected, and changes are attributed to those that made them. Changes must also be submitted back into the original code base. Any software that is based on this Software must also be open-source.

THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COLLEGE OF NEW JERSEY OR THE AUTHORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

 

Michael E. Locasto is a Computer Science student at The College of New Jersey. He has participated in various undergraduate research projects for the past two years and enjoys hacking Java and other web-related technologies.
Michael Hulme is a Computer Science student at The College of New Jersey. He is talented in a variety of areas.
Copyright © 2001-2002 Michael E. Locasto http://www.tcnj.edu/~assistf/clay/