Current Projects
While numerous tools have been developed to visualize dynamically executing programs, little attention has been paid to visualizing the correlations among different behaviors. The software system's multifaceted characteristics and diverse behaviors make it difficult for a developer to acquire sufficient knowledge about the real behavior of a system without observing its multiple executions. A visualization framework for cross-referencing multiple program executions would provide tremendous helps. This project investigates the design issues of a program correlation analysis framework, and uses a multi-plane visualization approach. To effectively visualize the correlations, we formalize the plane placement problem and decide the spatial arrangement of multiple execution representations based on the proximity of their semantic relationships using the Bond Energy Algorithm (BEA). To automate the process, a visualization tool called SoftLink has been developed, and experiments have been conducted on an open source project. (Zhao, Zhang, Hao, and Wong: IEEE SSIRI'09)
There is a dramatic increase in the population who use mobile computing devices. Though the hardware becomes more powerful, effective support for information rendering on small screens very much lags behind. To display hierarchical information, researchers have proposed many algorithms for desktop screen visualization. Such algorithms are generally divided into connection and enclosure. Connection approach displays hierarchy with a clear structure but consume display area. Enclosure can maximally utilize the screen space but the layout is essentially implicit. This project aims at developing an approach for visualizing hierarchical data on palmsized devices, by combining the advantages of both connection and enclosure approaches. While maximizing the space usage, our approach can achieve structural clarity of layout by arranging location of non-overlapping regions. We plan to adapt our techniques for visualizing structural information for various applications. (Hao, Zhang, and Hsieh: IEEE ISCE'07)
The Reserved Graph Grammar (RGG) is a general graph grammar formalism that expresses a wide range of visual languages. This project investigates an extension to RGG with the capability of spatial specification, called SGG. Graph transformation satisfying the spatial specification will be performed in the process of parsing. The RGG with spatial specification can be applied to various types of applications, such as mathematical expression recognition, Web transformation (see the next project), multimedia authoring and presentation, and software engineering. The following two projects are two on-going effort in the applications of the extended formalism. (Qiu et al.: IEEE VL'03, Zhang et al.: ACM/Springer Multimedia Sys., Kong, Zhang and Zeng: ACM Trans on CHI, Zhao, Kong and Zhang: IEEE Trans on SE)
Many heuristic approaches have been developed to group semantically related visual elements underlying a graphical interface. However, those approaches cannot address the challenging issue of uniformity satisfactorily, and are not capable of tagging a semantic role of each element. This project investigates a robust and scalable approach to extracting interface semantics using graph grammars. We use the state-of-the-art graph grammar technology, i.e. the Spatial Graph Grammar (SGG), to perform semantic grouping and interpretation of the segmented screen objects. SGG introduces spatial notions to the abstract syntax. The direct representation of spatial information in the abstract syntax can make productions easy to understand since grammar designers often design rules with similar appearances as the represented graphs. In other words, using spatial information to directly model relationships in the abstract syntax is coherent with the concrete representation, which avoids converting spatial information to edges. Furthermore, the power of spatial specification in the abstract syntax allows designers to model interface semantics from the perspectives of both visual effects (e.g., a topological relation between two visual elements) and structural relations (e.g., a DOM structure relation in a Web interface).
Recently Completed Projects
As a commonly acceptable standard for guiding Web markup documents, XML allows the Internet users to create multimedia documents of their preferred structures and share with other people. The creation of various multimedia document structures, typically as trees, implies that some kinds of conversion mechanisms are needed for people using different structures to understand each other. This project investigates a visual approach to the representation and validation of multimedia document structures specified in XML and transformation of one structure to another. The underlying theory of our approach is a context-sensitive graph grammar formalism. The project takes advantage of the conciseness and expressiveness of the graph grammar formalism in specifying XML structures and transformations from one structure to another.The graph grammar formalism with spatial extension (SGG, see above project) can be used to transform desktop Web presentations to suit small screen mobile devices such as PDAs. The work is underway for applying SGG to the transformation of Web page structures and Web graphics to suit PDA devices. We will perform the translation of XML pages to WML cards through three steps: transforming the original XML file into a host graph automatically or using a Web graph constructed using a graph editor; using the SGG to transform the layout of host graph into a presentation suitable for multiple small pages; and finally, generating the equivalent WML document. (Zhang et al.: IEEE HCC'01)
To generalize the above concept, we propose a unified framework to manage model-based information on the Web in a hierarchical structure. The framework allows models, schemas, and data instances to be represented explicitly and uniformly. The uniform representation of the framework also enables simple user-defined graph transformation rules for different Web data models to translate schemas and data instances between different formats. In addition, the framework implements a set of prototype tools for users to identify meta-primitives at the meta-model level, to define a model or schema by specifying a set of graph grammar rules and to draw the structure of data instances. These features promote a wide scope of Web-related applications, such as information exchange between different organizations, and integration of data coming from heterogeneous information sources. (Song and Zhang: IEEE ITCC'04, Song et al.: ACM/IEEE WI'04)
The project currectly focuses on a visual representation for model management operators based on graph transformation. Graph transformation formalisms, as the theoretic foundation of many visual programming languages, can formally represent model management operators by visual and intuitive expressions. By using graphical representations users can easily comprehend and manipulate the operators and desired outputs. (Song et al.: IEEE VL-HCC'04, Song et al.: IEEE EEE'05)
Clustering techniques have been widely used in many application areas due to the necessity of grouping closely related entities together in order to identify coherent groups (i.e. clusters). It is difficult, if not impossible, to find a universally applicable clustering technique for all kinds of applications. Thus the involvement of domain knowledge to the general clustering methods becomes inevitable. Unfortunately, the utilization of domain knowledge is still a difficult job with current clustering techniques. Aiming at solving part of the domain knowledge utilization problem, this project proposes a customizable framework for data clustering based on a set of analysis criteria. The framework includes knowledge representation, constraint solving, quality measurement, and cluster visualization. We propose a set of criteria based on a sparse graph model that has four features: first, the criteria can represent the user's requirements. Second, they measure the quality of a clustering result. Third, they combine the application-independent criteria and user's specific needs. Last, they provide a guideline for more customizable clustering algorithms. (Qian and Zhang: ACM SAC'03, Qian and Zhang: ACM SAC'04, Qian and Zhang: DMKD'04, Qian, Zhang and Qiu: ACM SAC'05, Qian and Zhang: ACM SAC'05)
Visual programming is a key to successfully enhancing the computer utlisation. As the first part of the overall VESTIGE project, this project aims to design and develop an object-oriented visual language for parallel programming. The language supports direct manipulation in a graphical user-interface, and allows the user to construct a high level domain-specific language for either parallel programming or simulating concurrent systems. It will assist, graphically, program editing, verification, debugging, and performance tuning. The platforms we are working on include Java and MPI ( Zhang, et al.: J. Visual Languages and Computing).
The design of Web sites has been largely ad hoc, with little concern about the effectiveness of navigation and maintenance. This project proposes a general framework that addresses the issues in Web site design, navigation, and maintenance through an integrated graphical approach. The focus of the framework is a user-friendly interface for Web site/page design, navigation and maintenance, capable of graphical design, scalable structuring, information grouping and filtering, and XML customisation. Web application resuse is achieved through automatic generation of Web graphs from existing sites, re-design and re-generation of new Web sites. The user-interface uses a simple and intuitive graph formalism throughout the development life cycle so that a single mental map is maintained for Web designers and users. (Zhang et al.: IEEE WECWIS'02)
Previous research on building performance visualisation tools for parallel programs has a major shortcoming. Many tools collect performance data by instrumenting application programs with data recording probes. Such probes significantly perturb a program's original behaviour and thus make it difficult to measure the performance of the program.As part of the overall research programme in the development of an integrated parallel program visualisation tool, this project aims at minimising the perturbation of instrumentation by inserting probes at the assembly level. Apart from less intrusiveness to the original program, there are other advantages with lower level instrumentation. The program performance affected by the architectural organisation can be measured, and the low level instrumentor has also a better control over data organisation, memory arrangement, and communications among multiple processors. (Cai, Zhang, et al.: J. Parallel Computing, Stankovic and Zhang: IEEE Trans. on SE)
A performance measurement and tuning tool is an important part of an integrated parallel programming environment. Although there have been important advances in performance tuning tools in recent years, evidence has shown that users do not find the current generation of tools useful for their program development needs as they offer no guidance as to how performance may be improved. Today's state-of-the-art performance analysis systems offer little more than basic measuring and analysing facilities. We believe that an ideal tuning tool should serve as a ``performace advisor", using observed performance characteristics and stored knowledge to direct performance optimisations. The main aim of this project is to develop a visualisation environment for parallel programmers to tune the performance of their programs. The performance tuning is performed by providing information on the performance losses and advising for improvement, rather than by providing only volumes of complex graphs and tables that require an expert to interpret. (Li and Zhang: J. Supercomputing)
The implementation of visual programming languages and their supporting environments is time-consuming and tedious. To ease the task, researchers have developed some high level tools to reduce the development effort. None of such tools, however, can be easily used to create a complete visual language in a seamless way like the lex/yacc tools for textual language constructions.This project aims at the design, construction, and application of a domain-specific visual language (DSVL) generation environment, called VisPro. The VisPro design model improves the conventional Model-View-Controller framework in that its functional modules are decoupled to allow independent development and integration. The VisPro environment consists of a set of meta visual programming tools. Using VisPro, the process of DSVL construction can be divided into two steps: lexicon definition and grammar specification. The former step defines visual objects and a visual editor, and the latter step provides language grammars with graph rewriting rules. The compiler for the DSVL is automatically created according to the grammar specification. A target DSVL is generated as a programming environment which contains the compiler and the visual editor. ( Zhang, Zhang, and Cao: Computer J., Zhang and Zhang: IEEE VL'98, Zhang, Zhang, and Cao: IEEE Trans. on SE)
