Truth is stranger than fiction, because fiction has to make sense.

:: CLUE Lab ::


Computer Languages and User Experience Laboratory

Università degli Studi di Salerno
Dipartimento di Informatica
 

Visual Languages

A visual language is a set of practices by which images can be used to communicate concepts.

In the ambit of Visual Languages, our research has focused on theoretical, methodological and applied aspects of the traditional visual languages and has deepened on the study of the sketched visual languages, that is, the languages defined on free-hand drawn sketched visual languages.

As for the theoretical aspects, extensions to the model of picturel languages to the colored, drawn and pixel pictures case have been defined, studying some of their computability and decidability aspects.
As for the methodological and implementing aspects, a technique, based on positional extended grammars, for the creation of the compilers and the visual languages and then of the visual modeling environment through the use of technologies tradiotionally employed for the textual programming languages, has been define. Some applications focus on the use of visual languages both in the e-learning and in the software engineering fields.

In the ambit of sketched visual languages, our research has brought to the definition of a new grammar formalism, called Sketch Grammar, capable of describing both the free hand drawn symbols and the syntax of the languages they compose. On the sketch grammars several procedures have been defined, similar to those defined for LR Parsing, for the automatic generation of recognizers for the described sketch languages.
In order to improve the efficiency and the accuracy of the built recognizers, several strategies have been defined, such as, the introduction of a training phase, the use of a framework based on agents for the low level recognition of symbols, the use of the ambiguity resolution context during symbol recognition, the use of techniques for error recovery to capture information on missing or erroneously recognized strokes.
The results gained in this ambit and further studies on the definition of incremental parsers based on GLR parsing have been used for the automatic generation of syntax-aware visual language editors, which allow the user to easily construct a hand drawn visual phrase. In these editors, the user is interactively aided in the correct building of the phrase with suggestions on the possible extensions of the phrase with techniques of automatic completion of incomplete symbols.
An initial experimental prototype has been integrated in the VLDesk system, already used for the generation of classical visual environments. This has allow the development of empirical studies on real languages, such as UML, to verify the effectiveness of the proposed approaches.

 

Publications

Our recent publications in Visual Languages research

2010

  • Gennaro Costagliola, Tracy Hammond, Beryl Plimmer. JVLC special issue on sketch computation J. Vis. Lang. Comput. 21(2). Pp. 67-68

2009

2008

  • Giovanni Casella, Vincenzo Deufemia, Viviana Mascardi, Gennaro Costagliola, M. Martelli. An agent-based framework for sketched symbol interpretation Journal Of Visual Languages And Computing, Vol. 19 (2). Pp. 225-257
  • Gennaro Costagliola, Alberto Greco. On Semantic Labeling of Hand-Drawn Sketches Workshop On Visual Languages And Computing. Pp. 288-291
  • Gennaro Costagliola, Alberto Greco. Towards Semantic Role Labeling of Hand-drawn Sketches VL/HCC Workshop On Sketch Tools For Diagramming . Pp. 29-36
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. Using Error Recovery Techniques to Improve Sketch Recognition Accuracy GREC 2007 (Selected Papers). Pp. 157-168

2007

  • Gennaro Costagliola, Vincenzo Deufemia, Giuseppe Polese. Visual Language Implementation through Standard Compiler-Compiler Techniques Journal Of Visual Languages And Computing, Vol. 18. Pp. 165-226
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. Using Grammar-based Recognizers for Symbol Completion in Diagrammatic Sketches 9Th International Conference On Document Analysis And Recognition, ICDAR'07 - IEEE Cs Press. Pp. 1078-1082
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. Towards Robust Sketch Recognizers through Error Recovery Techniques 7Th IAPR International Workshop On Graphics Recognition GREC'07. Pp. 59-65
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. Improving the Robustness of Diagrammatic Sketch Recognizers International Workshop On Visual Languages And Computing, VLC'07. Pp. 241-246
  • Gennaro Costagliola, Vincenzo Deufemia, Giuseppe Polese. Modeling and Implementing Visual Languages with Extended Positional Grammars Visual Languages For Interactive Computing: Definitions And Formalization. Pp. --

2006

  • Gennaro Costagliola, Vincenzo Deufemia, Filomena Ferrucci, Carmine Gravino,. CONSTRUCTING META-CASE WORKBENCHES BY EXPLOITING VISUAL LANGUAGE GENERATORS IEEE Transactions On Software Engineering, Vol. 32 (3). Pp. 156-175
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. A Recognition Framework based on LR Parsing for Hand-drawn Diagram International Workshop On Visual Languages And Computing - VLC'06. Pp. 123-128
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. A Multi-layer Parsing Strategy for On-line Recognition of Hand-drawn Diagrams IEEE Symposium On Visual Languages And Human-Centric Computing - VL/HCC 2006. Pp. 103-110
  • Giovanni Casella, Gennaro Costagliola, Vincenzo Deufemia, MMartelli, Viviana Mascardi. An Agent-Based Framework for Context-Driven Interpretation of Symbols in Diagrammatic Sketches IEEE Symposium On Visual Languages And Human-Centric Computing . Pp. 73-80
  • Gennaro Costagliola, Andrea De Lucia, Vincenzo Deufemia, Carmine Gravino, Michele Risi. Case Studies of Visual Language Based Design Patterns Recovery The 10Th European Conference On Software Maintenance And Reengineering - CSMR 2006. Pp. 165-174

2005

  • Gennaro Costagliola, Vincenzo Deufemia, Giuseppe Polese, Michele Risi. Building Syntax-Aware Editors for Visual Languages Journal Of Visual Languages And Computing, Vol. 16 (6). Pp. 508-540
  • Gennaro Costagliola, Filomena Ferrucci, Giuseppe Polese, Giuseppe Scanniello. A Visual Language Based System for Designing and Presenting E-learning Courses International Journal Of Distance Education Technologies, Vol. 3. Pp. 43466
  • Gennaro Costagliola, Filomena Ferrucci, Carmine Gravino. Adding symbolic information to picture models: definitions and properties Theoretical Computer Science, Vol. 337(1-3). Pp. 51-104
  • Gennaro Costagliola, Vincenzo Deufemia, Giuseppe Polese. Towards Syntax-Aware Editors for Visual Languages Electronic Notes In Theoretical Computer Science, Vol. 127(4). Pp. 107-125
  • Gennaro Costagliola, Andrea De Lucia, Vincenzo Deufemia, Carmine Gravino, Michele Risi. Design Pattern Recovery by Visual Language Parsing 10Th European Conference On Software Maintenance And Reengineering - CSMR 2005. Pp. 102-111
  • Gennaro Costagliola, Andrea De Lucia, Filomena Ferrucci, Giuseppe Scanniello. Visual Languages for Non Expert Instructional Designers: A Usability Study The 11Th International Conference On Distributed Multimedia Systems - DMS'05. Pp. 46-51
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. A Trainable System for Recognizing Diagrammatic Sketch Languages IEEE Symposium On Visual Languages And Human-Centric Computing - VL/HCC 2005. Pp. 281-283
  • Gennaro Costagliola, Vincenzo Deufemia, Michele Risi. Sketch Grammars: A Formalism for Describing and Recognizing Diagrammatic Sketch Languages Eighth International Conference On Document Analysis And Recognition - ICDAR 2005. Pp. 1226-1231
  • Gennaro Costagliola, Vincenzo Deufemia, Filomena Ferrucci, Carmine Gravino,. A USER-CENTERED METHODOLOGY TO GENERATE VISUAL MODELING ENVIRONMENTS ICEIS 2004 (Selected Papers). Pp. 219-226

WebLab Site Search

Upcoming Events

No current events.