Cross, Marilyn
Marilyn Cross
Senior Research Scientist
DSTO C3 Research Centre
Fernhill Park
DEPARTMENT OF DEFENCE
CANBERRA ACT 2600
Australia
Phone: +61 6 265 8026
Fax: +61 6 265 8010
Email: marilyn.cross@dsto.defence.gov.au
marilyn_cross@macitd2.dsto.gov.au
STRAND: Semiosis in Language and other Modalities
Making Meaning in Multimodal Contexts
The Information Age is bringing an increasing and diversified set
of sources for finding and making meaning. The diversity and
volume of accessible on-line information from a variety of
sources, including the Internet, is increasing dramatically.
Information is now available in a multiplicity of media that
includes texts, photos, video clips, and films. Not only are
unstructured data in the form of free text and structured data in
the form of databases on-line, but non-text data such as images,
audio and video clips are increasingly available. In order to
take advantage of this increasing diversity and availability of
multimodal sources new theoretical approaches and technologies
are required. Providing some degree of automated assistance in
the task of exchanging, coordinating and making meaning in
contexts that span different modalities is a major challenge.
The technology of multimedia does provide a means of connecting
different sources of information using reference links, but it
does not assist in the translation of meaning between different
representations. Multimodality is described in the context of
communicable diseases using a systemic-functional model of
ideational meaning. Using the domain model, computational
approaches to retrieving, manipulating and generating multimodal
sources are explored.