Author:

Abstract:

“Bringing text to life” through 3D interactive storytelling is a new way of exploring and understanding text information, achieved by transforming documents into virtual worlds where the user can interact with the content in a game-like environment. It requires a deep understanding of text and a mapping from text to a formal knowledge representation that can be then used to generate characters, objects, and events etc. of the virtual 3D world. This challenge is addressed by this dissertation from a Natural Language Processing (NLP) point of view, with a focus on children’s stories. Due to the lack of training data, we perform an NLP analysis on the text, then translate the obtained NLP structures into the targeted formal representation. This dissertation identifies three key natural language processing tasks needed to solve the problem: Semantic Role Labeling (SRL), coreference resolution and Implicit Semantic Role Labeling (iSRL). We found that it is very important to develop domain adaptation techniques to adapt standard NLP systems to children’s stories. This is due to an observation of significant performance drops when applying supervised systems are trained on one domain to another domain. Our primary goal is to develop light-weight domain adaptation techniques for those above three NLP tasks. Here, we call a technique light-weight when it limits the need of extra human efforts. A common idea through this dissertation is to investigate the uses of automatically extracted knowledge from unlabeled data, available human knowledge and existing manually-constructed resources. Our contributions are as follows: First, for SRL, we propose a method based on a recurrent neural network language model and several linguistic filters to automatically generate additional SRL training data that is closer to the target domain, for circumstance semantic roles – location, time, manner and direction. By this work, we show the effectiveness of automatic knowledge acquisition from a language model. Second, a collaboration of word embeddings and co-training facing the most difficult case of SRL – to predict unseen semantic frames in an out-of-domain scenario – is presented. This work shows the positive effect of using the distributional word representation (word embeddings) on aiding a traditional technique in an uncommon and difficult scenario. Our experiment on the benchmark CoNLL 2009 data shows an improvement over the other state-ofthe-art SRL systems in the out-of-domain scenario. Third, we extend the state-of-the-art coreference resolution system of UC Berkeley to narratives by designing a novel global inference algorithm that adopts narrative-specific constraints known by humans. When testing on the UMIREC and N2 corpora, our proposed inference substantially outperforms the original inference on the CoNLL 2011 metric. This is a proof that human knowledge is still useful during the era of automatic knowledge acquisition. Fourth, an unsupervised iSRL approach based on a recurrent neural semantic frame model to overcome the lack of training data, is introduced. This work proposes a novel technique to learn semantic knowledge automatically from a huge amount of unlabeled data, and shows its successful application to deal with an emerging NLP task. On the NomBank iSRL test set, our approach improves state-of-the-art performance on implicit semantic role labeling with much less reliance than prior work on manually constructed language resources. Finally, after domain adaptation techniques, this dissertation focuses on a framework, which tackles the “bringing text to life” through 3D interactive storytelling problem. The framework consists of an NLP pipeline to process children’s stories, a Bayesian network to map the obtained NLP structures to the 3D world’s formal representation, and a graphical engine to render the 3D animations. The main contributions of this dissertation include developing the NLP pipeline and extracting evidence for the Bayesian network. This dissertation lays the initial foundation to develop an innovative idea of “bringing text to life”. We successfully employ automatically-extracted knowledge, human knowledge and knowledge from existing manually-constructed resources to our problem, suggesting a promising research direction in developing light-weight techniques.