Hi, everyone. This is my Ph.D. dissertation. It is available on Amazon.com as an eBook for Kindle. The full citation for this document is:
Self-Improvement through Self-Understanding: Model-Based Reflection for Agent Adaptation. J. William Murdock. Ph.D. Thesis, College of Computing, Georgia Institute of Technology, July 2001.
The systems featured in this thesis are SIRRINE and REM, q.v.
The ability to adapt is a key characteristic of intelligence. This dissertation explores techniques for enabling intelligent software agents to adapt themselves as their functional requirements change incrementally. In the domain of manufacturing, for example, a software agent designed to disassemble physical artifacts may be given a new goal of assembling artifacts. As another example, in the internet domain, a software agent designed to browse some types of documents may be called upon to browse a document of another type.
This research examines the use of reflection (an agent's knowledge and reasoning about itself) to accomplish adaptation (incremental revision of an agent's capabilities). Reflection in this work is enabled by a language called TMKL (Task-Method-Knowledge Language) that supports modeling of an agent's composition and functioning. A TMKL model of an agent explicitly represents the tasks the agent addresses, the methods it applies, and the knowledge it uses. TMKL models are hierarchical, i.e., they represents tasks, methods and knowledge at multiple levels of abstraction. These models are used in a reasoning shell called REM (Reflective Evolutionary Mind). REM provides support for the execution and adaptation of agents which contain TMKL models of themselves.
This dissertation presents a variety of strategies for adapting agents. Some of these strategies are purely model-based: knowledge of composition and functioning directly enables adaptation. This model-based approach is combined with two traditional artificial intelligence and machine learning techniques: generative planning and reinforcement learning. The combination of model-based adaptation, generative planning, and reinforcement learning constitutes a mechanism for reflective agent adaptation which is capable of addressing a variety of problems to which none of these individual approaches alone is suited. The work described in this dissertation has demonstrated the computational feasibility of this mechanism using experiments involving a variety of intelligent software agents in a variety of domains.
For more information on this and other research, see the publications section of my home page. Some of the papers which are particularly relevant to this work from that page are:
Meta-Case-Based Reasoning: Self-Improvement through Self-Understanding. J. William Murdock and Ashok K. Goel. Journal of Experimental & Theoretical Artificial Intelligence, 20(1):1-36, March 2008. (PDF, WWW)
Using Model-Based Reflection to Guide Reinforcement Learning. Patrick Ulam, Ashok Goel, Joshua Jones, and J. William Murdock. Proceedings of the IJCAI 2005 Workshop on Reasoning, Representation and Learning in Computer Games. Edinburgh, UK, 2005. (PDF)
Localizing Planning with Functional Process Models. J. William Murdock and Ashok K. Goel, Proceedings of the Thirteenth International Conference on Automated Planning & Scheduling (ICAPS'03). Trento, Italy, June 9-13, 2003. (PDF)
Learning about Constraints by Reflection. J. William Murdock and Ashok K. Goel, Proceedings of the Fourteenth Canadian Conference on Artificial Intelligence (AI'01), Ottawa, Ontario, Canada, June 7 - 9, 2001. (PS, PDF) [*]
Meta-Case-Based Reasoning: Using Functional Models to Adapt Case-Based Agents. J. William Murdock and Ashok K. Goel, Case-Based Reasoning Research and Development. Aha, D.W., Watson, I., and Yang, Q. (Eds.). Proceedings of the 4th. International Conference on Case-Based Reasoning (ICCBR'01). Vancouver, Canada, July 30 - August 2, 2001. (PS, PDF) [*]
Model-Based Reflection for Agent Evolution. J. William Murdock, Ph.D. Defense of Research, Georgia Institute of Technology College of Computing Technical Report GIT-CC-00-34, 2000. (PS)
* Indicated publication is published in the Springer-Verlag Lecture Notes in Computer Science series or a related sub-series, q.v.