Sketching

Sketching is a natural activity that people engage in while thinking and communicating, especially when spatial content is involved. Sketching is heavily used in STEM disciplines, such as engineering and geoscience. This working group is creating a sketch understanding system, called CogSketch, which is both a new research instrument for cognitive science research and a potential platform for sketch-based educational software. As a research instrument, we are using CogSketch to model spatial reasoning and learning, extending it using new results from SILC, and contributing to that science base in turn (e.g., a computational model of mental rotation). We are also exploring the use of CogSketch to gather and analyze data in behavioral experiments. Since the software can keep track of properties such as the timing of what is drawn, it can provide information that formerly would require video analysis to glean. Moreover, as SILC research progresses and its cognitive simulations become both broader and higher fidelity, we hope to be able to use it for scoring data, such as making similarity judgments. In education, we are exploring two paths to support STEM education. The worksheet model is designed to support students doing sketch assignments in class work. We are designing this to be as simple as possible, and as broad as possible, to support a variety of disciplines. To complement this effort, we are going in depth into an extremely difficult area, engineering design education. By exploring both in parallel, we maximize the likelihood of creating a truly versatile, useful platform for sketch-based intelligent educational software. Our vision is that, in ten years, sketch-based educational software could be as widely available to students as graphing calculators are today. Our hypothesis is that incorporating sketch understanding into educational software could provide revolutionary benefits for spatial education.

The aims of the Sketching Working Group are:

1. To model spatial reasoning and learning with CogSketch.
2. To explore the use of CogSketch in gathering and analyzing data in behavioral experiments.
3. To develop a worksheet model for CogSketch that can enhance STEM education.
4. To explore the use of CogSketch to support engineering design education.

Point of Contact:

Downloads

• ♦Download CogSketch
• ♦Slides for CogSketch Tutorial

Relevant Background Publications

• ♦Alvarado, C., & Davis, R. (2001). Resolving ambiguities to create a natural sketch based interface. Proceedings of IJCAI-2001, August 2001.
• ♦Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., et al. (1997). QuickSet: Multimodal interaction for distributed applications. Proceedings of the Fifth Annual International Multimodal Conference (Multimedia '97), (pp. 31-40), Seattle, WA. ACM Press.
• ♦Falkenhainer, B., Forbus, K.D., & Gentner, D. (1989). The structure-mapping engine: Algorithm and examples. Artificial Intelligence, 41, 1-63.
• ♦Feist, M.I., & Genter, D. (2001). An influence of spatial language on recognition memory for spatial scenes. In J.D. Moore & K. Stenning (Eds.), Proceedings of the 23rd Annual Conference of the Cognitive Science Society (pp. 279-284).
• ♦Forbus, K. (1995). Qualitative spatial reasoning: Framework and frontiers. In J. Glasgow, N. Narayanan, and B. Chandrasekaran, Diagrammatic reasoning: Cognitive and computational perspectives, AAAI Press.
• ♦Forbus, K., Ferguson, R., & Usher, J. (2001). Towards a computational model of sketching. IUI'01, January 14-17, 2001, Santa Fe, New Mexico.
• ♦Forbus, K.D., Gentner, D., & Law, K. (1994). MAC/FAC: A model of similarity-based retrieval. Cognitive Science, 19, 141-205. (Abridged version to be reprinted in T. Polk & C.M. Seifert (Eds.), Cognitive Modeling. Boston: MIT Press.)
• ♦Forbus, K., Nielsen, P. and Faltings, B. (1991). Qualitative Spatial Reasoning: The CLOCK Project. Artificial Intelligence, 51 (1-3).
• ♦Forbus, K., & Usher, J. (2002). Sketching for knowledge capture: A progress report. Proceedings of IUI'02, San Francisco, California, January 13-16, 2002.
• ♦Klenk, M., Forbus, K., Tomai, E., Kim,H., and Kyckelhahn, B. 2005. Solving Everyday Physical Reasoning Problems by Analogy using Sketches. Proceedings of 20th National Conference on Artificial Intelligence (AAAI-05), Pittsburgh, PA.
• ♦Kuehne, S., Forbus, K., Gentner, D., & Quinn, B. (August 2000). SEQL: Category learning as progressive abstraction using structure mapping. Proceedings of CogSci-2000. Philadelphia, PA.
• ♦Kuipers, B. (2000). The spatial semantic hierarchy. Artificial Intelligence, 119, 191-233.
• ♦Lockwood, K., Forbus, K., & Usher, J. (2005). SpaceCase: A model of spatial preposition use. In Proceedings of the 27th Annual Conference of the Cognitive Science Society. Stressa, Italy.
• ♦Lockwood, K., Forbus, K., Halstead, D. & Usher, J. (2006). Automatic Categorization of Spatial Prepositions. Proceedings of the 28th Annual Conference of the Cognitive Science Society. Vancouver, Canada.
• ♦Regier, T. (1996). The human semantic potential: Spatial language and constrained connectionism. Cambridge, MA: MIT Press.
• ♦Regier, T., & Carlson, L.A. (2001). Grounding spatial language in perception: An empirical and computational investigation. Journal of Experimental Psychology, 130, 273-298.
• ♦Tomai, E., Lovett, A., Forbus, K., & Usher, J. (2005). A Structure Mapping Model for Solving Geometric Analogy Problems. Proceedings of the 27th Annual Conference of the Cognitive Science Society, Stressa, Italy, 2190-2195.