Students Build 3D Chess Set for Virtual Reality Studies
Do people reason and learn in immersive virtual environments in different ways than in the real world?
If you could immerse yourself in a computer puzzle rather than viewing it on a flat screen, would you play any differently? Colby students are exploring this idea through a virtual reality (VR) chess environment, where players experience the game using a headset that transports them to play in a three-dimensional space.
The chess project is one path into the larger question of how people reason and learn in immersive virtual environments in different ways than in the real world. Clare Boothe Luce Assistant Professor of Computer Science Stacy Doore is collaborating on the research project with Assistant Professor of Math Education and Instructional Technology Justin Dimmel at the University of Maine.
The first step in exploring how people interact with objects such as chess pieces in VR is to build the board. Linn Cao ’24 and Izge Bayyurt ’22 built multiple virtual environments this summer and presented their work at Colby’s Undergraduate Summer Research Retreat. Cao developed three standard chess boards at different spatial scales (tabletop, room, and environment), while Bayyurt built a three-dimensional chess cube that could be rotated in space and explored from inside and outside of the cube. Players can interact with both of the virtual environments by wearing a VR headset and moving virtual chess pieces by using a hand-held controller.
Doore and Dimmel came up with the VR chess project to bring together their respective research interests. Doore conducts research in spatial cognition and non-visual navigation, and Dimmel is exploring different ways to represent mathematical concepts. Both are interested in using emerging technologies such as VR to create unique learning environments that do not exist in the real world.
At her Immersive Navigation Systems and Inclusive Technology Ethics Lab (INSITE), Doore explores how assistive technologies can expand information access, particularly for people with blindness and vision impairments. Assistive technologies such as screen readers and hearing aids are familiar to many of us, but assistive technologies development can also involve VR. For example, a navigation system using spatial audio cues to help blind users move through an unfamiliar indoor space could first be tested and refined in VR with sighted participants.
“I’m always looking for new ways to convey information that will engage people in different ways or provide new insights into the ways people think and reason about objects in space.”
Stacy Doore, Assistant Professor of Computer Science
“VR is very visual, but there are so many other types of multimodal interfaces that can be developed and tested in a virtual environment, such as haptics and natural language cues,” Doore said. “I’m always looking for new ways to convey information that will engage people in different ways or provide new insights into the ways people think and reason about objects in space.”
For Dimmel, who runs the Immersive Mathematics in Rendered Environments (IMRE) Lab at the University of Maine, VR could be a way to convey complex mathematical ideas. Geometric concepts, for example, can be hard to grasp when compressed into two-dimensional formats such as diagrams or equations. VR could render the same concepts as objects within a 3D space, like a sculpture that could be inspected from all angles.
“It’s often nice to have a constellation of representations for students,” Dimmel said. “The barriers to entering and engaging with higher-level mathematics could potentially be lowered when we have students interacting with these spatial representations.”
“I never imagined that I could learn a new language and lots of new tools, and then basically create a set of complex environments using all of them in just six weeks. I’m very proud of it.”
Linn Cao ’24
VR is also a good training ground for both beginning and advanced computer science students, Doore said. With oversight from both professors, Cao took on the task of rendering standard chess boards at different scales, while Bayyurt built on his advanced programming experience to create the more complex three-dimensional chess cube.
Cao built on existing code libraries of chess objects, using them to create her own environment and writing scripts for moving chess pieces. Her virtual world looks like three rooms, each with a different size chessboard. The rooms with the larger boards have stairs that allow a user to investigate an overhead view of the board while considering their next move.
Cao said the project helped her learn the programming language C# and build experience in VR development, which will boost her applications to software engineering internships.
“I never imagined that I could learn a new language and lots of new tools, and then basically create a set of complex environments using all of them in just six weeks,” Cao said. “I’m very proud of it.”
And the project was a natural fit for Bayyurt, who not only was interested in VR but has been playing chess since he was four years old. In his three-dimensional chess cube, the pieces look like they are floating in space. A player can adjust the free-form playing field’s size, and eventually, the environment will allow a player to freely rotate the cube to see the game from all angles.
Building the chess environment was a good lead-in to his current study abroad program in Copenhagen, Denmark, where he is focused on video game design and interning at a company that builds meeting-room software in VR.
Both students were struck by the difference between coding their environments and actually putting on a VR headset and trying it out. “I was mesmerized by how real the objects were around me,” Bayyurt said. “It gives you a different perspective, and I think it’s hard to appreciate that without actually trying it.”
Now that the virtual chess environments are designed and developed, the next step is to design a study protocol and run experiments with it. Cao and other INSITE lab students will collect eye tracking and movement data to better understand how people use vision to consider strategies, move in the same space presented at different scales, and manipulate objects outside of real-world boundaries. Their first study, which will be conducted over the next few months, will ask participants to think aloud as they perform navigation tasks and perform a sequence of chess moves at different spatial scales. The goal is to better understand how the virtual environments affect participants’ spatial reasoning strategies and their ability to reproduce the task sequence.
This study is just one part of a larger research agenda for Doore and Dimmel. Both professors acknowledge VR is a compelling tool, but they note that the science demonstrating its value in educational settings is still in the early stages. However, VR is often the “hook” that gets students interested in assistive technology research, Doore said, and she has no shortage of interested students. Funding for the student development work was provided from the Colby Computer Science Department, the Maine Space Grant Consortium, the Spencer Foundation, and the Office of the Vice President of Research at the University of Maine. “Ideally, anything we develop using VR to enhance learning or teaching should be a value-added experience, as opposed to a replacement experience,” Doore said. “That’s really the main thing. Justin and I are most interested in what we can do to enhance learning only in a VR environment that we can’t do in real-world learning environments.”
