A Memorial University-grown, cutting-edge interactive learning tool is bridging the gap between tactile play and digital learning.
The augmented reality (AR) sandbox, developed by Memorial University’s AI and Games Lab in the Department of Computer Science, is also proving to be a valuable asset for multiple academic disciplines and an exciting outreach tool for young learners.
How it works
The AR sandbox combines a physical box of sand with an Intel RealSense depth-sensing camera and an overhead projector.
As users manipulate the sand, the camera detects the changes and updates in real time, displaying color-coded contour lines and other environmental simulations back the sand.
Memorial’s AR sandbox has been designed from the ground up, using modern computer graphics techniques that update 90 times per second, making interactions seamless and immediate.
Most existing AR sandboxes can take up to a second to respond to user inputs, breaking user immersion and lowering the quality of the experience, says one of the sandbox developers.
Dr. David Churchill, a professor in the Department of Computer Science, says the ability to shape landscapes and see immediate topographical and environmental changes makes the experience “more real and fun” and helps keep students engaged and able to grasp complex concepts intuitively.
“This more efficient implementation also enables research and teaching in areas where real-time feedback is required, such as video games and robotics,” he said.
Used across disciplines
This innovative tool has found applications beyond computer science, already benefiting students in the departments of Earth Sciences, Geography and Environmental Science.
A recent earth sciences lab used the sandbox to teach topographical mapping, allowing students to gain a hands-on understanding of contour lines in a dynamic environment.
Benjamin Stanley, a master of science (computer science) is using the sandbox to interface with video games such as Minecraft.
“When players move the sand in the sandbox, the Minecraft world reshapes in real time to match,” said Mr. Stanley. “This opens a new frontier in gaming and interacting with virtual worlds. I am excited to use AR to study and develop new ways of playing games.”
Dr. Joseph Fitzgerald, an assistant professor in the Department of Physics and Physical Oceanography, is co-supervising student Ethan Denny, who are using the AR sandbox in their climate and ocean science research.
Dr. Fitzgerald says when teaching and learning about the physics of meteorology and climate, it’s “crucial” to be able to visualize dynamic phenomena such as the flow of air over hills, mountains and even glaciers.
“AR sandboxes provide a totally new approach that we think will benefit students, here at Memorial and beyond, as the technology is incorporated into the classroom,” he said.
Future of STEM learning
Beyond academia, the AR sandbox plays a significant role in the Faculty of Science’s community outreach work.
It was most recently featured at Whale of a Day 2024, an event aimed at engaging young learners with science.
Children spent hours sculpting mountains, valleys and rivers while learning about geography, weather patterns and water flow in an interactive, gamified way.
“We saw kids completely immersed in playing and experimenting with the sand, creating structures that interact with the visualizations,” said Dr. Churchill, who assisted with the demonstration. “Parents actually had to pull them away because they were so engaged.”
He believes the AR sandbox has the potential to make STEM education more accessible, particularly for children who may struggle with traditional learning tools.
“Let’s face it, we live in the TikTok age where it’s hard to get students to pay attention to anything for more than a few seconds at a time. By blending physical and digital interactions, we can make science more engaging and accessible. This kind of hands-on learning sticks with students far longer than just reading about these concepts in a textbook.”
A lasting impact
With increasing interest from students, researchers and the broader community, the AR sandbox is proving to be more than just a technological novelty: it’s a powerful educational tool that fosters curiosity and deeper scientific understanding.
Dr. Churchill invites others at the university to develop their sandbox project.
“All of the AR sandbox software is written by the AI and Games Lab and is completely open source for anyone to modify and create their own experiences. We welcome all collaboration from researchers, game developers and educators who want to come get their hands dirty — literally.”
