The College of

Arts and Sciences

 

One of our lab's and Center's thrusts involves encouraging autistic students towards STEM careers and to participate in STEM educational activities. We conduct outreach activities as well as are involved in a unique project funded by the state of Oklahoma through a grant from OCAST (Oklahoma Center for Science and Technology). The objective of this project is to study the potential of teaching science and engineering concepts to children with autism using Virtual Reality based Learning Environments (VLEs).

We are working with psychologists and autism therapists to design VLEs based on Human-Computer Interaction (HCI) principles. We are also exploring the potential of adopting Applied Behavioral Analysis (ABA) techniques (such as introducing positive reinforcers) to help autistic students learn.

Medical and educational specialists agree that children with Autism and teens learn best using Applied Behavior Analysis (ABA) which follows a single subject experimental analysis of behavior design. The long term interest is to throw more light and assess patterns and technology preferences that influence learning behavior. Research has consistently shown that positive reinforcement, as opposed to negative reinforces (or penalties) punishment, is the most effective method to teach an individual a desired behavior. Positive reinforcement provides opportunities for reward when participants perform desired learning behaviors. For example, when teaching a child to match the correct computer action to a computer-based request, every correct application earns the student reinforcement. Some children may need an extrinsic reinforcer (such as a token or playing a computer game the student likes); others may gain reinforcement by a computer display of fireworks or some other visual experience they receive after successfully completing a learning task. Other children may be intrinsically motivated solely by the computer learning experience itself and feel reinforced by successful completion of the computer learning task itself.

We have been exploring the role of Affordance, Visual density and Cognitive load on the design of 3D content and scene layout for learning. We have proposed the notion of dynamic affordance.

Affordance is ‘what the environment offers to the individual’. Other researchers including J.J. Gibson have described them as action possibilities that can be perceivable readily by an actor in an environment or as the properties of the world which are defined with respect to how people interact with them. We propose dynamic affordance (DA) as a function of comprehension of the data/information in a 3D scene by a user moving along a specific path P (within that target 3D environment) over a fixed period of time (T). DA seeks to throw light on the comprehension and understanding of a target 3D VR environment from various positions and perspectives as a user navigates or traverses along certain paths within that environment; this includes understanding relationships of objects of interest (OOI) in that scene; we contend that such understanding of relationships may be key to comprehension of the learning content. Such OOI can be entities on the lunar surface, the lander, various tools to be used in collecting science samples, menu buttons, vital sign monitors (of astronauts), training avatars, etc.; such an understanding of target simulated scenes will vary based on various elements and factors such as density of objects in a given scene (visual density), audio interruptions/distractions (from heart monitors, etc.), presence of cues, introduction of avatars, etc. (which can be included or removed, varied or modified).

For autistic students, the amount of information presented has to be carefully designed as it may

lead to information overload. Each autistic student may have different levels of 3D data/information they can process and comprehend. In that context, we propose another new term referred to as visual density of a scene.

The visual density in a 3D scene is a measure of the number of objects/ cubic unit. A user’s understanding can be affected by the visual density and appearance/characteristics of the OOIs (color/contrast/lighting and textures). We continue to explore the role of visual density in impacting autistic student learning.

Some of our initial results are outlined below.

During all learning interactions, positive learning behavior was emphasized through congratulatory messages and rewarded through positive reinforcers. For these interactions, the participants were allowed to select their preferred learning reinforcement from a menu of reinforcers (the successful completion of a learning activity resulted in their obtaining this reinforcement or reward). Examples included tokens that when collected earned the student a choice of activities ((such as giving them an opportunity to play a computer game, try to assemble a robot, or create computer based art). 

 

Assessment of affordance

TEST 1: 2 groups of 10 students (5 middle and 5 high school students) participated. One group interacted with scenes with lower visual density; other with scenes of high visual density

Affordance was measured based on understanding of scene tasks and target concepts learned through interactions.

TEST 2: We introduce interruptions and disturbances to study impact on student learning; ambulance siren, someone talking in the back of the room (in scene); 2 groups of three middle school students were involved: one group interacted without interruptions and distractions; the second group interacted with interruption and distractions

Assessment focused on how such interruptions and distractions affected understanding and grasp of the target learning concepts for autistic students.

Null Hypothesis: Interruptions and distractions does not influence Affordance and Learning. Based on t-test, the null hypothesis was rejected.

Finding #1: students who interacted with low visual density environments demonstrated high affordance, measured by understanding of target concepts compared to those who did not receive any reinforcers

Finding #2: students who interacted with VLEs without disturbances scored higher on the post tests of knowledge regarding assembly and path planning

An Applied Behavioral Analysis (ABA) based approach was adopted to support and encourage learning interactions. For the majority of the participants, interacting with a 3D computer-based learning environment served as positive reinforcement itself. Three of the high school participants did not want additional reinforcers and were able to complete their learning interactions without such additional reinforcers (in essence, the 3D based graphics environments acted as the positive reinforce itself). Two of the high school participants elected (as a reinforcer) to create computer-based art twice during the learning interactions (other reinforcer options included completing a VR based assembly of a robot, playing VR games, surfing the web using a tablet). Three of the middle school participants chose to play VR based computer games (wearing headset and controller); of these three, Two middle school participants played one computer game before coming back to complete his last set of learning interactions successfully, One middle school participant was able to complete the learning interactions after playing two different computer games. 

 

Impact of Positive Reinforcers

2 groups of 6 students were involved in study related to positive reinforcers.

One group did not receive any positive reinforcers; the other group received positive reinforcers during their interactions with the VLEs. 

 Results from the study showcasing the role of positive reinforcers

 

The result of our initial study is shown above. The improvements in the group which received positive reinforcers is higher compared to the group who did not receive any reinforcer. The results underscore the importance of using positive reinforcers in teaching and training using science concepts using VR based environments. A t-test was performed which showed a significant difference in the mean scores of the two groups. The alternate hypothesis which stated that positive reinforcers influence learning was accepted.

 

Our work is continuing. We are also comparing the effectiveness of learning through Virtual Reality versus learning through Mixed Reality environments. We have completed some preliminary work involving learning of density concepts for middle school students.  In the figure below, a View of a 3D MR environment to help study density concepts is shown (the view on the left is what the user sees – both the physical environment and the virtual). The image on the right shows a student interacting with such a  MR density environment.

Designing Virtual Learning Environments (VLEs) to teach STEM concepts to autistic K-12 students,

 

Some of our findings have been published:

1. Cecil, J., Mary Sweet-Darter, Aaron Cecil-Xavier, Avinash Gupta, Role of Affordance, Visual Density and Other HCI Criteria in Designing Virtual Learning Environments to Support STEM Learning for Autistic Students, Frontiers in Education (FIE) Conference, Oct 13-16, 2021, Nebraska, USA
2. Cecil, J., Sweet-Darter, M., Gupta, A., Design and Assessment of Virtual Learning Environments to Support STEM Learning for Autistic Students, In Frontiers in Education (FIE) Conference, Oct 21-24, 2020, Uppsala, Sweden
3. Cecil, J., Sweet-Darter, M., Cecil-Xavier, A., Exploring the use of Virtual Learning Environments to support science learning in autistic students, Proceedings of the 2017 Frontiers in Engineering FIE / IEEE conference, Indianapolis, Oct 18 – 20, 2017.

 

We are inviting parents of autistic students (grade 1 to 12) to contact us regarding partcipating in this innovative project. Students from local Stillwater schools as well as other schools in Oklahoma are encouraged to participate.
Students can be in any of the following grades:
Elementary (grades 1 – 5), Middle (grades 6 – 9) and High School (grades 10 – 12).

KOCO TV (Oklahoma City, ABC news affiliate)  carried a news report about this project recently. Here is the link.
Dr. J. Cecil (PI) is working closely with Dr. Mary Sweet-Darter (educational psychologist) in this project.
Examples of these VLEs are shown below.
Parents of children with autism can email Dr. Cecil to learn more about this project (j.cecil@okstate.edu)

In this project, we are exploring the design of different types of Virtual Reality (VR) based environments to teach science and engineering to autistic children. These environments are called Virtual Learning Environments (VLEs). If you are a parent or student, please consider participating in this project.
In these VLEs, students will learn science/engineering concepts in an exciting VR environment using game controllers and haptic devices (the latter will allow them to touch and feel objects inside a learning environment).  There will be 3 categories of learning environments (called non immersive, haptic based and fully immersive environments), each dealing with a specific set of science topics. Interaction sessions with these modules typically can range from 30 minutes to 1 hour.  These interaction schedules will be on different days spread over several months. Depending on the grade level of the autistic student, they will be able to interact and learn a variety of science topics; examples of these science modules can range from learning about our solar system to robotics (see images below).

Immersive Virtual Learning Environment
In a fully immersive VLE, students can wear a 3D headset and interact using gaming type controllers. In the image below, a student is interacting using such an immersive environment. The controllers and headset can be seen on the system below, which has been implemented using the HTC Vive™ platform.

Interacting with a VLE using an immersive platform (Vive™)

 

Haptic based Virtual Learning Environment
In the image below, a student interacts with a VR based learning environment using a Haptic device (the student is holding this haptic device with his left hand). This device allows a user to touch and feel objects virtually inside a learning environment. The VLE below introduces students to concepts in density.

Our lab and Center also is equipped with a VR Powerwall, shown below. Here, a user is interacting with a semi-immersive VLE to explore our solar system virtually.

Interacting with a virtual solar system

 

SPECIAL REQUEST TO PARENTS
Parents of autistic students (grades 1 – 12) who are in the Stillwater and surrounding areas (Perkins, Perry, etc) are invited to contact Dr. Cecil to know more about this project and how it can benefit your child (j.cecil@okstate.edu)

Your child’s participation and other data will be kept completely confidential during the study. Any research findings will only be aggregated and published in conference and journal papers without any names or identifiable data pertaining to your child. 

OUR PAST WORK
We have been conducting pilot studies for the past few years in this area of special needs. Our initial study involved 10 autistic children (kindergarden to grade 6). They explored 2D/3D geometrical shapes using both an immersive environment and a haptic interfaced environment (fig 1); The Virtual Learning Environments (VLEs) were related to a virtual solar system (using an immersive environment) and properties of matter (using a haptic interface). While the children were able to learn basic concepts, their levels of engagement (and preferences) when using the VLEs was different: some preferred the 3D stereo based environment while the other enjoyed interacting with the haptic interfaced VLE where they could ‘feel’ objects and their responses.  Figure 1 (left) shows a student who preferred interacting with a haptic device while Figure 1 (right) shows a student who enjoyed learning using the 3D graphical environments. As our study continues, we plan to investigate the impact of the types and levels of immersion on learning and engagement.

The Next Generation of Internet Technologies holds great potential in making the learning activities more accessible to autistic children. A quiet revolution involving the Next Internet is underway, which can phenomenally impact education and engineering worldwide. We are one of the pioneering groups currently exploring these Future Internet technologies for engineering and education (we are actively involved in the GENI and US Ignite initiatives). Software Defined Networking and Cloud computing are some of the emerging technologies which are poised to impact Cyber Learning and Engineering practices. As part of the Global Cities Teams initiatives, we are expanding use of Next Internet technologies and principles to link computers from schools and homes through a variety of interfaces including haptic devices and smart phones. In spring 2013, as part of US Ignite and GENI demonstrations, we successfully highlighted the use of software defined networking (SDN) technologies in which an autistic student at one location was able to interact with a virtual learning environment at another location using a haptic interface; further, a milestone was achieved when a student and teacher were able to interact from 2 different locations using their respective haptic devices (the teacher and student were able to take control and assume the role of leader during their interactions) (fig 1). A small part of this GENI demonstration along with an interview with an autistic student and his parent can be found here
https://www.youtube.com/watch?v=BAfd2ax6tk4
Note that this youtube video shows only the technology elements of this demonstration where one student at one location interacts with a Virtual Learning Environment (VLE) at another location through a haptic (touch) interface. in this example, a student is interacting with a duck model in a virtual tank of water; we were attempting to find out how the Next Internet framework would perform learning activities involving the student (who was using mouse based interactions and a haptic device to move and push the duck).  Using such an environment, a teacher can expose students to concepts related to buoyancy, floatation and density of materials.

We are also excited about the potential of next generation Internet technologies which holds the promise of ushering a new era in cyber learning for all students. We have been using Global Environment for Network Innovation (GENI) related networking concepts to support learning activities not only for helping children with autism but also to support STEM education for elementary, middle and high school students. We have posted one of these interactions involving GENI networks at http://youtu.be/EIlNqpCAIu4.

Dr. Cecil has been mentoring and encouraging students with autism to explore STEM learning using Virtual Reality mediums. An Oklahoma STEM Alliance has been created to support autism students towards STEM programs and degree programs. Some of our past work has been carried by TV and newspapers in Oklahoma and New Mexico. (Link)

1.