Even as our world begins to open back up, the use of Virtual Exchange (VE) in universities across the globe has become a prevalent model to allow for the intercultural exchange of people, ideas, and programs without the participants needing to travel across borders. Virtual Exchange allows people to learn from other cultures without actually traveling. While having a virtual model of exchanging cultural ideas and information sounds promising compared to shutting down student exchange programs, assessing the effectiveness of virtual exchanges is essential to ensure that VE programs actually provide students with an intercultural experience without leaving their home cities.
How can programs be sure that their virtual exchange efforts are achieving their goals?
A collaborative study with three countries found that one of the biggest challenges facing VE programs is fostering reflection on perceived cultural differences between learners before setting up any VE. They suggest introducing students to learn about different cultures, discard stereotypes, and foster communication and an openness to respectful dialogue. It is thought that implementing these cultural dimensions helps grow students’ intercultural understanding and abilities. But without actual quantification or measurement of these increases in understanding, how can programs be sure that their VE efforts are achieving their goals?
In 2022, researchers from Georgia State University published data to address this exact question. While it was thought that VE programs provide high-impact programming to increase students’ intercultural understanding, there was no established data on this. The specific goals and impacts of VE the research team sought to address were:
- the development of intercultural effectiveness in students
- whether there was a difference in intercultural effectiveness for minority and non-minority participants
- the effect of a program’s duration on the development of intercultural effectiveness skills
The study utilized data from 131 undergraduate and graduate students participating in VE programs around the world across various disciplines and program foci. The research team measured the students’ intercultural understanding using The Kozai Group’s Intercultural Effectiveness Scale (IES). Due to the known difficulties in measuring intercultural competence, having a quantifiable tool to assess students’ gains in their knowledge, skills, attitudes, and awareness is an essential part of being able to truly assess the effectiveness of VE programming.
Measuring Intercultural Competence
The researchers chose to use the IES largely due to its use and importance in measuring intercultural competence. An additional benefit of the IES is that after completing the IES, test takers receive a 24-page self-report that includes detailed results, common profiles with behavioral and attitudinal descriptors, guides to assist students with determining their profiles, and a development plan for self-improvement. Using Kozai’s IES, the researchers in the Georgia State study were able to compare students’ scores on the IES before and after engaging in VE activities.
Without access to a tool such as the IES, the researchers would not have been able to quantitatively determine whether VE programs are indeed effective at meeting gain of function and understanding in intercultural fluency, fluidity, and cultural competency. By using the IES, the researchers were able to determine that each student had higher post-test scores for the overall test as well as for each subtest. Students were shown to have a starting mean score of 3.60 before participating in VE and a mean score of 3.71 afterward. Additionally, they found that there was a statistically significant increase resulting from participation in VE programs in continuous learning, which reflects the degree to which you seek to understand and learn about activities, behavior, and events that occur around you, and interpersonal engagement, the factor that assesses one’s interest in other cultures and importance of developing relationships with people from other cultures (see chart below).
Table 3. Descriptive statistics and paired t-test results for IES scores (N=131)
The IES showed the researchers, too, that while these aspects of intercultural competency increased, VE did not significantly increase the hardiness dimension of the students. This dimension of the IES measures the ability to manage intercultural situations and view new ideas and behaviors with an open mind and without judgment. Since VE is less experiential than learning abroad, the students’ scores in this dimension inform VE programs to carefully review their curricula to ensure that it will simulate intercultural situations to stimulate students’ hardiness dimension.
| IES Dimension | Pre-test | Pre-test | Post-test | Post-test |
| M | SD | M | SD | |
| Continuous Learning | 4.09 | 0.37 | 4.23 | 0.42 |
| Self-Awareness | 3.93 | 0.42 | 4.13 | 0.45 |
| Exploration | 4.26 | 0.39 | 4.34 | 0.39 |
| Interpersonal Engagement | 3.53 | 0.50 | 3.70 | 0.55 |
| World Orientation | 3.01 | 0.73 | 3.30 | 0.88 |
| Relationship Development | 4.06 | 0.50 | 4.10 | 0.53 |
| Hardiness | 3.17 | 0.51 | 3.21 | 0.57 |
| Positive Regard | 3.22 | 0.55 | 3.26 | 0.65 |
| Emotional Resilience | 3.12 | 0.71 | 3.17 | 0.7 |
| Overall IES Score | 3.60 | 0.34 | 3.71 | 0.37 |
Results using the Intercultural Effectiveness Scale (IES)
By using Kozai Group’s IES, the Georgia State researchers in this study were able to determine that students demonstrated an overall upward trend in intercultural competency after participating in VE. Validating the effectiveness of Virtual Exchange programs is one milestone in the utility of Kozai’s Intercultural Effectiveness Scale (IES) and its applications and effectiveness will continue to grow as our world continues to grow, change, and globalize.
The information gleaned from this study’s use of the IES has validated VE programming and shown how intercultural understanding through VE programming can become an even higher impact program in the future. The full research article can be read online and was published by UNICollaboration Journal of Virtual Exchange.
The IES provides multidimensional feedback for assessing intercultural effectiveness, competencies. The individualized information gained from the IES provides usable data to assess any program’s or individual’s strengths and weaknesses in intercultural understanding. Learn more about the IES and how it can impact your organization here.