CSCLearning? : participation, learning, activities and knowledge construction in computer-supported collaborative learning in higher education

<strong>Background of the research</strong>Recent developments in Information and Communication Technology (ICT) offer many opportunities to reorganise education according to constructivist principles. In contrast to more traditional education, education organised by constructivist principles is not teacher-centred, but student-centred. Students can influence their education and are not only consumers as in traditional education. Students work in collaboration to solve tasks and importance is attached to their own ideas; reproducing facts is becoming less important. Students are expected to be active and independent. They have to search for information by themselves and are expected to process this information critically. The accent is not on testing reproduction of facts but much importance is attached to creating own ideas and theories. <em>Chapter 2</em> of this PHdissertation outlines the theoretical framework which was based on constructivism and which determined the design and conduction of the research.The assumption is that supporting education by ICT can increase the quality of learning. This PHdissertation studies one specific ICT-application, namely Computer-Supported Collaborative Learning (CSCL). In CSCL, students learn collaborative by using a CSCL-system. A CSCL-system can be considered to be a discussion forum in which students can contribute messages and can read each other's messages. A computer network connects students and therefore, students can read all messages and react to all the messages contributed to the discussion forum. Synchronous as well as asynchronous systems are available. In synchronous systems, students can work from different places in real time. In asynchronous systems, work is independent of time and place. In the research described in this PHdissertation, only asynchronous systems are used. Students could work in the system at any moment.The central idea of CSCL is that it supports shared knowledge building by the learners (Scardamalia & Bereiter, 1994). The principles of shared knowledge building and CSCL are consistent with a constructivist view of learning. From a constructivist point of view, learning is a dynamic process of knowledge construction. In this PHdissertation, collaborative learning is described as a learning situation in which participating learners exchange ideas, experiences and information to negotiate about knowledge in order to construct personal knowledge that serves as a basis for common understanding and a collective solution to a problem. Research shows that collaborative learning can be useful to reach intellectual goals such as critical thinking or debating. People learn by interaction (Erkens, 1997; Gokhale, 1998; Kanselaar & Van der Linden, 1984; Lethinen, Hakkarainen, Lipponen, Rahikainen & Muukkonen, 2001; Newman, Johnson, Webb & Cochrane, 1999). Characteristic to collaboration is the interaction between people and people learn through interaction with each other (Biggs & Collis, 1982). Discussion is important because we will only 'give words to our thoughts' when we use these words to communicate with others, and this in turn may be related to our ability to clarify and remember ideas (Johnston, 1997); understanding is achieved through interaction (Veerman, 2000). Besides, CSCL seems to be an effective tool because students have to write down their ideas. Writing can be seen as the most important tool of thinking, and it has a crucial significance in explication and articulation of one's conceptions (Bereiter & Scardamalia, 1987; Rijlaarsdam & Couzijn, 2000; Tynjälä, 1999).<em></em><strong>Research questions</strong>Literature shows that there is a reasonable amount of published experiments indicating positive learning effects when CSCL-systems have been used in education (De Laat & De Jong, 2001; Koschmann, Feltovich, Myers & Barrows, 1997; Lethinen <em>et al</em> ., 2001; Lipponen, 1999; Salovaara, 1999; Tynjäla, 1999). Despite developments in research and educational practice, much is still unclear about students' learning processes in CSCL. It is unknown how students use a CSCL-system, which learning activities they use and how CSCL supports students' learning. The aim of this research is to gain insight into students' learning processes in CSCL, focused on both the amount and the quality of knowledge construction. The underlying assumption is that understanding students' learning processes will be helpful to use CSCL effectively in education. Inspired by this research problem the following main research questions were addressed ( <em>chapter 1</em> ):1) How can students' learning processes in an asynchronous CSCL-system be characterised in terms of participation and interaction?2) How can students' learning processes in an asynchronous CSCL-system be characterised in terms of cognitive, affective and metacognitive learning activities?3) Do students construct knowledge and what is the quality of that knowledge constructed by students in an asynchronous CSCL-system?4) What are the effects of moderating a CSCL-discussion on students' learning?<strong>Method</strong>To find an answer to our research questions, first a review study ( <em>chapter 3</em> ) was carried out to find out if a method was available to analyse students' activities in a CSCL-system on participation, interaction, types of learning activities and amount and quality of knowledge construction. The reviewed methods supplied many ideas we could use to develop a new method and helped us to clarify our view on analysing CSCL-data. However, studying a number of methods did not result in finding a workable, ready-made method to answer our research questions. Therefore, a new method was developed on the basis of the theoretical framework outlined in chapter 2, on the ideas supplied by the reviewed methods described in chapter 3, and on our experiences with CSCL in pilot projects. <em>Chapter 4</em> describes the developed method used to analyse students' learning processes in this PHdissertation. The method consist of three steps (see Figure I): 186<CENTER><img src="/wda/abstracts/i31871.gif" WIDTH="578" HEIGHT="233"/></CENTER>Figure I: Three steps of the method used to analyse students' contributions in a CSCL-system.<em></em> The method consists of three steps: (1) Analysing students' participation and interaction, (2) analysing cognitive, affective, and metacognitive learning activities, and (3) assessing the amount and quality of knowledge constructed by students and expressed in written contributions. Students' participation was operationalised as the number of written notes (new notes or build-on notes) and number of different read notes. To indicate interaction, density was calculated twice, based on read notes as well as on linked notes. Density describes the general level of linkage among the students in a discourse. In other words, density refers to the extent of interaction between students.The second step of the method concerns the use of learning activities. The classification of learning activities of Vermunt (1992) was used as a frame to create a coding scheme divided into (1) Cognitive, (2) affective, and (3) metacognitive learning activities. Next, these main categories were divided into several subcategories. The main category 'cognitive learning activities' consists of three subcategories: (a) Debating, (b) using external information and experiences, and (c) linking or repeating internal information. <em>Debating</em> refers to the process of negotiation, critical thinking, asking questions and discussing subjects with other participants in the database. <em>Using external information and experiences</em> was inserted into the scheme because in an asynchronous CSCL-system students have time to search for information to support their ideas with explanations and to elaborate their questions. Information can be used to evaluate contributions thoroughly. Types of information contributed to the CSCL-system are for example articles found on the Internet, notes made in a lecture, a summary of a book chapter, results of running a specific tool, or a summary of another discussion. A third subcategory is <em>Linking or repeating internal information</em> . Internal information concerns information found in the discussion view students are working in. Referring to and linking notes were considered to be important because of increasing coherence in the database. It was assumed that more coherence between notes means more interactions between students. By 'affective learning activities', students' feelings expressed in their notes while working in the learning environment are meant. An affective category was included in the coding scheme to provide information about the kinds of feelings and was expected to be useful in interpreting the nature of the interactions between students. In this coding scheme affective learning activities are not related to content of subject matter, they are non-task-related. The category 'metacognitive learning activities' consists of three subcategories: (a) Planning, (b) keeping clarity, and (c) monitoring. <em>Planning</em> refers to practical issues such as making appointments, subdividing parts of the task, appointing a group member as chairperson or to theoretical issues such as choosing a definition after discussing a concept or deciding to run a specific tool. Characteristic of these content-related approaches is their effect on the process of the task performance. The subcategory <em>Keeping Clarity</em> refers to messages written in order to keep the structure and the content of the notes clear. The last subcategory of metacognitive learning activities is called <em>Monitoring</em> . While conducting the task, students will keep watching the learning process. Next, a number of codes are distinguished within all the subcategories.Because the third main research question could not be answered by means of the first coding scheme, it was necessary to add a step to the process of analysis, step three. Therefore, knowledge construction was first operationalised as adding, elaborating and evaluating ideas, summarising or evaluating external information and linking different facts and ideas. In line with this definition, six codes from the first coding scheme were selected to indicate the amount of constructed knowledge. To measure the quality, a second coding scheme was developed on the basis of the Structure of the Observed Learning Outcome (SOLO) taxonomy of Biggs and Collis (1982). This scheme consists of four levels of quality, increasing from level D to level A. Both coding schemes were validated by calculating Cohen's Kappa to determine the inter-rater reliability of the scheme. The first coding scheme (step 2) was applied to units of meaning. In other words, several types of learning activities could be decoded within one message. The second coding scheme (step 3) was applied to complete messages; a contribution was assessed in its entirety. The coding schemes were developed to understand students' learning processes. Standards were formulated to judge students' learning processes and to compare results of different studies.<strong>Six studies</strong>From 1998 until 2001, CSCL was implemented in six university courses. Three studies were conducted at the Wageningen University, two studies at the University of Nijmegen and in one study we made use of data collected in a course organised at the University of Toronto. All data were analysed by means of the method developed. Besides the similarity of a university context, the six studies were comparable in some other aspects. All studies took place as part of a real course in which students had to work collaboratively on complex tasks by the use of a CSCL-system. All studies were planned in the final phase of the educational programmes. Another similarity was the CSCL-system used, namely Knowledge Forum. In none of the studies, students were charged with rules concerning the use of Knowledge Forum. They were expected to log in regularly, but were not obliged to read all notes or to write a certain numbers of notes. However, there were differences between the studies as well.Sometimes, the course was required; sometimes it concerned an optional course. The period of the use of CSCL varied substantially (2 to 17 weeks), just as the number of hours students were expected to spend in the CSCL-system weekly (2 to 20 hours). Besides, courses differed in their testing of learning. Sometimes, the participation in Knowledge Forum was assessed, but sometimes only a final test determined the course's grade. The discussions analysed in studies 1, 2, 3, and 4 ( <em>chapter 5)</em> differ from the discussions analysed in studies 5 and 6 <em>(chapter 6</em> ) in being not moderated; students in studies 1-4 were self-regulated. In studies 5 and 6, a teacher was active in writing contributions focused on stimulating collaboration between students or triggering critical thinking. Beforehand, teachers were instructed on how to moderate half of the students in their courses. Guidelines were discussed and notes were available to try out the guidelines.Each study of chapter 5 first answers the main research questions. Besides these overall research questions, three more specific questions were formulated related to specific characteristics of the different educational tasks and settings in studies 1, 2 and 3. The sub-question in study 1 concerns group size, the sub-question in study 2 concerns having a specific discussion role and the sub-question in study 3 concerns students' learning style. Chapter 6 answers the fourth main research question. Comparable to the studies 1-4, first students' learning processes were analysed. Additionally, moderators' activities were analysed to survey how the moderation was carried out. Moderators' activities were analysed on types of actions, percentage of read notes, number of written notes, the percentage of students to whom the moderator directed notes, response time and relation between number of students' and moderator's notes contributed per week.<em></em><strong>Results</strong>Table A shows the main results of the six studies: mean participation per student, density of interaction, mean number of used learning activities per student and knowledge construction on average per group.Table A: Mean participation per student, density of interaction, mean number of used learning activities per student and knowledge construction on average per group in the six studies<TABLE BORDER="0" cellpadding="5" WIDTH="100%"><TR><TD VALIGN="TOP"></TD><TH VALIGN="TOP" COLSPAN=4>Chapter 5</TH><TH VALIGN="TOP" COLSPAN=3>Chapter 6</TH></TR><TR><TD VALIGN="TOP"></TD><TD VALIGN="TOP">Study 1(N=15)</TD><TD VALIGN="TOP">Study 2(N=13)</TD><TD VALIGN="TOP">Study 3(N=24)</TD><TD VALIGN="TOP" COLSPAN=2>Study 4(N=7)</TD><TD VALIGN="TOP">Study 5(N=28)</TD><TD VALIGN="TOP">Study 6(N=9)</TD></TR><TR><TD>Participation (mean per student)</TD><TD></TD><TD></TD><TD></TD><TD></TD><TD></TD><TD></TD></TR><TR><TD VALIGN="TOP"># Written contributions# Read contributions</TD><TD VALIGN="TOP">19.1376.13</TD><TD VALIGN="TOP">11.3851.30</TD><TD VALIGN="TOP">10.3158.53</TD><TD VALIGN="TOP" COLSPAN=2>24148.14</TD><TD VALIGN="TOP">16.7665.44</TD><TD VALIGN="TOP">46.32192.67</TD></TR><TR><TD>Interaction (density) based on:</TD><TD></TD><TD></TD><TD></TD><TD></TD><TD></TD><TD></TD></TR><TR><TD VALIGN="TOP">Read contributionsWritten contributions</TD><TD VALIGN="TOP">1.0.60</TD><TD VALIGN="TOP">1.0.48</TD><TD VALIGN="TOP">.61.13</TD><TD VALIGN="TOP" COLSPAN=2>.95.67</TD><TD VALIGN="TOP">**</TD><TD VALIGN="TOP">**</TD></TR><TR><TD VALIGN="TOP">Learning activities (mean per student)</TD><TD VALIGN="TOP"></TD><TD VALIGN="TOP"></TD><TD VALIGN="TOP" COLSPAN=2></TD><TD VALIGN="TOP"></TD><TD VALIGN="TOP"></TD><TD VALIGN="TOP"></TD></TR><TR><TD VALIGN="TOP" height="15">Cognitive</TD><TD VALIGN="TOP" height="15">9.07</TD><TD VALIGN="TOP" height="15">22.89</TD><TD VALIGN="TOP" height="15">20.50</TD><TD VALIGN="TOP" COLSPAN=2 height="15">62.86</TD><TD VALIGN="TOP" height="15">7.33</TD><TD VALIGN="TOP" height="15">48.11</TD></TR><TR><TD VALIGN="TOP" height="15">Affective</TD><TD VALIGN="TOP" height="15">7.73</TD><TD VALIGN="TOP" height="15">1.70</TD><TD VALIGN="TOP" height="15">2.33</TD><TD VALIGN="TOP" COLSPAN=2 height="15">3.43</TD><TD VALIGN="TOP" height="15">6.75</TD><TD VALIGN="TOP" height="15">27.78</TD></TR><TR><TD VALIGN="TOP" height="15">Metacognitive</TD><TD VALIGN="TOP" height="15">13.47</TD><TD VALIGN="TOP" height="15">3.78</TD><TD VALIGN="TOP" height="15">7.38</TD><TD VALIGN="TOP" COLSPAN=2 height="15">7.14</TD><TD VALIGN="TOP" height="15">10.29</TD><TD VALIGN="TOP" height="15">11.56</TD></TR><TR><TD>Knowledge construction (mean per group)</TD><TD></TD><TD></TD><TD></TD><TD></TD><TD></TD><TD></TD></TR><TR><TD VALIGN="TOP" height="87">AmountQuality</TD><TD VALIGN="TOP" height="87">LittleLow</TD><TD VALIGN="TOP" height="87">ReasonableHigh</TD><TD VALIGN="TOP" height="87">LittleHigh</TD><TD VALIGN="TOP" COLSPAN=2 height="87">LittleReasonable</TD><TD VALIGN="TOP" height="87">LittleReasonable</TD><TD VALIGN="TOP" height="87">ReasonableReasonable</TD></TR></TABLE>* In this study interaction was not calculated because a teacher intervened in one half of the group, and besides, interaction was not part of the research question in this study. It is striking that the results of the different studies vary enormously. The only similarity between the studies is that in each study, students read many more notes (passive participation) than they wrote (active participation). Concerning the use of cognitive, affective and metacognitive learning activities we see large differences. Except for study 6, in each study students used affective learning activities least. Next, students in the studies 2, 3, 4 and 6 used more cognitive than metacognitive learning activities. In the studies 1 and 5, it was just the other way around. In four of the six studies, students constructed little knowledge, in the remainder of the studies, a reasonable amount of knowledge was constructed. The quality of the knowledge constructed varied between low and high, but in most of the studies, quality of knowledge was assessed to be reasonable.Based on the results, it was not possible to create a pattern of the way in which students learn in a CSCL-system. Comparable to more traditional settings, students learned on their own way. In study 3, students were asked to fill in a part of the Inventory of Learning Styles (Vermunt, 1992) to search for a possible relationship between students' learning style on the one hand and students' learning processes in a CSCL-system on the other hand. No correlations were found between students' learning style and their participation. Between students' learning style and their learning activities a few significant correlations were found. To keep the discussion clear and to monitor the task, it seems to be good to have students with an application-directed learning style or meaning-directed learning style in the group. However, because of the lack of explicit learning styles of most students, a Pearson correlation test was also executed on the level of scales. There, some interesting correlations were found. It will bear fruit to stimulate a positive attitude towards collaborative learning. Another correlation consisted between the scores on the scale deep cognitive processing and amount of knowledge construction. Students who scored high on the scale deep cognitive processing constructed more knowledge than students who scored low on this scale. Additionally, students who lack regulating strategies will have some problems in working with CSCL.Among others, Webb and Sullivan Palinscar (1996) and Dillenbourg (1999) wrote about the complexity of educational contexts. They argued that because of the multiple interactions between factors such as group size and task characteristics it is very difficult to set up initial conditions that guarantee the effectiveness of collaborative leaning. This research also confirmed the complexity of factors in setting up a successful course. As mentioned above, we deepened one factor in our research, namely moderating discussions. The results gave reason to assume that students that are moderated critically construct on average more and qualitative better knowledge than self-regulated students. Critical moderation was among other things concretised by asking questions, checking answers and contributing statements. Critical moderation triggered students to deep learning which means that students interact critically with the learning content, relate contributions within the discussion or to information found in other sources, use organising principles to integrate ideas and examine the logic of the arguments used (MacFarlane Report, 1992). Although an effect was found for the use of cognitive learning activities and knowledge construction, the results indicate that the quality of teacher interventions determines the success. It is difficult to instruct teachers to moderate asynchronous discussions in the short term. Although guidelines can be given and trained, teachers must become familiar with CSCL and moderating discussions. Factors such as using the right tone, moderating according to a personal way of teaching, real involvement in the course and pleasant contact with students are of importance to let moderation succeed.Another factor that was manipulated is solving a problem from a certain perspective. In study 2, students conducted two tasks. In contrast to the first task, students working on the second task played a specific role (for example economist, tourist or farmer); students worked in a multidisciplinary team. Having other information and contradictory interests stimulated active as well as passive participation, which resulted in more knowledge construction.<strong>Conclusions</strong><em>Chapter 7</em> summarises our most important findings and discusses the results from both theoretical and methodological perspectives. Among other things, comments are given on the definition of knowledge construction used, the standards used to assess the amount and quality of knowledge construction and the number of students participating in the studied courses.In our opinion, the developed method was useful to analyse students' learning processes in a CSCL-system. In other words, the executed analyses increased our insight into students' learning processes and helped us to survey the activities students use when taught by CSCL. Beforehand, we assumed that CSCL could be useful to support students' learning processes, especially in higher education. After carrying out the research, we still believe CSCL offers opportunities to support the process of knowledge construction. However, we think it is getting time to consider how to increase the amount and quality of knowledge construction by students in a CSCL-system. Students do not make optimal use of the opportunities. It is true that students constructed knowledge, but the amount of knowledge often was not large and the quality of the constructed knowledge left to be desired. Thus, we can conclude that CSCL can lead to learning indeed. However, we do not have to expect miracles from CSCL. Besides teacher interventions and working in multidisciplinary teams, the following aspects appear to be of importance for the use of CSCL in education successfully:<UL><LI>experiences and motivation of students;</LI><LI>a well-organised course;</LI><LI>organising face-to-face meetings regularly;</LI><LI>a complex task that provokes discussion and negotiation about knowledge;</LI><LI>a transparent and user-friendly CSCL-system;</LI><LI>enough time to become familiar with the ICT-program;</LI><LI>enough time to work on the task, to read each others' contributions and to react to contributions;</LI><LI>task structures that regulate organisational and planning issues;</LI><LI>brainstorming about the task individually first and comparing ideas next;</LI><LI>assessing the learning process.</LI></UL>As with other educational appliances, the use of CSCL must very thoroughly be considered. Although this remark seems to be obvious, it is an important finding. Nowadays, people often think too easily that the use of ICT stimulates students to learn and that this learning automatically results in positive results. Our research shows that this assumption is far from the reality. When considering the use of CSCL in education, examples of important questions are: What is the aim of the educational course? Which task is needed to reach the aim and is that task appropriate to work on in a CSCL-system? Is it desirable that students learn collaboratively in this course? To what extent does the task have to be prestructured? Are the subject matter and the task useful to negotiate about knowledge? Do students have experience with CSCL and if not, can we train them in a short term? Is a user-friendly CSCL-system available? Are enough computers available? Do we prefer students working from a distance or working in one room? How much time do we expect students to work on the task? Do we assess students' participation in the CSCL-system and/or the content of the contributions? Do we charge students with rules? Finally, it is important to consider whether moderation is desirable, and if so, how to moderate discussions; what is your aim with moderating a discussion? Do you want to stimulate students to participate or do you want to increase their critical thinking and knowledge construction?<strong>Future research</strong>Besides conclusions and practical implications to use CSCL effectively, chapter 7 gives suggestions for future research. A line of research suggested is systematically analysing the relation between the conditions of using a CSCL-system and the depth of learning. By conditions one can think for example of additional or integral use of the CSCL-system, different types of CSCL-systems, and different types of tasks. Besides, it would be interesting to analyse the extent of transfer of acquired knowledge and skills compared to similar task situations. Another interest is analysing participation, interaction, use of learning activities and knowledge construction during the course as well. Therefore, the instrument developed and used in this PHdissertation can be used in further research again, elaborated or not. When repeating the research, attention must be paid to the used standard to check whether students construct little knowledge in other settings, too. If so, the core of further research must concern the question of how to increase knowledge construction in CSCL. It would be wise to involve students' experiences more intensively.