1. What are the similarities and differences between face-to-face and online collaboration? Which strategies for instructional design and delivery can be transferred from in-person workshops to the new medium, and which need to be modified for reap the benefits of online professional development?
2. What facilitation strategies encourage engaged and reflective interactions? How do we encourage participants to voice their thoughts? How do we ensure that participants interact with each other, rather than interacting primarily with the facilitator?
3. How can online learning connect to off-line activities such as local study groups, action planning teams, and classroom experiments born from ideas developed in online collaborations?
4. What instructional design strategies best exploit the strengths and avoid the weaknesses of this new technology? What combination of factual information, provocative discussion questions, investigative activities, and other techniques promote meaningful learning?

Information in regard to these questions was collected from participants in four ways:

• Web-based surveys distributed to participants before and after the running of each workshop (see Appendices D and E for sample surveys). In some workshops we also used a midpoint questionnaire and conducted phone interviews of a sample of participants.
• Online discussions, during the last session of each workshop, in which participants were asked to reflect upon their online learning experiences.
• Quantitative data of participant log-ins, number of messages, and time spent online.
• Analyses of discussions and products produced by participants.

The results from data that were gathered in the six online workshops run within the NetTech and LNT projects (see Section 2.1) are summarized below. The results are a compilation of the responses participants provided on the post-workshop surveys. Results are organized by the research areas designated by the questions above. The response rate for the surveys ranged from 50% to 75%. More detailed data from two sample workshops are provided in Appendix F. Our interpretations of the data into guidelines for effective workshop design and facilitation are provided in Section 6 below.

5.1 The Online Medium's Effect on Participation

When asked what time management strategies they would recommend to a new participant in an online workshop, participants overwhelmingly commented that new participants should schedule time for an online course the way they would for any other commitment. The following strategies for time management are representative of the replies received:

• Review weekly assignment early in the week to know how much time to allocate.
• Print out required readings so they can be read and responses drafted off-line.
• Try to do a little work online every day.
• Schedule specific times during the week to go online.
• Participate with a colleague for motivation and collaboration.
• Decide ahead of time whether you can devote the required time.
• Take advantage of the practice session so that you are very comfortable with the online environment before the workshop gets into full swing.

Participants were asked to comment on the advantages of learning online. Their replies fell into the following categories:

• More time to be reflective
• Could work according to own schedule
• Web resources could be bookmarked for future reference and easy access
• Opportunity to learn with experts who would not otherwise be accessible
• Opportunity to engage in dialogue with educators from diverse regions and types of schools and districts
• Saves travel time and costs

When asked to list the disadvantages of taking a workshop online, participants’ responses were very similar across all workshops. Replies to this question often listed the lack of liveliness in the discussion and an expectation that conversations online would have more depth. A second disadvantage cited was the pace and that operating in an environment that was primarily reading and writing took longer. Most other comments mentioned technical difficulties and the learning curve needed to master the technology.

5.2 Workshop Elements and Facilitation Strategies Designed to Promote Interaction

When asked whether there were periods of time during which they would read messages but not contribute to the discussion by posting, most respondents indicated that at times they felt that they had nothing new to add to the discussion. For example, one participant commented that if he got to the discussion early he had something to post, but otherwise he felt that there was too much to read and too much already covered to add anything. Another participant wrote that reading in general took more time than expected so it was difficult to find time to write messages.

Despite the fact that for some participants there may be times at which they don’t appear to be "interacting", i.e., they are not posting messages to the discussions, our research has shown that by itself, the number of messages posted is not an accurate measure of participation, learning or benefit. Many participants reported engagement and learning from the workshop at times when we could not see this if we were only measuring their (lack of) message postings. This engagement is evident in the final products that participants produce for the workshops, as well as in participants’ comments on the final feedback surveys about the overall benefit of the workshop to them.

When participants were asked whether there were periods of time when they were neither posting nor reading, a minority of participants responded in the affirmative; of these participants, most of them explained that their temporary lack of participation occurred when off-line commitments took precedence. Further, one respondent commented that it was difficult to get back into the workshop once he "had been away" for a period of time.

5.3 Incorporating Small-Group and Team Activities

All the workshops included online group discussions around a topic or issue, or the sharing of information around practice. Often online collaborations would informally develop when participants were asked to research resources and to share what they learned. In addition, several of the workshops incorporated specific small-group or team activities, and explored ways to accomplish these activities off-line and/or in the online environment.

During the MUVE workshop (see Appendix A for further description), participants engaged in online activities where they were taught how to use a tool almost entirely online (some phone support was provided). Participants were divided into small groups, and each group would "meet" online in a particular MUVE at a particular time, along with one or more facilitators. Learning a tool together provided a shared learning experience in the group similar to that experienced if they were attending a face-to-face workshop together. Participants felt strongly that the group support was essential for learning the tool, and they preferred this method rather than relying exclusively on self-paced individual exploration. They found that they were more confident returning to the MUVE for further individual experimentation after their initial group explorations. Conversely, initial individual explorations made the group time more productive. One challenge was scheduling times during which participants in different time zones could meet. This can be a problem even in the same time zone, due to participants’ busy schedules.

The IDEA workshop (see Appendix A for further description) incorporated a collaborative action-planning activity that was conducted both online and off-line. Participants for this workshop had registered in district teams, and during the workshop each team worked on an action plan that could be implemented in their respective districts. Each team was designated a private online discussion area and assigned a facilitator to help with the planning process. Teams were also encouraged to arrange off-line meeting times. Of the ten teams that participated in the workshop, three completed action plans and shared them with the rest of the group; three teams produced drafts of action plans; and two teams indicated that they wanted to continue the action planning process on their own. Two teams responded that they did not engage in the action planning process primarily due to the off-line logistics required.

5.4 Instructional Design Strategies

When asked which assignments and readings participants found the most useful during the workshops, the majority indicated that their choice for "best" assignment was linked to how applicable the activity was to their own practice. Readings that were highlighted also tended to be those that were related to participants’ own issues in their school or district. Participants said the following elements of workshops were most valuable to them:

• Information on how to design their own learning activities
• Readings that provided practical examples of what was being done by others
• Assignments that asked participants to evaluate examples of web sites or actual projects designed by others in the field
• Looking at models of practice
• Hearing stories from other teachers or administrators who had first-hand experience
• Any assignment or reading that helped focus on problem-solving
• Activities that were most applicable to professional development needs