Guidelines for effective online feedback in e-Portfolio learning

Receiving feedback is a key determinant for a successful e-portfolio implementation. When introducing an e-portfolio, one not only needs to prepare the students, but it is also important to support the supervisors in writing effective online feedback.

The following guidelines are based on our own experiences with providing online feedback and on discussions with supervisors during a workshop on effective online feedback. The guidelines are related to the role of the feedback giver, the tone of the feedback, its content and style. We conclude with the importance of investing in training for e-portfolio supervisors. The experience-based format of the workshop seemed to be very useful.

Introduction

An e-portfolio can fulfill several functions like documenting, structuring and evaluating the learning process (Deketelaere et al. 2007). Receiving feedback on an e-portfolio assignment is one of the determinants for a successful e-portfolio implementation (Driesen et al. 2007).  An electronic portfolio creates ample opportunities to share portfolio assignments (or reflections) and to communicate about the learning process among peers, (clinical) supervisors and/or teachers (Lawson, Nestel, & Jolly 2004).

Because of this online feedback experience, students realize that their assignments are taken seriously. Further, when given in a proper way, this feedback is perceived helpful to deepen their learning (Van Eylen & Deketelaere 2018). The feature of asking for and receiving online feedback is therefore crucial. Access to the internet is the only prerequisite.

Particularly in the context of workplace learning, online feedback can be an appropriate tool to support a student’s learning process. Firstly, because workplace supervisors do not always have the time and space to give face-to-face feedback during working hours. Secondly, it gives the learning institute (the school) the opportunity to support, steer and adjust the learning process from a distance (Deketelaere 2010).

Online feedback, just as written feedback, has some specific characteristics. The most important one is that the feedback is archived and is permanently accessible. Thus, it gives the student the opportunity to take it in and then reread it. It also makes it easier for the student to see his/her own progress. However, online .eedback can also be subject to unintended misinterpretations. Therefore, the way feedback is phrased is crucial. That is one of the reasons, why feedback givers find this time-consuming. Sufficient attention to an appropriate formulation is needed.

Face-to-face feedback is more volatile, but one can immediately see how feedback is interpreted by the receiver. This makes it easier to adjust wrong interpretations.

Table 1. Overview of characteristics of written and online feedback versus face-to-face feedback.

Characteristics of written and online feedback	Characteristics of face-to-face feedback
Postponed	Immediate
Archived (remains available)	Volatile
Considered formulating	Effect is visible
Time-consuming	Adjustable
Evolution is visible	Spontaneous (conversation)
Documents the supervision process

The specific added value of online feedback, compared to written feedback in a paper portfolio format, is that it is accessible at any time and place. There is no hassle with ring binders that have to be transported from one place to another. Everyone who has access to the portfolio can write or read the feedback, from anywhere and at any time.

As stated above, online feedback offers plenty of opportunities to support and supervise e-portfolio learning in an improved way. Unfortunately, including an online feedback button is not enough to assure high quality feedback, especially when feedback givers are medical doctors with only limited pedagogical training. We discovered that for them one of the barriers in supervising an e-portfolio was the struggle with the content and the formulation of the online feedback. To answer this need, we developed a workshop in providing quality online feedback.

In this article, we want to share some guidelines based on these experiences.

Context

A few years ago, the medical school at KU Leuven (Belgium) introduced an online portfolio for its clerkships. This online portfolio, called Toledo My Portfolio, was developed within the university as part of the Blackboard learning environment. Consequently, the university chose to make the student the owner of the e-portfolio. The supervisor coordinator has full access to the e-portfolio to overview students’ assignments. But it is the student who requests feedback and thereby gives permission to others to write in the e-portfolio. With one click the student can ask for feedback for a particular assignment from whomever he wants and the one requested for feedback will automatically receive an invitation in the inbox of the e-mail. This person does not need to remember a password to get access to the e-portfolio. This e-portfolio environment is created for the different feedback applications a feedback provider receives. In this way the feedback giver maintains access to all assignments and feedback s/he has already provided in the past.

Every student keeps, for all clerkships, a clerkship-portfolio in which his learning goals are documented, and makes reflective assignments on several topics, like a clinical case report, a case presentation, an observation of breaking bad news, a perceived ethical dilemma, and an interview with a paramedic. All these assignments can be sent to peers, supervisors or other staff members for feedback. One assignment, a self-evaluation in preparation of a feedback talk, must be sent to the supervisor.

Method

The following guidelines are based on two main sources: our own experiences as feedback givers and our experiences as trainers of the workshop ‘online feedback for clinical supervisors’. Additionally, we explored the literature on this topic to get input for the workshop.

The workshop was designed in such a way that the clinical supervisors, further named “feedback givers”, could experience the process of feedback giving and receiving. It consisted of three rounds. In the first round, the participants had to write an individual feedback on a fictive e-portfolio assignment. In the second round they compared and discussed each other’s written feedback. It quickly became clear that each feedback differed a lot in content and style. We discussed the characteristics of the feedback and the rationale behind writing it. In the third round we had to reread the feedback from the perspective of the student as the feedback receiver. This was carried out to answer the key question: “What kind of feedback would best support effective learning?” This workshop was also given as part of the EEP meeting in Leuven, on February 22th 2018, at UCLL.

Results

We present the results in the form of specific guidelines concerning the role of the feedback giver, the tone of the feedback, and its content and style. We end with some reflections on the context of online feedback.

Supervising with different roles: feedback is not an evaluation

Among students and supervisors there are still many misunderstandings about feedback and evaluation. It is important to make the difference clear according to the aim, the time frame, the focus and the role of the supervisor (see table 2). It is also important to make supervisors conscious that what they say as feedback can be perceived by the student as ‘evaluation’. It all has to do with the atmosphere and the role in which they present themselves (Jakube et al. 2016). Especially in the context of online feedback it might be important to articulate that the comments are meant as supportive and not as evaluation. Separating both roles is especially difficult when the supervisor has to evaluate the student at the end too.

Feedback is always subjective and it reflects the values and norms of the feedback giver. When this difference became clear during the workshop, supervisors felt safe that it was all right that the content of their feedback differed.

Table 2. Overview of the characteristics of feedback versus evaluation.

	Feedback	Evaluation
Aim	Support learning (formative)	Decide on reaching the prerequisite level/criteria (Summative)
Time frame	Looking forward	Looking backward
Focus	Mastery	Performance
Role supervisor	Coaching Subjective	Assessor Objective

The tone of the feedback: reinforcing and supportive

Most written feedback is positive, self-oriented and without information for further actions (Canavan et al. 2010). Quotes like “well done!”, “good progress!” or “You are an added value in our team!” are reinforcing but not very supportive for further development. Just adding one sentence telling what and why something is “well done” already gives more information on the strong points of the student.

Reinforcing feedback is valuable for a new, unassured student who fears failure. It strengthens self-confidence. In this way, it works on the affective dimension: the well-being of the student, which is a condition for further learning.

But supportive feedback is also actionable: it is behavior oriented and contains specific information of what and how to improve. For instance, compare both examples below:

Well done! You received lots of autonomy and used it well to prepare the next day’s patients.

Despite the high workload, you succeeded to work efficiently. Try in the near future to take 2 minutes extra to fill out the patient files properly and finish the report before going on to the next patient.

Gulbas, Guerin, and Ryder (2016) analyzed the written comments given by supervisors on e-portfolio entries and asked students to categorize these comments in terms of helpfulness. They found out that comments that were related to specific behavior and that contain specific suggestions for improvement were perceived as most helpful. Students find it important that the feedback giver indicates that he knows and has observed the student-in-action, so he can give specific examples of appropriate behavior to reinforce, or unsuitable behavior to eliminate. A statistical analysis revealed that the high-quality feedback mostly contains more words/sentences (Gulbas et al. 2016).

The content of the feedback

Feedback that only consists of praise, reward or punishment is hardly effective. This feedback confirms what one already knows, but hardly pushes the learning process to a deeper level of understanding. Hattie and Timperley (2007) developed an alternative model, consisting of Feed Back, Feed Up and Feed Forward on distinctive levels, guaranteeing more deep level learning.

• Feed Back is an answer to the question: how am I going?
• Feed Up is an answer to the question: where am I going?
• Feed Forward is an answer to the question: where to next?

The four levels of feedback are:

1. Comments on the task: is the task done well or not (output)?
2. Comments on the process: were the used strategies adequate?
3. Comments on the self-regulation: what did you learn about steering your own learning process?
4. Comments on the self: how are you as a person?

Following this model, the online comments can be past or (nearby) future-oriented. They can also be related to different aspects of the learning process and/or the self. Radstake (2018) found in her self-study, based on Hattie and Timperley’s model (2007), that students learned most from Feed Up comments on process and on self-regulation, because this feedback helped them to stay focused on the goal. She also added a sentence explaining the reason for a specific comment as part of the online feedback.

Style of feedback: online feedback as a conversation

Literature does not offer an exact definition of feedback (Van de Ridder et al. 2008). However, even in the case of online feedback, when your partner is a computer screen, it is interesting to consider the following definition by Lefroy et al. (2015, 297): “Helpful feedback is a supportive conversation that clarifies the trainee’s awareness of their developing competencies, enhances their self-efficacy for making progress, challenges them to set objectives for improvement, and facilitates their development of strategies to enable that improvement to occur.”

Following this definition, online feedback should be addressed to the person (“Dear Jean…”) and be written in the second person (you). In addition, questions are recommended, as Dekker et al. (2013) mentioned. They investigated the characteristics of written feedback that stimulated students’ reflective competence. Written comments phrased as a question were judged as stimulating reflection, more than comments phrased as statements.

The context of online feedback: making online and face-to-face feedback complementary

Online feedback and face-to-face feedback have different characteristics and can be applied complementarily. Online feedback on assignments can be given in preparation of a reflection seminar in which these assignments are discussed in a more general way. Here, the online feedback gives the opportunity to respond in a more personal way and connect to each individual case a student brought in.

The online self-assessments during clinical clerkships are always followed by a face-to-face supervision talk. The self-assessments by the students help the supervisor to define the focus of this talk. It provides him with good information on the ‘person’ of the student: how realistic is the self-image of the student? Does he under- or overestimate himself? Is he rather profound or superficial in his reflections? Can he make a distinction between core and details? This meta-information can also be an element of feedback during the face-to-face talk. These are comments on level four (person) according to Hattie and Timperley’s model (2007).

During the workshops, it became clear that the moment of filling out the online feedback differs. Some supervisors fill in a ‘draft’ feedback in preparation of the talk and send it afterwards to the student. Others fill in their feedback during or after the talk (similar to filling in a patient file during a consultation). During the workshop, we discussed the advantages and disadvantages of the different moments of writing the online feedback. The main message is that each supervisor must find a way that works most efficiently and uses the least time.

Students prefer more detailed online feedback than frequently-given ‘general’ feedback. They also expressed a desire for critique of their work rather than non-specific general comments. (Bleasel et al. 2016.)

Students all agreed to the importance of combining online feedback with face-to-face contact. This helps a supervisor to fill out timely feedback. The face-to-face contact moment functions as a kind of “deadline” and means that the feedback request in the mailbox is not forgotten or does not disappear.

Discussion: the characteristics of high quality online feedback

The previous considerations will make clear that training in giving qualitative online feedback is not a superfluous luxury! Based on this experience new insights on high quality online feedback were formulated, for example:

• Address the student with his/her name (like in a letter).
• Write in the second person (to the student), instead of the third person (about the student).
• Rephrase specific quotes, as a sign that you read the assignment.
• Write about specific observations or actions by the student, as a sign that you know him and paid attention to his performance.
• Add a sentence explaining why you give a certain comment.

The characteristics of good, qualitative online feedback are often derived from the SMART principle (specific, measurable, acceptable, realistic and timely). In Table 3, this is summed up and added with a few more characteristics to improve the pedagogic implementation of it.

Table 3. Hightlights the important characteristics of qualitative online feedback.

Characteristics of qualitative online feedback
Timely
Specific
Actionable
Reinforcing
Supportive
Task- and process-oriented
Supports self-oriented learning

There are plenty of technical opportunities to make an audio or video recording as an alternative form to give online feedback. The main points of these recommendations on online feedback is that they are experience-based and supported by evidence from literature.

The format of the workshop was perceived as very valuable to broaden beliefs on online feedback and to practice and get experience with online feedback. Supervisors became conscious about their intuitive style of writing feedback and received examples of alternatives. In this way, they can apply a specific style of feedback related to the students in front of them. One general conclusion was to read the feedback before sending it to the student, from the perspective of the feedback receiver: “How would I perceive this feedback: nice or informative? How do I have to (re-) formulate it so that it contributes even more to my own learning process?”

The exercise made them conscious about the fact that a feedback receiver can interpret the feedback in an unexpected way. One of the main conclusions was that the formulation of the feedback needs to be considered and nuanced. It demands more time and mental peace of the feedback provider to reflect on the exact formulation. Here as well, it is important to reread the feedback from the perspective of the receiver: how would I interpret this feedback if I had received it?

Otherwise, one always has to bear in mind that these recommendations are based on the perceptions of the usefulness of feedback. Students’ satisfaction with feedback is not a good indicator of its effectiveness to change behavior. Boehler et al. (2006) concluded that student satisfaction was higher in the group receiving compliments, whereas the improvement in the task was greater in the group receiving constructive feedback. There is also no relationship found between the valence (positive/negative) of the feedback and the perception as ‘helpful’ (Gulbas et al. 2016). A more sophisticated kind of research is needed to investigate whether and how online feedback effectively leads to change in behavior and thinking.

An interesting side-effect is that through the online feedback an institution receives information on the quality of the online supervision process by the supervisors. The amount and quality of the feedback is visible, one can see if the feedback was provided in a timely fashion. This information can be used to set up a training for supervisors to enhance their online feedback skills. Such a training was one of the sources for the reflections in this article.

Acknowledgements

Thanks to Sabien Van Moorter for her comments on a previous version.

This article was produced in the Erasmus+ (KA2 action) funded project “Empowering Eportfolio Process (EEP)”. The beneficiary in the project is Häme University of Applied Sciences (FI) and the partners are VIA University College (DK), Katholieke Universiteit KU Leuven (BE), University College Leuven-Limburg (BE), Polytechnic Institute of Setúbal (PT) and Marino Institute of Education (IE). The project was implementated during 1.9.2016–30.11.2018.