EEP Impacts 2 – Students’, teachers’, and employers’ perspectives​​

Using ePortfolios in higher education can have strong impacts on improving competence-based education and lifelong learning. The Empowering Eportfolio Process (EEP) project, with six European higher education institutions, achieved many impacts on different stakeholders important in developing ePortfolio practices. The present article describes the impact evaluation findings of the EEP project at the students’, teachers’ and employers’/the world of work level based on a special impact evaluation model (further described in the article EEP impacts 1 ‒ Framework for impact evaluation). The model investigates the principal research question “How did the partners experience the impact of the EEP project?”The findings of the first three out of the five impact levels are described in this article. Figure 1 shows the propositions that describe the expected impacts at the levels of students, teachers and representatives of the world of work, like employers’ perspectives. The findings of the two other levels will be described in the article EEP impacts 3 – Organizational and networking perspectives.

Methods

The impact evaluation model and its implementation are described in detail in the article EEP impacts 1 – Framework for impact evaluation (Erdei, Kunnari & Laurikainen 2018). All the partners in the EEP project were interviewed; one from Marino Institute of Education, MIE, Ireland; one from VIA University College, Denmark; one from KU Leuven and one from University College Leuven-Limburg, UCLL, Belgium; two from Polytechnic Institute of Setubal, IPS, Portugal; three from Häme University of Applied Sciences, bachelor level education (HAMK1) and two from teacher education (HAMK2). The previous analysis published by the partners were combined to the impact evaluation:

1) Collection on Engaging practices on ePortfolio Process (Kunnari & Laurikainen 2017a),
2) Students’ perspectives on ePortfolios (Kunnari & Laurikainen 2017b),
3) Employers’ perspectives on ePortfolios (Laurikainen & Kunnari 2018), and
4) Higher education perspectives on ePortfolios (Kunnari 2018b).

The first author of this article was responsible for the analysis and the second author provided coordinators’ perspectives in writing the findings. The frame consisted of the following subdivisions: 1) expected impacts; 2) unexpected impacts; 3) challenges, and 4) ideas for improvement.

Findings

Although the contexts of the five partner HEIs are different, the impacts at the students’ and teachers’ levels, as well as at the level of the world of work show great similarities. The impact on students is of utmost importance, as the ownership of ePortfolios must be in students’ own hands. Teachers are key actors in providing a supporting learning environment and encouraging students to build their own ePortfolios in collaborative processes. Representatives of the world of work must be invited and integrated to the process, as competence-based education needs to be connected to the world of work. In the following, we describe the impacts of EEP project in these three different, but interconnected levels.

Impact at the level of students

Partners’ analysis highlighted some major benefits of ePortfolios for students’ professional development. Students appreciated visibility, assessment, personal development, reinforcement of learning, user-friendliness and freedom given by teachers. Among competences developed they mentioned digital and technological skills, transversal and transformative capabilities like creativity, autonomy, reflection, logical thinking, selecting and organizing information, systematization, collaboration and sharing. Peer guidance and assessment were also observed to be motivating. (Kunnari, Laurikainen, Pires, & Rodrigues 2017a; Kunnari, Laurikainen, & Torseke 2017d; Korhonen, Ruhalahti, & Torseke 2017; Van Eylen & Deketelaere 2018; Pires, Rodrigues, & Pessoa 2018; Choistealbha 2018; Rodrigues, Pires, & Pessoa 2017; Pires & Rodrigues 2018a; Devaere, Martens, & Van den Bergh 2017.) Some students also mentioned respect for ecology (Van Eylen & Deketelaere 2018; Pires & Rodrigues 2018a).

Some partners emphasized that students’ core competences and career possibilities were strengthened from the perspective of lifelong learning. Most partners emphasized the improvement of digital and organizational skills, while some of them referred to other skills. HAMK1 experts highlighted transversal and visual skills, organization, communication, patience and the possibility of combining digital tools and environments, for the final point this HEI is unique compared to the partners. The Portuguese experts also mentioned creativity, cooperation, autonomy and responsibility. Based on all the above, the expected impacts outlined in Proposition 1 of the EEP evaluation model proved to be realized.

Some experts underlined the importance of reflection in the ePortfolio working process. The MIE partner mentioned that the ePortfolio work process might have opened students’ eyes to what reflection actually is. The VIA partner referred to the specific double perspective of working with students who are themselves teachers, which raised the awareness of the challenge of reflection for all participants. He also highlighted that students were extremely satisfied that their prior learning was taken seriously in the ePortfolio process, and teachers helped them build on their previous knowledge.

Peer learning, as some experts noticed, yielded unexpected results, which motivated them to further enhance this learning method. The MIE partner observed that peer-on-peer learning was almost more effective than teacher-led workshops, which made her consider strengthening communities of practice. The HAMK1 partner also considered enhancing student collaboration: “I have been thinking that we should suggest students to build their ePortfolios in teams because collaboration is so important.”

EPortfolios can powerfully promote one’s professional success. Some interviewees evidenced impact by referring to interesting cases where students managed to benefit from using ePortfolio in their lives in various ways:

“…as part of the project, I did research on a student’s case, and she said that by using an ePortfolio across both her undergrad and postgraduate programs, she had more of an understanding of herself as a professional, and she was better able to represent herself at interviews because she had engaged in that level of reflection that the portfolio allowed. And she also noticed that you really grow in comfort with it as a tool.” (MIE)

“I know of one ex-student who had a portfolio and now works as an educational developer at another faculty, so she merged the importance of using ePortfolios within the educational program and is now responsible for developing it.” (KU Leuven)

“And today I’ve heard from one of my teacher students that she’s going to use the ePortfolio concept with her social and healthcare students. It’s one of our targets as well; that they will learn how to use this concept with students.” (HAMK2)

Partners’ analysis revealed also some main challenges related to ePortfolio use. These concerned the lack of students’ and teachers’ digital confidence, technical issues, digital support, feedback, guidance, dynamics, privacy concerning the openness of ePortfolios and resistance to change. (Kunnari et al. 2017c; Kunnari et al. 2017d; Poulsen & Dimsits 2017; Van Eylen & Deketelaere 2018; Pires et al. 2018; Choistealbha 2018; Rodrigues et al. 2017; Pires & Rodrigues 2018a) Some students raised concerns about ePortfolios as assessment tools (Korhonen et al. 2017; Kunnari et al. 2017d; Choistealbha 2018), while others failed to understand what ePortfolios are good for or how to reflect with or combine it with other tools (Kunnari et al. 2017c; Kunnari et al. 2017d; Poulsen & Dimsits 2017). Furthermore, interviewees mentioned difficulties with technical issues and students’ digital competences, and they also pointed out areas to be improved. The MIE partner revealed that students expressed the need for more institutional support. Students at KU Leuven found it important to know how and why they use the platform, while they made remarks also on teachers’ ePortfolio-using competences. As for improving the process, the expert revealed that the organization wishes to involve students in the evaluation of the tool.

Impact at the level of teachers and study counsellors

The partners’ analysis about teachers’ perspectives of the EEP project highlighted the importance of collaborative processes that help to understand how students’ reflection can be improved and competences visualized (Kunnari & Salmia 2018; Pires & Rodrigues 2018c; Devaere 2018a). Teachers need to guide students with the ultimate goals to create the feeling of ownership (Kunnari & Salmia 2018). Keywords concerning the impact of ePortfolios at the teachers’ level were also highlighted: dissemination, synthesis, openness, workspace, assessment, feedback, reflection, repository, skills, equality, competencies, opportunities, showcasing, learning styles (Choistealbha, Mullally, & McMorrough 2018). It was also underlined that ePortfolios must become rooted in the personal pedagogy of teachers, which also raises the question of prioritisation and leading by example (Poulsen 2018).

Most partners referred to teachers gaining better competence in guiding student-centred and collaborative learning and building better assessment processes, which was an expected impact outlined in Proposition 2 of the EEP evaluation. According to HAMK1 partners, the main impact has been that teachers recognized the benefits of the process. Some experts also highlighted the importance of teachers’ collaboration in the ePortfolio process and the need to build new knowledge of ePortfolio practices. That is why, for example, IPS has established new training programs, while MIE has built their own body of research on using portfolios.

Some interviewees referred to another impact outlined in Proposition 2 of the EEP model, i.e. educational work supporting lifelong learning for teachers and counsellors. The double perspective of working with teachers as students will continue the discussion on treating the validation of prior learning beyond the end of the project, which will in turn help study counsellors think about it more as a working process, as our VIA partner revealed. Partners’ motivation and professional development have been and is expected to further increase, which will also provide organizational understanding. The IPS partner noted that the teachers involved in their pilot activities started using ePortfolios more systematically and showed great interest in deepening their knowledge and skills to work with it. The HAMK1 expert revealed that after she introduced and piloted the ePortfolio work other teachers and study counsellors became interested in working with it in other groups. The KU Leuven partner also pointed out that after their workshop on ePortfolios some participants showed great interest to implement them to their educational programs. The UCLL partner believes that their two-year participation in the EEP project helped teachers recognize ePortfolio benefits leaving them wanting to learn more, which is a great impact of the project. The MIE and HAMK1 experts also mentioned the beneficial effect of teachers using ePortfolios.

Some experts highlighted areas for improvement at the teachers’ and study counsellors’ level to succeed in ePortfolio work. The MIE expert revealed her intention to put supports in for staff and students to build digital competences. The HAMK1 expert referred to the risk of some teachers’ resistance to do things differently, because of the time-management issues. As HAMK2 experts noted, working with ePortfolios may mean extra work for teachers in an initial phase, which might discourage some, but later on it does not require so much extra time from teachers. Students can be the key actors in building new practices, as they are the owners of ePortfolios.

Impact at the level of the world of work

The partners’ analyses showed various existing or possible benefits of ePortfolios resulting from cooperation with the world of work. Employers recognized the importance of ePortfolios and are available to provide support in this creation process (Pires & Rodrigues 2018d). Working field experts revealed that ePortfolios could serve as a showcase to demonstrate professional experience and competences, evidence personal strengths, and present individual interests and characteristics (Kunnari 2018c; Pires & Rodrigues 2018b, Rodrigues & Pires 2018, Pires & Rodrigues 2018d, Van Eylen 2018; Devaere 2018b;). ePortfolios can also give a more dynamic perspective on a candidate’s profile (Pires & Rodrigues 2018d)). Perspectives from the world of work can also motivate students to be creative and innovative in building their ePortfolios (Pires & Rodrigues 2018b, Kunnari 2018c).

Measuring impact was not easy at the level of the world of work, as some partners noted, because the project is so young. However, cooperation and interviews with employers led to interesting observations. As an important impact of the project, HAMK1 experts highlighted that employers’ awareness related to the benefits of ePortfolio has risen. For example, students can utilise work-life environments and feedback from the world of work in their ePortfolios. Similarly, to what was outlined in Proposition 3, it was confirmed that better visibility of future applicants will help employers evaluate applicants’ competences and skills: “as employers, they get an idea of who the students are, and whether they want to have them at their workplace.” (KU Leuven.)

In addition, the collective knowledge and research of the EEP project received an important role when cooperating with the world of work. As our MIE partner concluded: “when I speak to work-life representatives I have the collective knowledge of the entire project that I’m speaking on behalf of. So, that has certainly been a very positive impact for me.” People from the world of work can also learn about research and development projects. Similarly, the VIA expert who is the head of the R&D program, referred to their special forum, where many representatives from the world of work, relevant to their programs meet twice a year to discuss the status of the various R&D projects. The preliminary results of the EEP project have also been presented to them, which can possibly influence them.

Partners’ analysis mentioned various unexpected impacts for the world of work, which were complemented at the interviews. Digital portfolios could serve as a tool for professional development in the workplace (Kunnari 2018c;). The HAMK1 expert mentioned the discovery of using ePortfolios in continuous professional development at Helsinki University Hospital, while the VIA expert referred to some fruitful discussions resulting from the EEP project: “I think a lot of our results prove that the questions of meaning and motivation come before technology. That’s a discussion I couldn’t have had, I think, with for instance, these work-life representatives beforehand.”

In addition, two interviewees referred to most recent state educational policy changes concerning portfolios, which has remarkably raised the importance of the international EEP project. In Denmark the portfolio was put into new legislation on transition from comprehensive to upper secondary school. As a great benefit of the international project the VIA partner concluded; “We could use the experiences from EEP, as the starting point of discussion on how to make experiments with ePortfolio meaningful. We could take, present and discuss other partners’ experiences with our work-life representatives.” Furthermore, in Ireland, primary and secondary level teachers are now required to have a professional portfolio: “it’s called taisce, an Irish word for ‘treasure box’. So, from the project’s point of view, I’ve been able to talk about how portfolios are used in different countries and the different approaches to it, and research that we’re doing.” (MIE)

Partners’ analyses and interviews revealed challenges that students may face while working with ePortfolios with the world of work. They suggested ideas for improvements, for example, it was reported that mistakes students made during their studies should not necessarily be shown to recruiters (Devaere 2018b). Ensuring the privacy of students while being open and transparent with ePortfolios during job recruitment can also be challenging (Devaere 2018b). Job recruiters would like to see the differentiating aspects, and relevant, synthetic, simple but non-superfluous information, which students still find difficult to understand. Recruiters expressed a need to make ePortfolios more visible (Pires & Rodrigues 2018d, Rodrigues & Pires 2018).

Representatives of the world of work see the benefits of a more intensive future collaboration between workplaces and higher education (Kunnari 2018a). An initiative about giving feedback was suggested during the interview: “…we would also like internship mentors to use ePortolios to give feedback. That’s why the impact of the project is really big because the way we introduce the project was a way to introduce the ePortfolio to the working field, and so we can build on it to go further.” (UCLL)

Conclusions

As a conclusion, we can recognise that the EEP project has had strong impacts in developing new practices to competence-based and student-centred higher education. With a dynamic approach to the ePortfolio process, students can develop various competences like digital and transversal skills, transformative capabilities and collaboration, and this can powerfully promote students’ future professional success. Teacher and study counsellors can achieve better professional competences, which can increase their motivation and contribute to their lifelong learning. Furthermore, the use of ePortfolios can have an impact on strengthening collaboration and communities of practice in higher education. This in turn, can improve pedagogical practices, for example, as teacher collaboration proved to be effective in understanding how students’ reflection can be improved and competences visualized (Kunnari et al. 2018; Pires & Rodrigues 2018c; Devaere 2018a). In the ePortfolio process, students can be key actors. Students’ peer learning can be more effective than teacher-led workshops and students can be included in evaluating and developing ePortfolio tools and practices.

This impact evaluation demonstrates that higher education students and teachers need time and space to learn and understand the benefits of ePortfolios, and they need new competences. Among the challenges that students faced, digital confidence, privacy issues and possible resistance to change were mentioned, while occasionally students failed to understand the purpose of or ways to work with ePortfolios. (Kunnari et al. 2017c; Kunnari et al. 2017d; Korhonen et al. 2017; Van Eylen & Deketelaere 2018; Pires et al. 2018; Choistealbha 2018; Rodrigues et al. 2017; Pires & Rodrigues2018a; Devaere et al. 2017.) Students also noted that they needed more institutional support or guidance. The importance of leading by example was referred to be effective when working with ePortfolios. Challenges at this level may arise because of the lack of various crucial conditions. Improving digital support for staff and students may solve problems occurring from insufficient digital competences, while increasing student ownership may ease teachers’ time management problems.

Based on the findings of this impact evaluation, representatives of the world of work can find new ways to collaborate with HE institutions and utilize students’ competences. Incorporating the feedback from the world of work was suggested to increase the fruitful cooperation (In addition, ePortfolios can serve as showcases to demonstrate professional experience and competences, evidence personal strengths and present individual interests and characteristics, while also motivating students’ creativity and innovation. (Kunnari 2018c, Pires & Rodrigues 2018b, Rodrigues & Pires 2018, Pires & Rodrigues 2018d, Van Eylen 2018, Devaere 2018b). The collective knowledge and research of the EEP -project also made impacts on representatives of the world of work. Employers recognized that ePortfolios can be used as more meaningful recruitment tools to evaluate applicants’ competences, interest and personal characteristics. Furthermore, employers discovered that ePortfolios can contribute to continuous professional development of staff, while the project even received an elevated role in collaborating with recent local state educational policy changes. The impacts on the world of work level are hard to measure at present because the project is so young, therefore some of the aspects outlined in Proposition 3 are yet to be seen. The impacts discovered in the analysis though raise hopes that working life will have better possibilities to find more suitable applicants for open positions and build fruitful collaboration with HE students and teachers in competence-based education.

This article was produced in the Erasmus+ (KA2 action) funded project “Empowering Eportfolio Process (EEP)”. The beneficiary in the project is Häme University of Applied Sciences (FI) and the partners are VIA University College (DK), Katholieke Universiteit KU Leuven (BE), University College Leuven-Limburg (BE), Polytechnic Institute of Setúbal (PT) and Marino Institute of Education (IE). The project was implementated during 1.9.2016–30.11.2018.