EEP Impacts 3 – Organizational and networking perspectives

What kinds of impacts did the Empowering Eportfolio Process (EEP) project achieve at the organizational and networking levels? This article describes the impact analysis findings of EEP -project at the organizational and networking levels based on a special impact evaluation model (described in detail in the article EEP impacts 1 ‒ Framework for impact evaluation). The model investigates the principal research question “How did the partners experience the impact of the EEP project?” The findings of the last two out of the five impact levels are described in this article. Figure 1 shows the propositions that describe the expected impacts at the organizational and networking levels. The findings of the other levels are described in the article EEP impacts 2 ‒ Students’, teachers’, and employers’ perspectives (Erdei & Kunnari 2018).

Methods

The impact evaluation model and implementation are described in detail in the article EEP impacts 1 – Framework for impact evaluation (Erdei, Kunnari & Laurikainen 2018). All the partners in EEP project were interviewed; one from Marino Institute of Education, MIE, Ireland; one from VIA University College, Denmark; one from KU Leuven and one from University College Leuven-Limburg, UCLL, Belgium; two from the Polytechnic Institute of Setubal, Portugal; three from Häme University of Applied Sciences, bachelor level education (HAMK1) and two from teacher education (HAMK2). The previous analysis published by the partners were combined to the impact evaluation:

1) Collection on Engaging practices on ePortfolio Process (Kunnari & Laurikainen 2017a),
2) Students’ perspectives on ePortfolios (Kunnari & Laurikainen 2017b),
3) Employers’ perspectives on ePortfolios (Laurikainen & Kunnari 2018), and
4) Higher education perspectives on ePortfolios (Kunnari 2018).

The first author of this article was responsible for the analysis and the second author provided coordinators’ perspective in writing the findings. The frame consisted of the following subdivisions: 1) expected impacts; 2) unexpected impacts; 3) challenges 4) ideas for improvement.

The findings

The impacts at the organizational and networking levels showed various similarities with some contextual differences. The EEP project yielded various positive impacts on the individual levels of students and teachers and the world of work (see Erdei & Kunnari 2018). This article describes the project’s impacts on more abstract levels; elaborating on various organizational aspects and networking connections.

Impact at the organizational level

Partners’ analyses highlighted important institutional aspects to empower the use of ePortfolios. It was observed that organizational and managerial recognition and prioritisation of time are essential to create space and the possibility for teacher motivation and engagement. A professional community collaborating with an ePortfolio will enable the development, sharing and supporting of competences relevant for working with it, the use of which should depend on teachers’ professional judgement (Poulsen 2018). In UCLL the introduction of ePortfolio is connected to huge organizational changes and will affect the entire curriculum (Devaere 2018). In order to support teachers in the embedding of ePortfolios in practice, MIE intends to provide opportunities for staff to create their own ePortfolios so as to experience the meaning, autonomy, growth, impact and connections resulting from authentic engagement (Choistealbha, Mullally & McMorrough 2018). Teachers should change their practices through peer discussions and the ePortfolio should be integrated in the curriculum (Pires & Rodrigues 2018). Common ePortfolio working methods and models enhance the process, where both students and teachers’ empowerment is achieved by encouraging them to create their own applications (Kunnari & Salmia 2018).

In accordance with the expected impacts outlined in Proposition 4 of the EEP impact evaluation model, interviewees noticed that new innovative models concerning ePortfolio work were created at their HEIs. UCLL’s institutional change after the merger of two HEIs was seen to be a great momentum to introduce ePortfolio over an entire educational program. Therefore, the impact of the EEP project is expected to be enormous.

Most experts mentioned the huge impact of dissemination events. The events increased organizational activities and cooperation, as IPS, KU Leuven and UCLL partners noted. The MIE partner believes that the dissemination event allowed MIE to showcase what they do with ePortfolios to a wider education community. Based on the positive experiences of the other partners, the VIA expert is expecting a broad impact on the organizational level after the multiplying event in Denmark in October, which will be visited also by many representatives of VIA’s extended organization.

Some partners reflected on how their organizations could develop assessment processes. The project resulted in cohesion and more conversation at MIE about how assessment in different modules occurs and how different approaches are used. As an expected impact outlined in Proposition 4, the project impacted managements to better understand and support new assessment methods. HAMK1 experts found that the development of ePortfolios can make the old guiding and assessing methods transparent and impact on how the improvements of new assessment practices in competence-based education can be improved. At HAMK, the development of the ePortfolio process has been in line with management targets and decisions both in bachelor and professional teacher education, and the EEP-project has supported this development. HAMK1 experts also mentioned that ePortfolios can increase transdisciplinary activities and the combinability of modules and programs. HAMK2 experts, who highly appreciate their pedagogical freedom referred to HAMK’s digital orientation, which the ePortfolio process can promote. The management also expressed their intentions to widen this approach in teacher education.

Some experts highlighted unexpected impacts, various changes, trainings or events that could not have been created without the EEP project. Our MIE expert revealed that a new on-going document was created at MIE solely as a direct result of the project, in which three colleagues have been documenting how they use and assess ePortfolios, and what support is provided for students. This process has resulted in a more efficient use of time, in relation to preparing students to use portfolios, assessing portfolios and sharing a rubrics assessment methodology, which has been a great impact on the organization. At HAMK, a special group was established for people involved in using ePortfolios, to meet regularly, discuss and share their experiences and good practices. As a result of the EEP project, several teacher training programs that were organized at IPS (e.g. CNAPES) created a need for a change to engage teachers to participate in collective and supporting discussions, which can eventually yield a great impact.

As a somewhat abstract impact, our VIA partner pointed out a controversial misconception concerning the relationship of digitalization and teachers’ workload, which the EEP project helped to dissolve: “the idea that you can reduce the amount of teachers’ time spent on learning processes, seems, at least from the results that I’ve seen also from the partners, to be wrong. You spend your time in another, very interesting way.”

Challenges at the organizational level were mentioned during the interviews and highlighted in partners’ analyses. Analysis highlighted that the lack of proper conditions at the organizational level may result in teachers facing difficulties, for instance, to have impact and provide important connections between theory and practice (Choistealbha et al. 2018). Another challenge may be if teachers and students consider the creation of ePortfolio as something extra and they fail to understand the idea and broader benefits (Kunnari & Salmia 2018). Difficulties may also occur in case of resistance to change or in the absence of formal guidelines and established models for ePortfolios, pedagogical or digital skills, culture of reflective dialogue, collaborative learning and growth, shared values and practice (Pires & Rodrigues 2018). Most interviewees referred to critical steps for the successful functioning of ePortfolios. As for improvements, the KU Leuven partner emphasized the need for having all necessary ICT and educational information about ePortfolios in one-place on their web-page. The MIE partner revealed their intention to have greater support for staff to use portfolios; “So, it’s sort of identifying our strengths and weaknesses, as a result of involvement in the project.”

Impact on networking

Interviews revealed that the networking impact has already been remarkable and is expected to grow even after the project ends. The dissemination events provided a possibility to introduce the EEP project to a great educational audience, while the training weeks served to exchange ideas and increase impact through collaboration. It was mentioned that all publications of the EEP project will increase the project impacts to a wider audience, which cannot be measured now but will contribute to the networking impact of the project.

Some interviewees referred to the well-functioning partnership between the participating organizations, as a result of the EEP project. A coordinator of the project noted: “…we have managed to create a very nice, personal, interactive and collaborative network and partnership, these qualities are really important in the future. I find it easy to turn to these people and propose something new.” (HAMK1). KU Leuven and UCLL partners also pointed out that the project created an excellent, intense and fruitful collaboration between their two HEIs in Flanders.

Most partners highlighted various new relationships and cooperation nationally or internationally, which opened new discussions, topics and insights. As the HAMK partner revealed at an event organized by the funding agency, they met representatives of another funded project with whom they started to cooperate immediately. Later, other Finnish HEIs joined the network through various dissemination activities. Moreover, external writers from other Finnish HEIs decided to participate in the final publication of the project, which also increases the networking impact. The coordinators of the project find that opening the platforms can make the network operational and visible, which can also serve for future cooperation. HAMK’s Professional Excellence Research Unit is believed to be a great place to make impact as well as through HAMK’s other projects that have extended and strong national and international networks.

Some aspects of Proposition 5 have already been realized and are expected to develop further. Partners can have an impact on other HEIs within the region and even globally. Our HAMK partner noted that an ePortfolio can work as a guidance tool with which foreign partners of the university can make developments in their contexts transparent, which is one way to make the impact of the project visible to different stakeholders globally.

In order to spread its key messages, the EEP project was highlighted in certain countries through various channels or forums. As Proposition 5 of the EEP evaluation model suggested, the project can have an impact on the world of work. Measuring this impact is difficult because the project is so young. However, the above channels opened the gate for employers, local policy makers and administrators to step in and learn about this new assessment practice. Our IPS partner revealed that national and international seminars were organized in Portugal which provided possibilities for networking with other HEIs, and where the results of the EEP project were presented. At the regional level the project appeared in the media. At this year’s Employability Week of ESE-IPS, which is open to the community, the IPS partner presented the ePortfolio project to an audience of employers, students and teachers from other schools. The VIA partner referred to the importance of VIA’s special forum, where they can discuss their projects with different actors of their field; like guidance counsellors, work-life representatives, local policy makers and administrators. As for the experiences of working with ePortfolios in the Danish perspective, the work done within the EEP project is breaking new ground.

Many experts referred to the impacts of the dissemination events on their or other experts’ work, which further created various unexpected results. The MIE partner mentioned two people from Ireland who started cooperation in Finland and Belgium around portfolios after the Irish dissemination event. UCLL and KU Leuven partners noted that before and after the dissemination event in Leuven, they started new collaboration through receiving new national and international contacts and invitations to workshops. Some of the contacts also greeted the idea of a conference on ePortfolios because there had been no platform for exchanging ideas about them before. The UCLL interviewee also highlighted the UCLL’s new highly important connection with the Student Committee of the Biomedical Laboratory Technologists in Belgium, who also works with other European countries and steering committees and has projects around ePortfolios. She also mentioned the personal benefit of working and sharing ideas in English with different cultures, which she found to be an enriching experience. Based on the positive experiences in other countries, a great impact is expected from the dissemination event in Denmark, at which national actors like representatives of the eGuidance Portal will also participate. VIA also revealed their intentions to continue the pilot’s perspectives and work with the validation of prior learning more progressively and reflectively after the project ends by sharing it with other Danish HEIs.

Among somewhat unexpected impacts, the networking of the EEP project had a huge local influence in Ireland. The Irish partner revealed that the “EPortfolio Ireland” network was created as a direct result of the project. The initiative was born after the Irish dissemination event in September 2017, following a conversation around the topic. The aim of the group has been to try and encourage the sharing of practice between Irish universities and institutes that are using portfolios across disciplines. Interinstitutional research on the use of ePortfolios has also taken place and the group is presenting at conferences, where the networking of the EEP project also continues because: “when I’m talking about our use of network to ePortfolios I always reference the project, and our involvement in it.” (MIE) Furthermore, as the expert underlines, the project has further impacted on the level of the Irish steering committee of the EEP project with the nomination of Roisin Donnelly, renowned in Ireland for using portfolios. She also sits on the National Forum for Teacher Education in Ireland, which has widened the reach of the project.

Unexpected impacts of the EEP projects at the level of networking have already been detected and are expected to increase in the future, while challenges are hard to imagine at this level at present. Nevertheless, networking can greatly contribute to pinpointing areas for improvement and lead to fruitful discussions around the challenges of various stakeholders’ perspectives. Networking, therefore can be very effective in improving ePortfolio work. The challenges at this level can be related to difficulties to comprehend differences in competence-based higher education in different countries.

Conclusions

The EEP impact evaluation showed that various organizational and managerial conditions are necessary to enable the development, sharing and supporting of competences relevant for working with ePortfolios (Poulsen 2018; Devaere 2018; Choistealbha et al. 2018; Pires & Rodrigues 2018; Kunnari & Salmia 2018). In the impact evaluation, organizational and networking level improvements were recognized according to expected impacts. Partner organizations shaped new innovative models to create, support and guide ePortfolio processes in the organizational level. Experts noted more organizational cooperation and collaboration resulting from the process, which created space for further developments as well. The dissemination events were also perceived to have yielded increased activity, cooperation and the showcasing of the project to a wider education community. Furthermore, management were experienced to better understand and support new assessment practices, which can be an empowering asset at the organizational level. Challenges at this level may be connected to the lack of proper conditions for change, renovating the pedagogical practices, integrating ePortfolio in the curriculum, improving the training context, strategies and rethinking teacher’s training models or resistance to change (Choistealbha et al. 2018; Kunnari et al. 2018; Pires & Rodrigues 2018). Partners suggested the improvement of digital conditions and support for staff and students to create a well-functioning environment for ePortfolio work.

The interviews revealed numerous and versatile impacts on networking, some of which are expected to have a far-reaching influence on the future as well. The project had an impact on other HEIs regionally and globally and through various channels, it also influenced the world of work by raising awareness of the benefits of ePortfolios. The dissemination events at the partner HEIs provided the possibility to introduce the EEP project to a wide educational and work-life audience, while the training weeks contributed to increasing impact through expert collaboration. The key messages of the project were spread also through various channels; in the Portuguese media, at the Employability Week of ESE-IPS or through VIA’s special forum for discussing R&D projects.

All publications describing the results of partners’ analyses conducted during the project on different stakeholders’ perspectives will widen the reach of the impact (available at unlimited.hamk.fi).

Experts also highlighted the well-functioning partnership within the project and the establishing of new relationships and cooperation with other organizations or experts nationally and internationally. The new discussions, topics and insights that were born from these conversations contributed to the unexpected impacts of the project. Similarly, local initiatives like “Eportfolio Ireland” were created as a direct and unexpected result of the project and have increased the networking of the EEP project through continuous referencing. Through its extended routes, networking can yield countless unexpected impacts and be an effective way of discussing challenges from various stakeholders’ perspectives to improve ePortfolio work.

This article was produced in the Erasmus+ (KA2 action) funded project “Empowering Eportfolio Process (EEP)”. The beneficiary in the project is Häme University of Applied Sciences (FI) and the partners are VIA University College (DK), Katholieke Universiteit KU Leuven (BE), University College Leuven-Limburg (BE), Polytechnic Institute of Setúbal (PT) and Marino Institute of Education (IE). The project was implementated during 1.9.2016–30.11.2018.