The Careers Service of the University of Pennsylvania (2003) goes so far as to say an electronic portfolio is “essential” not only when seeking web development and similar jobs but if “you want to demonstrate your technical & design skills as well as your comfort level with web-based technologies”. They clearly recognise difference.

Access is an important difference that links process and outcome. A recognised hallmark of the mature portfolio is growth not only over a reasonable period but also in response to feedback. Web ePortfolios allow access to a global readership that may respond where there are invitations and ready ways to do so. Some commercially available platforms foreground the seeking of feedback, the evidencing that this has occurred and its further connectivity with the reflection and growth of the portfolio.

As Batson (2002) avers:

ePortfolio developers are making sure that their platforms can accept the full range of file types and content: text, graphics, video, audio, photos and animation. The manner in which student work is turned in, commented on, turned back to students, reviewed in the aggregate over a semester, and certified can be – and is being - deeply altered and unimaginably extended. (¶6)

Where, as discussed below, the audience has the capacity to respond appropriately (and, demonstrably, this is not a given) then, for students from the University of Alaska, the ePortfolio (as distinct from the portfolio) set them apart. Not only did it allow them to show ‘proof’ of their knowledge, experiences and abilities, but, as students from Southeast Missouri State University found, it also showed their command of advanced technologies and their appropriate use (ALTec, 2002).

Communication online is a different experience from reading print. While there are enriched possibilities for interaction, as through providing feedback, those using the web almost invariably have a considerably reduced tolerance for the time it takes to download and to find what they are seeking than we would, for instance, when we are seeking to locate the right book and then, through the index, the right page and the precise reference within it. Further,

Users of web documents don't just look at information, they interact with it in novel ways that have no precedents in paper document design. The graphic user interface (GUI) of a computer system comprises the interaction metaphors, images, and concepts used to convey function and meaning on the computer screen. It also includes the detailed visual characteristics of every component of the graphic interface and the functional sequence of interactions over time that produce the characteristic look and feel of Web pages and hypertext linked relations. Graphic design and visual "signature" graphics are not used simply to enliven Web pages — graphics are integral to the user's experience…. In interactive documents graphic design cannot be separated from issues of interface design. (Lynch & Horton, 2004, ¶1)

Usability heuristics (see, e.g., National Cancer Institute, n.d.) and the numerous stages of presenting an ePortfolio reinforce the point that skills of a different type and a different order are required to produce a satisfying mature electronic portfolio.

The ‘mature’ ePortfolio

As Love et al. (2004) so correctly contend, “The promise webfolios hold – a richer educational experience for all – will not be realized … unless educators embrace webfolio concepts and apply them at the highest level of maturation” (¶1).

Hence, the first step in achieving the potential of ePortfolios is to have a mature sense of what an ePortfolio is, can be and should be. As with any discrete element, it is how it is used – the context in which it is placed – that becomes its defining characteristic. If it seen as an additive, a nice extra, if it is not integrated into the whole learning process, it is highly unlikely to achieve its potential in shaping the entire approach to learning. As Tosh and Werdmuller (2004) recognise, “One thing is certain: if ePortfolios are viewed as a new, separate entity as opposed to an integrated part of one’s learning experience, they may never fulfil their potential, becoming a tool that alters learning pedagogy” (p. 9)

Love et al. (2004) provide useful tables and discussion about their designated five levels of maturation: scrapbook, curriculum vitae, curriculum collaboration between student and faculty, mentoring leading to mastery and authentic evidence as the authoritative evidence for assessment, evaluation and reporting. Their work has the strength of setting the ePortfolio in its context, an issue that should be stressed. Their characterising of the mature portfolio (level 5) is a useful illustration of how essential it is to integrate it with the institution and, where appropriate, the wider community.

At level 5, webfolios are organized by curricular requirements and electives or by standards established by a cadre of educators or the institution. Students generate portals for displaying work samples and achievements within the same curricular structure or institutional standard. In addition, educators link standards, departmental goals, and other descriptors – such as higher-order thinking taxonomies – to specific webfolio items, including student-generated work samples and achievements. Level 5 webfolios allow multiple opportunities for students to receive feedback from mentors and educators and to redeem their work. (pp. 9–10)

What, then, is required for the actual ePortfolio? The following summary (see Table 1) has been drawn from the literature (see, for example, Barrett, 1999, 2000; and Vandervelde, 2004) and from experience.

The shaded section of the table deals specifically with electronic portfolios. While the unshaded section of the table relates to all portfolios, the functionality of an ePortfolio enhances the ability to select and edit material, and to interact with others. The interrelated nature of a mature ePortfolio is facilitated digitally, especially through the web, for as a technology and an interface, there is an ease in assembling, (re)organising and integrating narrative captions as links (Bitter & Pierson, 2005).

A primitive ePortfolio is simply a collection of artefacts that has no coherence beyond the fact that the person responsible for the collection considered they were worth archiving. Stone (1998) recognises that the methods of selecting items in terms of their relevance is an important part of the development of the portfolio and the use of databases allows efficient sorting and clustering as well as facilitating comparison over time. It is, as Wiske (2005) reminds us, inherently linked with reflection: “Usually, reflection requires representing experience in some way, often with language but perhaps nonlinguistically, with images or video, or in music or dance, or with gesture. Representing experience entails selecting aspects of experience, relating them, and expressing them…” (p. 104).

The concept of the portfolio as a ‘reflective tool’ has been clearly identified (see, e.g., Barrett, 1999, 2000; Paulson, Paulson, & Meyer, 1991). A mature ePortfolio offers ways of making meaning from information through reflection ( Cambridge, quoted in LDP, 2003–2004). In its ePortfolio Report (LDP, 2003–2004) the University of Berkeley concluded that it was the practice of reflection (with social construction) that represented the “key rationale for implementing ePortfolios for students and specifically address(ed) the manner in which the information age has changed human ecology”. This report quoted an AAHE self-reported survey of 51 universities and colleges that currently implement ePortfolios and found reflection to be the highest primary purpose listed (36%).

Table 1: A checklist for a mature ePortfolio

While reflection is a recognised attribute of the mature ePortfolio, what is more challenging for educators, is how that reflection is fostered and recognised. As Brookfield (1995) writes, “Waxing rhapsodic about the benefits of critically reflective teaching is of limited use unless we have a specific focus on how it actually happens” (p. 28). While his context is that of teaching, his contention is apposite here. Wiske (2005, p. 104) usefully defines reflection as a process of standing back from experience and examining it in ways that generate meaningful knowledge. While this can be accomplished alone, input from peers and mentors is likely to enhance the reflection process. In this regard digital portfolios can be more readily and widely accessed than their paper-based counterparts.

Key requirements for achieving the potential of ePortfolios

If ePortfolios are to achieve their potential in the higher education context they require skilful integration with curriculum, for students, with review and promotion for staff, and with personal growth and career aspirations for both. How difficult this is in reality can be quickly demonstrated by considering one aspect for students and one for staff.

For students

If we accept that the mature ePortfolio evolves, that it will be refined and reshaped in response to growth and opportunity and for specific purposes, then restricting it, as so frequently is the case, to a few opportunities added to one subject/unit for one semester for a few students presents it in a basic, immature sense. To take it beyond this requires considerable change. First, the curriculum (probably including assessment as this is the known driver for student participation) needs to incorporate portfolios in a purposeful way. Second, students need to see this as part of their holistic growth, so some connectivity with portfolios should be made across their entire program. This requires understanding of what is involved by much more than one isolated enthusiast within a Faculty: it requires a Faculty, or at least a recognised area, to embrace the concept and its values. As Love and Cooper (2004) contend, “Portfolios are complex educational tools that require full integration into carefully crafted course designs” (p. 66). But, even if this happened, it would not be enough when we are considering ePortfolios. Our experience (Challis et al., 2003) has shown that students will not engage with ePortfolios unless they have some confidence that the system they are using will be accessible in the long-term, and that includes when they have left the institution. If they are to fully engage, they (and the same is true for staff) want to be able to access their portfolio at other times, in other places and for different purposes. While they can archive it by using CD-ROM or DVD, if they are accustomed to the functionality of a specific platform this is a major deterrent to uptake beyond that which is enforced. In this regard web-based portfolios promote seamless access by eliminating software and platform incompatibilities when ePortfolios are created with multiple authoring tools (Bitter & Pierson, 2005).

If the portfolio is to have the characteristics delineated above, those developing and reviewing them need to have an informed base. Yet it is a common complaint (see, e.g., Ramsden, 2003) that there is insufficient time to carry through all the complex duties and demands of a lecturer without adding ‘training’ in the pedagogy of portfolios and the related software. There is also common complaint that there is not enough time within the semester to teach the perceived core elements of the discipline. In practice, then, taking time to learn about and then explain the dimensions of a mature portfolio, carefully integrate it into a reconsidered and accredited curriculum, then provide formative feedback as well as possibly summative assessment are all unlikely to happen unless there is a fundamental commitment and appreciation of what the ePortfolio will contribute to the students’ learning and, probably, career opportunities. If it happens without this very real underpinning it will almost certainly not have the outcomes that are desired and so, in turn, reaction may change from a neutral, possibly positive stance by both staff and students to negativity. In this regard, Batson (2002) referred to the experience of one unidentified campus where 100 per cent of the staff teaching a program adopted ePortfolios but “students may not see the value because the faculty have not re-thought their courses to accommodate electronic portfolios.” He concluded that “unless they do, the standards initiative in that program may be undermined.”

For staff

Turning to staff, our experience when portfolios were introduced to the University as part of the promotions process revealed variable understanding by those compiling and those reviewing of what was involved and differing views of what was desirable. Most staff have no experience of compiling a portfolio. In some disciplines, and for some cultures, the practice of reflection that is so fundamental is quite alien. Our experience (Challis, 2003) suggests that portfolio writing tends to suit those involved in the more discursive disciplines and who have developed skills in English language writing. Of special note was the fact that portfolios that revealed insight into less-than-ideal situations, even when they were accompanied by carefully thought through strategies that clearly evidenced growth and improved student feedback, were dismissed as “not up to it” as they showed weakness. At this stage conventional paper-based portfolios were what was anticipated and provided and it was of concern to observe that in one instance a CD-ROM that accompanied the portfolio was readily dismissed. This is similar to an anecdotal report posted on the web:

The first question that faculty ask about e-portfolios—as we found in recent workshops—is “how do I print this?” Promotion and Tenure committees don’t take electronic submissions … (Michelle’s weblog, 2004).

In a similar context, using ePortfolios when seeking employment, the University of California-Riverside’s Graduate School of Education found that the barrier at the moment was “that people aren’t really ready to do it. The process just doesn’t seem to allow much time for viewing the contents of ePortfolios” (Reilly, quoted in Sloan Consortium, 2003). Although there is demonstrated adoption of ePortfolios, as with California moving towards this model for granting teaching credentials, Reilly (2003) claims that his experiences indicate that “considerable preparation and changes in existing practices in credential programs will be required if [this] is to develop into a tool for authentic assessment” with the most important barriers to change being not “the new technologies, but the old practices”.

Further, despite encouragement to participate in an early trial of ePortfolios, not one staff member at Deakin was prepared to make what they considered an additional effort to work in a different environment, especially as it was improbable they could access anything they completed after the trial was ended. Where staff have a considerable collection of paper-based material, even if they are accustomed to using computers for word-processing, communication and routinely search the web, it is a significant mind shift to work electronically in terms of capturing evidence and most have neither the skills nor interest to digitise such material. Even where portfolios were carefully explained and discussed, integrated into a program and sensitively reviewed within a negotiated space, as we found when they were part of our University’s Induction/Orientation program from 2001–2003, this did not ensure success. The pressures of normative academic work can quite readily become a rationalisation for failure to devote time to portfolios. If a significant number of new staff did not complete portfolios when they were mandated (8 of 45 in 2001, 9 of 42 in 2002 and 6 of 33 in 2003, the final year this program was offered), and many others completed the requirement in a minimalist way that would not satisfy the requirements of a mature portfolio identified above, how less likely is committed involvement when this is entirely optional, when there is little security in how the portfolio will be reviewed, where there is uncertainty of purpose and outcome, and when the additional layers of an electronic portfolio are introduced?

Technical issues

Vendors of related software, as well as those who market ePortfolio platforms, understandably privilege access to the technology, stressing the ease with which items can be incorporated within an audio and video enhanced environment. This is an important element. If the tool is not reasonably intuitive and readily available it is far less likely to be accessed. Moreover, it is the incorporation of such multimedia devices that enables the richness of the artefact. The problem occurs when the technology(ies), the tool(s) become the entirety of the exercise and concern with the presentation, rather than the process, the gloss and the gimmick, rather than the content, dominates the attention of the person developing and producing the portfolio and the person reviewing it. Unless the person preparing the portfolio understands the need to use the captured data in a meaningful way and can provide the thoughtful interpretation and contextualisation, the result will inevitably be disappointing.

If ePortfolios are seen in their broadest context and as a life-long activity the situation becomes more fraught. Managing that volume of data, ensuring accessibility coupled with standard integration are essential issues to address. There are also issues of security and related ones of certification/verification of data as well as copyright/intellectual property. Institutions, such as universities, are justifiably concerned by what visual images and so on are posted on Internet sites and privacy issues are important considerations. In a world of hackers and litigation these are not to be dismissed lightly. It is of note that the fifth annual EDUCAUSE survey of current IT issues (Spicer et al., 2004) found that security and identity management not only remained in the top-ten issues for all measures (strategic importance, growing in significance, demanding the campus IT leader’s time and expenditure of human and fiscal resources) but, with terrorism, viruses and denial of service attacks it had “risen dramatically in the last (resource consumption) category—from eighth in 2003 to third in 2004”.

Conclusion

EPortfolios have much to offer the higher education sector as the summary of their possible roles provided at the start of this paper indicates. It is readily apparent from such key indicators as the plethora of websites devoted to the subject, vendor commitment, documented initiatives by governments and within the higher education sector, a developing academic literature and even conferences devoted to the subject, that there is considerable interest in ePortfolios. It is, however, a very different thing to find substantive material about the actual use of ePortfolios in the higher education sector in a mature and systematic way. In this regard, ePortfolios are not alone. Maddux and Cummings (2004), for instance, expose and evidence the weak role of research and theory in information technology in education. They quote Grossen who claims that “research currently plays an extraordinarily weak role in educational decision-making” and that “cited research often is not research at all, but merely published opinion” (pp. 515, 517). This seems representative of much of the literature of the higher education sector concerned with ePortfolios. Carney’s (2004) annotated bibliography of the empirical electronic portfolio literature is, without exception, restricted to K-12 and teacher education and in her most recent publication (2005) she restates concerns regarding the paucity of rigorous research on the efficacy of portfolios. Maddux and Cummings (2004) consider that the jury is still out regarding the fate of electronic portfolios which, like other ‘fads’, are susceptible to disrepute and abandonment. Carney (2005), who also quotes Maddux and Cummings, concludes similarly:

Electronic portfolios show promise for enhancing learning, but if we fail to critically evaluate our uses of the device, we may find that they will go the way of Papert’s Logo turtles and become yet one more educational fad – an innovation poorly understood and often implemented in ways contrary to its theoretical underpinnings. (p. 4)

From what research there is, Love and Cooper (2004) conclude that “most online portfolio systems fall significantly short of their potential” (p. 65). They found in the majority of instances of online portfolios they reviewed that they consisted “only of a single essay, project report or term paper presented as a web-based electronic facsimile of a conventional document” (p. 78) – a far cry from a mature ePortfolio as delineated here. While more examples of higher education use are being disseminated each quarter (see, e.g., EDUCAUSE, 2005), as Murphy (2003) points out, “The operative word here is ‘potential’ as e-portfolios are still in their infancy and have not yet proven to have a transformational impact on higher education. In fact, they are in very limited use on …campuses” ( ¶ 5)

However, as Greenberg (2004), for instance, reminds us, entry tertiary students come with experience with ongoing, self-directed learning. Moreover, they are accustomed to communicating and interacting online, using basic applications programs and multimedia. Increasingly they come from schools where electronic portfolios are part of their learning and personal development. Hence, as Greenberg suggests, ePortfolios may well provide a starting point for the type of learning communities that higher education needs to offer its future students.

Although, as Bates and Poole (2003) argue, technology should not drive our teaching, it does drive change as we rethink pedagogy and achieve previously unreachable learning goals. However, as they caution, “usually with technology it is three steps forward and two steps back” (p. 282). EPortfolios are an exciting possibility, but they are a possibility, not yet a proven product. There is clearly a need for rigorous research and also for that research to be translated into practice. There is a need to recognise and deal with the complexity of disparate elements as ePortfolios are integrated into curriculum and culture. Fundamentally, those who are concerned with the education of tertiary students need to be informed about what ePortfolios entail for students, staff and the institution before major commitments of any sort are made. Educators need to be open to the promise but aware of the implications for their adoption.

References

ALTec: The Advanced Learning Technologies - Center for Research on Learning at the University of Kansas (2002). Kick It Up A Notch: Taking professional portfolios from Black-and-White to 3-D Color. Preparing Tomorrow’s Teachers to Use Technology. Retrieved May 21, 2004 from http://www.pt3.org/stories/eportfolio.html

Arter, J., & Spandel, V. (1992). Using portfolios of student work in instruction and assessment. Educational Measurement: Issues and Practice, Spring, 36–44.

Baird, D. (2003). ePortfolios: A Pocket Full of Ambition. Syllabus: Technology for Higher Education. Retrieved May 17, 2004 from http://www.syllabus.com/article.asp?id=7478

Barrett, H. (1999, 2000). The Electronic Portfolio Development Process. Retrieved April 15, 2004 from http://electronicportfolios.org/portfolios/EPDevProcess.html

Barrett, H. (2005). White Paper. Research Electronic Portfolios and Learner Engagement. Retrieved September 17, 2005 from www.taskstream.com/reflect/whitepaper.pdf

Bates, A., & Poole, G. (2003). Effective Teaching with Technology in Higher Education. San Francisco: Jossey-Bass.

Batson, T. (2002). The Electronic Portfolio Boom: What’s it All About? Syllabus: Technology for Higher Education. Retrieved May 17, 2004 from http://www.syllabus.com/article.asp?id=6984

Bitter, G. & Pierson, M. (2005). Using Technology in the Classroom (6th ed.). Boston: Pearson.

Brookfield, S. (1995). Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass.

Careers Services, University of Pennsylvania. (2003). Retrieved May 24, 2004 from http://www.vpul.upenn.edu/careerservices/college/electronic_portfolio.html

Carney, J. (2001). Electronic and Traditional Portfolios as Tools for Teacher Knowledge Representation. Unpublished doctoral dissertation, University of Washington, Seattle, WA.

Carney, J. (2004). An Annotated Bibliography of Empirical Electronic Portfolio Literature. Retrieved September 2, 2004 from it.wce.wwu.edu/carney/Presentations/AERA04/annotatedbib.pdf

Carney, J. (2005). What Kind of Electronic Portfolio Research Do We Need? Retrieved September 2, 2004 from it.wce.wwu.edu/carney/Presentations/SITE05/ResearchWeNeed.pdf

Challis, D. (2003). Recording the past to prepare for the future: the role of the teaching portfolio. In C. Bond & and P. Bright (Eds.), Research and Development in Higher Education: Learning for an Unknown Future (pp. 92–99). HERDSA, ACT.

Challis, D. et al. (2003). Professional Learning Portfolio (PLP). Internal report from the Working Party to the Joint Committee on Teaching & Learning, Deakin University, Australia.

CRESST: National Center for Research on Evaluation, Standards, and Student Testing. (n.d.). Creating and using Portfolios on the Alphabet Superhighway. The Alphabet Superhighway. Retrieved May 17, 2004 from http://www.ash.udel.edu/ash/teacher/portfolio.html

EDUCAUSE. (2005). E-Portfolios. Retrieved September 19, 2005 from http://www.educause.edu/E%2DPortfolios/5524

Fitch, K. (2004). Spinning Webfolios: Online Portfolios for Learning, Teaching, and Assessment. Retrieved May 12, 2004, from http://www.kafkaz.net/SpinningWebfolios2_files/v3_document.htm

Greenberg, G. (2004). The Digital Convergence: Extending the Portfolio Model. EDUCAUSE Review, 39(4), 28–37. Retrieved August 22, 2005 from http://www.educause.edu/apps/er/erm04/erm0441.asp?bhcp=1

LDP: Leadership Development Program 2003-2004. (2003-2004). LPD e-Portfolio Report. Regents of the University of California. Retrieved May 12, 2004 from http://bearlink.berkeley.edu/ePortfolio/html

Love, D., McKean, G. & Gathercoal, P. (2004). Portfolios to Webfolios and Beyond: Levels of Maturation. EDUCAUSE Quarterly, 27(2), 1–14. Retrieved May 10, 2004 from http://www.educause.edu/pub/eq/eqm04/eqm0423.asp?print=yes”

Love, T. & Cooper, T. (2004). Designing Online Information Systems for Portfolio-Based Assessment: Design Criteria and Heuristics. Journal of Information Technology Education, 3, 66–81.

Lynch, P. & Horton, S. (2004). Web Style Guide (2nd ed.). Retrieved 23 May, 2004 from http://www.webstyleguide.com/interface

Maddux, C. & Cummings, R. (2004). Fad, Fashion, and the Weak Role of Theory and Research in Information Technology in Education. Journal of Technology and Teacher Education, 12(4), 511–533.

Michelle. (2004). Michelle’s Online Learning Freakout Party Zone: Learning technology, with a particular focus on resources for the LT initiatives at UBC. Retrieved May 15, 2004 from http://careo.elearning.ubc.ca/weblogs/michelle/

Murphy, P. (2003). E-portfolios: Collections of Student Work Move from Paper to Pixels. TLTC News, University of California. Retrieved April 21, 2004 from http://www.uctltc.org/news/2003/02/feature.html

National Cancer Institute. (n.d.). Research-Based Web Design & Usability Guidelines. Retrieved May 24, 2004 from http://usability.gov/guidelines/

Paulson, F., Paulson, P. & Meyer, C. (1991). What makes a portfolio a portfolio? Educational Leadership, 58(5), 60–63.

Ramsden, P. (2003). Learning to Teach in Higher Education (2nd ed.). London: Routledge Falmer.

Reilly, B. (2003). Electronic Portfolios in the UCR Graduation School of Education Teaching Credential Program. TLTC News, University of California. Retrieved May 5, 2004 from http://www.uctltc.org/news/2003/02/reilly.html

Sloan Consortium of Institutions and Organizations Committed to Quality OnLine Education. (2003). Electronic Portfolios in a Teaching Credential Program. Retrieved May 21, 2004 from http://www.aln.org/effective/details2.asp?ACC_ID=46

Spicer, D., DeBlois, P. & the EDUCAUSE Current Issues Committee. (2004). Fifth Annual EDUCAUSE Survey Identified Current IT Issues. Current Issues,27(2). Retrieved May 21, 2004 from http://www.educause.edu/ pub/eq/eqm04/eqm0422.asp

Stone, B.A. (1998). Problems, pitfalls, and benefits of portfolios. Teacher Education Quarterly, 25(1), 105–114.

Tosh, D. & Werdmuller, B. (2004). ePortfolios and weblogs: one vision for ePortfolio development. Retrieved May 17, 2004 from http://www.eradc.org/papers/ePortfolio_Weblog.pdf

Vandervelde, J.M. (2004). A+ rubric: Rubric for electronic portfolio. Retrieved July 18, 2005 from http://www.uwstout.edu/soe/profdev/eportfoliorubric.html

Wiske, M.A. (2005). Teaching for Understanding with Technology. San Francisco: Jossey-Bass.

© Canadian Journal of Learning and Technology

ISSN: 1499-6685