Key Performance Indicators for OER

One of the things I’ll be looking into as part of my new role is key performance indicators for open educational resources.  At the University of Edinburgh we have a Vision and Policy for OER that encourages staff and students to use, create and publish OERs to enhance the quality of the student experience, enrich the University and the sector, showcase the highest quality learning and teaching, and make a significant collection of unique learning materials available to Scotland and the world.

Staff and students at the university are already making open educational resources available through a range of channels including Open.Ed, Media Hopper, TES, SketchFab, Youtube, Wikimedia Commons and Wikipedia, and there are a number of initiatives ongoing that promote and support the creation of OER including 23Things, Board Game Jam, various MOOC projects, our Wikimedian in Residence programme and others.

So how do we develop meaningful key performance indicators to measure and assess the success of these initiatives?

Quantitative indicators are relatively simple to measure in terms of OER produced. It’s not difficult to gather web stats for page views and downloads from the various platforms used to host and disseminate our OERs.  For example our open educational resources on TES have been viewed over 2,000 times, and downloaded 934 times, a Wikipedia article on Mary Susan MacIntosh, created during a UoE editathon for International Women’s Day has had 9,030 page views, and UoE MOOCs have reached two and a quarter million learners.

Measuring OER reuse, even within the institution, is much less straightforward.  To get an of idea of where and how OERs are being reused you need to track the resources. This isn’t necessarily difficult to do, Cetis did some research on technical approaches for OER tracking during the UKOER Programme, but it does raise some interesting ethical issues.  We also discovered during our UKOER research that once authors create OER and release them into the wild, they tend not to be motivated to collect data on their reuse, even when actively encouraged to do so.

There is also the issue of what actually constitutes re-use.  Often reuse isn’t as straightforward as taking an OER, adapting is and incorporating it into your course materials.  Reuse is often more subtle than that.  For example, if you are inspired by an idea, a concept or an activity you ome across in an OER, but you don’t actually download and use the resource itself, does that constitute reuse?  And if it does, how do we create KPIs to measure such reuse?  Can it even be measured in a meaningful way?

And then there’s the issue of qualitative indicators and measuring impact.  How do we assess whether our OERs really are enhancing the quality of the student experience and enriching the University and the sector?  One way to gather qualitative information is to go out and talk to people and we already have some great testimonies from UoE students who have engaged with UoE OER internships and Wikimedia in the Classroom projects. Another way to measure impact is to look beyond the institution, so for example 23 Things lornwas awarded the LILAC Credo Digital Literacy Award 2017 and has also been adapted and adopted by the Scottish Social Services Council, and the aforementioned article on Mary Susan McIntosh featured on the front page of English Wikipedia.

I know many other institutions and organisations have grappled with the issue of how to measure the impact of open education and OER.  In the US, where OER often equates to open textbooks, the focus tends to be on cost savings for students, however this is not a particularly useful measure in UK HE where course are less reliant on astronomically priced texbooks.  So what indicators can we use to measure OER performance?  I’d be really interested to hear how other people have approached this challenge, so if you have any comments or suggestions please do let me know.  Thanks!

Standard Measures, CC BY SA 2.0, Neil Cummings, https://flic.kr/p/aH8CPV

New Role

The view from the office isn’t bad…, CC BY, Lorna M. Campbell

Now that I’ve cleared the inevitable post-holiday e-mail backlog I’m ready to start a new role as Senior Service Manager – Learning Technology within Education Design and Engagement (EDE) at the University of Edinburgh. I’ve actually been transitioning into this role for several months now and have been working closely with colleagues in EDE, where the OER Service is based, for some time now.  For the last year I’ve been working as OER Liaison – Open Scotland in the Learning Teaching and Web directorate within Information Services.  This role involved co-chairing the OER 16 Open Culture Conference together with Melissa Highton, promoting the Open Scotland initiative and the Scottish Open Education Declaration, disseminating the University of Edinburgh’s open education activities, working with our Wikimedian in Residence, and liaising with other international open education initiatives and organisations.  Open education will still be central to my new role but I’ll be more focused on embedding open education and OER within the university.  In addition to continuing with some of my existing activities I’ll be working more closely with the OER Service and getting more involved in supporting institutional programmes and initiatives and liaising with other departments within the University, such as the Institute of Academic Development, to ensure that openness and OER are embedded across the institution.  I’ll also be wrapping up the two IS Innovation Fund Projects that I’ve been managing recently and disseminating their outputs.  And of course I’ll still be actively involved with the Association for Learning Technology, Wikimedia UK, and the Open Knowledge Open Education Working Group.  I’ll continue to share my experiences here on my blog and on twitter, so watch this space!

 

How is it almost August?

This is another of those blog posts that starts “Where the hell have the last two months gone?!”  I’ve been sorely neglecting this blog since early May, not because I’ve got nothing to write about, quite the opposite, I’ve been so busy I haven’t had a chance to get near it!  I’m about to go off on annual leave for a couple of weeks but I wanted to post a quick round up of the last two months before I go, so here’s wot I have been up to.

Innovation Projects

UoE OKN, CC BY Natalie Lankester-Carthy

A lot of my time has been tied up with two Information Services Innovation Fund projects.  The UoE Open Knowledge Network was a small project that aimed at drawing together the University’s activities in the area of Open Data, Open Access, Open Education, Open Research, Open Collections and Archives, to support cross-fertilisation and promote the institution’s activities in these areas. We ran three events, with the last one taking place in early July.  This event focussed on discussing priorities, ideas for the future and how we can sustain the network going forward.  You can read about the first two events on the project blog here: UoE Open Knowledge Network and I’ll be writing up the July event when I get back from leave in August.

The aim of the second project was to develop a MOOC for entrepreneurs, creative individuals, and SMEs to help them develop the knowledge and skills to find and access free and open licensed research, data and content produced by universities and higher education. I was lucky enough to recruit Morna Simpson of Geek Girl Scotland to work on the project however despite our best efforts and an incredible amount of work on Morna’s part the project faced a number of challenges which we struggled to overcome.  Rather than go ahead with a MOOC we will be releasing a series of twelve case studies on the theme of Innovating with Open Knowledge demonstrating how individuals and organisations can access and use the open outputs of University of Edinburgh research.  These case studies should be finished by early August so watch this space!

Media Hopper Replay

The University of Edinburgh is in the process of rolling out a new state of the art lecture recoding service, Media Hopper Replay, which will see 400 rooms enabled to deliver lecture recording by 2019.  As part of a training programme for staff, my colleague Charlie Farley and I have been developing training sessions on preparing for lecture recording covering accessible presentation design, copyright basics, and using open educational resources.

ALT

City of Glasgow College, CC BY Lorna M. Campbell

I was honoured to be invited by ALT to join the selection panel for the prestigious Learning Technologist of the Year Awards.  The quality and diversity of the entries was really inspiring and while I thoroughly enjoyed reading all the entries it wasn’t easy to pick the best from such a strong field.  The winners of the awards will be announced at the ALT Annual Conference which this year takes place at the University of Liverpool.  I’ll be there rejoining my old partner in crime Richard Goodman to provide social media coverage of the conference for the third year running.

In June I also helped to organise ALT Scotland’s annual conference which focused on sharing strategy, practice and policy in learning technology.  We had really interesting talks on lecture recording policy and practice from the Universities of Edinburgh and Glasgow and Joe Wilson reported back from two European open education policy events he recently attended on behalf of Open Scotland.  The real star of the show however was City of Glasgow College’s new state of the art campus where the event took place.

Celtic Knot Conference

In early July I was busy helping UoE’s Wikimedian in Residence, Ewan McAndrew, plan the University of Edinburgh / Wikimedia UK Celtic Knot Conference.  The conference showcased innovative approaches to open education, open knowledge and open data to support and grow Celtic and Indigenous language communities, and explore how our cultural heritage can be preserved as living languages.  The conference was attended by delegates from all over Europe and was an enormous success.  It was a real privilege to be involved in this event and as a Gael, I found the conference to be both moving and inspiring.  I may have got a little starry eyed listening to delegates talking animatedly in Gaelic, Welsh, Breton, Basque and too many other languages to mention.  And as an indication of the collaborative and supportive nature of the event, it was great to see all 50+ delegates come together to provide input and advice to Wikimedia Norge on how to support Sami language Wikipedia.

 

Wikimedia UK

Last weekend I was at the Wikimedia UK AGM and Board Meeting in London where it was a real pleasure to see Josie Fraser voted in as new chair of the Wikimedia Board and our very own UoE Wikimedia in Residence Ewan McAndrew awarded a very well deserved joint Wikimedian of the Year award together with Kelly Foster.  It was also great to hear that Sara Thomas has been appointed as the new Wikimedian in Residence at the Scottish Libraries and Information Council.

CMALT

And on top of all that I somehow managed to submit my CMALT portfolio at the end of May! Although it was a lot of hard work, and although I went right to wire (of course), I actually enjoyed the process of putting my portfolio together and I found it really useful to step back and reflect on my experience of working as a learning technologist in the broadest sense of the word. I would still like to write a proper post reflecting on my experience of developing my portfolio in the open but that will have to wait until the autumn.

That’s just a few of the things that have been taking up most of my time over the last couple of months.  I’m now off for a fortnight’s holiday during which we are going to attempt to coax our aged VW van to take us all the way to Brittany.  If we make it to the Borders we’ll be lucky!   I’ll be back in early August with a new role at the University of Edinburgh as Learning Technology Team Leader in the Department of Education Development and Engagement.

Mary Susan McIntosh and the Women in Red


I was chuffed to discover today that English Wikipedia’s main page features a link to sociologist, feminist, and campaigner for lesbian and gay rights Mary Susan McIntosh.  It’s always great to see women featured on Wikipedia’s main page, which is viewed by around 4 million people, but I confess to being doubly pleased because I created the article on Mary at a recent editathon to mark International Women’s Day here at the University of Edinburgh.  This editathon was facilitated by Ewan McAndrew, Wikimedian in Residence here at the University, and Ewan was also instrumental in nominating Mary to appear on the main page.

Wikipedia 11 May 2017

Only last week I had been complaining on twitter about the lack of gender balance on English Wikipedia’s main page which happened to feature 18 named men but only 4 named women that particular day. The main page changes on a daily basis but you can see the edition from 4th May on archive.org here.

WikiProject Women in Read

Of course this is not particularly surprising; Wikipedia has a well known problem with gender imbalance, only 16% of biographical entries on the English Wikipedia are of women, and the main page is a pretty accurate reflection of this imbalance.  The Wikimedia Foundation and the various Wikimedia chapters around the world, including Wikimedia UK, are well aware of this problem and are attempting to address it through a range of projects and initiatives.  WikiProject Women in Red raises awareness of this issue and aims to turn red links blue, by creating new biographical articles about women who are referenced on Wikipedia but who do not have their own pages. And here at the University of Edinburgh, one of the objectives of our Wikimedian in Residence is to encourage more women to get involved with editing Wikimedia.  Ewan regularly runs editathons focused on addressing the coverage of articles about women in general and Scottish women in particular.

Before I went along to the International Women’s Day editathon, I confess knew nothing about Mary Susan McIntosh, I picked her name at random from a list of “Women in Red” because she sounded interesting.  It didn’t take me long to realise what a hugely significant and influential woman Mary was.  In addition to being one of the early members of the UK Gay Liberation Front, and sitting on the committee that lowered the homosexual age of consent in the UK from 21 to 18, Mary published important research arguing that homosexuality should be regarded as a social construct, rather than a psychiatric or clinical pathology.  Mary’s paper The Homosexual Role helped to shape the concept of social constructionism, later developed by Michel Foucault.  Mary’s contribution to shaping this important philosophical construct has of course been largely overlooked.  My Wikipedia article barely scrapes the surface of Mary’s life and academic career and her important contribution to social theory and political activism.  I hope to do a bit more work on Mary’s Wikipedia page sometime in the future but it would be great if there are any philosophers, sociologists or critical theorists out there that could help with editing to ensure that Mary gets the recognition she deserves.

“What do you do?” – Starting out on CMALT

“So what do you do?” can be a bit of a difficult question to answer when you work in the domain of learning technology.  And depending on which area of learning technology you work in it can be a harder question to answer for some than others.  My default answer tends to be “I work at a University” followed by “I work in education technology”, often with the added explanation “It’s about the use of new technology in education.”  “Open education” tends to get you blank looks outwith academia (now there’s a topic for discussion), and thank god I don’t work in “education technology interoperability standards” these days.

My family have defaulted to telling people that I’m a spy on the basis that they don’t actually know what I do, other than travel a lot and disappear for days at a time. It’s hard to argue with them tbh.

Lorna Campbell – Spy

Sometimes I think it’s easier to explain what I don’t do; I don’t teach, I don’t do formal academic research, I’m not a programmer, I don’t develop or implement systems, I don’t provide help desk services, I don’t run the VLE.   I do manage projects and provide advice to colleagues. I provide input to policies. I support networks and disseminate practice.  I write a lot, talk a lot and present a lot.  I facilitate events and chair conferences.  I sit on boards, steering groups and executive committees. Maybe it is easier to tell people I’m a spy.

The reason I’ve been thinking about this is that after months of procrastination, I’m finally making a start on my CMALT application.  I had hoped to do this towards the end of last year but two new projects took precedence, so CMALT went on the back burner.

I had mixed feelings about CMALT for a number of years, primarily because for a long time I didn’t really seem to fit any recognisable definition of what a learning technologist is.  I tried to explain this anxiety in a blog post I wrote in 2014 Thoughts on ALT’s CPD Rebooted.  That post also refers to a brilliant piece written by Amber Thomas Perhaps I’m not one,  which I identified with strongly at the time, and still do.  The main point I was trying to make in CPD Rebooted was that formal certification can be difficult for people whose roles don’t neatly fit into the kind of boxes that make up accreditation frameworks.   This is doubly true for those on short term contracts, who have to jump from project to project and rarely have much time for formal CPD.  I ended that blog post with a question I asked on twitter:

Things have changed a lot for me since 2014, both professionally and personally.  Our understanding of what it means to be a learning technologist has matured and become more inclusive, and although contracts in higher education have become increasingly precarious, I’m very lucky that my own employment situation is more secure than it was three years ago.  In fact I’m incredibly fortunate to work for an institution that not only allows dedicated time for CPD but that also actively promotes and supports CMALT membership. Information Services at the University of Edinburgh offer bursaries to enable learning technologists to become Certified Members of ALT and my colleague Susan Grieg supports colleagues to help them prepare their portfolios.

Having spent the day pouring over the CMALT guidelines I can see that ALT have worked hard to create an accreditation framework that is as broad as it is inclusive.  However I’m still sitting here sifting through projects, webinars, presentations, papers, twitter conversations and reflective blog posts wondering how the hell I’m going to fit all this into that. How on earth can I demonstrate an “understanding of my target learners” when I don’t actually teach?  Of course the answer is that I’m going to have to think creatively.  I may not have a teaching role, but hopefully all those webinars and talks and blog posts do help my peers and colleagues to learn and to develop their professional practice.  I’m still at the stage where I’m struggling to fit my experience into the CMALT framework, but hopefully if I keep thinking about it and reflecting on what I actually do, it will all start to fall into place.  Having access to the CMALT Portfolio Open Register is already proving to be enormously helpful but I’d be very interested to hear how others have approached this.

Organising my CMALT portfolio like

(Belatedly realising I have no idea how to licence memes….)

Crossing the Field Boundaries: Open Science, Open Data & Open Education

Last week I was invited to speak at the International Open Science Conference in Berlin which this year had a special focus on OER.  My talk featured a case study of the University of Edinburgh’s Geosicence Outreach and Engagement Course so I’d like to thank Colin Graham and all those involved in the course for allowing me to present their inspirational work.

This talk focuses on the interface between OER, open data and open science and our experience at the University of Edinburgh of promoting open education through the School of GeoSciences Outreach and Engagement course.

The title of this paper, “Crossing the field boundaries”, comes not from the domain of GeoScicences though, but from Maryam Mirzakhani, professor of mathematics at Stanford University and the first female winner of the Fields Medal.  In a 2014 interview Maryam said

“I like crossing the imaginary boundaries people set up between different fields—it’s very refreshing. There are lots of tools, and you don’t know which one would work. It’s about being optimistic and trying to connect things.”

A Tenacious Explorer of Abstract Surfaces, Quanta Magazine, August 2014

I am not a mathematician, or a scientist, but I do have some experience of crossing field boundaries, and since open education is all about breaking down boundaries and cutting across fields, this seems like a nice metaphor to hang this talk on. Continue reading

23 Things: Thing 5 Diversity

“A lot of communication online is now via the mode of emoji/emoticon images. Traditionally these have been displayed as a yellow standard, but recent releases of more diverse emoji choices have raised a number of conversations. Read the two articles on reactions to the Apple and Facebook release of diverse emoji/emoticons in 2015 and 2016. Now consider the emoji alternative Bitmoji
Thing 5 

To be honest I’m not big on avatars and emjois.  I used the same twitter avatar (a rather fetching picture of the back of my head) for nine years and only got round to changing it a couple of months ago :}  I also don’t use emojis very often so I’ve never really given much thought to who they may or may not represent.  Now I stop and think about it though, that lack of regard is a clear reflection of my own position of privilege.  I may not use emojis, but if I ever wanted to, it wouldn’t be difficult to find plenty that would broadly represent me.  So the article about the furore surrounding Apple’s multicultural icons certainly gave me pause for thought.  It also made me think of the recent news articles about Rayouf Alhumedhi, a Saudi teenager living in Germany who has submitted a proposal to the Unicode Consortium’s emoji subcommittee for the inclusion of hijab and keffiyeh wearing emojis. Motherboard Alhumedhi as saying

“Emojis can seem like a trivial topic but people use emojis to represent themselves and their lives. When the different couples and different skin tone emojis were added there was a huge buzz, and this was because people finally felt represented and acknowledged, which is the same case with the headscarf emoji.”

Rayouf Alhumedhi

Rayouf Alhumedhi

There are several things I find really inspiring about this story. Firstly it’s about choice and empowerment.  Here is a young woman who felt she lacked representation online and took it upon herself to change that.  And secondly it’s about diversity and engagement with standards bodies.  The way that Alhumedhi went about creating an icon that represented herself was by submitting a proposal directly to the formal standards body that governs unicode emojis.  That takes some doing.  I worked with technology standards bodies for many years, though admittedly not the Unicode Consortium, and to say that women are underrepresented in these bodies would be something of an understatement. I got so used to being the only woman in the room that I stopped even noticing and I don’t think I ever encountered a woman of colour in any of the standards working groups I was involved with over a period of about fifteen years.  So more power to Alhumedhi for taking her campaign for representation straight to the body that governs the standard.  If we had more people like Alhumedhi involved in the the development of standards and software perhaps the web would be a more diverse and inclusive place and companies like Apple wouldn’t find themselves in such a mess when it comes to dealing with issues of race, representation and diversity.

Links
The Hijab Emoji Project
The Unicode Consortium
Unicode Emoji Subcommittee

 

23 Things: Bonus Thing A – About Me page

Consider creating a definitive ‘About Me page’. This is a space where you can tell the world who you are, what you do, where your interests lie, and link your online presences all together in the one place.

This one’s easy.  I’ve got an about me page right here lornamcampbell.org/about-2/ I first posted this page in 2013 but because my job tends to change pretty rapidly I update it regularly.  Looking at it today though I realise I could do with updating it again.

I also recently started keeping a google doc for short bios, because I’m always being asked for bios for one thing or another and tend to end up writing variations of the same thing over and over again.

I did used to keep an About.me page but I haven’t updated it for ages and since the system changed it now looks a mess so I should probably delete that account.

23 things is proving to be very useful for highlighting all these little digital housekeeping jobs that I never seem to get round to doing!

23 Things: Thing 4 Digital Security

So Thing 4 is all about checking your digital security and privacy settings.  This isn’t actually something I’ve done before but I’m glad I didn’t have any nasty surprises.  I actually don’t use many apps, I tend not to link them together, I don’t turn on location services unless it’s absolutely necessary and I keep bluetooth off.  That might make me sound very careful about my digital privacy and security but I have to confess, it has more to do with the fact that I have a rather old iPhone with hardly any storage capacity and crap battery life!  In actual fact I’m a bit blase when it comes to this kind of thing so it would probably do me no harm to follow up with one of the recommended digital security courses.

Credit: Wowser, CC BY NC 2.0, https://www.flickr.com/photos/wowser/2915951282/

Credit: Wowser, CC BY NC 2.0, https://www.flickr.com/photos/wowser/2915951282/