Three Days of Focus on Curricular Alignment, the Reusability Paradox, and Offline OER

[cogdog]

Our first Three Days of Focus is planned for June 30 - July 2, 2021 Our guest Conversation Starters include Werner Westermann (@wernerio) , David Wiley (@opencontent) and Vahid Masrour (@vahidm from Learning Equality, providers of Kolibri Studio) .

Technically the three days have passed but the conversation here remains open. Thanks for participating!

Join us for three days of focused discussion on the relevance of the Reusability Paradox, curricular alignment of OER, using Kolibri Studio for delivering offline OER, and more. Participate when it works for you.

Background on the Focus Topic

From his work in Chile aligning K-12 curriculum with offline OERs delivered via Learning Equality’s Kolibri Studio, Werner Westermann shares a new perspective on David Wiley’s Reusability Paradox, published in 2002. Is there more to confronting the paradox than just open licenses?

• Read: “Offline OER and the Reusability Paradox” (Werner Westermann)
• Review: The Reusability Paradox (2002, David Wiley)
• Listen: OEG Voices 10: Werner Westermann Juárez
• Watch: Werner talks about Curricular Alignment as part of a February 2021 webinar panel on Curriculum Alignment co-hosted by Learning Equality, EdTech Hub, and UNHCR
• Explore: Kolibri Studio (Learning Equality)

What to Do in Advance

• Read Opening replies from our guest experts that will be posted here before we open the space for responses.
• Join OEG Connect: If you do not have an account, create one in advance via the Join button in the top right
• Browse OEG Connect: Say hello in our Welcome Area in the Plaza. Explore other topics.
• Prepare Draft some questions or comments for posting on Day 1.

How to Participate

Our Three Days of Focus will open with a few statements, positions, and questions from our co-hosts. They will appear in the main Three Days of Focus area.

Reply to any of these with comments, counter arguments, relevant resources that support your statement (including web links, videos, documents). We are looking for constructive discussions that can lead to some summary, recommended course of action, or clarity on a topic. Reply also to messages from others. This is a conversational space…

If you have something new to state or pose as a question, create a New Topic.

OEG Connect is an open space for discussion about Open Education, and is also a place we ask that you act with respect and courtesy for others. Please review the participation guidelines created for our online conference.

For assistance with this platform, please see our Guide to OEG Connect. If you need assistance, send a message to our Help team or email us connect@oeglobal.org

[wernerio]

Thank you Alan, Marcela and the whole OEGlobal team. So much to talk about in these special and challenging times around education, specially in K-12, where children and youth have been negatively impacted respected to their learning paths. They need urgent answers from us, moreover in socially and economic vulnerable contexts.

How can OER, its use and the openness in our educational practices can emerge as alternative and new ways of learning and teaching? That was our initial questions which triggered many more, those that we will discuss here, super excited!!!

Quienes deseen comentar o iniciar un hilo de discusión en español, ningún problema!! Feliz de comentar a los hispanoparlantes. Abrazos!!

[vahidm]

As i was reflecting on Werner’s post, a description of two stages in the adoption of OERs emerged for me, which i wanted to offer for discussion:

• Stage 1: Aligning the OER with specific (national) curriculum outcomes. In the cases he mentions, Chile and Honduras, sense-making is required to sort out which resources are relevant to which “curriculum outcome”. We could describe this as “curation to the curriculum”.
• Stage 2: Teachers selecting the resources they can put to use in their classrooms, which we could describe as “curation in the classroom”.

This first stage can impact at least two groups:

1. The authorities that manage said curriculum: the alignment work makes it easier to build trust with the resources, at least because it becomes easier to examine the materials’ alleged relevance.
2. Teachers in their classrooms: The output of stage 1 provides teachers -especially those that are pressed for time [i’m imagining there are teachers out there that are not pressed for time]- with readily available resources that have been neatly “sorted out”. This facilitates the work at the next stage.

In the second stage, the teachers are provided with a “limited” number of resources (instead of a large library where they need to conduct time-consuming searches) that have been sorted to fit explicitly with the outcomes they are expected to work on with their students. The educators can now make decisions about which materials might be more pertinent to certain groups within their classrooms, personalizing the interactions with the groups as needed including making adaptations and providing context for their use to the students.

Another impact in that “second stage” could be observed during the pandemia: At Learning Equality, we observed a growing interest by different actors in education -a need even- for students to become better able to select materials they could use to enrich their learning. For many students, living in “remote” areas, contact with the teachers became at best limited. Thus, having access to a collection of materials that were aligned to the curriculum became relevant to students and their families.

[opencontent]

Thanks to Alan and others for the invitation to participate in this conversation.

If you didn’t make it all the way through my 2002 article linked above (and I wouldn’t blame you!), the Reusability Paradox can be stated in its simplest form as follows:

• There is an inverse relationship between the amount a student can learn from a resource and the reusability of the resource.
• “Larger” resources like a complete module - which may include text, images, interactives, practice with feedback, etc. - can support significant student learning BUT are reusable only in a very small number of circumstances. (How many places can you reuse an entire chapter on sonata form?)
• “Smaller” resources like an individual image are reusable in a wide range of circumstances BUT support very little student learning. (How much can a student learn from an isolated photograph of an orchestra?)

In the 1990s when I first posited the reusability paradox, the assumption was that learning resources would be traditionally copyrighted. (I didn’t release the OpenContent License until 1998, and Creative Commons wouldn’t come along until the 2000s.) In the context of traditionally copyrighted resources - that is, when others could not make edits to the resources - the instructional designer had to choose between creating resources that were more effective but less reusable, or resources that were more reusable but less effective. Neither choice was ideal, and the “Goldilocks zone” in the middle of these two options turns out to be extremely narrow.

Open licensing resolves the reusability paradox. When a larger resource (like a complete module) is openly licensed, it can be designed to support meaningful student learning AND - thanks to the 5R permissions that come with open licenses - be customizable to fit into different contexts.

Problem solved, right?

The resolution of the reusability paradox by means of open licensing leads to two new problems. The first problem is that not everyone has the time, resources, and technical expertise necessary to engage in the significant amount of revising and remixing necessary to tailor a larger resource to a specific context. Plenty has been written about this issue elsewhere, so I will not linger on it here.

The second, far less discussed problem is that not everyone has the instructional design and learning science expertise necessary to understand why a specific larger resource might effectively support student learning. For example, a person who is not familiar with the principle of spaced rehearsal wouldn’t be able to “see” spaced rehearsal at work in the design of the resource, and their revising and remixing could easily undo this aspect of the original instructional design. In other words, even if a person has the time, resources, and technical expertise to revise and remix an open educational resource, the localization process may very well reduce the effectiveness of the resource rather than improve it.

Here we can name and state a new paradox. I will call it “the Localization Paradox.” The localization paradox states that when a person without a deep understanding of instructional design and learning science revises and remixes an open educational resource to better fit their specific context, they may unknowingly remove (or otherwise render ineffective) the instructional design features of the resource. What’s worse, the detrimental effects of their revising and remixing will be just as invisible to them as the instructional design features of the original resource were. (They will never know they’ve broken it.)

Will any increase in student learning attributable to the localization outweigh any decrease in student learning attributable to the localization?

Note that some OER are immune to the localization paradox. (Before you proceed, pause to ponder why that might be true.) This is because these OER are created by people with a deep understanding of their discipline (e.g., physics, accounting, literature) but no training in instructional design or learning science. This results in OER that are largely informational and lack any instructional design features. For example, take an open textbook that contains accurate explanations of key facts, concepts, principles, and procedures in a discipline. This is essentially a reference work without much that is explicitly instructional in its design. Because OER like this include few instructional design features there is little to “break” during the localization process, and little harm can be done by revising and remixing them.

This leads us straight back to something like the reusability paradox. Should authors create OER that function more like reference works, without explicitly instructional features - minimizing their educational effectiveness in order to maximize their localizability? Or should authors create explicitly instructional OER that will support better student learning, knowing they will require significantly more expertise to localize effectively?

I would suggest that we should always design the most educationally effective resource we can. If its instructional design features are removed or rendered ineffective during localization, the result will be an informational resource fit for use the new context. In other words, in the worst case scenario the result of the localization process would be the same (a reference work) regardless of whether the original OER was informational or instructional.

[Mackiwg]

Kia ora @opencontent

Thanks for your gift of time and knowledge, and great to connect again with you in this forum. Learning is contextual and culturally bounded and the notion of the Reusability Paradox is a powerful framework for thinking about reuse in OER.

At the risk of clouding the waters, I think there is also a cultural challenge - specifically the culture of sharing. One of our big challenges, imo is to cross the chasm from sharing to LEARN → learning to SHARE.

When a group of educators get together with the express purpose to design and develop OER-enabled online courses in open environments, we can succeed in amazing examples of reuse that facilitate ease of remix. The great thing about open collaborative development, is that we learn together how to design for remix.

With the will to cooperate for the benefit of learners - we can succeed with remix “in-spite” of the paradox ;-).

Look forward to the ongoing conversation.

[wernerio]

Hello Vahidm, thanks for your time. Let me hang over your thoughts to expand on the potential of reuse practices and how and why open is the best way to go, having K-12 as the educational context in mind:

@vahidm

The authorities that manage said curriculum: the alignment work makes it easier to build trust with the resources

I’ve come to conclude that alignment work is indeed a type of reuse by repurposing, but also triggers other open practices (5R’s) that build a broader or engaged trust from the end user that can better lead to effectiveness in learning. But from the “curriculum provider”, there’s a huge potential for curricular development, a process that traditionally has taken many years, even decades, to evolve, its been shaped by a group of “experts”, without major participation of end-users. With tools and environments like Kolibri Studio you can efficiently create curriculum, managing and selecting different sources of resources, structured in a intended sequence, in a participatory framework. Moreover post-pandemia, specially for our young ones, is most important to rethink today why and what should we learn in a fast paced changing world.

@vahidm

The educators can now make decisions about which materials might be more pertinent to certain groups within their classrooms, personalizing the interactions with the groups as needed including making adaptations and providing context for their use to the students.

Isn’t it mind blowing!! There was a teacher movement in the 70 that pushed teachers to “Re-write the Curriculum”, as a way to “land” the curriculum in your classroom (or curate the curriculum as you say) so it could meet specific needs and personalized learning response. Again, open tools and environments like the Kolibri Studio can make this dream come alive.

@vahidm

growing interest by different actors in education -a need even- for students to become better able to select materials they could use to enrich their learning.

This is also a long heart-felt dream, to see our students actively participating in open educational practices. Do you see K-12 students working over the Kolibri Studio?, improving existing channels, creating their own, new channels, creating others for peer or community-based learning? Why not? Thanks again, attentive on your thoughts.

[wernerio]

@opencontent

Here we can name and state a new paradox. I will call it “the Localization Paradox.”

Dear David, thank you for your time and your endless food for thoughts!! Decades may pass by but your contributions have a relevant echo in our work and hopes. Please consider that it has taken 20 years to better deal with the Reusability Paradox, and now … the “Localization Paradox”.

In hopes to shorten this paradox-solving: it is important the degree of instructional design (a technical capacity) embedded in a resource to better fit the learning outcome, so, how could this be transparent to the user? I do not recall many OERs explicit their instructional design solutions, should they be digitally readable or accessible? Instructional design is also very much evolving, specially due to digital learning, from where do we understand instructional design? Dick & Carey? LOM Learning design? I think it would be great to know that a resource was built for Project-based or Problem-solving learning setting or Disposable assignments!! Sincere thanks again.

[wernerio]

@Mackiwg

One of our big challenges, imo is to cross the chasm from sharing to LEARN → learning to SHARE.

Hello Wayne @Mackiwg great to read you, quite a challenge you leave us here!! a sort of “Flipped Open Learning” commitment, thank you once again for your tireless open efforts, before any OER paradox.

Totally agree with you on how about overcoming paradoxes getting educators involved in open practices that lead to Learning to Share. As @opencontent points out, openness can lead for an educator to reuse an OER by adapting the resource to the specific context. That solves the Effectiveness side of the paradox (more context and impact), but what about the other side (the provider or who delivers) of the tradeoff (more reusable and scalable)? In other words, what is necessary and preferable to foster Learning to Share? Questions with K-12 context in mind.

[cogdog]

Interesting idea here, thanks for putting this into the mix, David. This should make for some ripe conversation.

But hasn’t what you described as a new paradox always existed on its own in terms of reuse of materials, not matter what kind they are? All teaching is localized. If not, it’s transmission.

The suggestion here is that the reuse is degraded, is it always much more instructionally sound or superior at the upstream end?

And I have some challenges in dividing OERs into categories of purely informational vs instructional. Got an example I can chew on?

And what if a revision and remix improves or adds an element to spaced rehearsal that the creators did not envision? How do the 5Rs matter if we worry that someone is going to remix / reuse it in a way that one thinks is demeaned?

What I do agree completely with is:

I would suggest that we should always design the most educationally effective resource we can.

And also, we should do this when we revise and remix.

[Mackiwg]

Hi @wernerio

Its been a few years since we had contact - great to see you here and catch up with the amazing work you are doing.

@wernerio

In other words, what is necessary and preferable to foster Learning to Share? Questions with K-12 context in mind.

Been thinking a lot about this in the K-12 context as we embark on capacity development initiatives in the Pacific for school teachers.

I think a productive starting point is to have two or more teachers working on the same OER resource for reuse in their respective classrooms from different contexts, eg two different countries. This provides an authentic context to design for remix.

Interested to hear your thoughts on how would you tackle this challenge in the South American context?

[Mackiwg]

@cogdog

But hasn’t what you described as a new paradox always existed on its own in terms of reuse of materials, not matter what kind they are? All teaching is localized. If not, it’s transmission.

That may be true for conventional face-to-face teaching. It does become interesting when developing full OER-enabled online courses for independent study. Bring on the H5P customizable pedagogical elements

[wernerio]

@Mackiwg

I think a productive starting point is to have two or more teachers working on the same OER resource for reuse in their respective classrooms from different contexts, eg two different countries. This provides an authentic context to design for remix.

Indeed @Mackiwg ! That’s basically has been our work that has sparked all these reflections, taking a resource and repurposing to a new context (national curriculum). In the context of Kolibri, the idea to align OER to our national curriculum comes from alignment efforts of Khan Academy in México and Perú, efforts that will trigger many more: A proposal for open educational resource adoption through a curriculum alignment hub - EdTech Hub This proved true in our work, as we started to first sneak, but later reuse these other alignments, remixing reused (aligned) OER. Let me say that the Kolibri Studio would be wonderful fit for design for remix, an open context to unleash 5R’s and emerging open practices.

@Mackiwg

Interested to hear your thoughts on how would you tackle this challenge in the South American context?

South America has incredible potential for cross-country reuse and remixing mainly due to have the same official language (leaving out my friends from Brazil, its a really a continent!!). Can Khan Academy videos and exercises, or any quality OER be relevant to any South American country? Of course it can!! In fact, they are in use now, pandemic has made Khan popular and much more used. Even more relevant for our region, that has fallen behind in open practices: in our work we have spotted scarcity of OER available in Spanish working in math, even more in science. But in other subjects or domains, there’s little close to nothing. Many of them give you the opportunity to push “design for remix”, there’s much to be done in more locally driven subjects (Language learning, History, Civics, etc.), so there’s so much to be done.

One of the takeaways in our work is that with a framework to support open educational practices (curricular alignment, design for remix, for example) is critical. The design and co-construction of OER by teachers and learners, among many other possible open practices, needs to be supported by an open infrastructure that has efficient and distributed tools and platforms, to create, import, manage and steward quality OER, made with open technical standards, in collaboration, licensed with shared open licensing schema.

Why not dream about a shared infrastructure of Spanish K-12 Offline OER for low-tech or no-connectivity contexts? With Kolibri I think we have a good shot!! Why not dream about Wikieducator being the infrastructure for K-12 OER in the Pacific? Thanks again for leading the way in open for so many years.

[Mackiwg]

@wernerio

In the context of Kolibri, the idea to align OER to our national curriculum comes from alignment efforts of Khan Academy in México and Perú

How did you deal with the licensing issues? As far as I can see the videos on the Khan Academy channel on YouTube use the standard Youtube license and the terms of service on the Khanacademy.org site states:

Khan Academy grants you, a limited, non-exclusive, non-transferable, non-sublicensable, revocable license to download, install, view and use the Downloadable Content, in object code form, on devices owned or controlled by you, solely for your personal, non-commercial purposes. You agree not to (i) modify or create derivative works of the Downloadable Content …

So technically these resources are not OER. Curious to know how you were able to integrate these resources into your initiative?

[WayUpNorth]

@wernerio

it is important the degree of instructional design (a technical capacity) embedded in a resource to better fit the learning outcome, so, how could this be transparent to the user?

This was similar to a thought I had. If a specific learning design, e.g. spaced rehearsal, is embedded in a larger OER, could there be meta data to explicate this for potential localization projects? It seems, IMHO, such meta data would allow for an informed choice whether to revise it as a simple resource or to invest additional resources in also localizing the embedded instructional design.

[wernerio]

Hello @Mackiwg … yes, another big challenge!! Legal rigorousness can come to determine that Khan is not an OER, its has gotten more restrictive over time (CC BY-SA in early days). But Learning Equality’s founder was an intern in Khan Academy and took the challenge to bring it offline, resulting in KALite which had huge success around the world. KALite led to Kolibri expanding the range of learning functionalities and content sources (OER). So Khan is much part of what Kolibri is today, I imagine Khan does not get mad that their resources gain value of this reuse (offline, curriculum aligned).

But licensing is a key aspect of OER infrastructure, where many efforts can be jeopardized for legal issues, when reusing and remixing. Its important to avoid non-permitted reuse because of compatibility problems when remixing or creating collections of OER. So recommendable to “harmonize” the licensing by advocating for a specific license(s), maybe defined by an explicit statement or policy around content for educational use, hopefully using the “more” open license(s) aimed at maximizing the flexibility and reusability capacity of the resources. Would you agree to this? Best wishes.

@WayUpNorth

This was similar to a thought I had. If a specific learning design, e.g. spaced rehearsal, is embedded in a larger OER, could there be meta data to explicate this for potential localization projects?

This is great suggestion, as I recall there is a Learning Design field in Dublin Core. Also agree on applying it to larger collections of resources, the more granular resource is, e have found its more reusable, at least for alignment efforts. But again, the problem for me is that learning design models are so much changing and mixing, it will be difficult to keep up with that flexibility.

[paulstacey]

I’m so delighted with this topic and format. Thanks to you all for engaging in discussion on this, to my mind, essential but poorly understood paradigm - reusability. Big shout out to Werner for such amazing work and a fantastic example use case.

I read the reusability paradox several times with great interest. So interesting to flashback to “learning objects”. Let me make a couple of remarks about the argument David originally laid out. I found myself questioning some aspects of the logic. Is it really true that a large learning resource is less reusable than a small one? I’m not so sure it is. Here’s a corollary. What is more reusable a car or a steering wheel. A car is an assembly of smaller parts and I’d argue that cars as a whole are more reusable than the deconstructed parts. Look at the size of the used car market. Certainly it has higher “value” as a whole unit than uncoupled into parts. Learning resources are similar. A course is designed as a complete thing like a car. While it is possible to de-constuct it into parts it’s integrity and value are greater when kept together.

David asserts that open licensing introduces a new problem in that not everyone has the time, resources, and technical expertise necessary to engage in the significant amount of revising and remixing necessary to tailor a larger resource to a specific context. The same is true of cars thats why we have mechanics. And this issue has in my view had little impact on the reusability of cars.

I find the issues being raised about the visibility of instructional design really quite fascinating. Is instructional design similar to the engineering of a car that assembles the parts into a fully functional whole? Does revision and localization really result in degradation or destruction of instructional design? As Alan points out I question whether upstream instructional design is really sound and superior. And let me suggest something else - I think most faculty and teachers see themselves and their interactions with the leaerners as the instructional design. It is not necessarily a tangible visible thing embedded in the resource. Continuing with my car analogy hen someone buys a used car and customizes it for their own use (soups it up, custom paint job, etc.) are they degrading it? I don’t think so.

And finally I really like Mackiwg’s push for a move from sharing to learn to learning to share. One thing I discovered in my work supporting open education during my time at BCcampus is that faculty are more comfortable sharing and reusing a smaller learning resource than a larger resource. Oh my did I just undo my earlier arguments? I don’t really think so. It’s just that it takes time to build a culture of sharing and that culture is in direct opposition to societal norms that commodify resources into things bought and sold.

[Mackiwg]

@wernerio

But Learning Equality’s founder was an intern in Khan Academy and took the challenge to bring it offline, resulting in KALite which had huge success around the world. KALite led to Kolibri expanding the range of learning functionalities and content sources (OER).

Aaah, so I assume Kolibri has custom permissions to use Khan Academy videos (which are not extended to other initiatives). It’s unfortunate that the Khan academy dropped CC-BY-SA which from memory coincided with a large funding grant.

I was recently asked to assist Fiji in identifying resources they could copy into a local national repository for school children who are grappling with another COVID-19 lockdown. The use of Khan videos has worked well in your initiative, but I can’t advise a government to ignore copyright because I don’t think the rights holder will object. Unfortunately the Fiji initiative will need to incur additional transaction cost and time to get permissions to use those video resources .

[JoshHalpern]

In my opinion there are emerging/emerged technological solutions to both the reusability and localization paradoxes. Although it may seem amusing, copy and paste enables reuse on the most granular level, and automatic translation shatters language localization.

Remixers do this on a larger level but require much less expertise and time for the rebuilders. As with automatic translation, the results are not perfect but often more than useful and can be polished with comparatively little effort against starting from scratch.

This is all doable as we know from our interactions with such programs as are used to fill out tax forms. Constant reminders of the rules from the software may be annoying but in the end are useful. Hints about where to find more information also.

This does not take away from the fact that it always helps to know somebody in accounting but that a great deal of the work may be supported by building better tools and then using the tools to build content

[jamalex]

Hi folks! Learning Equality co-founder (and that “former KA intern”) popping in. Thanks for hosting these great discussions!

First thing I wanted to clarify is that the majority of the Khan Academy videos are still CC-licensed – in particular, the vast majority are CC-BY-NC-SA. The exceptions to this are for content that came from KA partners (e.g. MoMA) or co-created with partners. It’s more hidden than it used to be, but if you look on a video page on the KA site, way at the bottom below the comment section, you can see text like “Creative Commons Attribution/Non-Commercial/Share-Alike” (e.g. here).

That said, there are a few things that have always been a bit muddy:

• The KA exercises don’t explicitly specify their licensing on the site, as of 2014. We have verbal clarification from them that the intention is for them to still be open, with the same caveats as the videos (certain partner-provided content, e.g. College Board). The rule of thumb being “if a video is CC, then the exercises alongside it are also CC”. We pushed to have this made more explicit, but it looks like it’s been left implicit so far.
• Their licensing metadata on YouTube hasn’t always been accurate. We pushed for some cleanup and it got better. I believe we primarily refer to API results to determine licensing, when bringing content into Kolibri, as it’s more canonical.
• I too find their Terms of Service at times confusing, and in some cases possibly contradicting itself. The “Downloadable Content” pieces referenced above I think are largely (in terms of intentions) targeting their mobile applications. Section 7.2 first sets up a generic “personal, non-commercial” license as the default for “Licensed Educational Content”, and then 7.2(a) specifies that some content is made available under an “Alternate License” (with the example of CC being given). Despite this being phrased here as the “exception”, it’s actually the case for the majority of their videos.
• One shift that has definitively happened over time is a move away from open-source, on the code side. When I interned at KA in 2012, the full KA website codebase was available for download, with many components (e.g. khan-exercises) released as explicitly open-source components. Later that year or in 2013, that download was removed, but some elements stayed open-source (and these were the pieces we incorporated into KA Lite and Kolibri). In 2018, they archived the Perseus exercise renderer codebase (GitHub - Khan/perseus: Perseus is Khan Academy's new exercise question editor and renderer.), which is required in order to render their exercises, and moved it into their internal repositories for ongoing development. So far we haven’t had issues using the 2018 version of the exercise renderer, but issues may arise as it continues to diverge.

Hope that provided some useful context and background! In terms of Learning Equality’s approaches to licensing, we always advocate for as open a license as possible. And in KA’s case we’re operating under the explicit CC license. In some cases, however, we do negotiate “special permissions” for distribution through Kolibri from a content provider that is not otherwise fully open.