The OER Adoption Impact Explorer

impactI’m very excited to announce the launch of the OER Adoption Impact Explorer. This interactive tool lets users adjust a range of Institutional Settings to match their local context and estimate what the impact of adopting OER would be on their students and campus. Users can also tinker with a group of Research-based Settings to make the estimates more conservative or more aggressive.

The goal of the Explorer is to provide OER advocates with rigorously modeled, data-based arguments that they can use in conversations with a wide range of stakeholders (faculty, administration, students, policy makers, etc.).

We’d love your feedback! Let us know how we can make the Explorer more useful to you in your advocacy work.

The Review Project Launch

At #OpenEd14 John Hilton presented a summary of all the empirical research on the impact of OER adoption that we could find. The presentation was extremely well received, and John has turned it into a nice journal article which is currently under review at a journal which shall remain nameless (for the time being).

By the time the article appears, however, it will almost certainly be out of date. As a service to the field, we’ve published an abstracted version of John’s article on the OEG website under the label The Review Project. We’ll keep this page up-to-date as we come across additional articles that actually take a solid empirical look at the impacts of OER adoption. Do you know of one we’ve missed? Head over to The Review Project and leave a reference in the comments.

Ed Researcher Article is Live!

We’re extremely excited about our newest publication:

Robinson T. J., Fischer, L., Wiley, D. A., & Hilton, J. (2014). The Impact of Open Textbooks on Secondary Science Learning Outcomes. Educational Researcher. DOI: 10.3102/0013189X14550275

Putting something in Ed Researcher has been a “bucket list” item for many of us. It is, arguably, the highest ranking education journal which accepts this kind of research. And yes Jared, the first author on this paper, is a student member of the OEG!

We’re just completing another paper like this one, but with post-secondary data. It’s going to be even more awesome than this article was! Which journal should we target next? PLOS One? Nature?

Gates Grant is a Go

The Bill and Melinda Gates Foundation has approved a grant to the Open Education Group to apply our COUP framework to post-secondary adopters of open textbooks and open educational resources. This study will examine each of the following questions:

1. What is the relationship between OER-adoption by teachers of post-secondary courses and their students’:

a. rates of course completion?
b. rates of course success (i.e., completing a course with a C or better grade)?
c. enrollment intensity (i.e., the number of credit hours they take during the semester they are taking the OER course)?
d. rates of persistence (i.e., registering for classes again in the semester following the OER course)?

2. What is the relationship between the total cost of required instructional materials on course syllabi (whether the course uses OER or not) and the outcomes listed in question 1? Do very inexpensive but non-OER materials result in similar changes in academic success?

3. Do the answers to questions 1 and 2 vary according to students’ Pell eligibility?

4. Do the answers to questions 1 and 2 vary according to the degree to which teachers exercise the reuse, revise, remix, and redistribute (4Rs) permissions granted by OER licenses?

We will initially conduct this research with Project Kaleidoscope participants and Open Course Library materials adopters. If you’re using open textbooks or OER in your course in place of traditional textbooks and would like to participate in this study, contact David Wiley.

Utah Open Textbook Project Goes Statewide!

Something very exciting happened today.

The Utah State Office of Education announced that (1) it will be supporting the development of Utah-specific open textbooks for all secondary language arts, mathematics, and science courses, and (2) that the USOE recommends that all schools across the state consider these open textbooks for adoption in their secondary language arts, mathematics, and science courses for this fall (2012). The math and science books will be remixes of CK-12 materials (as per our existing pilot program), while the Language Arts books will be produced locally. The Hewlett Foundation is providing partial funding.

Yep.

This potentially impacts all 275,000 6th-12th graders in the state of Utah. The cost savings will be astronomical, but I don’t have exact figures yet. More on that in the days to come. My team and I will continue to research the impact on learning outcomes and the actual cost savings associated with the move, as we have with the pilot program the past two years.

The full text of the release is below. This is a historic day for Utah students, schools, and taxpayers. It’s also a historic day for open education. Congratulations to everyone involved.

= = = = =

January 25, 2012

For Immediate Release

Contact:
Sydnee Dickson, Teaching and Learning director
801-538-7739 :: sydnee.dickson@schools.utah.gov

Utah State Office of Education to Create Open Textbooks

SALT LAKE CITY — The Utah State of Office of Education (USOE) today announced it will develop and support open textbooks in the key curriculum areas of secondary language arts, science, and mathematics. USOE will encourage districts and schools throughout the state to consider adopting these textbooks for use beginning this fall.

Open textbooks are textbooks written and synthesized by experts, vetted by peers, and made available online for free access, downloading, and use by anyone. Open textbooks can also be printed through print-on-demand or other printing services for settings in which online use is impossible or impractical. In earlier pilot programs, open textbooks have been printed and provided to more than 3,800 Utah high school science students at a cost of about $5 per book, compared to an average cost of about $80 for a typical high school science textbook.

“Utah’s open textbooks are a great use of technology,” said State Superintendent of Public Instruction Larry K. Shumway. “Texts get into classrooms quickly and can be updated as needed rather than on a publishing schedule – something that’s particularly important in science. The open textbook also adds to Utah’s reputation as the most cost-efficient school system in the country. This is a fantastic way to get the latest textbooks into the hands of Utah’s nearly 600,000 public school students.”

“We’re thrilled that the State of Utah is encouraging school districts to consider adopting open textbooks,” said Barbara Chow, Education Programs director at The William and Flora Hewlett Foundation, which helped fund the project. “At a time when education budgets are under increasing stress, digital technology in the form of open textbooks now offers the potential to save school systems millions of dollars.”

Later this spring the Utah State Office of Education will invite all districts and charter schools across the state to attend informational meetings and professional development designed to help open textbook adoptions succeed.

The decision to pursue open textbooks at scale comes after two years of successful open textbook pilots led by David Wiley of Brigham Young University’s David O. McKay School of Education. Each pilot was conducted by the BYU-Public School Partnership in partnership with the Utah State Office of Education. The William and Flora Hewlett Foundation provided funding. Mathematics and science textbooks will be based on books originally published by the CK12 Foundation, a not-for-profit organization based in California founded with the mission to produce free and open source K-12 materials aligned to state curriculum.

In new research soon to be published in the International Review of Research in Open and Distance Learning, Wiley and his colleagues found that Utah high school students learn the same amount of science in classes using the $5 open textbooks as they do in classes using the $80 traditional textbooks.

The $5 Textbook – Now For Less Than $5

From time to time I re-review the costs from various print-on-demand vendors to see if we can’t get the cost down a little further on our just-as-effective-as-$100-big-publisher-books $5 textbooks. Good news! CreateSpace, one of the first sites I reviewed this time around, can print our books (8.5×11, B&W, paperback, 250 pages) for $3.85. Add $0.40 per book for shipping, and the $5 dollar textbook is now a $4.25 textbook, delivered. What else will we find as we continue to review vendors in preparation for 2012-2013?

If you’re not familiar with the $5 textbook, these are digital textbooks that use an open license (a Creative Commons license, to be more specific) originally published by CK12. These books are free to adapt, revise, improve, and use in any format. The Open Education Group at BYU works with districts to help them develop their own custom versions of these high quality high school science textbooks (and, starting Winter 2012, high school math textbooks) that are individualized for their students’ specific needs. Where infrastructure allows, these books can be used digitally for no cost. For other settings, we work with print-on-demand vendors to provide printed versions of these books that cost about $5 each.

When we compare the Utah state CRT scores of students who use the $5 custom books with students who use traditional, expensive textbooks, we find no difference in the percentage of students who are proficient at end of year. That’s a savings of over 50% of textbook costs (when you buy one per student each year and give it to the student to keep forever, highlight in, take notes in, etc. – things they aren’t allowed to do in their traditional textbooks) for the same amount of learning. We have anecdotal evidence supporting the conclusion that when students use these books proactively – taking notes, highlighting, etc., the amount they learn increases. While this finding would agree with previous research, we need more data to be able to make this claim with certainty.

And what can a district do when it saves 50% on textbook costs? Well, textbook money can only be used for textbooks, unfortunately, so districts can’t turn these savings into increases in teaching staff or teacher salary. But there are still interesting things that can be done. As one example, these savings can be used to purchase hardware (like iPads) on which digital versions of open textbooks could be read. The digital versions of books can still be annotated, highlighted, etc., and students can keep their digital versions forever. And the digital versions of these books are completely free. In other words, one way to view the financial savings gained from using open textbooks is as a completely self-funded way of acquiring the hardware needed to transition an entire district from static print material to interactive, digital content. And that’s just one example – a district is only limited by its imagination.

So far about 4000 Utah high school students have used these open high school science textbooks, with the support of the Open Education Group at BYU, CK12, and the William and Flora Hewlett Foundation. This year almost the entire Nebo district is participating. If every high school science class in the state were to save 50% for the same (or better) learning outcomes, the annual statewide savings would approach $1.5M.

We’re hoping to “greatly expand” the program for next year, and should be able to make a specific announcement soon. Keep watching!

Efficacy Data Are In!

The Big Take Away:

Schools can save over 50% on textbook costs without negatively impacting student learning.

The Short Version: Simply substituting open textbooks for proprietary textbooks does not impact learning outcomes.

The Longer Version:
If there’s one thing this project has taught me about data, it’s that they’re messy. I knew this already, but I assumed that with our relatively small year one pilot group (n=7 teachers) things would be easier and cleaner. Instead, what we ended up with was the 2011 CRT (state standardized test) scores for each teacher (these scores are a percentage indicating what proportion of their students demonstrated proficiency on the exam as judged by the state), the 2010 scores for each teacher, and the 2009 scores for only four of the teachers. We had hoped for 2011 plus three years back for every teacher, but some of our teachers are new (no data beyond 2010), some have moved schools (“difficult” to get data beyond 2010), etc. Fortunately there doesn’t seem to be any hidden systematicity to our missing data.

So what did we find? Table 1 gives raw scores.

Teacher T Teacher U Teacher V Teacher W Teacher X Teacher Y Teacher Z
2009 64 N/A 54 59 100 N/A N/A
2010 69 62 44 59 99 88 89
2011 61 61 58 82 100 83 85

There are two straightforward ways of asking these data about the impact on student learning of substituting open textbooks for proprietary ones.

The first, noisier way is just to subtract the 2010 scores from the 2011 scores (remember, these scores are the percentage of students achieving proficiency). When we do that, we get a distribution that looks like this: -8% (i.e., 8% fewer students achieving proficiency), -5%, -4%, -1%, 1%, +14%, +23%. The mean of this distribution is +2.86% and the mode is -1%. By either measure of central tendency, there is almost nothing happening in this data.

The second, slightly more stable way of looking for the impact on student learning is to subtract either the average of the 2009 and 2010 scores (when both are available) or the 2010 scores otherwise from the 2011 scores. This gives a slightly better picture of what the “true” previous scores were. This method provides the following distribution: (-5.5%, -5%, -4%, -1%, +0.5%, +9%, +23%), which has a mean of +2.43% and a mode of -1%. Again, there is nothing to see here folks. These aren’t the droids you’re looking for. You can go about your business. Move along.

As a side note, I should point out that we’re keenly interested in what the teachers who saw 23% and 14% jumps did with their open textbooks last year. One of these teachers told me, “the betters students write in their textbooks more.” If this casual observation turns out to be true, and this particular change in pedagogy can be propagated broadly, perhaps we can see wide increases in proficiency scores. We’re looking at just what exactly students are doing with their books more closely this year (with 20+ teachers in this year’s group).

We’ll be running more sophisticated analysis next year with the larger data set, and collecting data on this for a few years to come to improve the stability of the findings, but for the first year pilot this is a fabulous outcome. The implications for students, schools, and districts are “large” indeed. A more formal writeup to come.

The $5 Textbook

We went through lots of pain and suffering last year trying to find the “right way” to print open textbooks – the way that makes them super affordable. We printed with different vendors, printed books whole and in sections, bought and stuffed three ring binders and tried paperback print on demand, etc. Some of these approaches were more expensive than traditional textbooks, as we explain in our upcoming article. But we also discovered the “killer app” – a process for localizing textbooks to meet local student needs and print them at the super low cost we were looking for.

How well does it work? Here’s the receipt from this year’s textbook purchase:

When you divide the total purchase price ($14,400 including shipping) by the total number of high school science textbooks we purchased (2690), you will discover that the average printing + shipping cost for one of our textbooks this year is:

$5.35.

Heck yea! This is what we’ve been looking for. Localized books, printed at scale, at ridiculously affordable prices, just as our online Open Textbook Cost Calculator predicted. Oh, and by the way, they’re not just less expensive – they also enable new pedagogies.

Effectiveness data will be forthcoming soon – how much science do you think the kids using $5 textbooks learn compared to kids whose schools adopt $100+ textbooks?

Cost paper finalized and submitted

The paper we’ve written outlining all the cost-related lessons we learned during the first phase of the Utah Open Textbooks project has been submitted to the International Review of Research in Open and Distance Learning! While you’re waiting for an opportunity to read the prose, you might enjoy interacting with the Open Textbooks Cost Calculator we created based on the lessons we learned.

We’ll be announcing information about Phase 2 of the UOT project in the coming weeks. Stay tuned!

UPDATE: Apparently open textbooks is just a little too far away from distance learning to be a good fit for IRRODL. Consequently, we’re rolling up the year two cost information together with year one efficacy data and submitting a new version of the paper elsewhere. More to come…

March Meeting

We had a great meeting with the Teachers in March. They reported that their students are loving the textbooks and many students from other classes are envying the textbooks. We’ve also finalized our cost analysis and are putting the finishing touches on a paper to publish about it so that everyone can benefit from what we’ve learned. Finally, David Wiley has put together a very cool calculator that estimates the cost of switching to open textbooks. Anyone can get online and try it out at http://opencontent.org/calculator/.

We asked our teachers if they find themselves doing anything differently because they are using open textbooks and they shared several good examples. One of our teachers said that having the textbooks the students can write in saves him a lot of time since they don’t have to go back and forth between the textbook and a worksheet and he doesn’t have to spend extra time preparing worksheets. One teacher said that she doesn’t have to have her tables scrubbed like she used to because the kids don’t doodle on the tables anymore, now they doodle in their textbooks (and the kids love that it is okay because the textbook belongs to them). Several of our teachers agreed that they present content in a different way because now the students can follow along in the textbook and it makes content presentation much easier. They also mentioned that when a student misses class, it is so much easier to get the student caught up–the teacher simply tells them which chapters they missed. Since the teachers didn’t use traditional textbooks as regularly as they do the open textbooks, the students weren’t able to follow along in their textbooks and a few chapters in the textbook wouldn’t help the student catch up with what they missed in class. One teacher mentioned that his students are learning much more from the science labs than they used to because the book now matches what they are doing in their lab.

When discussing how their students use the books, one teacher said that initially she thought that she would have to remind the students to highlight their books frequently but she has found that they grab the highlighters and mark up the books all on their own.

In the final analysis the costs proved more difficult to calculate than we originally thought. The reason is that we gave the teachers several choices and we used two different printers. But we were able to analyze them and discover that with the experience we’ve gained (which printers to work with and which ways to print it) people can potentially save money by using open source textbooks. But what we’re most excited about is the potential educational benefit. And we’ll have more information coming on that in the next few months.

For now, we’re going to be getting our results published and we have the calculator up for folks to play around with.