Recent Survey Shows...
It's that time again. Time to debate the age-old question of "how many hours does it take to develop one hour of training?" This has always been one of the more interesting questions in training development, e-learning in particular. It came up while going through my subscriptions and came upon the blog of a colleague in the field, Robin Defelice. Her blog, DishingDesign, is an excellent source of information for the true, waist-in-the-deep-muddy instructional designer. I first got to know Robin a few years ago when, in my previous role as Director of E-learning development at Infologix, I hired her to be a sub-contractor ID. She was marvelous. She was a tremendous asset to our team and I adopted some of her approaches to our collective process along the way. I'll also mention that she's way cool, a trait that never gets enough credit professionally.
A few months ago, she reached out to me to fill out a survey about hours to develop training. I still had the metrics in my head from my previous job so I was happy to oblige. She worked on this survey with Dr. Karl Kapp, another contact of mine at Bloomsburg University's Institute for Interactive Technology. Karl's resume in the business is impressive and he has done a lot of work recently in the gaming area. I highly recommend subscribing to Karl's blog.
Here are the Results
Their results were just recently published on ASTD's Learning Circuits website. I've taken the survey results and placed them here (Robin is not sure why the editors at Learning Circuits went with this color scheme either). Karl had administered this survey 6 years ago so results are compared to that 2003 data. Give this a quick look:
Type of Training per 1 hour | Low Hours (2009) | High Hours (2009) | Low Hours (2003) | High Hours (2003) |
Stand-up training (classroom) | 43 | 185 | 20 | 70 |
Self-instructional print | 40 | 93 | 80 | 125 |
Instructor-led, Web-based training delivery (using software such as Centra, Adobe Connect, or WebEx-two-way live audio with PowerPoint) | 49 | 89 | 30 | 80 |
E-learning Developed without a Template | ||||
Text-only; limited interactivity; no animations | 93 | 152 | 100 | 150 |
Moderate interactivity; limited animations | 122 | 186 | 250 | 400 |
High interactivity; multiple animations | 154 | 243 | 400 | 600 |
E-learning Developed within a Template | ||||
Limited interactivity; no animations (using software such as Lectora, Captivate, ToolBook, TrainerSoft) | 118 | 365 | 40 | 100 |
Moderate interactivity; limited animations (using software such as Lectora, Captivate, ToolBook, TrainerSoft) | 90 | 240 | 150 | 200 |
High interactivity; multiple animations (using software such as Lectora, Captivate, ToolBook, TrainerSoft) | 136 | 324 | 60 | 300 |
Limited interactivity; no animations (using software such as Articulate) | 73 | 116 | NA | NA |
Moderate interactivity; limited animations (using software such as Articulate) | 97 | 154 | NA | NA |
High interactivity; multiple animations (using software such as Articulate) | 132 | 214 | NA | NA |
Simulations | ||||
Equipment or hardware (equipment emulation) | 949 | 1743 | 600 | 1000 |
Softskills (sales, leadership, ethics, diversity, etc.) | 320 | 731 | NA | NA |
- The 2009 numbers seem much more 'averagy' (yes, I just used that word). The 2003 numbers are all rounded nicely, almost as if a sales person was providing a 'range' of hours estimates to a client.
- In a number of important 'sweet' spots of development, the number of hours has actually gone up in the past 6 years.
So Why is This?
Based upon my experience in 17 years in this 'business' of instructional technology/design/e-learning development or whatever else we call it these days, I believe the 2009 numbers are fairly accurate to each category.
I think the reason these numbers are more accurate is that training development organizations are putting more effort and resources into actually tracking true development related hours per project. I figured you might might ask me some question to back this assertion up so here are my answers to those perceived questions:
But haven't training development resources been doing hours tracking for a long time now?
- Yes. Sort of. Hours tracking in our business has sort of an odd history. Way back in the day (early 90's), hours tracking for e-learning (back then called CBT or TBT) development was done under an organization's more formal hours tracking system. These systems did not easily conform to the unique needs of e-learning development but it wasn't that important. For e-learning teams within bigger organizations,if tracking was done, it was to ensure we had 40 hours on our timesheet. It was a requirement to get paid. Whether it was detailed to a project was important, but not critical. As long as you had 40 in there, you were good. If you were only at 35 hours for the week, you simply sprinkled 5 hours on a few different projects and you were good to go. Who would really know? Or care?
- If e-learning leaders wanted more realistic data, they would have their own separate system so they could a) give more realistic price quotes for future projects and b) prove how efficient they were in the 'time to develop one hour of training' metric. Part b was particularly troublesome because you could change assumptions on your numbers to come up with absurdly low ratios that showed how "awesome" you were. Bottom line: tracking was done to get Paid, determine Price and ego-massaging Pride.
So why have training developers become more precise in the past six years?
- I'll boil it down to one key event: Enron. The Enron scandal exposed the failure of the somewhat self-regulatory nature of how companies did their accounting, most notably how they recognize revenue. Companies would 'accelerate' revenue recognition in order to make the next quarter look good. Some companies, like Enron, took this to outlandish boundaries. To illustrate with an e-learning example:
- Let's say Enron (all these years later, it's still OK to pick on them) had an e-learning division and that team landed a project that was signed on April 1 and was expected to take 6 months to complete. The contract value was worth $100,000. If Enron wanted their second quarter revenues to look good (quarter ending June 30), they would just recognize the full $100,000 of the contract as revenue for that quarter, justifying it by saying they landed it in second quarter. But by that time, barely half of the project would actually be 'done'. You see the problem. And so did Congress. Thus, Sarbannes-Oxley (SOX) was born. And this is where the project tracking science picks up.
OK, I sort of follow. But why did Sarbannes-Oxley force more accurate project tracking?
- Three words this time: "Fear Of Noncompliance". (Note: I will tell you right now, I'm not qualified to discuss Sarbannes-Oxley. What I am relating is my experience at Infologix with SOX's impact). While SOX was to only impact public companies, many private companies re-evaluated their own accounting practices and adjusted "just in case". At Infologix, like many companies, the CFO's role became critical in defining and enforcing financial responsibility all the way down the chain. Infologix had just gone public so the need to "do it the right way" was VERY important because now regulators were watching carefully. One end result was the implementation of an organization-wide project tracking system that tied directly into the financials system at many levels. For Finance, this meant that at any time, they could see the relation between budgeted hours burned vs. revenue recognized on any project. Sales people and executives naturally hated this but this is a new era in accountability and transparency.
- The finance function is more powerful than ever in this new environment. But if it was just finance telling e-learning developers to stay the course, it wouldn't be enough. The key driver to the furthering exact science of hours tracking is executive leadership. Yes, the big bosses. Why? Because it has a direct, immediate impact on revenue. In this new world, executives have a direct interest in how many hours an e-learning (or any other of course) project will use and how many have been 'burned' at any given point. Here's the bottom line on that: on any given day now, a project manager or even director must be ready to explain that's days status of hours burned and revenue recognized. That 'motivator' has spurred this more exact science in. The world has demanded more transparency and accountability on a daily basis and e-learning developers are included.
OK, Matt, I get the financial accountability thing. But why are so many of the 2009 hours higher than 2003? Were previous estimates really that far off? Why?
- Yes, previous estimates were off. I personally feel the main reason was the under-reporting of hours from two main resources: graphics/media development and programming. In my opinion, both of these disciplines are highly creative but in different obvious ways. These disciplines have also chronically under-estimated how long something will take them to do and also under-reported how long something did take them to do. Why? These are fiercely proud folks and also competitive within their disciplines. For example, when asking a flash developer or a programmer how long something will take to create, they are typically very aggressive in their estimates, meaning they give a low estimate. Let's say the person said "4 hours". He or she is not low-balling; it is what they believe. More often than not, it takes longer to do than they thought. Let's say it took 8 hours. When they report back how long it took, they might budge and say, "mmm, about 6 hours". Now that financial accountability has spread to the front lines, these resources spend more time ensuring accuracy and not worrying about the pride factor related to time.
- Outsourcing has also contributed to more accurate hours tracking. As increasing numbers of organizations rely on the outsourcing model for development, the eye is out for 'over-billing' constantly. If you know how many hours a task should take, say 40, and a sub-contractor give you an invoice for 60 hours, you are all over that. You would investigate heavily before agreeing to pay for 20 more hours than you expect something to take.
So, there you have my first thoughts on this recent study. In case you want the correct answer to the question "How much time does it take to develop one hour of training" the CORRECT answer is, "It depends".
I saw this when it came out, and two things came to mind. First, the time to develop is increasing, not decreasing. Why?
Second, we need to figure out how to leverage resources to hit the low end of the time-to-develop. How do we do that?
Two questions well worth visiting by the community.
Aside from more accurate tracking measures for time to develop (in my blog), I think there a couple of other reasons time is increasing:
1) Pushing the envelope. There is a tendency to try to squeeze out incremental functionality and interactivity in projects that are already heavily templatized. We simply can't help ourselves.
2) As tools and methods for development become more 'mainstream', you end up getting non-designers/developers performing roles traditionally performed by IDs and developers. These novices end up on the e-learning development learning curve and that adds to the time to develop. The more mainstream development tools become, the more 'novice' resources will try (or be forced) their hand at e-learning development.
If we're overly concerned about time to develop we would:
a) stop innovating interactivity and stick to our current tools and functions- no more 'trying one more cool thing'
b) only have seasoned e-learning development professionals handle development
That being said, we all know those are two trends that will not occur.