Thursday, May 29, 2008

My take on analysts and their reports

What I have seen is that there are a few different categories of
analysts. Keep in mind that they have businesses to run (and if they
can't keep their businesses operating, we don't get the benefits of
their work):

1. The analysts who reflect what the "big boys" are doing. This group
is considered vendor neutral, except that they will tell you that the
most widely used tools are the best. By doing this, they will not
offend any of their customers who bring them in to review their
approaches. Currently, they are showcasing Microsoft PowerPoint, because
that is what so many trainers are already using. There is no discussion
of learner satisfaction (which is pretty poor when it comes to
asynchronously delivered presentations.) They are promoting the status
quo, and driving the decision makers to follow the pack: "driving by
looking in the rear view mirror". This approach is "safe" from a
business point of view. (They would have recommended horse-and-cart as
the best car in the 1900-1920 time period.)

2. The analysts who reflect what they are paid to. This group masks
paid advertisements as "white papers" and "studies". Our company has
been approached by some of these companies who are writing reports about
all the tools in the market. We are told that if we don't purchase a
$20K "case study" from them about us, there is a chance that our tool
won't show up in their study. They have followed through on their threat.

So, when people review analyst reports, it is important that they keep
this in mind. Just because an analyst has written about a tool in their
report, does not mean that the tool is being unbiasedly endorsed as the
"best" approach or even as an "effective" approach.

The Brendon Hall authoring tools reflect what the vendors submit to them. They
provide an excellent service as a collection of what is available (and
are worth the money), but should be considered in this light. Many
vendors will market-spin when it comes to what they submit, e.g. they
will say they are completely ADA compliant when they produce a separate
single file that is the text from the entire course, but the course
really delivered is all chained graphics (not ADA compliant). Similarly
they will say they are SCORM conformant, but they track no student

Regarding W3C standards, I am glad that finally people are asking about
building web courses that actually consider "web" design. The W3C
standards really do lead to learner satisfaction and accessibility (e.g.
look at Google - does it use flying bullets?). Learner satisfaction and
accessibility lead to return on investment and to lower
maintenance/support costs.

Tuesday, May 27, 2008

five thoughts on eLearning

1. The "tell-test" model works best when it is a
"tell-repeat-test-tell-repeat another way-test..." method. It has been
proven over the years to work better. Replacing it with Wikis,
podcasts, walkman-casts, virtual campuses, etc. doesn't replace it. It
just delivers it differently. Can you do tests from Wikis or PodCasts?
(I don't think so.)
2. I believe people have become disenchanted with e-learning because
companies put the "cart before the horse". I see that billions of
dollars were spent on LMSs before any courses had been created. This is
like building a railway network without having any locomotives. There
is nothing wrong with the web technologies - just with the way they were
sold. Similarly, courses were built as evolutions from presentations
instead of evolutions from web pages. A self-propelled presentation is
a bad way of training, just like correspondence courses, cassette-based
courses, VCR courses, ... A web page with multiple navigation paths,
repeating the information several ways, followed by frequent tests is
much more compelling. It is called "Instructional Design". That is
what has been lacking from many of the products out there. No matter
how good the technology, if you present content poorly, it will not give
3. Should we jump to new technologies? New technologies have their
place. However, if we use them without considering instructional design
- or what the end-user is actually doing - they will be no more
effective. For example, we have seen that if you have audio in a
course, the student's eyes drift away to other tasks. Soon their mind
drifts away too. If they are forced to read, without other
distractions, knowledge retention increases dramatically. Wikis are
great as reference material look-ups (as long as they don't get clogged
with garbage.) PodCasts allow asynchronous delivery of speeches, but I
would be cautious because of the easy distraction factor. Social
networking will work great for improving chances of the employees
getting their next job. Wikis, PodCasts, Bulletin Boards, Chat Rooms,
Multimedia etc. are excellent technologies to augment the basic content,
but should not be considered a replacement. Moving to the next level
of technologies reminds me of a phrase I once learned: "I'm working on
my second million dollars....I gave up on the first".
4. Page-turners, if done right can be effective. PPT by itself does not
make a compelling presentation - it takes content and an effective
presenter. If you get rid of the live presenter (recordings don't
count), it is harder to make the material complete. If you take
advantage of web navigation and hierarchical design, you can fill in for
the missing live instructor. Leaving all your testing for the end of
the course is less effective than mingling multiple questions in the middle.

Page-turners can be really bad if you are limited to putting 3 bullets
on a page. The web allows you to structure your content so that you
break it up in instructionally meaningful ways. PPT and tools like it
force you to break up your content so that it fits on a screen. Don't
blame "page-turners". Many authoring tools were designed and
effectively sold because they look so much like PPT with a test at the end.
5. Yes... see #4. Distracting multimedia and flying bullets are
exciting for the course author, but they are really annoying for the
student who has to see them 3 or 4 times.

5.I think many LMSs and instructors have been looking at the web
technologies to fit their current instructional models instead of
molding their instructional approaches to fit the web. That is, LMSs
force courses to be a one-time event. You take the course, you take the
test, you're evaluated, you're done, you never see the content again.
The web should be used to flip this around. Once you know where the
content is (e.g. google), you go there when you need it. You should be
able to take the tests as many times as you want. You should be able
to use the material in a just-in-time fashion. Why memorize the
material, if you can find it quickly? Proper web instructional design
considering "just-in-time use" makes this possible. A linear "page
turner" discourages this approach. A "page turner" with additional
navigation and proper tables of contents can achieve the advantages of
web technologies. But if the content is bad, it doesn't matter how you
deliver it.

Friday, May 16, 2008

Why use a web authoring tool for eLearning?

Why use a web authoring tool for eLearning? I use MS-Word, but not to create courseware.
I don't use MS-Word for standup presentations either. Why? because
there is no instructional design or web delivery structure inherently
built into it. It reminds me of the early days of PowerPoint, when
everyone said, "why do I need another tool? I can just use WordPerfect
to create the presentations. You see, they're just as good as your
PowerPoint ones." This is why a WordProcessing staff was still needed -
because the WordPerfect built presentations were lousy. Just as today,
the courseware built in PowerPoint or even your beloved MS-Word leave a
lot to be desired. Once people started using PPT, they saw the benefit
of a tool designed for the purpose. I believe that once people start
using proper eLearning tools designed for web delivery they will see the
benefit of a tool designed properly for the purpose.

To your questions: "Does it work? Does it say time and money?" I would add:
1. Has employee (rather than course builder) productivity been increased?
2. Are the learners giving positive feedback?
3. Are the learners re-using the content on an as-needed basis?

Without positive answers for all these questions, the approach/tools
will only lead to corporate frustration, and a long delay in adoption of
eLearning in a manner where it will succeed. By success, I mean become
a part of everyday processes and have tangible measurable benefits beyond:
1. Course developer didn't have to learn a new tool
2. Course developer got deliverable off their desk in record time
3. Course developer has no more courses to build because rest of staff
found the content boring, insulting, and difficult to use.

Wednesday, May 14, 2008

Why are people asking for XML?

I wouldn't mind if everyone was saying "MS-Word, MS-Word, MS-Word".
What concerns me is that XML is a behind-the scenes recommendation. It
is like saying "roman alphabet". It gives you some structure, but does
not solve your problems. Solutions can be built utilizing it as an
intermediate step. The same solutions could be built using any other
data storage mechanism. If you look at the "X" in "XML", it stands for
"eXtensible". This means that any one can make their own proprietary
version (like Microsoft has done), immediately defeating the
interchangeability that everyone praises XML for. I think XML is good,
but it is only a step towards people being able to talk to each other
(or in this case machines or programs). Currently I find HTML to be a
better standard because its purpose is more tightly defined, and there
is wider agreement on how it works.

XML by itself does not save any money. If I sent you a SCORM manifest
written in XML, MS-Word would have no idea what to do with it. If I
send this same document to a browser, all it can do is display it. If I
send it to a SCORM conformant LMS, now there is a system that can do
something with it. Just because something is XML does NOT mean that it
will work everywhere with everything or even that it will save any
money. If XML were a standard, maybe it would solve the problems that
need to be solved. As it is, the communication is only improved when
both sides speak the same variant of XML.

Monday, May 12, 2008

Applying Classroom training to eLearning

Over the web, it is much easier for the extroverts to
dominate the conversation and for the introverts to hide. One of the
big advantages of eLearning is precisely that the students can study at their own
pace and NOT be forced into someone else's as is the case in classroom
training. What you need to think about is how, when, and if you want to apply classroom metrics to web
based training.

I have talked to very many customers who are dealing
with regulating bodies that are applying the following requirements from
classroom training. Perhaps they need to think a little more deeply
than just transferring classroom concepts to the web.
1. Track how long a student takes a course. (This is supposed to be
self-paced. Does everyone have to work to the average time? It should
be about what they learned; not how long it took them to learn it.)
2. Ensure that the person didn't cheat. (If you assume everyone is
dishonest, then DON'T do distance-based testing. If you're willing to
accept people's affidavits of being who they are, taking the test alone,
and taking the test "closed book", then distance-based testing is
acceptable. Most people are honest.)
3. Make sure people don't steal the content and re-use it. (Just as
with published books, there is no sure-fire way of protecting published
material. These days, printed material can just be run through an
optical character recognition package with a cheap scanner, and it is
now easily reproducible.)

Some ideas that will work on the web:
1. Test test test. Give several tests throughout the course (not just
at the end of the chapter/course). It is OK to repeat the questions.
At the minimum the students will learn the material because they had to
answer the question so many times.
2. Test questions should be tied to the content. If you want to force
the students to read the content (rather than just skipping to the
tests), make the quesitons content-sensitive. That is, instead of "What
regulation applies to a company exporting widgets of Type B to India?"
the question should be "What regulation applies to Widget's
Incorporated, discussed on page 3.6?"
3. Randomize questions and answers to that answer keys posted by the
copier are useless.
4. Require that material be presented in 3 or 4 formats: Bullet points,
white-paper, step-by-step procedure, interactive exercise, etc. Since
different people learn from different expository styles, a variety of
styles will make the content more effective.
5. Blend self-paced study with moderated sessions. It would be easy to
require a 1 hour session with a live moderator as part of a training
program. The classic requirements can then be applied to the moderated
6. Web-based tests must capture what the student answered. Many systems
only provide a "score". I believe this is lame. By tracking all
answers provided, the students can be evaluated better, and the test
questions can also be evaluated.

Tuesday, May 6, 2008

Wouldn't it be great ...

Wouldn't it be great if LMSs and course
developers got away from the print paradigm (fixed, pre-defined size)
and evolved to the web paradigm (size based on end-user
preference/needs). Nothing in SCORM or AICC requires fixed screen sizes
- these are just a compromise because so much eLearning is implemented
by graphic artists (rather than subject matter experts or instructional

It is nice that Flash content can resize, however, I believe it still
has serious accessibility and searchability problems. Blind readers
cannot obtain the content nor can search engines without an external
(XML, for example) dump of all the text. Typically this dump does not
contain the content instructional organization that an HTML-based web
course would contain. For fully sighted students, a search engine that
can bring them to content based on keyword search is an enormous
productivity boost. We should really be tapping the power of the web
(just-in-time research/training) rather than trying to hammer presential
training precepts (fixed size content, fixed seat time, test only at the
end, etc.) in order to have successful eLearning.