Randomizing test questions with SCORM

With SCORM/AICC tracking, you can do limited randomization based on pooling. That is, on the test page, create more questions than what you want to display. Then, set the number of questions to display to the desired quantity. When the page displays, certain questions will be randomly hidden. So there is some level of randomization without the need for our Server Side Testing module. ReadyGo's SST module provides a more complete randomization ability, and more complete result storage/reporting (every answer to every question every time it is submitted.)

True randomization cannot be done with SCORM/AICC because this requires involvement from a web server. Prior to SCORM 2004, there was no LMS support for anything approaching randomization. With SCORM 2004, we see from the specification that it is technically possible, but it is not pretty. Each question has to be on a separate page, and the LMS would have to randomize the order of page delivery. I don't know of any LMSs that support this. As a side note, ReadyGo spent a large effort implementing as complete a version of SCORM 1.2 as we could, and we then found that the majority of LMSs only supported a very minimal subset of what we transmitted. In many cases, the extra information we sent the LMSs caused the LMS to crash. The customers who experienced this typically came to us to fix the problem since their LMSs dont give them support. The irony is that our tool costs $500, and they are paying upwards of $150,000 for the LMSs.

new terms for existing technologies

Discussion groups have been around since before the web (they were
called "bulletin boards").  You could access them through gopher before
http existed.  Wikis are just bulletin boards that are organized more
randomly (less by thread).  Blogs are just server-based authoring
through a browser with a limited number of pages.  I would not
categorize either of these as "Web 2.0".

I fear the "Web 2.0" is being hijacked by vendors just as "Rapdid
eLearning" has been.  The terms have been distorted to fit marketing
purposes.  I would refer you to the following:
For a more practical definition of Web 2.0 as it would apply to
eLearning: http://www.readygo.com/e-learning-2.0.pdf
For a more practical definition of Rapid eLearning:
http://www.readygo.com/isd/index.htm?start_file=ele02/01ele02.htm
(Note that the eLearning portions are part of a larger course that has
next/back buttons, tables of contents, and other links.  Also, if you
start from http://www.readygo.com/isd your responses to test questions
are tracked.  Like a Wiki, but with tracking and navigation.  This was
built and is maintained using the ReadyGo authoring tool.)

PENS - A better SCORM?

PENS will be a good complement to SCORM.

SCORM covers the following:
1. What page do you launch as the start of each SCO.
2. What files are needed for each SCO
3. How the SCO should send information (and receive it) to the server so
that the server can store it.
SCORM 2004, adds conditions for going from one SCO to another. (I
believe this is what Christie was asking about.)

There are still several other issues that are currently handled in
proprietary manners. The main one is "how does the LMS know that there
is a new package available?" This is what PENS would address. In terms of security, I cannot envision a condition where the fact that a new course is available to an LMS would
need to be encrypted - there is no personal data in this transaction.

I have seen tech support requests where customers need to know how to
load a course built with our tool into their LMS. We have to refer them
to their LMS vendor.

Future issues:
1. How does the LMS report the data it has received or export it for
post-processing?
2. What return access does a learner have once they have taken
(completed) a course?

"The more things change, the more they stay the same."

Are younger people different then us? Todays pervasive networked communication has to affect their learning behaviors. I suspect that it could even change the 'learned' elements of their cognitive processes.

Early versions of Sesame Street had a lot of very short
(1-2minute) segments. This meant that the kids were trained to have
short span attention, then jump to another topic. The newer Sesame
Street episodes have longer segments, e.g. "Elmo's World", that last
15-30 minutes. Anecdotally, I notice that my 4 year old's attention
span tends to mimic that of the activities and adults around her.

When I was in college, everyone was walking around plugged into a
newfangled revolutionary device called the "Walkman". Everyone was
talking about what a difference this was (as if the transistor radio in
the '50s didn't exist). I watched its prominence fade away when I was in
grad school, and everyone today seems to have forgotten about it.

It just reminds me of the old expression (and I tried to Google it to
find out who said it first, but there were too many links without
attribution)

"The more things change, the more they stay the same."

streaming class content ...why?

I am curious as to why so many people want training to be live, streamed content
over the internet? Isn't this a little bit like asking "what is the best
way to listen to radio using my television set?"

Much of the power of the Internet is that the content does not need to
be delivered live. In order to deliver live video content, everyone
needs a very high bandwidth connection (or a tiny display screen for the
content). Textual content contains more information/knowledge per byte
than any other form of delivery, and it can be searched so that people can
get to the content they need much faster.

I am hopeful that trainers/instructors will evolve past the concept
equating PowerPoint slides with narration to web based training. This
is like equating McDonalds(TM) food with nutrition. There is so much
more power that the Internet provides beyond streaming an in-class
presentation.

The power of online learning that most people miss

The point I'd really like to drill into the learning community's mind is that
presential (face-to-face) training achieves limited knowledge retention.
I believe the typical measure is that 20-30% of the material presented
is remembered. A PPT-Flash, Video, or PodCast element has similar if
not lower effect. the power of eLearning is that if done like web
content, the students can come back to it over and over. This raises
the efficiency from 20-30% up to 70-80% (my estimate from anectodal
evidence). Then, because the learners know where they can find/access
the content, the remaining retention is unnecessary because they can
find it quickly, either through hierarchical navigation or a search
engine. A PPT or video movie does NOT provide this easy indexed access.

Most SCORM and AICC-based LMSs (and the SCORM community at large) seems
to have missed this point. This is represented by the behavior of most
LMSs where the student is measured on "completion" of the course, and
previous tracking results are overwritten when a new session occurs.
Also the examples where each page of content is a Unit/SCO, and the LCMS
puts the courses together for the sake of author re-usability severely
handicaps the more important learner reusability. When the LMS or LCMS
has to deliver each page as a separate session, rapid student access and
continuity of concentration are impeded. The AICC and SCORM
specifications do not force these implementations, but a large portion
of the community has assumed these approaches because they fit into
their large database paradigm.

Getting video's to work in an eLearning course

When it comes to video and multimedia, the biggest hurdle is the end-user's computer/connection. You may find that if you use the latest
Codecs and Windows Media Player 11 generation, many of your students won't get to see the video. Similarly if you make the video 1600x1200 pixels, the students will give up before the multimedia has a chance to download. While there are no absolute numbers, here are a few recommendations:

1. Most end-users have Flash installed on their computer. Other media players (like Windows Media Player) are not as consistently installed. So, if you can convert the videos to flash, you'll get a better success rate. Keeping the size around 300 pixels gives a good compromise between view-ability and download speed. (These fall under plug-in/embed).
2. I recommend that every video be justified. That is, is it necessary to present the material as a video? or would it be sufficient to present a series of snapshots? A video is presented at the author/producer's pace. Self-paced training is most effective when the student can control the pace of delivery. If the video is a welcome message from the president of the company, does this add any value to the course? I limit video usage to showing assembly procedures or body gestures that can't be shown cleanly any other way. There is a high cost both in production and delivery time for videos, so they need to be thought through well.
3. Java applets are good if you are getting learner interaction. Otherwise, their cost is difficult to justify.
4. In terms of a camera, I don't have experience with them. Since you'll be reducing the size to something that can be delivered quickly over the web, however, the down-sizing will remove any advantage that one camera has over another. You may want to get a video capture card so that you can record at high quality, and then edit the video on your computer to reduce the size.
5. The main software I recommend is a format converter that can take the output of your camera and convert it to Flash or WMV (if you know all your users will be using Internet Explorer on MS-Windows) format. I would definitely try a bunch of different editing packages before purchasing one.

FUN VS. DEVELOPMENT

If the focus is on "fun and food", you will attract people interested in "fun and food" rather than the other people who are more interested in improving productivity, advancing careers, networking within the company, etc. I would first try to find out why people aren't interested in the courses you give. My guess is that the top reasons are that they are not convenient given the worker's busy schedules, and that they don't seem worth the effort. By "worth the effort", I mean that the topic is either too broad or too narrow. For example, if you offer a course on Excel, it will probably be too broad in scope for the engineers (and therefore boring). If you specialize is about doing database type queries using Excel, it will be too narrow for the administrative assistants (and therefore boring).

I am a strong believer in web-based training because it allows you to overcome the problems related to convenience and scope (if done correctly). First of all is convenience. If the course is built so that the student can get to it any time they need it and so they can jump quickly to the topic of interest, it becomes more attractive to them. When they can get to the topic of interest in 3-4 clicks, their time on course will be 5-10 minutes, and they will have gotten what they needed. If you instead give them a bunch of 1 hour recorded PowerPoint slide shows, you will forever repel your audience from your course offering. People just don't have blocks of 1 hour to sit down and watch a narrated "PPT-mentary" about the topic you're trying to present, even if it has lots of flying bullets and dancing pigs. Additionally, content retention is typically 20-40% from the first presentation of the material. If it takes 1 hour to get to a specific piece of material, you are guaranteed that the learners will only visit it once. (Flying bullets and dancing pigs will ensure that the 20% they retain is about the dancing pigs.) If the material is really convenient, they will come back to it over and over. In the repetition, their knowledge retention will increase.

Secondly, structure your content so that it can reach different audiences based on their needs. Include pages with bullet points that give the highlights of the subject. On this page, add links to sub-pages that give the topic in more details. That way, if someone really wants to delve into it, they can. The sub-pages should provide alternate presentations: A screenshot simulation, a step-by-step table, an interactive simulation (e.g. Flash), a case study, a homework task, a quiz, a test. Each of these presentation styles will reach a different audience, thereby making your courses more attractive - people will be able to control their own learning.