Question:
From an AICC course can a score be saved for each page that has test questions? If so, can you save one score per course, one score per test, or one score per test question? If a student re-takes a test does the first score get stored or overwritten?
Answer:
In theory, a score can be stored for each question. To do this we will need to make each page of questions a unit/SCO. That way you can store a score for each page of questions. Alternatively, within the course, we could define an objective that is a composite score for several questions. Once again, I haven’t seen an AICC-based LMS that would store/report the objectives and interactions, so this is a moot point. Generally, I only see one score per unit. So, if you want more granularity, you have to break your content up into one question=one unit. When you throw in the delays for launching units, this becomes instructionally painful for the student. (Most LMS's take 8 to 20 seconds to store a unit and launch the next unit) If a student retakes a test, it is up to the LMS to decide what to do. My preference, as you see with ReadyGo SST, is to append the result to the report. However, most LMSs only report a single score, so they will tend to overwrite results from previous sessions. I have never seen an LMS that will store more than one score for a single student session. So, the student could take the test 15 times until they pass, and only the final score is reported (without recording that the student took the test 15 times.) Of course, the course itself could also track how many times the student took the test, but there is no way to report this except in the “suspend_data”, which the LMS does not view since there is no standard for it. SCORM 2004 addresses this, but for it to work, the LMS has to relaunch the unit each time the student wants to retake the test.
Friday, June 12, 2009
Wednesday, June 10, 2009
LMS/SCORM Complete status
Question:
If the LMS manager and the LMS provider are clear that the status “complete” shall appear when for example the student took a test and passed. Why is this still so complicated to implement that? Is this a problem of the SCORM definition, or a mismatch of field names?
Answer:
It is all about WHEN the completion status is sent. Some LMSs assume that a SCO contains only one page. Therefore, if the SCO has been completed, the student session is finished, and there is no more to do. However, if the SCO has multiple pages, and after the student completes, they visit other pages, the LMS could decide to stop tracking any further results (I have seen this). Sending a status to the LMS is not hard, but it can be hard to calculate it, especially if it depends on scores or behaviors from previous sessions. Also, there can be timing issues. Typically, one wants to report status type information as the user leaves a page or loads a page. With some LMSs, if the messages arrive too quickly, the LMS drops messages. In the case we were seeing with the AICC Basic IE6+, FF+ for frameless, the reporting problem existed because as each page of content is unloaded, the course reports the time on the page and the current lesson status. When the course gets to the Exit page, it has more to do. When the page loads, it sends the course completion status (putParam/ LMSSetValue). Then, separately, it has to send the ExitAU (equivalent to LMSFinish() and LMSCommit() in SCORM) message. With AICC, these are messages sent directly to the server. You can’t send more than one at a time, so we had to split them into onunload for one page, onload for the exit page, and onclick for the exit page. We were possibly seeing a timing issue that the onunload from the previous page and the onload for the exit page were arriving too soon (or in the wrong order) at the server. The onclick for the exit button wouldn’t be such a problem since the user will take at least 0.5seconds before they click it.
Another big issue is what to do when the user “exits” the course. This can either occur with an explicit “exit” button or if the user closes their browser. Many authors don’t want the exit button, and many users simply won’t use it. Therefore, if the user clicks “X” to close their browser window/tab, the course must send the exit messages. This can get difficult. So there needs to be an overall onunload for the course/frameset. Some newer browsers are preventing sending of messages directly to servers when pages unload because of security. Also, how does one send two or 3 messages simultaneously as the browser is shutting down?
Aside from the theoretical issues, there are some simple practical ones. When the course is built, it needs to be tested in the delivery environment. We often find problems in sequences we didn’t expect the student to follow – they start the course, jump to the end, jump to a test, exit, come back… With the AICC course we were working with there was an additional issue that it seems that the browser behavior changed over time. We saw that the onunload message from the prior page was arriving at the LMS after the onload from the final page. So, the onload message set the status to complete, but this was overwritten when the previous page’s onunload message arrived. We did expect the messages from the previous page to arrive after the message from the current page.
If the LMS manager and the LMS provider are clear that the status “complete” shall appear when for example the student took a test and passed. Why is this still so complicated to implement that? Is this a problem of the SCORM definition, or a mismatch of field names?
Answer:
It is all about WHEN the completion status is sent. Some LMSs assume that a SCO contains only one page. Therefore, if the SCO has been completed, the student session is finished, and there is no more to do. However, if the SCO has multiple pages, and after the student completes, they visit other pages, the LMS could decide to stop tracking any further results (I have seen this). Sending a status to the LMS is not hard, but it can be hard to calculate it, especially if it depends on scores or behaviors from previous sessions. Also, there can be timing issues. Typically, one wants to report status type information as the user leaves a page or loads a page. With some LMSs, if the messages arrive too quickly, the LMS drops messages. In the case we were seeing with the AICC Basic IE6+, FF+ for frameless, the reporting problem existed because as each page of content is unloaded, the course reports the time on the page and the current lesson status. When the course gets to the Exit page, it has more to do. When the page loads, it sends the course completion status (putParam/ LMSSetValue). Then, separately, it has to send the ExitAU (equivalent to LMSFinish() and LMSCommit() in SCORM) message. With AICC, these are messages sent directly to the server. You can’t send more than one at a time, so we had to split them into onunload for one page, onload for the exit page, and onclick for the exit page. We were possibly seeing a timing issue that the onunload from the previous page and the onload for the exit page were arriving too soon (or in the wrong order) at the server. The onclick for the exit button wouldn’t be such a problem since the user will take at least 0.5seconds before they click it.
Another big issue is what to do when the user “exits” the course. This can either occur with an explicit “exit” button or if the user closes their browser. Many authors don’t want the exit button, and many users simply won’t use it. Therefore, if the user clicks “X” to close their browser window/tab, the course must send the exit messages. This can get difficult. So there needs to be an overall onunload for the course/frameset. Some newer browsers are preventing sending of messages directly to servers when pages unload because of security. Also, how does one send two or 3 messages simultaneously as the browser is shutting down?
Aside from the theoretical issues, there are some simple practical ones. When the course is built, it needs to be tested in the delivery environment. We often find problems in sequences we didn’t expect the student to follow – they start the course, jump to the end, jump to a test, exit, come back… With the AICC course we were working with there was an additional issue that it seems that the browser behavior changed over time. We saw that the onunload message from the prior page was arriving at the LMS after the onload from the final page. So, the onload message set the status to complete, but this was overwritten when the previous page’s onunload message arrived. We did expect the messages from the previous page to arrive after the message from the current page.
Friday, June 5, 2009
Here is another great question I got. This is a continuation of my last blog.
If someone, like the E-Learning Manager at a company, would make up his mind that for a group of courses the "completion" of a test is given when All tests are taken and for another group of course when 1 test is taken-- then two different behaviors need to be programmed into the course. The LMS would need to work differently. Can the e-Learning manager define all sorts of behaviors? How do you know if the LMS can do this? Can the course deliver the data the way I need to?
Answer:
This is a difficult issue. Keep in mind that we have to program for any LMS. This would mean even the most minimal LMS. Therefore, the course itself has to handle all the logic. With SCORM 2004, the navigation between SCOs and the calculation of course completion can be assigned to the LMS. In SCORM 1.2, the concept of course completion did not exist – only lesson completion did. So, from this point of view, it is best to think of the LMS as a data storage device only.
When approaching people who have this question, it is best to have a series of examples of what behaviors are available. Then, based on the customer preferences, it is necessary to choose or create the LMS-pack based on what they want. Some examples:
Course completion: Take final test; pass final test; visit certain pages in course; total score across tests above 70%; pass every test with 70%; pass certain tests with 70%; be on course for a certain amount of time; or a combination of these.
Lesson status on restart: If course was previously completed, don’t change status; restart with incomplete every time; send pass/fail instead of complete/incomplete; if course has tests, send pass/fail. If it has no tests, send complete/incomplete; Hide some tests if the student is not taking the course for credit.
Currently ReadyGo provides multiple packs. The course creator then chooses the pack that meets their needs. This way we can trick an LMS to provide more behaviors then they were designed for.
If someone, like the E-Learning Manager at a company, would make up his mind that for a group of courses the "completion" of a test is given when All tests are taken and for another group of course when 1 test is taken-- then two different behaviors need to be programmed into the course. The LMS would need to work differently. Can the e-Learning manager define all sorts of behaviors? How do you know if the LMS can do this? Can the course deliver the data the way I need to?
Answer:
This is a difficult issue. Keep in mind that we have to program for any LMS. This would mean even the most minimal LMS. Therefore, the course itself has to handle all the logic. With SCORM 2004, the navigation between SCOs and the calculation of course completion can be assigned to the LMS. In SCORM 1.2, the concept of course completion did not exist – only lesson completion did. So, from this point of view, it is best to think of the LMS as a data storage device only.
When approaching people who have this question, it is best to have a series of examples of what behaviors are available. Then, based on the customer preferences, it is necessary to choose or create the LMS-pack based on what they want. Some examples:
Course completion: Take final test; pass final test; visit certain pages in course; total score across tests above 70%; pass every test with 70%; pass certain tests with 70%; be on course for a certain amount of time; or a combination of these.
Lesson status on restart: If course was previously completed, don’t change status; restart with incomplete every time; send pass/fail instead of complete/incomplete; if course has tests, send pass/fail. If it has no tests, send complete/incomplete; Hide some tests if the student is not taking the course for credit.
Currently ReadyGo provides multiple packs. The course creator then chooses the pack that meets their needs. This way we can trick an LMS to provide more behaviors then they were designed for.
Thursday, June 4, 2009
More about ReadyGo LMS packs
ReadyGo provides different LMS interface packs based on the behaviors of the different LMSs. For example some LMSs will prevent the student from returning to the content once they have "Completed" it. If you are using that LMS and you want to reuse your courses then you need to set up the interface so that the student's status is reported as "Passed" letting them reuse the course. I am unaware of any tool other then ReadyGo WCB that lets you choose your behavior by changing one option in a pull down menu.
Question:
Don't I need a description of what is behind each LMS pack? If I want the student to be able to return after he has completed a course does that mean I need to have you write a new LMS-pack?
Answer:
The “descriptions.txt” file found with each LMS pack contains a brief description of each LMS’s behavior. ReadyGo will write a new LMS-packs if the LMS does not behave like one that we have already dealt with. Typically it is just easier to write a new LMS-pack and test it with the specific LMS. This way we don’t run into timing or lockout problems.
Question:
Don't I need a description of what is behind each LMS pack? If I want the student to be able to return after he has completed a course does that mean I need to have you write a new LMS-pack?
Answer:
The “descriptions.txt” file found with each LMS pack contains a brief description of each LMS’s behavior. ReadyGo will write a new LMS-packs if the LMS does not behave like one that we have already dealt with. Typically it is just easier to write a new LMS-pack and test it with the specific LMS. This way we don’t run into timing or lockout problems.
ReadyGo's philosophy on course progress
Question:
What is ReadyGo's philosophy on course progress?
Answer:
Our philosophy is that the course is reporting both a score and a completion status. These should be handled independently of each other. However, a large number of customers want score>90% = completion. So, you’re throwing away one of the very few reported data available. Each LMS-pack has a different philosophy. Some tie the completion to the score; some do completion when the student reaches the exit page; some do completion based on visiting a certain number of pages. This way course creators can choose a pack that meets their needs.
What is ReadyGo's philosophy on course progress?
Answer:
Our philosophy is that the course is reporting both a score and a completion status. These should be handled independently of each other. However, a large number of customers want score>90% = completion. So, you’re throwing away one of the very few reported data available. Each LMS-pack has a different philosophy. Some tie the completion to the score; some do completion when the student reaches the exit page; some do completion based on visiting a certain number of pages. This way course creators can choose a pack that meets their needs.
Wednesday, June 3, 2009
Can my authoring tool support more then one behavior?
Question:
How can I verify that my authoring tool supports more than one behavior?
Answer:
Most authoring tools simply have 1 SCORM behaviour and 1 AICC behaviour. If the author wants a different behaviour, they have to break the courses down to page=SCO. Then, they can pay for the programming effort. In ReadyGo WCB, all the behavior is programmed through JavaScript. Each LMS interface has a file that lists the component pieces used for it during course generation. There are components for the "index.htm" page, for the main/first page, for the bullet pages, for the test pages, for the sidebar, for the services/menu bar, and for the exit page. These different pages play their own role in the lifecycle of a course. The listing is in a .des file. The contents of the .des file are described in a file called “desfile.txt” in the LMS folder. The .des file contains the names of the files that are used in the different places throughout a course (e.g. when a test loads, within the index page, etc.) There is a file called "descriptions.txt" in the "lms" folder that contains a summary of every LMS pack.
How can I verify that my authoring tool supports more than one behavior?
Answer:
Most authoring tools simply have 1 SCORM behaviour and 1 AICC behaviour. If the author wants a different behaviour, they have to break the courses down to page=SCO. Then, they can pay for the programming effort. In ReadyGo WCB, all the behavior is programmed through JavaScript. Each LMS interface has a file that lists the component pieces used for it during course generation. There are components for the "index.htm" page, for the main/first page, for the bullet pages, for the test pages, for the sidebar, for the services/menu bar, and for the exit page. These different pages play their own role in the lifecycle of a course. The listing is in a .des file. The contents of the .des file are described in a file called “desfile.txt” in the LMS folder. The .des file contains the names of the files that are used in the different places throughout a course (e.g. when a test loads, within the index page, etc.) There is a file called "descriptions.txt" in the "lms" folder that contains a summary of every LMS pack.
What does a SCORM LMS actually do?
I was just asked some great questions on how LMS's actually work with a course. I thought others may be interested in this conversation:
Question:
I have worked with a number of LMS's. Why is it that all the work is done on the course side? The LMS must do something so that the course can send data. What does the course do, what does the LMS do? Is the LMS the data receiver and then processes the data in a given way (how flexible?. Does the LMS create the reports?
The course sends the data. I need an example here for example the status complete:
Does an LMS has a datafield that is set to “completed – not completed” depending on what the course sends? How many datafields are there for a course to be “SCORM”-compatible? Which fields? What does an LMS need to do to be SCORM-compatible?
Start a course, notice the exit, write a status, store the test results ----?
Answer:
For most LMSs, what the LMS has to provide is a User ID when they launch the course, the “suspend_data” (2048 characters that the course set the previous time it was taken), the user name, the previous time on course, and a few other “core” fields. That is what is provided with a good SCORM course. There is no logic that the LMS is supposed to perform (with SCORM 1.2 and earlier): An LMS only provides data storage and retrieval. With AICC, since the course couldn’t really get to the content sent by the LMS, the requirement for data retrieval was minimal.
For SCORM 1.2, conformance, all that a course needs to do is send LMSInitialize and LMSFinish. This would make a course SCORM “conformant”. That is why you will see tools out there that can turn a Word document into a SCORM conformant “course”. Of course, instructionally this has no value except to track that someone has taken the course. A simple web statistics tracker could tell you that (you can see what users based on their IP address have seen what pages of content.)
For an LMS, to be SCORM 1.2 conformant, they must handle LMSInitialize(), LMSCommit(), LMSSetValue(), LMSGetValue(), and LMSFinish(). The important work is set by LMSSetValue, which specifies the variable name and the value. To be 1.2 conformant, there are 10 “core” values that must me supported. Most 1.2 LMSs have traditionally only supported about 5 of these (student_id, student_name, lesson_location, credit, lesson_status, entry, score, total_time, exit, session_time). There are some other groups of values called “objectives” and “interactions” where a course author could put in the score for individual questions. In SCORM 2004, support for these is required. The ReadyGo courses have used these in SCORM 1.2, but there are only a handful of LMSs that have tracked them and even fewer that have been able to report on them.
For continuity of behavior from one session to another, therefore, the course generally has to put the data it might need into the “suspend_data” field, and then it can parse it when it loads. With the better LMSs, the course could ask for a listing of the objectives and interactions that were previously provided along with previous lesson status and score. The difficulty is if you have 20 questions in 4 tests within one SCO (unit). It is hard to retrieve this type of detail from most LMSs since they only support the “core” set. So, with WCB, we use the suspend_data field to store/retrieve previous scores that have been set by the course. Then, in terms of logic for certificate generation, the course has to take care of it.
Because of the difficulties in retrieving data, there is a perception that a SCO should be designed to only have one question in it. That is why you will see many authoring tools that set up every test question, and usually every page of content to be a separate SCO. The golden rule with SCORM is that one SCO cannot send the user to another SCO, so this means that after each SCO (or “page” in most cases), the course has to exit from the LMS, and the LMS has to launch the next SCO. In some cases, this can take a noticeable time since the LMS has to retrieve the user’s previous data and package it so that it can be obtained by the SCO. In a good LMS, this may be a lot of data. So the end user suffers a delay between each SCO. If the SCO is designed as a chapter or an entire course, the delay will not be noticeable. However, if each page is a SCO, the course-taking experience could be painful.
Question:
I have worked with a number of LMS's. Why is it that all the work is done on the course side? The LMS must do something so that the course can send data. What does the course do, what does the LMS do? Is the LMS the data receiver and then processes the data in a given way (how flexible?. Does the LMS create the reports?
The course sends the data. I need an example here for example the status complete:
Does an LMS has a datafield that is set to “completed – not completed” depending on what the course sends? How many datafields are there for a course to be “SCORM”-compatible? Which fields? What does an LMS need to do to be SCORM-compatible?
Start a course, notice the exit, write a status, store the test results ----?
Answer:
For most LMSs, what the LMS has to provide is a User ID when they launch the course, the “suspend_data” (2048 characters that the course set the previous time it was taken), the user name, the previous time on course, and a few other “core” fields. That is what is provided with a good SCORM course. There is no logic that the LMS is supposed to perform (with SCORM 1.2 and earlier): An LMS only provides data storage and retrieval. With AICC, since the course couldn’t really get to the content sent by the LMS, the requirement for data retrieval was minimal.
For SCORM 1.2, conformance, all that a course needs to do is send LMSInitialize and LMSFinish. This would make a course SCORM “conformant”. That is why you will see tools out there that can turn a Word document into a SCORM conformant “course”. Of course, instructionally this has no value except to track that someone has taken the course. A simple web statistics tracker could tell you that (you can see what users based on their IP address have seen what pages of content.)
For an LMS, to be SCORM 1.2 conformant, they must handle LMSInitialize(), LMSCommit(), LMSSetValue(), LMSGetValue(), and LMSFinish(). The important work is set by LMSSetValue, which specifies the variable name and the value. To be 1.2 conformant, there are 10 “core” values that must me supported. Most 1.2 LMSs have traditionally only supported about 5 of these (student_id, student_name, lesson_location, credit, lesson_status, entry, score, total_time, exit, session_time). There are some other groups of values called “objectives” and “interactions” where a course author could put in the score for individual questions. In SCORM 2004, support for these is required. The ReadyGo courses have used these in SCORM 1.2, but there are only a handful of LMSs that have tracked them and even fewer that have been able to report on them.
For continuity of behavior from one session to another, therefore, the course generally has to put the data it might need into the “suspend_data” field, and then it can parse it when it loads. With the better LMSs, the course could ask for a listing of the objectives and interactions that were previously provided along with previous lesson status and score. The difficulty is if you have 20 questions in 4 tests within one SCO (unit). It is hard to retrieve this type of detail from most LMSs since they only support the “core” set. So, with WCB, we use the suspend_data field to store/retrieve previous scores that have been set by the course. Then, in terms of logic for certificate generation, the course has to take care of it.
Because of the difficulties in retrieving data, there is a perception that a SCO should be designed to only have one question in it. That is why you will see many authoring tools that set up every test question, and usually every page of content to be a separate SCO. The golden rule with SCORM is that one SCO cannot send the user to another SCO, so this means that after each SCO (or “page” in most cases), the course has to exit from the LMS, and the LMS has to launch the next SCO. In some cases, this can take a noticeable time since the LMS has to retrieve the user’s previous data and package it so that it can be obtained by the SCO. In a good LMS, this may be a lot of data. So the end user suffers a delay between each SCO. If the SCO is designed as a chapter or an entire course, the delay will not be noticeable. However, if each page is a SCO, the course-taking experience could be painful.
Subscribe to:
Posts (Atom)