ICT in Education Toolkit

Tool 4.4 Evaluation of Course Authorship & Management Systems
 
  1. Definition of Systems
 
An authoring system is not an automated instrument that authors contentware. It is a software package that assists developers in the "process" of producing contentware and gives them the means to create e-learning materials more efficiently and in a standardized manner. The authoring system is particularly helpful in creating multimedia modules that bring together different components: text presentations, graphics, interactive questions, simulation, video and audio clips, and animation.

Guidelines for designing and developing IVT-enhanced content are presented in Tool 4.5.

A Learning Management System (LMS) is a software program that streamlines the management of course offerings and learners. Typical features include:

  • Student register
  • Course register
  • Facility for tracking learners' progress
  • Systematic reporting
  • Scheduling

Most LMSs also include additional features such as

  • Communication features such as email, chat rooms and bulletin boards
  • Bookmarking, so that users can restart a module at the point where they left it last time they used it.

Combined authorship and LMS. Many instruments on the market combine features of authorship and learning management. They are sometimes referred to as LCMSs (Learning Content management Systems). They normally contain the following components: authoring application, data repository, delivery interface, and administration tools. In addition, some LCMSs offer collaboration tools including chat rooms, integrated email, and discussion groups.

Authoring or Management System Item


Authoring System Profile

Product Name:

Brief Description

System Type
 Authorware
 LMS


Source (where to find):

 
  2. Search for Existing Systems
 


Existing guides and portals can save considerable effort and time in identifying potentially useful software and websites. Most of them offer annotated lists, but many of them can be searched by product and features. Some include only the products that are evaluated by experts and deemed of satisfactory quality.

Click below for a list of sources broadly divided under the two categories of authorware and LMSs. Items under each category predominantly contain resources belonging to this category but sometimes they contain resources that belong to the other category or to both. It is therefore important to search both categories even if the search is intended for either type of software.

GUIDES TO EXISTING SYSTEMS

The Team may supplement the above search by searching catalogues of publishers and venders and surfing the web for authorware and LMSs.

As a result of the search and review process, the Expert Team may be able to narrow down the search to few potentially suitable systems that are deemed of good quality by external sources. The results may be indexed and recorded in a Team's database. To enter an item, click button "Create" button below.

Evaluate Authoring or Management System Item


Evaluation of Authorware or LMS Product

Evaluate this product by rating its suitability under each of the listed criteria.

1. Scope of System
To what extent does the system cover the functions expected from it? For example, in an authorware how far does the system provide features such as course template, web-based authoring, support of multimedia modules, easy course structuring, easy navigation, extendable architecture, etc? Similarly how far does an LMS offer features such as:
  • Student register
  • Course register
  • Facility for tracking learners’ progress
  • Systematic reporting
  • Scheduling
  • Communication features such as email, chat rooms and bulletin boards
  • Bookmarking, so that users can restart a module at the point where they left it last they used it.
 Below Average  Average  Good  Very Good 

2. Functionality
How well does the system integrate and is compatible with other parts of your network? What hardware, operating systems and related systems are needed, and are they different from what is already available or planned in Tool 5.2? Is there a need for additional investment?
 Below Average  Average  Good  Very Good 

3. Language
Is the main language of the system suitable? Does the system support additional languages?
 Below Average  Average  Good  Very Good 

4. Usability
How user-friendly is the system? Does it offer a HELP facility, documentation and opportunities for training? Is the delivery and presentation of material easy to use and highly intuitive? Can the user master the system within reasonable time? Does the system serve the diverse needs of different users simultaneously?
 Below Average  Average  Good  Very Good 

5. Scalability
Does the system have the capability to expand—or “scale”—to meet future growth, in terms of functions, volume of data, and size of users? How feasible is the expansion in terms of cost and programming?
 Below Average  Average  Good  Very Good 

6. Inter-operability
Does the system support content from different sources and hardware/software solutions from multiple vendors? Does it adhere to open industry standards for Web deployments (XML, SOAP or AQ) and support the major learning standards (AICC, SCORM, IMS and IEEE)? Does it allow for integration with other systems?
 Below Average  Average  Good  Very Good 

7. Security
Can the system limit and control access to online content, resources and back-end functions, both internally and externally, for its diverse user community?
 Below Average  Average  Good  Very Good 

8. Maintenance and Support
Is the system supported by the vendor? Will the system be maintained onto the future? If the system is Open Source, is there a strong community associated with it?
 Below Average  Average  Good  Very Good 

9. Licensing, Deployment and Implementation
There are basically three licensing arrangements:
  • The software is purchased and installed and managed in-house.
  • The acquired system is housed and managed remotely by a third party. Administrators, content builders, instructors and learners access the system over the Internet.
  • The customer buys the software and installs it on its own data platform, but the maintenance and upgrades are managed by the vendor or another third party.
There are fundamentally two kinds of Open Source licenses:
  • "Copylefting" license which allows anyone to change the program - but those changes must be provided to recipients under exactly the same conditions as the original. In other words, an Open Source system cannot be later turned into a proprietary program by a third party.
  • “Non-copylefting” license allows recipients of the code to do anything they want with the code - including making a modification and producing a proprietary version.
The vendor may offer other options for deployment and implementation.

How suitable are the licensing, deployment, and implementation arrangements of the system being evaluated?
 Below Average  Average  Good  Very Good 

10. Cost
When assessing cost, all costs must be considered: software, licensing, customization, training, maintenance contract, etc. Is the cost reasonable? How suitable is it to the Project’s budget?
 Below Average  Average  Good  Very Good 

11. Industry Standards
How well does the system adhere to industry Standards?
 Below Average  Average  Good  Very Good 

12. Strength of the Vendor and Product
How long has the vendor been in business? How strong is the vendor financially? Does the producer/vendor have a good reputation and staying power in the market? How long has the product been on the market? How widely is the product used?
 Below Average  Average  Good  Very Good 

OVERALL RATING
On the basis of the above individual ratings, make a judgment as the overall suitability of the product. Click one option. The rating will be recorded under the corresponding item in the List of Potentially Suitable authoring and Learning Management Systems
 Below Average  Average  Good  Very Good 
 
  3. Evaluation of Identified System
 


The Team needs to evaluate promising systems identified above for their quality and suitability. Review and evaluation may be conducted by the Expert Team or by a designated specialist.

To evaluate a system, select the item you wish to evaluate below and fill in the evaluation form.

Selection of a System


Select System
 System is suitable for deployment

 
  4. Selection of a System
 


There is no perfect system that is rated high on every criterion. Moreover, different potential users may give different weights to different criteria. To select an authoring system or an LMS, it is recommended that the Team retrieves List of Potentially Suitable Authoring and Learning Management Systems which contains overall rating for each evaluated product, thus allowing for rough comparison and creating a "short list" of suitable systems.

On the basis of these reviews and the weight given to each criterion, the Team will be able to make a value judgment as to the most suitable systems. Click on the corresponding title and register on the form tha the "System is suitable for deployment".  A appears next to the item in the List.

 
  OUTPUT OF THIS TOOL
 

The application of this tool generates the following files:

These files are saved in the Country's Filing Cabinet. They may be accessed anytime from the Filing Cabinet or from the above list by clicking on the specific file.

If any documents have been uploaded to the Team Bookshelf, they may be accessed at anytime.

Please go back to Toolkit Map/Home to continue to the next Toolbox.

 
  Reference Information
 

Guides to Existing Authoring and Course or Learning Management Systems

Authoring Systems

Evaluating Digital Authoring Tools

http://www.irrodl.org/content/v5.2/technote3.html
A wide range of inexpensive courseware authoring tools has emerged, enabling courseware designers to develop effective in-house materials and delivery platforms. Seven software tools are evaluated, representing the following categories: single purposes; activity creation; course development and presentation; general presentation; testing and assessment.

E-Learning Center

http://www.e-learningcentre.co.uk/eclipse/vendors/authoring.htm
E-Learning Center web site provides a long list of links to sources of content development tools for the production of online web-based courses.

Instructional Design: Choosing the Proper Authoring Tool

http://www.stc.org/intercom/PDFs/2004/2004011_24-27.pdf
This document contains lists of full-service authoring tools and software application simulation programs.

Overview of Authoring Systems and Learning Management Systems

http://www.peak.co.uk/AuthoringSystem.pdf
This regularly updated report contains annotated lists of the widely authoring systemsand learning management systems. PDF Version. Last updated December 2002

Moodle

http://moodle.org/
This is an Open Source authorship and management system.

Macromedia Authorware 7

http://www.macromedia.com/software/authorware/?promoid=home_prod_aw_082403
Macromedia Authorware helps create rich-media courseware for CD/DVD and the internet. It creates multimedia content with PowerPoint, and integrates graphics, sound, animation, text and video into engaging content for Web and online learning.

Macromedia Director MX  

http://www.macromedia.com/software/director/
This software allows you to integrate all sorts of multimedia, from video stream to audio to text, which can then be deployed on DVD or the Web. Text-to-speech capabilities are available for the visually-impaired, and users with disabilities can use the keyboard to maneuver rather than the mouse.
 

Course or Learning Management Systems

Evaluation of Learning Management Systems (Swiss Virtual Campus)
http://www.edutech.ch/
The purpose of this instrument is to assess the features and evaluate the strengths and weaknesses of course management systems used to deliver web-based distance courses. Six major systems are assessed in respect to the student environment that is provided, the modes of tutoring and didactics that are supported, course development features, and system administration. There is a detailed summary report for each system, and the user can also select two or three systems to compare them feature by feature.

Online Collaborative Tools: Evaluation Reports (Atabasca University)

http://cde.athabascau.ca/softeval/index.htm
This instrument compares the features of software packages that facilitate web-based collaborative learning. Information is provided on the following categories of software: course management systems for delivering web-based distance courses, asynchronous discussion/bulletin boards, chat messaging, voice and video e-mail, audio conferencing, video conferencing, polling with delayed feedback and polling with immediate feedback. Up to 30 features are reported for each software package within a given category. (Available in html or PDF format.)

COL Evaluation of open source learning management and content management systems

www.col.org/Consultancies/03LMSOpenSource.htm
This instrument includes criteria to evaluate open source LMSs. The criteria were applied to 35 products and the short list was systematically evaluated using hands-on testing.

EduTools Course management systems

http://www.edutools.info/course/
This site offers a database of about 40 content management systems that have been researched and reviewed. The site allows users to search by product name or by features, and to compare all or some products by some or all features.

E-Learning Center

http://www.e-learningcentre.co.uk/eclipse/Resources/choosing-lms.htm
E-Learning Center  web site provides a long list of links to sources of LMSs for online or CD-based web-based courses.

Overview of Authoring Systems and Learning Management Systems  

http://www.peak.co.uk/AuthoringSystem.pdf
This regularly updated report contains annotated lists of the widely authoring systems and learning management systems. PDF Version. Last updated December 2002

Open source Course management Systems

http://www.edtechpost.ca/pmwiki/pmwiki.php/EdTechPost/OpenSourceCourseManagementSystems

An annotated list of about 50 systems


intraLearn

http://www.intralearn.com/1000_Products.asp
IntraLearn technology can be used by teachers to develop and deliver online educational courses. With minimal IT skills and a standard browser, teachers can help students make the most of features such as email, chat rooms, streaming multimedia and other communication tools.