Intelligent tutoring system

[0002] This application claims the priority of Provisional Patent Application Serial No. 60/187,865, filed Mar. 8, 2000, the disclosure of which is expressly incorporated by reference herein.

[0003] A computer program listing is submitted in form of compact disk.

[0004] The present invention is directed to a computer implemented method and apparatus for simulating an intelligent tutor for interactive adaptive training of learners in any domain (that is, field of learning) under study.

[0005] Known computer-based training systems generally may be divided into two groups: classical Computer Based Training (CBT) and advanced Intelligent Tutoring Systems (ITS). Classical CBT systems are based on “manual” design of training “scripts”, which predefine what the CBT system should to do in any learning situation for each learner. Because of the enormous number of possible situations and diversity of learners, it is practically impossible to design quality training systems by manual scripting.

[0006] Intelligent Tutoring Systems, on the other hand, include two opposite approaches: Case-Based and Model-Based training systems. Case-Based Training systems are learnable and can be trained by expert-tutors how to teach learners. In the training stage, they memorize the most effective actions of the tutor in current learning situations from examples (cases) given by designer (author). After that “training”, in the implementation stage, such ITS can teach learners itself by taking the same teaching actions in the same and close situations by recalling the cases from memory. In practice learners encounter a large number of learning situations, so that the number of cases to train the system must be large as well. In contrast, short and affordable authoring can provide only very rough simulation of tutor's activity. Case-Based Training systems may be regarded as useful when applied complementary to the following model-based approach.

[0007] Model-Based Training systems need neither scripts to enter nor cases to learn. Rather, they are based directly on generalized tutoring knowledge and experience represented with models. Models can be represented in the form of empirical rules, decision tables, and theoretical relations. A part of these models can be reusable for different training paradigms/domains/learners. Reusability allows reducing the authoring labor. Examples are REDEEM, XAIDA, SIMQUEST, MOBIT.

[0008] Within Model-Based Training systems it is possible to distinguish empirical and theoretical modeling. Empirical model-based ITSs are designed usually by rule-based authoring. This is similar to rather high-level programming with production systems, which can be supported with some commercial tools for knowledge-based systems design. Such systems provide authors with capability to design and implement reusable rules and save their labor in this manner, but do not provide them with ready-made reusable models. See REDEEM, XAIDA, SIMQUEST.

[0009] Theoretical model-based ITSs are based on ready-made models that are specific for a training field and are generic within this field. These models comprise the best pedagogical experience materialized in a form of a reusable model-based Shell, which can be used by different authors to save on authoring for different training paradigms/domains/learners. However, there are at present no ready-made theoretical models, shells and tools for theoretical model based ITS's. Accordingly, some authors have partially solved this problem using a bottom up approach, for example, providing authors with specific templates within which blanks can be filled in. However, no known system has heretofore used a top=down approach and imprecise tunable modeling, as achieved by the system according to the invention.

[0010] Prior art model-based intelligent training systems have included:

[0011] a Domain Module for presentation and/or simulation of domain under study;

[0012] an expert system for presentation of ideal task performing;

[0013] a learner model defining personal knowledge, skills, and traits for training individualization,

[0014] an instructional or tutorial module defining objectives, strategy, style, and modes of training, and

[0015] an interface for system interaction with learner and author.

[0016] These prior intelligent tutoring systems suffer, however, from a number of important deficiencies. First, they are empirical (not theoretical) model-based systems. Moreover, their tutorial module does not comprise a generic and reusable ITS Shell, so that authors need to re-design the ITS tutorial module for different training paradigms/domains/learners from scratch Altogether this means a high cost of ITSs and an absence of their quality guarantee.

[0017] Accordingly, a primary goal of the present invention is to provide a reusable, training paradigms/domains/learners-independent, theoretical, actively adaptive, fuzzy, domain/learner/tutor model-based Shell with checking/prognosis/diagnosis-based dynamic planning capability.

[0018] Another object of the invention is to provide an ITS of the type described above, which, being filled in with concrete meta and media data by an author, can automatically realize intelligent, high quality, and efficient interactive adaptive training at low cost.

[0019] Yet another important objective of the invention is to provide a theoretical model based ITS on the basis of a top down approach and imprecise tunable modeling.

[0020] Additional objectives of the invention are:

[0021] to support learner-driven training;

[0022] to provide reuse of ready-made CBT, simulators, and games learning materials;

[0023] to supervise (control over) functioning of traditional ready-made CBT, simulators, games, and other training systems and applications;

[0024] to support plug in of existing web, CBT, simulation, game, and other applications authoring tools for learning/training content media design;

[0025] to support plug in of existing pre-authoring tools for job/task cognitive analysis;

[0026] to realize the system's self-improvement capability.

[0027] These and other objects and advantages are achieved by the intelligent tutoring system according to the invention which includes some of the same standard modules as other ITS's (Domain Module, Tutor Module, and interface) but with new properties, functions, structure, modes and in a new interrelation among them. In this regard, an important feature of the invention resides in the methodology used to structure and represent items to be learned and their pre-requisite and other dependency relationships (in any subject domain for any category of learners, and on the basis of any training paradigm and theory) in a (fuzzy) graph, together with a (fuzzy logic) computational engine (learner model) which dynamically adapts the available sequence of training actions (such as presentations/explanations, simulations, exercises, tasks/questions) to a current assessment of the learner's knowledge/skills, the level of difficulty of the presented material, and preferences and learning style of the individual learner. According to the invention, fuzzy logic is used as the basis of arc weightings and the computations, but the general methodology is applicable to other approaches to weighting and computation. Details of the programming functionality for important features of the invention are set forth in Table I (including a glossary) at the conclusion of this specification, of which it is a part.

[0028] The Intelligent Tutoring Systems Shell and Authoring Tool according to the invention is implemented in the form of a computer program for the following purposes:

[0029] For easy design of cost-effective Intelligent Tutoring Systems by filling in training paradigm/domain/learner—specific data into blanks of training paradigm/domain/learner—generic system Shell;

[0030] For executing cost-effective adaptive training of learners tailored to individual learner's current knowledge/skills, difficulty level, control level, preferences, and learning style in accordance with training objectives, strategy, and style of training pre-defined by the author.

[0031] The main system properties are:

[0032] The system can deliver realistic domain content in interactive multimedia, simulation, VR media.

[0033] It can be used any time and in any place, through the Internet as well.

[0034] It is a mixed initiative system providing a learner with capability to learn on his own and to use instructional support of an embedded tutor when it is necessary to resolve learning problems and/or achieve learning objectives by the most effective way,

[0035] The system provides a complete set of basic instructional modes: pre-test, instruction, practice, remediation, post-test.

[0036] The system makes learning/training logic transparent for learners by displaying the current assessment of learner's knowledge/skills mastery and corresponding system's training action.

[0037] High quality and effectiveness of training that can be expressed in terms of meeting objectives and learners' satisfaction in shortest time.

[0038] Low cost of system design, that can be expressed in terms of effort, time, and recourses of authors for course design and development.

[0039] Low cost of training that can be expressed in terms of consuming time and resource for achieving learning/training objectives.

[0040] The system simulates the best tutor that adapts to a learner.

[0041] The system pre-tests student to ensure that he/she is ready to learn the course unit selected.

[0042] The system dynamically plans and executes an individual training process in one-on-one interaction with learner.

[0043] The system gives to learner new knowledge/skills by providing him/her with necessary learning materials and/or domain situations for knowledge/skills construction.

[0044] The system realizes the optimal teaching strategy and style for the courseware available.

[0045] The system adapts to learner's personal knowledge, skills, preferences, and learning style.

[0046] The system can check learner's knowledge/skills tracking his/her progress and detecting errors.

[0047] The system assists learner by finding the causes of his/her errors and correcting them by appropriate re-training.

[0048] The system lets a learner control the pace and progress of his/her own learning.

[0049] The system post-tests a learner to make sure he/she successfully achieved the course or unit's training objectives.

[0050] The system supports the following author's operations:

[0051] (1) Login;

[0052] (2) Create a new course or open an existing course;

[0053] (3) Specification of a customer order for course design in terms of a target audience, particular domain, job/tasks, and training objectives;

[0054] (4) Pre-authoring session for an analysis of target audience, particular domain, job/tasks, and training objectives;

[0055] (5) Authoring session of the course synthesis (design and development) for target audience, particular domain, job/tasks, and training objectives;

[0056] (6) Course release, testing, and debugging;

[0057] (7) Implementation results analysis. Course correction.

[0058] (8) Close a course;

[0059] (9) Logout.

[0060] The system supports the following learner's operations:

[0061] Login;

[0062] Open course and unit;

[0063] Start unit

[0064] Learning/Training session;

[0065] Stop unit;

[0066] Close unit and course;

[0067] Logout.

[0068] Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.

[0069] FIG. 1 is a conceptual diagram which illustrates the composition of the overall ITS according to the invention;

[0070] FIG. 2 is a conceptual diagram which illustrates the composition of the Domain Module according to the invention;

[0071] FIG. 3 is a conceptual diagram which illustrates the composition of the Tutor Module according to the invention;

[0072] FIG. 4 depicts the functional interconnections among the components of the Tutor module and its functional scheme;

[0073] FIG. 5 is a conceptual diagram which illustrates the hierarchal structure of the tutoring modes according to the invention;

[0074] FIG. 6 is a flow chart which illustrates the operation process implemented by the Tutor module;

[0075] FIG. 7 is a conceptual diagram that illustrates the functional scheme of the behavior recognition component of the Tutor module;

[0076] FIG. 8 illustrates an example of the author's interface for the entry of unit properties;

[0077] FIG. 9 illustrates an instance of the author's interface for training (testing) action properties with embedded behavior (performance) prediction model;

[0078] FIG. 10 is a conceptual diagram which illustrates the cognitive interpretation model according to the invention;

[0079] FIG. 11 shows an example of the author's interface to enter the cognitive interpretation model;

[0080] FIG. 12 depicts a particular example of a fuzzy graph according to the invention, which embodies a learner's interface to present cognitive items assessment and a focus of the next training action in a particular domain;

[0081] FIG. 13 is an example of domain content presentation to the learner in the system according to the invention;

[0082] FIG. 14 is an illustration of an interface screen which shows the system adaptation of difficulty level according to the invention;

[0083] FIG. 15 is a diagram which illustrates the mapping of a training action into cognitive items in the system according to the invention; and

[0084] FIG. 16 shows the generic system architecture.

[0085] The system 1, which represents a computer program comprising a Domain Module and Tutor Module that interact with each other and with users (authors and learners), is illustrated in FIG. 1. It includes a CPU 2 (with memory 2A) operated by systems software 3, including a Domain Module 4 and a Tutor Module 25. Both of the latter modules include a graphic user interface (GUI) for communication with both authors and learners.

[0086] The Domain Module 4, which is illustrated in FIG. 2, provides users with interactive presentation of a part of the world (subject matter) under study that can include text, pictures, sounds, video, animation, multimedia, expert systems, simulations, games, 3D, VR, Internet resources. An authoring tool 4ais used to design and develop these interactive presentations. The Domain Module includes a user interface with learner and an author. In a preferred embodiment, the learner's interaction with domain occurs through domain windows, which can include control/navigation and task performance means. Authors can interact with the Domain Module through the same windows but extended with a capability to edit, check, verify, and debug data entered.

[0087] The Domain Module can be closed or open for tutor observation and control. In an open embodiment the Domain Module includes special observation and control means to support Tutor Module's control over the Domain Module.

[0088] The Tutor Module is illustrated in FIG. 3. It simulates a tutor's activity and provides monitoring and control over a learner's study of the domain and tasks performance. To design and develop these specific capabilities it includes an authoring tool 5a. The Tutor Module includes also a user interface with a learner and an author. Learner's interaction with the Tutor Module is realized through a tutor window, which can contain control and communication means as well. Authors can interact with the Tutor Module through the same window but extended with a capability to edit, check, verify, and debug course designed. The Tutor Module contains also means for capturing learner's behavior and control over Domain Module through direct and indirect communication means.

[0089] Overview of System Operation

[0090] An entire cycle of system operation consists of the following phases:

[0091] (1) Specification of customer order for course design in terms of target audience, particular domain, job/tasks, and training objectives;

[0092] (2) Pre-authoring session for an analysis of target audience, particular domain, job/tasks, and training objectives;

[0093] (3) Authoring session of the course synthesis (design and development) for target audience, particular domain, job/tasks, and training objectives;

[0094] (4) Course release, testing, and debugging;

[0095] (5) Course implementation on audience (with at least one learner);

[0096] (6) Course outcome analysis. If the outcome does not match the order specification, then returning to (2) for the next iteration of the system operation.

[0097] A course order specification includes at least definitions of

[0098] (1) target audience,

[0099] (2) domain,

[0100] (3) job/tasks, and

[0101] (4) objectives;

[0102] (5) Post-testing threshold.

[0103] A pre-authoring phase includes

[0104] (1) An analysis of the target audience and filling out and audience database with the audience profile data;

[0105] (2) Iterative Job/tasks decomposition aimed to meet predefined audience profile and filling out the job/task database;

[0106] (3) Mapping of job/tasks decomposition results into the audience profile to check mutual correspondence. If there is no mutual correspondence, return to (1-2) for corrections;

[0107] (4) Training objectives definition inside the job/tasks and outside of the audience profile;

[0108] (5) Decomposition of each training objective into sub-objectives;

[0109] (6) Splitting each sub-objective into behavioral sub-objective (behavior patterns) and cognitive sub-objective (cognitive items);

[0110] (7) Interactive mapping of behavioral sub-objectives into job/tasks to check their necessity and sufficiency for job/tasks. Corrections if it is necessary.

[0111] (8) Interactive mapping of cognitive sub-objectives into audience profile to check sufficiency of audience profile for these cognitive objectives. Corrections if it is necessary.

[0112] A course authoring includes the following steps

[0113] (1) Presenting the author a template “Course properties” to fill out. It includes course name, ID, author's name, version name, short description, course status, modes statuses, difficulty levels, style parameters, and association with corresponding Domain Module;

[0114] (2) Presenting to the author a generic template to design a specific map of the course. The template includes pre-test, introduction, units, post-test, and conclusion. An author should select at least one of them for the course.

[0115] (3) If the author selected a pre-test, then pre-test authoring is performed;

[0116] (4) If the author selected an introduction, then presenting him/her a set of introduction templates for choice. Each template may have a specific appearance, but the same content structure identifying pre-defined target audience, job/tasks, training objectives, and recommendations what to do;

[0117] (5) If the author selected a post-test, then post-test is performed;

[0118] (6) If the author selected a conclusion, then presenting to him/her a set of conclusion templates for a choice. Each template may has specific appearance, but the same content structure including congratulations to the learner, the objectives achieved, a list of job/task/needs relevant, and recommendations what to do next;

[0119] (7) If the author selected Units, then course structuring by entering and editing a unit list and prerequisite relations among units (unit flowchart);

[0120] (8) Checking, verification, and correction of entered data.

[0121] In the course implementation phase each learner goes through the login procedure to begin. After that the system presents him/her a list of available courses and units. The learner makes his/her choice and starts the selected course/unit. Within the course/unit run the learner has a choice to use the Domain Module only or together with the Tutor Module. Using the Domain Module only, the learner can navigate and act in it through a “domain window”. In this case he/she studies the domain via a self-driven mode without any control from the Tutor Module. It is not an effective way for learning and very often learners encounter learning problems. To resolve these problems in the present system the learner can activate the Tutor Module at any time (even before encountering any problem).

[0122] Even when not activated (in the self-driven learning mode of the Domain Module), the Tutor Module is not entirely passive. It observes the learner's behavior and makes its own “understanding” of what is going on with the learner and how he/she meets learning objectives and resolves learning problems. When the learner achieves terminal objectives, the Tutor Module informs the learner about it. So the present system in its passive state represents an “Intelligent Learning Environment”.

[0123] In its activated state, the Tutor Module begins to communicate with the learner through a “tutor window” which appears, navigating and directing the learner in the domain to meet learning objectives and resolve the learner's problems in the most effective way.

[0124] In the learning stage, the learner has an extra capability of control over the tutor's module as well as by selection of currently preferred tutoring modes and styles.

[0125] The Domain Module

[0126] Definition. The Domain Module (FIG. 2) is a computer application program representing a part of the system's computer program, which provides authors with the capability to enter and edit domain content, and learners with the capability to navigate through course content and study it in his/her own way; and in one preferred embodiment, it provides the Tutor Module with observations about domain situations and learner's actions in the Domain Module. The domain content generated by the Domain Module can include any text, pictures, sound, animations, and video. It can comprise multimedia interactive presentations, interactive simulations, games, expert systems, 3D environment, and Virtual Reality with appropriate control/navigation and task performance means. In some embodiments the Domain Module can include Intranet and Internet resources.

[0127] The Domain Module can be represented by another training system of lower level (such as CBT, case-based ITS, rule-based ITS, . . . ), which can be open enough for learner behavior observation and control over it by the present Tutor Module.

[0128] Properties. To be able to support easy authoring and high quality of learning/tutoring, the Domain Module possesses certain properties:

[0129] In the preferred embodiment the Domain Module can deliver to a learner realistic content in interactive multimedia, simulation, VR media form.

[0130] It can be used autonomously for self-driven learning;

[0131] For easy design by non-programmer authors, it allows plug in of the existing media editors and authoring systems.

[0132] For easy access and effective learning, domain content should be structured by author and provided with browsing and navigation capability.

[0133] It should include interactive task performance means as well to activate learning.

[0134] For the best learning/training, complex domains should be represented by authors on a different level of abstraction: from textual/graphic abstract presentations to more realistic simulations and virtual reality.

[0135] For adaptation to different learners, a domain should be represented in several levels of difficulty and be oriented to different learning styles and preferences. So, as a rule, the domain representation should be redundant.

[0136] It can contain pre-stored and/or generated materials; the former can increase the speed, the latter can reduce redundant storage.

[0137] In preferred embodiment Domain Module should allow embedding of specific “observation objects” for informing the Tutor Module about the domain situations and learner behavior.

[0138] A Domain Module should allow the Tutor Module control over the domain content presentation to the learner.

[0139] Functionality. The Domain Module has at least two main stages of functioning: content authoring support and content delivery to the learner. In addition, one embodiment includes the additional functionality of informing the Tutor Module.

[0140] Authoring function: the Domain Module supports the authoring process, in which authors enter and edit domain content data. The Domain Module checks entered data, and supports testing and debugging of domain content as a whole.

[0141] Delivery function: the Domain Module delivers specific content to learner in response to his/her control/navigation actions (and dependent on Tutor Module control actions).

[0142] Communicative function: in one embodiment the Domain Module delivers a domain situation, learner's control actions, and task performance data to the Tutor Module to inform it what is going on.

[0143] Composition. The Domain Module comprises an application program, a simulator, CBT, a low level ITS, or part of virtual world under study, including in a preferable “open” embodiment:

[0144] Domain data (content) defining specific domain content under learner's study;

[0145] A Domain shell representing generic computer program for the domain data organization and interactive domain simulation including:

[0146] A Domain database 4b(FIG. 2) defining generic (domain-independent relations) structure, and for containing specific domain data, its recording, storing, and retrieving capabilities. It is organized in Courses and Course Units (or at least allows mapping data into separate courses and course units). Data can represent domain objects and domain agents under learner's study.

[0147] A Domain player 4c(FIG. 2) defining generic domain procedures with domain data and its runtime engine. Different domain players are used to deliver media data such as sound, video, animation, multimedia. In preferred embodiment it includes a tunable computer program for interactive domain simulation, 3D, VR;

[0148] Domain-User interface including

[0149] Domain-author interface

[0150] Domain-learner interface

[0151] In one embodiment Domain-Tutor interface (as a domain part of Direct Communicator);

[0152] The Domain authoring tool 4asupporting the domain data input into the domain database by an author. The authoring tool comprises a diversity of media editors that are used to enter new and edit existing domain content in different media form.

[0153] Particularly the system's Domain Module can represent and/or include some autonomous systems (such as software packages, applications, simulators, expert systems, and co-learner models) and tutors (such as electronic textbooks, tutorials, tests, CRT, low level ITS) that can be under control of present system's Tutor Module. See FIG. 2.

[0154] The domain database 4borganizes and contains the domain data including in the most advanced embodiment the following fields within each course/unit data:

[0155] A plurality of domain objects capable of interacting with each other, domain agents, learner, and Tutor Module;

[0156] A plurality of domain agents solving their tasks by possibly interacting with each other, domain objects, learner, and Tutor Module;

[0157] In one embodiment, a plurality of observation objects providing the Tutor Module with data about domain situations and learner's behavior in these situations;

[0158] Specific interface among domain objects, domain agents, learner, and Tutor Module.

[0159] The domain player in a preferred “open” embodiment, defines generic domain procedures and provides runtime simulation of domain under learner study for every course/unit comprising

[0160] Players of domain objects, domain agents, and (in one embodiment) observation objects (generic object/agent's programs working with database filled in with specific said domain data);

[0161] Inter-object interface (generic interface to be filled in with specific interface from the said domain database);

[0162] The domain-learner interface (generic interface to be filled in with specific interface from the domain database);

[0163] Runtime manager;

[0164] Timer.

[0165] The Domain Module interface includes a domain-author interface, a domain-learner interface, and a domain-tutor interface. The Domain-Author Interface provides an author with a capability to enter, check up, and edit media data. The Domain Module can use plugged in existing media-authoring tool with ready-made computer-human interface that is familiar to potential authors. As a rule all media authoring tools include specific windows delivering content presentation, navigation, request, and interaction means (buttons, menus, multiple choice questions, filing in the blank, hot spots, and so on) with extra capability to edit them and use media templates available.

[0166] In one embodiment, to support Domain-Tutor interaction, the author enters specific “observation objects” into the Domain Module and predefines certain commands from Tutor Module to the Domain Module. So the domain-author interface provides the author with this capability as well.

[0167] The Domain-Learner Interface provides the learner with flexible access to any part of the Domain Module allowed by the Tutor Module. Because the present Domain Module can use plugged in media editors and multimedia authoring tools, they provide their own players, browsers with their own learner's interface. It includes the domain windows delivering presentations with navigation and interaction means (buttons, menus, multiple choice questions, filing in the blank, hot spots, and so on) but without editing capability of author.

[0168] In a preferred embodiment the Domain Module includes observation objects 4dto send control/situation/performance data into the Tutor Module. From the other side, the Domain Module is able to receive and fulfill commands from the Tutor Module.

[0169] The domain authoring tools 4asupport domain data input into the domain shell including

[0170] The domain-author interface including

[0171] Generic templates of domain objects, domain agents and (in one embodiment) observation objects to fill in with specific data;

[0172] In one embodiment, generic templates of observation objects to fill in with specific data;

[0173] Generic template of said inter-object/agent interface to fill in with specific data;

[0174] Generic templates of the said domain-learner interface to fill in with specific data;

[0175] Means to input and edit said domain data into domain database;

[0176] Means to check and verify the domain data;

[0177] Means to debug the course/unit of the Domain Module.

[0178] Operation of the Domain Module. There are two main stages of Domain Module operation: authoring and content delivery (or self-driven learning). In the authoring stage the author enters, checks, edits, and debugs a domain content with its control/navigation and interaction means. In the learning stage each learner can navigate through the domain and perform domain task using control and interaction means predefined by authors. In both stages an author or learner can browse the domain database through “domain windows”. They can also run any presentations, play sound and video records, run simulations and interact with them. In this manner, a learner can use the Domain Module on his/her own for self-driven learning. An author has the same capability with an extension to edit, check, test, and debug a domain content.

[0179] A control over the Domain Module that is “open” for authoring includes:

[0180] (1) In the authoring stage, definition of control commands to send to Domain Module;

[0181] (2) In the learning/training stage, commands to be sent to Domain Module for execution.

[0182] On the other hand, a control over Domain Module that is “closed” for authoring, includes

[0183] (1) In the authoring stage, capturing/recording interface messages of the user to the Domain Module during user control over the Domain module;

[0184] (2) If it is not enough, then capturing/recording of control messages from the operational system to the Domain Module during user control over the Domain Module;

[0185] (3) In the learning/training stage, emulation/playing back of captured/recorded messages.

[0186] In a preferred embodiment, the “open” Domain Module provides a learner (through a tutor interface) with a choice of

[0187] (1) Domain glossary (explain me words);

[0188] (2) Domain reference material describing (explain me);

[0189] (3) Presenting objects and agents in the domain field (show me);

[0190] (4) Interactive simulation of objects and agents behavior in the domain to investigate (let me investigate);

[0191] (5) Capability to select mission/job/tasks and perform selected mission/job/task by him/herself (let me try myself).

[0192] The Tutor Module

[0193] Definition. The Tutor Module (FIG. 3) is a computer program representing a part of a system computer program, which simulates human-tutor activity and can be authored by author and activated by learner at any time to manage his/her learning by the most effective way.

[0194] Properties. “Intelligence” is the most important property of the Tutor Module. Even before authoring, the Tutor Module has its own pedagogical “knowledge/skills”. An intelligent Tutor Module knows exactly “how to teach” learners (how to manage a learning process in order to deliver new experience to the learner, to test mastery, to diagnose and remedy learning problems) by itself without authors. The Tutor Module has also specific knowledge about “whom to teach” (the audience). Learners themselves, in the login step of a learning session, will deliver this information to the Tutor Module, inputting their ID, personal preferences and learning styles. All other personal data (learner's current assessment, achievements, pre-history) the Tutor Module can generate itself and keep in its memory.

[0195] To be ready to teach each particular learner, the Tutor Module needs to know concrete audience, domain, course/unit, training objectives, strategy, and style to use. It is going to get information from authors.

[0196] In addition to above-mentioned skills (how to teach) and knowledge (whom to teach), the Tutor Module embeds certain knowledge (templates, frameworks) for media domain-related data from authors. The templates and frameworks make authoring process much easier, providing a process of “filling in the blanks” of the shell. As a result, entered data can be automatically checked and interactively verified by the author for consistency and sufficiency. The author can also run and debug a course/unit designed on a different level of detail. All these features significantly simplify authoring, allow the input data to be verified, exclude mistakes and reduce the cost of course design.

[0197] In the training delivery session the Tutor can be in passive and active state. In the passive state, Tutor Module “watches over the learner's shoulder” and can capture, recognize, and interpret a learner's behavior in order to understand what is going on with learner's knowledge/skills. The present system with the Tutor Module in a passive state represents a new solution to the so-called “Intelligent Learning Environment”. In the active state, on the other hand, the Tutor Module can individually plan, re-plan, and execute training actions delivered through the Domain Module and/or directly to the learner, pursuing predefined training objectives, realizing predefined strategy of training, and realizing predefined tutoring style. The Tutor Module provides learners with a choice of its operations modes.

[0198] Functionality. The Tutor Module supports the main functions as follows.

[0199] Pre-authoring

[0200] Input: Raw data from an author entered through generic templates of author interface;

[0201] Result: Entered, tested, and debugged tutor data in a tutor database including target audience,

[0202] domain,

[0203] job/tasks,

[0204] objectives, and

[0205] Post-testing threshold.

[0206] authoring

[0207] Input: pre-authoring data from previous stage; Raw materials and data from an author entered through generic templates of author interface;

[0208] Result: Entered, tested, and debugged tutor data in a tutor database.

[0209] In the passive state the main Tutor Module monitors:

[0210] Input:

[0211] Domain situation data;

[0212] Learner's control actions;

[0213] Learner's task performance;

[0214] Output:

[0215] Performance comments, if requested;

[0216] Performance advice, if requested;

[0217] In a case of “fatal” faults, Tutor Module activation for a remediation session.

[0218] In the active state the main Tutor Module function is training:

[0219] Input:

[0220] Learner personal data and assessments;

[0221] Domain situation data;

[0222] Learner's control actions in the Domain Module;

[0223] Learner's task performance;

[0224] Learner's control actions in the Tutor Module;

[0225] Output:

[0226] The next training action (presentation of a learning material, exercise, task/question, . . . );

[0227] Learner's performance Comments;

[0228] Performance advice, if requested;

[0229] Results:

[0230] Switching the modes of training;

[0231] Updated assessments of learner's personal data.

[0232] Composition

[0233] The Tutor Module includes the following components:

[0234] Tutor data defining training paradigm/domain/learner/-specific knowledge of tutor;

[0235] a Tutor shell comprising ready-made training paradigm/domain/learner/-generic pedagogical knowledge and skills including:

[0236] Tutor database 5b(FIG. 3) for defining and organizing generic tutor data (domain-independent) structure and containing specific tutoring media and metadata (declarative models), its recording, storing, and retrieving capabilities;

[0237] Tutor player 5c(FIG. 3), for defining generic tutoring procedural models and runtime engine. It represents a run-able computer program for tutoring simulation;

[0238] Interface;

[0239] a Tutor pre-authoring tool supporting analysis of target audience, job/tasks, and training objectives;

[0240] Tutor authoring tool 5asupporting the tutor data design, input into the tutor database 5b, and editing by an author;

[0241] The tutor database includes sub-databases as follows:

[0242] Audience database provides the Tutor Module with a knowledge of a target audience to orient course to;

[0243] Job/tasks database that support preauthoring sessions and provides a basis for training objective definition;

[0244] Media database filled out by the author with a concrete course/unit structure, objectives, strategy, style parameters, training actions, and models provides the Tutor Module with paradigm/domain specific pedagogical knowledge;

[0245] Learner Database filled out by the author with data providing the Tutor Module with knowledge of its audience, person by person.

[0246] The tutor player 5c(FIG. 3) provides a runtime simulation of tutor activity for any course/unit. As shown in FIG. 4, it comprises the following components:

[0247] a Behavior recognition component 6, which compares any captured action of learner in the domain and/or tutor windows with predefined patterns, and decides which one of them has been realized;

[0248] a Cognitive Interpretation function 7 for mapping recognized performance patterns ID of the learner into his/her cognitive items assessment with Cognitive Interpretation model.

[0249] a Decision Making component 8 for making decisions about sufficiency of training in different modes (learning, pre-test, instruction, practice, remediation, and post-test) and sub-modes (delivery, testing, and diagnosing) and which sub-mode and/or mode to switch next;

[0250] an Actions Planning component 9 for planning and selection of the next training action (message to learner, command to Domain Module);

[0251] an Action Generation unit 10 for generation of the next training action in a media form;

[0252] The Tutor-Learner interface 11 (generic interface template to be filled in with specific interface from the said Tutor Database);

[0253] The Runtime Manager 22 (with a timer for a generation of an internal time flow) provides runtime environment for cooperative working of all components in any state (passive or active) and in different modes of active state (pre-test, instruction, practice, remediation, and post-test); and

[0254] An authoring tool 23.

[0255] The Interface 11 includes a Tutor-User interface that provides

[0256] Tutor-learner interface (available only in the active state of Tutor Module) A learner communicates with the Tutor Module using the same “tutor window”, through which he/she can get any messages such as introduction, task/question posing, explanation, hint, comment, and conclusion.

[0257] Tutor-author interface. An author communicates with the Tutor module through a “tutor window”, which contains edit, check, verify, and debugging functions. This interface allows the author to fill in all databases and models of Tutor Module.

[0258] The Interface also includes a Tutor-Domain Communicator capturing Domain Module situations and learner behavior, and for sending control commands to Domain Module including at least one of the following

[0259] Direct Communicator between the Domain and Tutor modules. The Tutor Module communicates with the Domain Module in the run-time sessions of course debugging or learning. In both of the Tutor Module states (active and passive), the Domain Module transfers the learner's control actions, domain situations, and learner's performance into the Tutor Module. In the active state, the Tutor Module transfers control events into the Domain Module. This interface is provided with tutor's observation objects 4d(FIG. 2) embedded in the Domain Module 4 (FIG. 2) and the domain's actions predefined in action library of Tutor Module. The observation objects send predefined events into Tutor Module. The Tutor Module captures them and its training actions send control commands to Domain Module.

[0260] Domain-Learner Interface Communicator providing interception and emulation of Domain-Learner exchange;

[0261] Finally, the Interface also includes a Domain-Operational System Communicator providing interception and emulation of Domain-Operational system exchange.

[0262] A tutor pre-authoring tool supports a detailed analysis of target audience, job/tasks, and training objectives and includes

[0263] The tutor-author interface including

[0264] Generic template for specific course order representation;

[0265] Generic template for specific target audience profile representation;

[0266] Generic template for specific jobs/tasks representation;

[0267] Generic template for specific training objectives representation including

[0268] Cognitive items template;

[0269] Behavioral patterns template;

[0270] A component for input, editing and checking of specific data in tutor database;

[0271] Analysis means supporting decomposition of audience profile, job/tasks, and objectives;

[0272] A component for classifying each leaf objective into cognitive item and behavioral patterns (behavior predictive model);

[0273] A component for mapping

[0274] job/tasks analysis results into audience profile to check mutual correspondence;

[0275] behavioral patterns of training objectives (behavioral predictive model) into job/tasks to check their necessity and sufficiency for job/tasks;

[0276] cognitive items of training objectives into audience profile to check sufficiency of audience background, experience, and other cognitive items for these objectives.

[0277] The tutor authoring tools 5a(FIG. 3) provide entry and editing capability for the tutor data including

[0278] The tutor-author interface including

[0279] Generic templates of the said metadata including

[0280] Template with audience data to orient authoring to;

[0281] Template with cognitive items to orient authoring to;

[0282] Generic templates of the said media data;

[0283] A component to structure a course into units and flowcharting them;

[0284] A component to input and edit specific said tutor data into tutor database;

[0285] A component to check and verify the tutor data;

[0286] A component to run and debug the course/unit of the Tutor Module.

[0287] Operation of Tutor Module. The main operating stages of the Tutor Module are pre-authoring, authoring, and learning/training implementation. In the pre-authoring stage, the authoring component of the Tutor Module supports the author's specification of course order, audience profiling, job/tasks analysis, and training objectives definitions. The author can edit, check, and verify entered data as well. On the other hand, in the authoring stage, the authoring component of the Tutor Module supports the author's course structuring, training strategy and style definition, entering of training actions (messages, commands), models, performance patterns, and comments into the tutor database. The author can edit, check, and verify entered data as well.

[0288] A course implementation phase includes the following steps:

[0289] (1) Providing a learner with a registration means;

[0290] (2) Providing a system with the learner personal data;

[0291] (3) Providing a learner with a choice of a course/unit to study;

[0292] (4) Providing a learner with a domain and tutoring module for the course selected;

[0293] (5) Providing a system with the learner's initial assessment within the course/unit selected;

[0294] (6) Providing an interaction of the learner with domain for course study;

[0295] (7) Providing an interaction of the learner with tutoring module for its activation, mode selection, objectives selection, and style adjusting;

[0296] (8) Checking if a current assessment of the learner matches the objective or the learner exits it, and stopping training if so; otherwise continue;

[0297] (9) If learner has not activated the Tutor Module, then the Tutor Module realizes a passive state;

[0298] (10) If learner has activated the Tutor Module, then the Tutor Module realizes an active state;

[0299] (11) Providing a learner with capability to stop the system anytime;

[0300] (12) Storing of updated learner's data.

[0301] In the passive state the Tutor Module “observes” learner's behavior in the Domain Module without any interruption and training actions, unless the learner requests it. It captures learner's behavior data transferred from domain observation objects through domain-tutor interface, including control/navigation actions by the learner in the domain, the current domain situation, and the learner's task performance in the Domain Module.

[0302] Learner activity (control/situation/intentionsà performance) is recognized by the behavior recognition component of the Tutor Module, on the basis of patterns predefined by an author. Recognized learner's behavior is interpreted by the cognitive interpretation component of the Tutor Module that provides mapping of each performance pattern into the assessment of learners' current knowledge/skills. The result of interpretation is a personal assessment of the learner's current cognitive items (knowledge/skills). The learner can ask the system to comment on his/her performance (comment me), to evaluate his/her performance (evaluate me), and to advise what to do next (advise me). The Tutor Module provides this service. The Tutor Module wakes up automatically in case of the learner's “fatal” faults.

[0303] In the active state the Tutor Module extends its passive state operation with decision making and planning capabilities in accordance with objectives, strategy, tactics, and style predefined by the author, and mode pre-selected by the learner (on strategic control level). The current assessment of the learner's knowledge/skills is used further in the decision making component to make (tactical control level) decisions on what to do next (begin, continue, or stop content delivery, checking knowledge, or diagnosing bugs). This decision and the current assessment are then used by the planning component to define the next tutoring action (on operational control level).

[0304] The active state of the Tutor Module includes f five operational modes, including pre-testing, instruction, practice, remediation and post-testing, as shown in FIGS. 5 and 6.

[0305] The goal of the pre-test mode 31 (FIG. 6) in the course/unit is to check that the learner has all necessary background for the particular course/unit of study. It is not necessary to have a pre-test for every course unit. But if the author has prescribed it for the particular course/unit, then all learners should pass it. The Tutor Module commences operation in this mode every time, independently of the learner's control (if the student did not pass it successfully before).

[0306] In the pre-testing mode the planning component uses tasks and questions as training actions assigned for “pre-test” in their properties windows. In accordance with the author's choice, pre-testing tasks and questions can be presented to the learner in differing styles:

[0307] at once (batch);

[0308] in a preset sequence that is the same for all learners (linear);

[0309] in a preset script that is the same for all learners, wherein the next task/question depends on previous performance of the learner (branching), and

[0310] by intelligent adaptive sequencing, wherein the next task/question depends on the current assessment of the learner's knowledge/skills.

[0311] It is up to the author which style of presentation to pre-define for learners. (In the absence of a selection, the system selects intelligent adaptive sequencing as a default).

[0312] Planning of batch testing is realized by the learner him/herself.

[0313] There is no need to plan linear tests because the whole sequence of tasks/questions is fixed. All that is necessary is to decide when the learner has passed the test. That is exactly what the decision component is doing by comparing a learner's current assessment of pre-tested items with a predefined pre-testing threshold.

[0314] Planning of the adaptive test is done in three stages:

[0315] Filtering of pre-test tasks/questions;

[0316] Weighting of each task/question's power (coverage of non-tested items yet);

[0317] Selection of the task/question with the maximal power

[0318] Each task/question is generated and presented to the learner; and the learner's performance is captured, and recognized. In the intelligent option it is also interpreted and assessed. In this case the decision component stops pre-testing when the current assessment for all pre-tested items becomes higher then the predefined threshold “Pre-testing threshold”.

[0319] The goal of the instruction mode 13 (FIG. 5; see steps 33-34 in FIG. 6) is to manage delivery and mastery of new content in accordance with predefined objectives, strategy and style.

[0320] This mode represents a combination of the Delivery sub-mode 13aand the Practice mode 14 (FIG. 5; see steps 37-39 in FIG. 6).

[0321] In the Delivery sub-mode of the Instruction mode 13a(FIG. 5; see step 34 in FIG. 6), the planning component of the Tutor Module can select all training actions assigned to the Instruction mode in the action properties window by destination “Instruction”. These training actions are called here “teaching actions”. In accordance with author's choice of Delivery Style they can be presented to the learner in differing style:

[0322] at once (batch),

[0323] in a preset sequence that is the same for all learners (linear),

[0324] in a preset script that is the same for all learners, wherein each next action depends of previous performance of the learner (branching); and

[0325] by intelligent adaptive sequencing, wherein each next action depends on the current assessment of the learner's current knowledge/skills, his/her personal preferences and learning style.

[0326] It is up to the author which presentation style to pre-define for learners. (Absent his/her selection, the last option takes place automatically).

[0327] The Batch instruction requires no planning at all. It is realized by the learner him/herself. Also, there is no need to plan a linear instructional sequence, because it is fixed. All that is necessary to do is decide when the learner has passed certain thresholds. This function is performed by the decision component, by comparing the learner's current assessment of expected knowledge/skills items mastery with a threshold “Delivery threshold”.

[0328] Planning of the intelligent adaptive Delivery on the other hand is done in three stages:

[0329] Filtering of teaching actions;

[0330] Weighting of each teaching action's current power; and

[0331] Selection of teaching action with the maximal weight and difficulty level (that is still appropriate for the learner).

[0332] The selected teaching action is realized, the learner's responsive performance is captured and recognized. In intelligent delivery, it is also interpreted and assessed. In this case the decision component 35 (FIG. 6) stops the instruction to begin the practice mode when a gap between current assessments of expected and demonstrated cognitive items becomes higher than a predefined threshold.

[0333] The goal of the practice mode 14 (FIG. 5; step 37 in FIG. 6) is to manage mastery of the unit knowledge/skills delivered before, detect mistakes of the learner and remedy their causes in accordance with predefined training strategy, training style, the learner's personal preferences and learning style. This mode represents a combination of Testing sub-mode 14aand Remediation mode 15. (See steps 36 and 40 in FIG. 6.)

[0334] In the Testing sub-mode of Practice (step 36 in FIG. 6) the planning component can use all training actions with destination “Practice” in an action Properties window. Usually these actions (“testing actions”) are of a testing type: tasks or questions. In accordance with author's choice they can be presented to the learner in differing styles:

[0335] All at once (batch),

[0336] in a preset sequence that is the same for all learners (linear);

[0337] in a preset script that is the same for all learners, wherein each successive problem choice depends on the prior performance of the learner (branching); and

[0338] by intelligent adaptive sequencing, wherein each next problem choice depends on the current assessment of the learner's knowledge/skills, his/her personal preferences and learning styles.

[0339] It is up to the author which one to pre-define for learners in the course unit. (As a default the system selects the intelligent option.)

[0340] Batch instruction requires no planning at all. It is realized by a learner him/herself. Also, there is no need to plan the linear instructional sequence, because it is fixed. All that is necessary is decide when the learner has passed mastery thresholds, which is done by the decision component by comparing a learner's current assessment of demonstrated knowledge/skills mastery with a threshold “Testing threshold”. (See step 41 in FIG. 6.)

[0341] Planning of the intelligent adaptive Testing is done in three stages:

[0342] Filtering of testing actions;

[0343] Weighting of each testing action's current power;

[0344] Selection of testing action with the maximal power.

[0345] The selected problem is realized, the learner's responsive performance is captured and recognized. In the intelligent testing it is also interpreted and assessed. In this case it continues until the current assessment of demonstrated knowledge exceeds an expected knowledge/skills assessment for all knowledge/skills items (step 41 in FIG. 6). Then the decision component stops the practice and switches to remediation mode when the learner has made more mistakes than set by the author in the parameter “Tolerance threshold” (step 42 in FIG. 6).

[0346] The goal of the remediation mode 15 (step 40 in FIG. 6) is to manage the search of learning problems (knowledge/skills bugs—causes of mistakes made) and fix them by an instruction at a reduced level of difficulty in accordance with a predefined strategy and style. For this purpose, remediation comprises a Diagnosing sub-mode 15a(FIG. 5; step 43 in FIG. 6) and corrective re-instruction mode 15bat a reduced level of difficulty that begins from causes of mistakes. (See steps 46-48 in FIG. 6.)

[0347] In the Diagnosing sub-mode of Remediation, the planning component can use all training actions with destination “Practice” and “Explanation” in their properties window. Usually these actions are of a testing type (tasks and questions) and explanation type (training actions and/or messages on reduced level of difficulty). In accordance with author's choice they can be presented to the learner in differing styles:

[0348] at once (as a batch of available explanations for self-remediation);

[0349] in a preset sequence that is the same for all learners (linear procedure);

[0350] in a preset script that is the same for all learners, wherein each next action depends of previous performance of learner (branching); and

[0351] by adaptive sequencing, wherein each successive action depends of current assessment of learner's knowledge/skills, his/her personal preferences and learning styles.

[0352] It is up to the author which style to pre-define for learners in the course unit. (The default is the last, intelligent choice).

[0353] Batch remediation requires no planning, being by learner him/herself.

[0354] There is no need to plan linear instructional sequences predefined by author as well, because they are fixed. All that is necessary to do is to decide when the learner has reached the required level of mastery. This is done by the decision component, by comparing the learner's current assessment of demonstrated knowledge/skills mastery with a predefined threshold “Testing threshold”.

[0355] Planning of the intelligent adaptive Diagnosing is done in three stages:

[0356] Filtering of diagnosing actions;

[0357] Weighting of each diagnosing action's current diagnostic and/or explanative power; and

[0358] Selection of diagnosing action with a maximal diagnostic power.

[0359] The selected diagnosing action is realized and learner's responsive performance is captured and recognized. In intelligent remediation, it is also interpreted and used for learner's knowledge/skills assessment. In this case the decision making component (steps 45 in FIG. 6) stops the remediation and returns to the practice mode, when all suspected knowledge/skills have been remedied.

[0360] The goal of the post-test mode 16 (FIG. 5; step 44 in FIG. 6) is to assure that the learner has mastered all knowledge/skills of the course unit. It is not necessary to have the post-test in every unit of the course. But if the author has designed the post-test in the course unit, then all learners can use it in order to bypass regular instruction, if they believe they know the unit material enough.

[0361] In the post-test mode the planning component uses training actions of testing type: tasks and questions with destination “Post-Test”. In accordance with author's choice they can be presented to the learner in differing styles:

[0362] at once (batch),

[0363] in a preset sequence that is the same for all learners (linear);

[0364] in preset script that is the same for all learners and depends on previous performance of the learner (branching); and

[0365] by intelligent adaptive sequencing, that depends on current assessment of the learner's knowledge/skills.

[0366] It is up to the author which style to pre-define for learners. (The last option is the default).

[0367] Planning of batch testing is realized by learner him/herself.

[0368] There is no need to plan linear tests because the sequence of tasks and questions is fixed. All that is necessary to do is just to decide when the learner has passed the test. The decision component does this by comparing the learner's current assessment of demonstrated knowledge/skills items with predefined threshold “post-test threshold”.

[0369] Planning of the intelligent adaptive post-test is done in three stages:

[0370] Filtering of post-test tasks/questions only (particularly, only a task/question with destination “post-test” in its properties window);

[0371] Weighting of each task/question;

[0372] Selection of the task/question with maximal weight.

[0373] Each task/question is presented to the learner; and his/her performance is captured and recognized. In intelligent post-test, it is also interpreted and assessed. In this case the decision making component stops post-testing, when the current assessment of demonstrated knowledge/skills items exceeds the threshold “post-test threshold” for every item.

[0374] The Tutor Module self-improvement capability is realized through the following steps:

[0375] 1. Delivery of new knowledge/skills to the learner;

[0376] 2. Testing of learner's knowledge/skills (and storing of unexpected answers of the learner);

[0377] If testing result is positive, then:

[0378] Increment fuzzy mapping (DM1j*l*v*, DM2j*l*v*, DM3j*l*v*,) of all teaching actions (which participated in this successful passage) outcomes into cognitive items predefined in cognitive interpretation model;

[0379] Update prospective fuzzy mapping (DM4) of these teaching actions outcomes;

[0380] Increment fuzzy mapping (OM1j*i*v*) of those parts of pre-stored testing actions/outcomes that are direct prerequisites (constitutes a background) to this success;

[0381] Update background fuzzy mapping (OM2) of these testing actions/outcomes;

[0382] Store fuzzy mapping (OM1j*i*v*) of all testing actions/outcomes which participated in positive assessment of new knowledge/skills;

[0383] If testing result is negative, then go to the next item; otherwise, repeat.

[0384] 3. Diagnosing of the learner's Knowledge/skills;

[0385] 4. Diagnosis posing;

[0386] 5. Corrective instruction;

[0387] 6. Checking of re-instruction success;

[0388] If Checking result is positive:

[0389] increment fuzzy mapping (OM2j*i*v*) of testing action/outcome that have detected this Diagnosis;

[0390] increment fuzzy mapping (OM2j*i*v*) of all testing actions/outcomes that have been used in Diagnosing procedure.(in distinguishing this diagnosis from others possible and localizing it);

[0391] increment fuzzy mapping (DM1j*l*v*, DM2j*l*v*, DM3j*l*v*) of all teaching actions/outcomes which have successfully corrected this Diagnosis;

[0392] update prospective fuzzy mapping (DM4) of these teaching actions outcomes;

[0393] save fuzzy mapping (OM1j*i*v*) of testing action/outcome which has positively checked Diagnosis correction;

[0394] If Checking result is negative, then go to the next item.

[0395] 7. Interactive author investigation and diagnosing of ITS Fault Reasons on the basis of automatically system recorded history of unsuccessful ITS solutions.

[0396] 8. Manual Correction of ITS metadata by an author.

[0397] 9. Interactive Testing of Correction Success.

[0398] Where:

[0399] DM k j*l*v*—fuzzy mapping of teaching actions (l)/outcomes (v) into cognitive items (j);

[0400] OM k j*i*v*—fuzzy mapping of testing/diagnosing actions (i)/outcomes (v) into cognitive items (j);

[0401] l—is a number of teaching action;

[0402] i—is a number of testing/diagnosing action;

[0403] v—is a number of performance pattern extended with a case of unrecognizable performance of learner (that considered as a fault performance);

[0404] k—(the first index) is a interpretation modality index having value: effect (expected knowledge/skills), background (demonstrated knowledge/skills), fault causes (suspected knowledge/skills);

[0405] k=1 means cognitive effect of learner's v-th performance predefined by the author;

[0406] k=2 means background of learner's performance learner's v-th predefined by the author;

[0407] k=3 means possible causes of learner's v-th performance predefined by the author;

[0408] k=4 means full-scale cognitive effect of learner's performance automatically calculated by the system for l-th or i-th training action;

[0409] k=5 means full-scale background of learner's performance automatically calculated by the system for l-th or i-th training action;

[0410] k=6 means full-scale possible causes of learner's performance automatically calculated by the system for l-th or i-th training action.

[0411] The Tutor Module Database

[0412] As previously noted the Tutor Module database includes the following sub-databases:

[0413] Audience database;

[0414] Job/tasks database including;

[0415] Meta database;

[0416] Media database; and

[0417] Learner's database;

[0418] Audience Database. The audience database defines the Tutor Module's knowledge of its audience including the following main record fields.

[0419] Registration data defining at least an average age, education, and experience of the audience.

[0420] Domain-independent data of the audience defining

[0421] Stable data to take into account in training process including

[0422] Learning preferences (theoretical, practical; . . . )

[0423] learning Style (for instance: verbal, visual, kinesthetic);

[0424] Unstable data to adapt training including

[0425] Background recall;

[0426] Generic knowledge/skills recall;

[0427] Prerequisite courses/units recall;

[0428] Initial difficulty/granularity level;

[0429] Initial control level.

[0430] Dynamic data to maintain in training process

[0431] Attention;

[0432] Intentions;

[0433] Motivation.

[0434] Domain-dependent data of the audience defining for each course the current status of course learning, including current status of unit learning for each unit within the course.

[0435] For each objective/cognitive item within the unit, this includes a personal assessment of

[0436] Expected mastery of cognitive item;

[0437] Demonstrated mastery of cognitive item;

[0438] Suspected mastery of cognitive item for single cause of errors made;

[0439] Job/Tasks database. The job/tasks database includes the record fields in following hierarchy:

[0440] Job including:

[0441] Task that may include a sequence of subtasks;

[0442] Subtask that may include a sequence of subtasks;

[0443] Meta database. The meta database comprises data about media data. Within each course it includes:

[0444] Course Properties including

[0445] ID,

[0446] Title;

[0447] author's name,

[0448] version name,

[0449] short description,

[0450] target audience profile (mapping),

[0451] job/tasks (mapping),

[0452] status (ready to use, under development),

[0453] Modes statuses:

[0454] Pre-Test (available, not available)

[0455] Instruction (available, not available)

[0456] Practice (available, not available)

[0457] Remediation (available, not available)

[0458] Post-Test (available, not available)

[0459] Mode/submode styles:

[0460] Pre-Test style (bunch, linear, branching, intelligent)

[0461] Delivery style (bunch, linear, branching, intelligent)

[0462] Testing style (bunch, linear, branching, intelligent)

[0463] Diagnosing style (bunch, linear, branching, intelligent)

[0464] Post-Test style (bunch, linear, branching, intelligent)

[0465] Sub-mode tactics:

[0466] adaptive delivery (teaching) planning:

[0467] Objectives are the first to achieve (challenging tactics);

[0468] Background are the first to create (grounding tactics);

[0469] Create minimally necessary background for the next available objective (by default)

[0470] adaptive testing planning:

[0471] The first items delivered—the first tested;

[0472] The last items delivered—the first tested;

[0473] The maximal coverage of delivered items (by default).

[0474] adaptive diagnosing planning for an author and a learner choice:

[0475] The closest causes—the first to check;

[0476] The deepest causes—the first to check;

[0477] The maximal distinguishing of all suspected causes (by default).

[0478] Style parameters

[0479] Delivery Threshold (0-100%)

[0480] Testing Threshold (0-100%)

[0481] Diagnosing Threshold (0-100%)

[0482] Delivery/Testing Threshold (0-1000)

[0483] Tolerance threshold (0-1000)

[0484] Difficulty range,

[0485] Difficulty switching threshold;

[0486] Association with corresponding Domain Module and media data;

[0487] Course pre-test (optional) metadata;

[0488] Course Introduction (recommended) metadata (audience, job, tasks, objectives);

[0489] Course post-test (optional) metadata;

[0490] Course Conclusion (recommended) metadata (job, tasks, objectives achieved, next courses for new objectives, tasks, jobs);

[0491] Units flowchart defining a course training strategy:

[0492] Unit list;

[0493] Unit prerequisite relations;

[0494] For each unit:

[0495] Unit Properties including:

[0496] ID,

[0497] Title;

[0498] author's name,

[0499] version name,

[0500] short description,

[0501] target audience profile (mapping),

[0502] job/tasks (mapping),

[0503] status (ready to use, under development),

[0504] Modes statuses:

[0505] Pre-Test (available, not available)

[0506] Instruction (available, not available)

[0507] Practice (available, not available)

[0508] Remediation (available, not available)

[0509] Post-Test (available, not available)

[0510] Mode styles:

[0511] Pre-Test style (batch, linear, branching, intelligent)

[0512] Instruction style (batch, linear, branching, intelligent)

[0513] Practice style (batch, linear, branching, intelligent)

[0514] Remediation style (batch, linear, branching, intelligent)

[0515] Post-Test style (batch, linear, branching, intelligent)

[0516] Sub-mode tactics:

[0517] adaptive delivery (teaching) planning;

[0518] Objectives are the first to achieve (challenging tactics);

[0519] Background are the first to create (grounding tactics);

[0520] Create minimally necessary background for the next available objective (by default).

[0521] adaptive testing planning:

[0522] The first items delivered—the first tested;

[0523] The last items delivered—the first tested;

[0524] The maximal coverage of delivered items (by default).

[0525] adaptive diagnosing planning for an author and a learner choice:

[0526] The closest causes—the first to check;

[0527] The deepest causes—the first to check;

[0528] The maximal distinguishing of all suspected causes (by default).

[0529] Style parameters

[0530] Delivery Threshold (0-100%)

[0531] Testing Threshold (0-100%)

[0532] Diagnosing Threshold (0-100%)

[0533] Delivery/Testing Threshold (0-1000)

[0534] Tolerance threshold (0-1000)

[0535] Difficulty range,

[0536] Difficulty switching threshold;

[0537] Association with corresponding Domain Module and media data;

[0538] Unit pre-test (optional) metadata;

[0539] Unit Introduction (recommended) metadata (audience, job, tasks, objectives);

[0540] Unit post-test (optional) metadata;

[0541] Unit Conclusion (recommended) metadata (job, tasks, objectives achieved, next units for new objectives, tasks, jobs);

[0542] Objectives flowchart defining a unit training strategy;

[0543] Objective list;

[0544] Objectives prerequisite relations;

[0545] For each objective:

[0546] Objective properties including

[0547] ID,

[0548] Title;

[0549] short description,

[0550] target audience profile (mapping),

[0551] job/tasks (mapping),

[0552] Style parameters

[0553] Post-Testing Threshold (0-100%)

[0554] Difficulty range,

[0555] Behavioral patterns:

[0556] Post-test tasks/questions;

[0557] Standard Performance;

[0558] Cognitive Items flowchart including:

[0559] Cognitive items list:

[0560] Prerequisite relations among cognitive items (training strategy);

[0561] For each cognitive item:

[0562] Cognitive Item properties including

[0563] ID,

[0564] Title;

[0565] short description,

[0566] Type:

[0567] Knowledge element with a desired mastery degree or

[0568] Skills element with a desired mastery degree or

[0569] Mental model aspect with a desired mastery degree or

[0570] Unintentional knowledge/skills deficiencies (misconceptions, misunderstanding, or bugs);

[0571] target audience profile (mapping),

[0572] Style parameters

[0573] Post-Testing Threshold (0-100%)

[0574] Difficulty range,

[0575] Learner control actions:

[0576] Navigation actions;

[0577] Domain controls (buttons)

[0578] Training Actions including the following record fields:

[0579] ID;

[0580] Title;

[0581] Type (teaching, testing);

[0582] Form (message, command, both);

[0583] Pre-conditions;

[0584] Post-Conditions;

[0585] Difficulty Level;

[0586] Learning Time;

[0587] Destination Modes (Pre-Test, Instruction, Practice, Remediation, Post-Test);

[0588] Sub-actions script (optional);

[0589] Association with a media level of the training action (message, command).

[0590] Behavior predictive model (may be case-based model, neural net model, . . . ):

[0591] Domain behavior model (may he case-based model, neural net model, . . . ):

[0592] Tutor's command ID list to the Domain Module;

[0593] Learner's control action ID list;

[0594] Domain situation pattern ID list;

[0595] Mapping of command IDs and control IDs into situation Ids (depends on the Domain Module state)

[0596] Learner behavior model (may be case-based model, neural net model, . . . ):

[0597] Learner intention model (may be case-based model, neural net model):

[0598] Tutor message ID list;

[0599] Learner's control action ID list;

[0600] learner's intention ID list;

[0601] Mapping of tutor's messages IDs and/or learner's control actions IDs into learner's intentions IDs;

[0602] Learner performance model (may be case-based model, neural net model):

[0603] learner's intentions ID list;

[0604] domain situations ID list;

[0605] learner's performance patterns IDs;

[0606] Mapping of learner's intention ID and/or domain situation ID into learner's performance patterns IDs (that depends on current assessment of learner's cognitive items).

[0607] Cognitive Interpretation model (may be case-based model, neural net model):

[0608] Behavioral patterns ID;

[0609] Cognitive Items ID;

[0610] Mapping of the behavioral patterns IDs into the cognitive items IDs;

[0611] Mapping of the behavioral patterns IDs into the job/tasks;

[0612] Particularly, the cognitive interpretation model can be represented in pre-stored form by two four-dimensional matrices of numbers from 0 till 100%:

[0613] DM[m][l][j][v] and OM[m][i][j][v].

[0614] Where:

[0615] l—is a number of teaching action;

[0616] i—is a number of testing/diagnosing action;

[0617] v—is a number of a behavioral pattern;

[0618] m—is a interpretation modality index having value: effect, background, causes;

[0619] m=1 means cognitive effect of learner's performance (expected knowledge/skills);

[0620] m=2 means background of learner's performance (demonstrated knowledge/skills);

[0621] m=3 means possible causes of learner's fault performance (suspected knowledge/skills);.

[0622] To simplify use of this model in a learning/training session, the system initially extends it into DM[k][l][j][v] and OM[k][i][j][v].

[0623] Where:

[0624] l—is a number of teaching action;

[0625] i—is a number of testing action;

[0626] v—is a number of performance pattern extended with a case of unrecognizable performance of learner (that is considered as a faulty performance);

[0627] k—is a interpretation modality index having value: effect, background, causes;

[0628] k=m=1 means cognitive effect of learner's v-th performance predefined by the author;

[0629] k=m=2 means background of learner's performance learner's v-th predefined by the author;

[0630] k=m=3 means possible causes of learner's v-th performance predefined by the author;

[0631] k=4 means full-scale cognitive effect of learner's performance automatically calculated by the system for l-th or i-th training action;

[0632] k=5 means full-scale background of learner's performance automatically calculated by the system for l-th or i-th training action;

[0633] k=6 means full-scale possible causes of learner's performance automatically calculated by the system for l-th or i-th training action.

[0634] Media Database. The media database comprises for each course/unit:

[0635] Course properties (how to display it);

[0636] Pre-test media;

[0637] Introduction media;

[0638] Post-Test media;

[0639] Conclusion media;

[0640] Units flowchart media (how to display the flowchart);

[0641] Within each unit:

[0642] Unit media data:

[0643] Unit properties (how to display)

[0644] Pre-test media;

[0645] Introduction media;

[0646] Post-Test media;

[0647] Conclusion media;

[0648] Objective flowchart media (how to display the flowchart);

[0649] Objective properties (how to display)

[0650] Cognitive items flowchart media (how to display the flowchart);

[0651] Cognitive item properties (how to display)

[0652] Training actions library:

[0653] Tutor commands Library (for injection into the Domain Module) with type of communication to use with Domain Module;

[0654] Tutor's Message Library (for presentation to a learner) with type of communication to use with learner;

[0655] Learner control actions library (for injection into the Domain Module);

[0656] Domain situation patterns Library/Simulation (for recognition);

[0657] Performance/Response patterns library (for a recognition);

[0658] Tutor comments library (to comment on the learner's performance, intentions).

[0659] The tutor commands library (above) keeps all necessary control actions of the Tutor to control the Domain Module. In the authoring stage, the author should predefine all these actions to the Domain Module. The library supports data entry and editing as well.

[0660] In the learning/training stage, the library provides a source of commands of the Tutor Module to control over Domain Module.

[0661] In the Tutor Message library, each message represents a partial case or a part of the training action that is being given to the learner directly through the tutor's window. The meta description of the message is in the training action library; but the media presentation of all messages is collected separately in the message library. This library, which is prepared by the author, provides a media form of introductions, explanations, hints, tasks posing, and conclusions to the learner within the course unit. Initially, the message library contains domain-independent message templates, which can be extended and specified by an author at will.

[0662] The control actions library keeps all possible control/navigation actions of the learner in domain and tutor's windows within the course unit. In the authoring stage, the author should predefine all these actions of learner. The control library supports data entry and editing as well. In the learning/training stage, the library provides control patterns for learner's control actions recognition.

[0663] The tutor's situation patterns library keeps patterns that are necessary for a current domain situation recognition. In the authoring stage, the author should predefine all these patterns on the basis of the available Domain Module. The library supports data entry and editing as well. In the learning/training stage, the library provides a source of media patterns for domain situation recognition.

[0664] The tutor's performance/response patterns library keeps patterns that are necessary for the learner's task performance recognition. In the authoring stage, the author should predefine all these patterns. The library supports data entry and editing as well. In the learning/training stage, the library provides a source of media performance/response patterns for learner's task performance recognition.

[0665] The comment library, which is prepared by the author, provides a media presentation of the tutor's direct reaction (such as confirmation, encouragement, . . . ) on the learner's performance and intentions within the course unit. So, each performance comment should be tailored to a particular performance pattern or its absence, predefined in the behavior predictive model. Initially, the comment library contains domain-independent comments, which can be extended and specified by author at will.

[0666] Learners database. Learners database provides the Tutor Module with the learners' personal data (which are not included in the audience database). This is a standard database, which is compatible with computer platform, operational system, and software environment of the system.

[0667] The learner database includes the following main record fields:

[0668] Registration data of the learner defining at least his/her ID, Name, User name, Password (and may be: Sex, Age, Affiliation, . . . )

[0669] Domain-independent data of the learner defining

[0670] Stable properties to take into account including

[0671] Personal preferences:

[0672] Personal learning Style (for instance: verbal, visual, kinesthetic);

[0673] Unstable capabilities to adapt training including

[0674] Current mastery of prerequisite courses/units;

[0675] Current difficulty/granularity level;

[0676] Current control level.

[0677] Dynamic features to maintain:

[0678] Learner Attention;

[0679] Learner Intentions;

[0680] Learner Motivation.

[0681] Domain-dependent data of the learner defining for each course the following records:

[0682] Current status of course learning (not permitted, permitted, ready, not attempted, browsed, delivered, incomplete, completed, failed, passed);

[0683] For each unit within the course defining:

[0684] Current status of a unit learning (not permitted, permitted, ready, not attempted, browsed, delivered, incomplete, completed, failed, passed);

[0685] For each objective/cognitive item within the unit, a personal assessment of

[0686] MB[j]—assessment of expected mastery of j-th knowledge/skills item;

[0687] MU[j]—assessment of demonstrated mastery of j-th knowledge/skills item;

[0688] MBOD[J]—assessment of knowledge/skills item mastery suspected to be a single cause of errors made;

[0689] MBDN[J]—assessment of knowledge/skills item mastery suspected to be one of multiple causes of errors made (the negative part only);

[0690] MDEF[j]—assessment of knowledge/skills item mastery suspected to be on e of multiple causes of errors made;

[0691] MI[j]—Membership of item to “start line”. (It is not stored in learner database);

[0692] MT[j]—Membership of item to “stop line”. (It is stored in learner database);

[0693] MH[j]—Membership of item to a set of necessary items to learn. (It is not stored in learner database);

[0694] P[j] is a weight of item in tutoring path taken;

[0695] MFLTR[l] is a teaching action filter value;

[0696] MFLTR[i] is a testing action filter value;

[0697] PrMFLTR is a previous value of a training action filter

[0698] An example of a cognitive items assessment display is shown in FIG. 10.

[0699] The current assessment of learner's knowledge/skills is represented to the learner for his/her inspection. It represents his/her knowledge/skills progress and deficiency, which serves as a deep-level feedback to learner from the system to support his/her self-driven learning. As shown, for example, in FIG. 12, which depicts a particular embodiment of the fuzzy graph in a particular domain, the nodes (vertices) of the fuzzy graph represent the items to be learnt and the arcs represent dependency relationships. Associated with each arc is a number between 0 and 100 representing the author's structuring the degree of dependency he/she thinks exists between vertices connected by the arc. (In FIG. 12 all such numbers are shown as 100. In practice, however, these numbers would vary between 0 and 100, as noted.) Thus a number of 50 between two nodes indicates that the item represented by the first node needs to be mastered only partly in order to master the item represented by the second node. When a presentation is shown relating to a particular node or a set of nodes, this is tracked by the engine and the nodes are marked “shown, mastery expected”. The level of difficulty or “granularity” of the material presented by the items is also noted, in terms of level of difficulty required and actual level of material available for instruction for that item. (Thus associated with each node is tutoring material at a few different levels of difficulty as shown in FIG. 14.) When questions are answered correctly or actions taken correctly, the nodes covered by those answers/actions are marked “mastery demonstrated”. However the mastery may not rise to a score of 100 until later answers/actions are recorded correctly. When an error is made at some node, all prior nodes that relate to that error are marked “mastery doubtful, or suspect”. The engine tracks and updates the state of the learner as depicted on the graph and directs the learner's navigating through the graph on his/her own individual path until mastery of the terminal learning objective is achieved.

[0700] The Tutor Module Player

[0701] The Tutor Player is a computer program realizing training functions by use of the Tutor Module database. It comprises the following components:

[0702] Behavior recognition;

[0703] Cognitive Interpretation;

[0704] Decision making;

[0705] Actions planning;

[0706] Actions generation;

[0707] Interface;

[0708] Runtime manager.

[0709] Behavior recognition. The behavior recognition component is illustrated in FIG. 7. It uses the behavior recognition model pre-stored in the database and generates an output in the form of a learner's performance ID, based on the learner's response on a training massage, his/her control actions, the domain situation, and his/her performance data (captured by the Tutor Module from the Domain Module). In a simulated-based embodiment, the behavior recognition component consists of the following sub-components:

[0710] (1) Control recognition 17;

[0711] (2) Domain behavior prediction 18;

[0712] (3) Situation recognition 19;

[0713] (4) Learner intentions prediction 20;

[0714] (5) Intention recognition 21;

[0715] (6) Learner performance prediction 22;

[0716] (7) Performance recognition 23.

[0717] Operation of behavior recognition sub-component includes the following steps:

[0718] (8) In the authoring stage, establishing a recognition threshold;

[0719] (9) Definition of a specific patterns set as a basis for a recognition;

[0720] (10) In the learning/training stage, receiving captured data;

[0721] (11) comparison of captured data with predefined patterns and calculation of similarity score;

[0722] (12) Making a decision whether or not captured data is similar to one of the patterns available with predefined threshold.

[0723] (13) If Not, then continue observation;

[0724] (14) If Yes, generate the result of recognition in terms of the most similar pattern ID.

[0725] The functional progression of the subcomponents of the Behavior Recognition component is shown in FIG. 7.

[0726] Control recognition 17 is a fully automatic sub-component of the Tutor Module. No authoring is needed. All actions of a learner in the tutor window and/or the domain window are captured and compared with controls patterns predefined in tutor media database. The ID of the nearest pattern represents a recognition result, and is output as the learner's control ID.

[0727] The role of the domain situations prediction sub-component 18 is to provide a Domain situation pattern set for the following domain situation recognition, based on the training message ID; and the Learner's control ID. This function is realized automatically based on the intention model, in the following steps:

[0728] (1) running of domain behavior model;

[0729] (2) Input tutor's command ID;

[0730] (3) Input learner's control ID;

[0731] (4) Output generation of domain situation patterns.

[0732] Situation recognition 19 is a fully automatic component of the Tutor Module. The domain situation depends not only on the actions of the Tutor Module, but also on some uncontrollable learner's actions and domain pre-history. In general, the current domain situation tends to go beyond the tutor's control and needs tutor's efforts to bring it back under control. The domain situation recognition is a computer program, part of the Tutor Module, which provides this capability, based on the current situation captured by the Tutor Module from the Domain Module and situation patterns from the domain situation prediction.

[0733] The situation recognition component consists of a comparison part and a decision making part. The latter can include an adjustable parameter (a sort of threshold) to adjust necessary reliability of recognition.

[0734] The Learner intentions prediction sub-component 19 provides a pattern set for learner intention recognition. It generates learner intention ID's, based on the training message ID, the learner's control ID, and the Domain situation ID. This function is performed automatically on the basis of learner intention model in following steps:

[0735] (1) running of learner intention model

[0736] (2) Input training message ID;

[0737] (3) Input domain situation ID;

[0738] (4) Input learner's control action ID;

[0739] (5) Output generation of learner's intention ID.

[0740] Intention recognition 21 is also a fully automatic component of the Tutor Module. The learner intentions depend on tutor's messages, domain situations, but also on some uncontrollable learner's actions and domain pre-history. In general, the current domain situation tends to go beyond the tutor's control, and needs tutor's efforts to bring it back under control. The intention recognition 21 is a computer program which provides this capability. It generates an output in the form of a learner intention ID, based on the learner's control action IDs pre-recognized and Domain situation IDs pre-recognized.

[0741] It consists of comparison part and decision making part. The latter can include an adjustable parameter to adjust necessary reliability of recognition.

[0742] The Learner performance prediction sub-component 22 provides a pattern set for the following learner performance recognition, based on the Learner's intention ID and the Domain situation ID. A learner performance prediction is realized on the

[0743] basis of the learner behavior model by the following steps:

[0744] (1) running of learner performance model;

[0745] (2) Input situation ID;

[0746] (3) Input learner's intention ID;

[0747] (4) Output generation of learner's behavior patterns

[0748] Performance recognition 23 is a fully automatic component of Tutor Module. In any domain situation, the learner can behave by following his/her own intentions. In order to understand better what and how learner is doing, the Tutor Module needs to know a domain situation and an intention of a learner in this situation. This task is solved by the performance recognition sub-component of the Tutor Module, which generates a Learner performance pattern ID, based on the Learner's actions in the tutor window and/or the domain window, and Performance patterns from Learner performance prediction sub-components.

[0749] This component consists of a comparison part and decision making part. The latter can include an adjustable parameter to adjust necessary reliability of recognition.

[0750] Here the index “v” (indicating the number of a behavioral pattern) is determined.

[0751] Cognitive Interpretation. Cognitive interpretation of learner's behavior provides an assessment of learner's current knowledge/skills based on the cognitive interpretation model (contained in the Meta database) and the learner's behavior ID pre-recognized.

[0752] Such cognitive interpretation includes the following steps:

[0753] (1) Generation of a conditional assessment of cognitive items:

[0754] Running cognitive interpretation model;

[0755] Input learner behavior ID into the cognitive interpretation model;

[0756] Output generation of conditional assessment of cognitive items by this model.

[0757] (2) Retrieval of the current assessment of the learner's cognitive items from the learner database;

[0758] (3) Calculation of new assessments by:

[0759] Combining of the assessment of expected cognitive items MB[j] with conditional assessment of expected cognitive items DM[3][l][j][v].

[0760] MB[j]=MB[j]+DM[3][l][j][v]−MB[j]*DM[3][l][j][v]/100;

[0761] Combining of the assessment of demonstrated cognitive items MU[j] with author's conditional assessment of demonstrated cognitive items OM [2][i][j][v]. MU[j]=MU[j]+OM [2][i][j][v]−MU[j]*OM [2][i][j][v]/100;

[0762] if for any item the assessment of expected cognitive item MB[j]happens to he less then assessment of demonstrated cognitive item MU[j], then expected assessment MB[j] is made equal to demonstrated one MU[j]. MB[j]=Max (MB[j]: MU[j]);

[0763] Intersection of the assessment of cognitive items single deficiency MBOD[j] with author's conditional assessment of suspected cognitive items OM[2][i][j][v];

[0764] MBOD[j]=MBOD[j]*(100−OM[2][i][j][v])/100 (for correct part of performance);

[0765] MBOD[j]=(MBOD[j]+OM[3][i][j][v])*(100−OM[2][i][j][v])/100. (for incorrect part of performance);

[0766] MBOD[j]=MBOD[j]*OM[2][i][j][v]/100. (for unexpected=incorrect performance).

[0767] Combining of the assessment cognitive items multiple deficiency with author's conditional assessment of suspected cognitive items.

[0768] MDEF[j]=MBDN [j]*(100−MU[j])/100.

[0769] (4) Replacing of all current assessments with new ones in the learner database.

[0770] In more detail the whole procedure of an assessment calculation looks like:

[0771] For teaching type of training actions:

[0772] MB[j]=MB[j]+DM[3][l][j][v]−MB[j]*DM[3][l][j][v]/100;

[0773] For testing/diagnosing type of training actions:

[0774] For Correct expected part of performance

[0775] MU[j]=MU[j]+OM[2][i][j][v]−MU[j]*OM[2][i][j][v]/100;

[0776] MBOD[j]=MBOD[j]*(100−OM[2][i][j][v])/100;

[0777] MB[j]=Max(MB[j], MU[j]);

[0778] For incorrect expected part of performance

[0779] MBOD[j]=(MBOD[j]+OM[3][i][j][v])*(100−OM[2][i][j][v])/100;

[0780] MBDN[j]=MBDN[j]+OM[4][i][j][v]−MBDN[j]*OM[4][i][j][v]/100;

[0781] For Unexpected=Incorrect performance

[0782] MBOD[j]=MBOD[j]*OM[2][i][j][v]/100;

[0783] MEDN[j]=MBDN[j]+OM[2][i][j][v]−MBDN[j]*OM[2][i][j][v]/100;

[0784] In any case

[0785] MDEF[j]=MBDN[j]*(100−MU[j])/100.

[0786] A learner model includes a set of the following sub-models:

[0787] 1. Learner behavior model:

[0788] Learner intention model;

[0789] Learner performance model;

[0790] 2. Learner database items;

[0791] 3. Multi-modal fuzzy relation (conditional assessment) of learner database items with learner's performance;

[0792] 4. Multi-modal fuzzy assessments of learner's database items (this is traditional “overlay learner model”).

[0793] Decision making. The decision making component of the Tutor Module Player is a computer program (a part of Tutor Module), which simulates decision making activity of a tutor and provides all necessary training decisions on the basis of a current assessment of the learner's cognitive items, objectives, strategy, and style parameters predefined by the author for the course unit.

[0794] This is a fully automated component of the Tutor Module. No authoring needed, but adjusting and tuning is possible.

[0795] The functionality of decision making component is as follows:

[0796] Input:

[0797] Current cognitive items assessment from the learner database or the cognitive interpretation component;

[0798] Course metadata from the tutor meta database;

[0799] Unit metadata from the tutor meta database;

[0800] Training objectives metadata from the tutor meta database;

[0801] Modes available in the course/unit from the tutor meta database;

[0802] Style parameters from the tutor meta database;

[0803] Learner control actions over the Tutor Module (selection of course, unit, objectives, state, Start-Stop conditions, mode);

[0804] Current state (passive, active);

[0805] Current mode (pre-test, instruction, practice, remediation, post-test)

[0806] Current sub-mode (Delivery, Testing, Diagnosing)

[0807] Result: Training Decisions:

[0808] Set the Tutor Module in a particular state, mode, sub-mode:

[0809] Passive state;

[0810] No intervention;

[0811] Commenting;

[0812] Advising;

[0813] Active state:

[0814] Set a mode:

[0815] Set a Pre-test;

[0816] Set a Instruction:

[0817] Set a Practice;

[0818] Set a Remediation;

[0819] Set a Post-test;

[0820] Set a sub-mode:

[0821] Set a Delivery;

[0822] Set a Testing;

[0823] Set a Diagnosing;

[0824] Set a difficulty level;

[0825] Set a control level;

[0826] Define global sufficiency of the training mode;

[0827] Define “Start-Stop” Sufficiency of the training mode;

[0828] Define local sufficiency of the training mode.

[0829] The training style of the system is flexible and the author can modify it at will. It is predefined by several variable parameters, which the author can set at the authoring stage, via an input window such as is shown in FIG. 8, by specifying the following information.

[0830] Delivery Threshold. Defines how thoroughly author wants the system to deliver new knowledge and skills to learners. The Delivery Threshold is defined as a percentage from 0 to 100. When the author sets the value higher, the system becomes more “persuasive”, and may repeat content presentation more often to confirm the same knowledge and skill items. When the author sets this value lower, the system presents content less thoroughly and more superficially.

[0831] Testing Threshold. Defines how thoroughly the author wants the system to check newly delivered knowledge/skills. The Testing Threshold is defined as a percentage from 0 to 100. When the author sets the value higher, the system will be more demanding, quizzing the learner more often to evaluate mastery; when the author sets this value lower, the system becomes more trusting and superficial.

[0832] Diagnosing Threshold. Defines how thoroughly the author wants the system to diagnose the causes of learner mistakes. The Diagnosing Threshold is defined as a percentage from 0 to 100. When the author sets the value higher, the system becomes more precise and exact, diagnosing frequently by using more questions; if overdone, this may prove annoying to the learner. On the other hand, when the author sets the value lower, diagnosis may be superficial or wrong; consequently, remediation may be redundant and annoying.

[0833] Delivery/Testing threshold. Defines how often the author wants the system to check newly delivered knowledge/skills. This threshold is expressed in integers from 0 to 1000, with each knowledge/skill item having a standard value of 100. Thus, if the threshold is set to 100, the system will check each new knowledge/skills item after it is delivered, at 200 every two delivered item will be checked, and so on. When the author sets this value at a higher level, corresponding to more knowledge/skills items, the system will deliver more knowledge/skills items before beginning to check their mastery. In this case the knowledge/skills assessment will be updated less often, meaning less individual adaptation, but more freedom for the learner. When the author sets the Delivery/Testing threshold lower, the control over learning is stronger, the assessment is updated more often, providing a higher degree of adaptation to the individual learner, but also less freedom for the individual learner to control his or her own pace and progress. This parameter defines a control level, in the other words.

[0834] Tolerance Threshold. Defines the system's tolerance to learner mistakes without interrupting the learning process. The Tolerance Threshold is measured in integers from 0 to 1000, with each error having a standard value of 100. Thus, when the Tolerance Threshold is set at 100, the system will interrupt delivery after detecting at least one error and begin to remedy it. Setting the value at 200 tolerates two errors without interruption, and so on. The higher the value, the more errors the system will allow before entering remediation; the lower the value, the fewer errors will be allowed.

[0835] Difficulty Range. Defines a diversity of difficulty levels an author wants a system to realize in audience training. (See FIG. 14.)

[0836] Difficulty switching threshold. Defines how much evidence an author wants a system to accumulate for switching a learner to the next difficulty level. This parameter thus defines the degree of inertia of the system in the decision to switch to the next level of difficulty.

[0837] Post-testing threshold. Pre-defines how thoroughly an author wants a system to post-test learners. It can be taken from the course order specification.

[0838] One of the roles of the decision making component is Training Mode administration. It includes the following steps:

[0839] (1) If an author predefined a pre-test, then the Tutor Module begins from the pre-test, and all other modes are unavailable for a learner's choice. (Self-driven learning and passive training is always possible). Only after the learner has passed pre-test successfully, all other modes become available for a learner's choice in the Tutor Module.

[0840] If the pre-test has not been passed, then the learner is advised which prerequisite course/units/items to learn;

[0841] If the pre-test has been passed (learner is ready for the course unit), then the Tutor Module presents to learner a menu to choose from: instruction, practice, remediation, and post-test (self-driven study is always possible), whichever has been provided for the current course unit by the author;

[0842] (2) The learner makes his/her choice;

[0843] (3) If the learner's choice was a post-test, then the post-test is administered;

[0844] If the learner passed the post-test, then active training of the course unit is over;

[0845] If the learner did not pass the post-test, the Tutor Module returns the learner to step (3) and recommends him/her certain mode for choice;

[0846] (4) If the learner's choice was the instruction (“teach me”), then the instruction is realized;

[0847] (5) If the learner's choice was the practice (“exercise me”), then the practice is realized;

[0848] (5) If the learner's choice was the remediation (“remediate me”), then the remediation is realized.

[0849] Actions Planning. The training actions planning component is a computer program which simulates the capability of a human-tutor. Simply speaking the component defines which training action to execute next. This is a fully automated component, no authoring is needed, but adjusting is possible.

[0850] The planning function is as follows:

[0851] Input:

[0852] objectives from the tutor meta database;

[0853] Strategy from the tutor meta database;

[0854] Training mode (pre-test, instruction, practice, remediation, post-test) from decision making component;

[0855] Sub-mode (delivery, testing, diagnosing) from decision making component;

[0856] Sub-mode style from the tutor metadatabase;

[0857] Adaptive tactics of sub-modes from the tutor metadatabase;

[0858] Style parameters (threshold set) from the tutor metadatabase;

[0859] Domain behavior model from the tutor metadatabase;

[0860] Learner behavior model from the tutor metadatabase;

[0861] Cognitive interpretation model from the tutor metadatabase;

[0862] Learner's personal data from the tutor's learner database;

[0863] Start-Stop conditions;

[0864] Current assessment of learner's knowledge/skills;

[0865] Current values of difficulty and control level;

[0866] Personal preferences and learning style of learner;

[0867] The previous training/learning actions;

[0868] Output: ID of the next training action to generate for an execution.

[0869] Planning is different for different modes (pre-test, instruction, practice, remediation, post-test), sub-modes (delivery, checking, diagnosing), and sub-mode styles (batch, linear, branching, and adaptive/intelligent).

[0870] For adaptive style of sub-modes there are different tactics as well. The author can predefine the Delivery tactics by selection from the list offered by the system in authoring stage:

[0871] Breakthrough to the terminal objectives is the first thing to do (Challenging, risky tactics);

[0872] Providing a general background is the first thing to do (Guarded, safe tactics);

[0873] Provide minimally necessary background for the next step on the way to the current objective (by default).

[0874] The author can predefine the Testing tactics by selection from the list offered by the system in authoring stage:

[0875] The first items delivered—the first checked;

[0876] The last items delivered—the first checked;

[0877] The maximal coverage of delivered items—the first.

[0878] The author can predefine the Diagnosing tactics by selection from the list offered by the system in authoring stage:

[0879] The closest causes—the first to check;

[0880] The deepest causes—the first to check;

[0881] The maximal distinguishing of suspected causes—the first to try.

[0882] By default the system selects the last tactics (above) automatically in each case.

[0883] Realization of the last (most efficient) tactics for all sub-modes (delivery, testing, diagnosing) comprises the same steps:

[0884] Filtering;

[0885] Weighting;

[0886] Selection.

[0887] Filtering allows exclusion of obviously inappropriate training actions from consideration, and this reduces the dimension of the planning problem. This is a strict pre-selection. Weighting on the other hand means evaluation of current pedagogic power of remaining training actions. (This is a soft pre-selection.) Finally, selection is a final decision of what to select from remaining and weighted training actions.

[0888] Planning of adaptive delivery (teaching) selects a next teaching action and includes the following steps:

[0889] (1) Filtering of training actions to reduce the scope of the planning task;

[0890] (2) Weighting of training actions;

[0891] (3) Selection of the next training action.

[0892] Teaching actions filtering represents a first (strict pre-selection) step of an adaptive delivery planning and in turn includes the following sequence of sub-steps:

[0893] 1. rejection of teaching actions with properties (target mode) that are inappropriate to current training mode;

[0894] 2. rejection of teaching actions with start-stop conditions that are inappropriate to current domain situation;

[0895] 3. rejection of teaching actions which deliver knowledge/skills (expected) outside of learner's current concern expressed him in the training boundaries (zone) predefined;

[0896] 4. rejection of teaching actions for which current difficulty level is higher then personal level of the learner;

[0897] 5. allowing teaching actions that will be able to remedy currently suspicious knowledge/skills items;

[0898] 6. rejection of teaching actions that are not able to contribute in learner's progress (in non-mastered (non-demonstrated) yet cognitive items within training boundaries and the nearest zone of delivery before mastery checking); and

[0899] 7. rejection of teaching actions that are not grounded on delivered (expected) or mastered (demonstrated) cognitive items.

[0900] An illustration of teaching action mapping into cognitive items is shown in FIG. 15.

[0901] Teaching actions weighting is a second (soft pre-selection) step of an adaptive delivery planning and includes a sequence of the following rules in a weight calculation:

[0902] 1. The more a teaching action is capable of contributing to new cognitive items delivery, the more weight it will have;

[0903] 2. The more a teaching action continues the path to the objective predefined by previous training actions, the more its weight will be;

[0904] 3. The more a teaching action matches the preferences and learning style of the learner, the more weight it will have. $\mathrm{Power}\left(l\right)=\mathrm{Fltr}\left(l\right)*\underset{j=0,1,\dots Jm}{\mathrm{Max}}\left[\left(\mathrm{DM}4\mathrm{jlVm}+1\right)*\left(100-\mathrm{MBj}\right)*\mathrm{MH}\left(j\right)*\sum _h^{\mathrm{Jm}}\mathrm{Ph}*\mathrm{PM2jh}\right]$

[0905] wherein:

[0906] Vm=number of total performance patterns

[0907] Ph=assessment value of learning on the tutoring path, initially set to 100 and updated during presentation.

[0908] Teaching action selection is a concluding step of delivery planning and includes a sequence of the following sub-steps:

[0909] 1. select training actions with maximum weight $L=\arg \underset{l}{\mathrm{Max}}\mathrm{Power}\left(l\right);$

[0910] 2. select a training action with maximum level of difficulty, which previously has been adaptively limited by a system for the learner.

[0911] Adaptive testing planning includes the following steps:

[0912] (1) Filtering of testing actions;

[0913] (2) Weighting of testing actions;

[0914] (3) Selection of testing actions.

[0915] Testing actions filtering represents a first (strict pre-selection) step of adaptive testing planning and includes a sequence of the following sub-steps:

[0916] (1) rejection of testing actions with properties (target mode) that are inappropriate to current training mode. (Only appropriate testing actions remain after that);

[0917] (2) rejection of testing actions with start-stop conditions that are inappropriate to current domain and training situation;

[0918] (3) rejection of testing actions that lie entirely outside of the predefined training boundaries;

[0919] (4) rejection of testing actions that cannot check cognitive items unchecked yet within the predefined training boundaries;

[0920] (5) rejection of testing actions presented before;

[0921] (6) allowing testing actions that cover delivered/expected cognitive items within the predefined training boundaries;

[0922] (7) If this is a remediation mode, then

[0923] scanning and passing of testing actions that are related to suspicious or reliably demonstrated cognitive items;

[0924] scanning and passing of testing actions that are inside suspicious cognitive items;

[0925] scanning and passing of testing actions that are related to suspicious cognitive items;

[0926] (8) rejection of testing actions that are not reliably grounded on demonstrated cognitive items;

[0927] (9) passing of testing actions that cover reliably delivered cognitive items within the predefined training boundaries;

[0928] (10) rejection of testing actions with difficulty level higher then learner's current (maximal) level of difficulty.

[0929] An illustration of testing action mapping into cognitive items is shown in FIG. 15.

[0930] Testing actions weighting represents a second (soft pre-selection) step of adaptive testing planning and includes a sequence of the following rules in a weight calculation:

[0931] (1) The more the testing action covers unchecked or ill-checked knowledge/skills items within the predefined training boundaries, the more weight it will have;

[0932] (2) The more the testing action covers the most expected knowledge/skills items within the predefined training boundaries, the more weight it will have;

[0933] (3) The more the testing action covers the path of previous training actions within the predefined training boundaries, the more its weight will be.

[0934] (4) The more the testing action matches the preferences and learning style of the learner the more its weight will be. $\mathrm{Power}\left(i\right)=\mathrm{Fltr}\left(i\right)*\sum _j^{\mathrm{Jm}-b}\mathrm{MHj}*\left(100-\mathrm{MUj}\right)*\mathrm{MBj}*\mathrm{Pj}*\mathrm{OM}2\mathrm{jiVm}+1$

[0935] Testing action selection is a concluding step of adaptive testing planning and includes a sequence of the following sub-steps:

[0936] (1) selecting testing actions with maximum weight;

[0937] (2) selecting testing action N with maximum level of difficulty adapted for the learner individually. $N=\arg \underset{i}{\mathrm{Max}}\mathrm{Power}\left(i\right)$

[0938] Adaptive diagnosing planning includes a cycle of the following steps:

[0939] (1) Diagnosis readiness check-up.

[0940] If Yes, then diagnosis posing and stop;

[0941] If Not, then continue;

[0942] (3) Diagnosing search;

[0943] (4) Return to (1).

[0944] Diagnosing search represents a core of adaptive diagnosing planning and includes the following steps:

[0945] 1. Filtering of diagnosing actions;

[0946] 2. Weighting of diagnosing actions;

[0947] 3. Selection of the next diagnosing action;

[0948] 4. Realization of the selected diagnosing action;

[0949] 5. Monitoring of learning with a cognitive assessment.

[0950] Diagnosing actions filtering is a first (strict pre-selection) step of diagnosing search and includes a sequence of the following steps:

[0951] 1. rejection of diagnosing actions with start-stop conditions that are inappropriate to current domain and training situation;

[0952] 2. rejection of diagnosing actions that cover only demonstrated knowledge/skills items;

[0953] 3. rejection of diagnosing actions presented before in current search pass;

[0954] 4. rejection of diagnosing actions that relate to unexpected or unsuspicious knowledge/skills items.

[0955] Diagnosing actions weighting is a second (soft pre-selection) step of diagnosing search including a calculation of a current power of diagnosing action $\mathrm{Power}\left(i\right)=\mathrm{Fltr}\left(i\right)*\sum _s^{\mathrm{Sm}}\sum _j^{\mathrm{Jm}}\sum _{r=j+1}^{\mathrm{Jm}}\left|\mathrm{MNjis}-\mathrm{MNris}\right|*\left(\mathrm{MBODj}+\mathrm{MBj}\right)*\left(\mathrm{MBODr}+\mathrm{MBr}\right);$

[0956] Or in a recursive form:

[0957] Sum=Sum+Math.abs(MN[i][j][s]−MN[i][r][s])*(MBOD[j]+MB[j])*(MBOD[r]+MB[r]);

[0958] A diagnosing action selection is a concluding step of diagnosing search and in the most simple embodiment it selects a testing action with maximal Power (i) $N=\arg \underset{i}{\mathrm{Max}}\mathrm{Power}\left(i\right);$

[0959] Training Actions Generation.

[0960] As a rule training actions do not exist in a form ready to be implemented. It wastes system's memory. When a time delay is not critical, then it is better to generate them when it is necessary. In one embodiment, it is possible to assemble training actions from pre-stored set of building blocks (such as templates, backgrounds, text windows, different kind of buttons, hot spots, and menus). In other embodiment, content of training actions can be automatically generated from a formal presentation of a domain under study, such as semantic networks, decision tables. In simulation-based embodiment, training actions can have a form of a command triggering certain processes and causing certain situations in the Domain Module.

[0961] Runtime manager with a timer. Runtime manager with a timer provides a synchronized interaction of all tutor domain components.

[0962] Generic System Architecture

[0963] A conceptual diagram of the generic system architecture is represented in FIG. 16. In this architecture all above-mentioned databases are combined into one integrated database. The generic player includes the domain player and the tutor-learner interface. The generic decision making component includes interpretation, decision making, and planning components of the Tutor Module.

[0964] The foregoing disclosure has been set forth merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to persons skilled in the art, the invention should be construed to include everything within the scope of the appended claims and equivalents thereof.