User interface educational database system for monitoring proficiency, performance and evaluation of student

[0002] The present invention relates to the assessment of a student's learning, and more particularly to mechanisms for tracking assessments of performances by a student which demonstrate progress in developing specific abilities.

[0003] In a traditional educational institution, students attend class lectures and read assigned materials outside of class. In order to determine the progress toward learning the subject matter of the course, the educators assign papers and other types of projects to the students, as well as periodically administer written examinations. These student performances then are evaluated by the teacher and a grade is assigned indicating the relative quality of the performance. At the completion of a course, the grades for all the performances are averaged to create an overall grade assessing each student's level of mastering the course subject matter.

[0004] Secondary schools, colleges, and universities traditionally calculate a cumulative grade point average by averaging the overall grade for each course taken by the student. A student is required to receive a specified number of credit hours of instruction and at least a minimum a cumulative grade point average in order to be graduated from these schools. In traditional educational institutions, other than observing that the student has taken specific courses and received a particular grade in each course, there is no mechanism for the student or teacher to track a student's progress in mastering abilities which are important to a major field of study.

[0005] An alternative technique for assessing students' learning has been developed by Alverno College in Milwaukee, Wis. That approach identifies eight abilities that students must use to demonstrate sufficient knowledge in order to receive a degree. With reference to FIG. 1, these abilities are: communication, analysis, problem solving, valuing in decision making, social interaction, global perspectives, effective citizenship, and aesthetic responsiveness. Four basic levels of proficiency have been defined for each of the abilities with specific requirements at each level. Additional levels are incorporated into outcome statements required for each student according to the nature of her major. For example, a student majoring in journalism has to demonstrate greater proficiency in the communication ability than a student majoring in chemistry.

[0006] Each course provides instruction which enables the student to develop different levels of each of these eight abilities integrated with the subject matter of the course. A student starts out taking courses which satisfy level one of various abilities and with each subsequent year of study takes courses which provide instruction at higher levels of the abilities. Although there are four levels with respect to each ability, the levels do not correspond to each year of a four year program toward receiving a degree. For example, a course taken by a student during her second year of study may provide instruction directed toward ability level one or level three.

[0007] During each course, the student is presented with several assignments to prepare performances, such as papers, examinations, speeches, and other projects, which develop and demonstrate proficiency in the associated abilities. For example, a course in a science may present a problem which the student must solve by defining and performing an experiment. Such an assignment would involve the abilities of analysis, problem solving and communication of the results.

[0008] Upon the completion of a performance, the student evaluates her performance based on criteria given by the instructor. The instructor performs a similar assessment of the performance which constitutes feedback to the student. The performance is evaluated in the context of whether the student has satisfactorily demonstrated proficiency at the appropriate level of the associated abilities. The students do not receive a traditional grade for each performance or for the course as a whole. Instead the faculty member provides an indication whether the course was completed satisfactorily or unsatisfactorily with a designation of the corresponding levels of ability.

[0009] As the student progresses through each course, she compiles a portfolio of information about significant, or key, performances with evaluations by herself, faculty members, sometimes student peers, and perhaps by persons from outside the educational institution who witness the performance. This portfolio should demonstrate a progression through the course to satisfying the requisite abilities. In a larger context, the portfolio of performance evaluations acquired during several years of instruction should demonstrate a progressive mastering of the eight core abilities, both in courses and in other contexts. The student and faculty members periodically review the student's portfolio of key performances in order to evaluate the student's overall progress and analyze where improvement is needed and what strategies might help her education.

[0010] In order to perform such an overall evaluation of the student's progress toward proficiency in each ability, the nature and quality (strengths and weaknesses in relation to criteria) of the performances completed inside and outside various courses must be readily available. This may be difficult as the related documents typically are retained in the files of several different departments within a college or university. Even if the performance evaluations are stored centrally, the amount of paper can be voluminous. In addition, it is desirable to organize the performance evaluations in several different ways, such as by ability, level of the abilities, and descriptions or subject areas. Such organization becomes difficult when the performance evaluations are kept as pieces of paper within file folders. Therefore, it is desirable to create a system for storing data regarding student performances from which the information can be presented a number of ways for analysis.

[0011] A database system records, organizes and displays information related to performances of a student which demonstrate her development of competency in the different abilities through which she uses her knowledge as required by an educational institution.

[0012] The database system has a data storage structure which for each student contains documentation about a plurality of performances completed by that student and which of the plurality of abilities each performance demonstrates. A user interface is coupled to the data structure and selectively presents information about an extent to which each student has demonstrated proficiency in each of the plurality of abilities within varied knowledge bases.

[0013] In the preferred embodiment of the database system, the information about a given performance completed by a student identifies the nature of the given performance, the criteria for successful demonstration, and specifies which of the plurality of predefined abilities are associated with the given performance. An evaluation of the given performance by a faculty member also is stored. The student submits a self assessment of her performance which also is retained in the data storage structure.

[0014] The preferred embodiment of the user interface enables the stored information to be organized and displayed in a number of formats. One of which is a two dimensional matrix with the plurality of abilities listed along one axis and different levels of achievement listed along another axis. The matrix has a plurality of cells each associated with one of the plurality of abilities and one of the levels of achievement. Within each cell is a list of courses which t he particular student has completed and other outside course contexts that demonstrate proficiency at the associated ability level. Another display format enables a student to review a given performance and the evaluations thereof.

[0015] The database system facilitates review of a student's progress toward satisfying proficiency in each of the abilities prescribed by the educational institution. It enables a student to quickly identify where further work is required and the types of courses that will meet the deficient ability levels. Faculty and administrators also are able to monitor the students' performance and progress via the performance portfolio database.

[0016] FIG. 1 is a matrix of the abilities and requirements at each level of those abilities which demonstrate a student's learning and development;

[0017] FIG. 2 is diagram representing a computer network for implementing the present diagnostic digital portfolio;

[0018] FIG. 3 is a chart of different data tables maintained for recording key performances with the relationship between the data tables being shown;

[0019] FIG. 4 is a flowchart depicting a process by which information is entered into the data tables;

[0020] FIG. 5 is a matrix created by the diagnostic digital portfolio which indicates satisfactory completion of courses and other performances that demonstrate proficiency in various ability levels; and

[0021] FIG. 6 represents a typical display of information related to a particular performance.

[0022] A diagnostic digital portfolio (DDP) of the student's key performances is maintained within a server on a computer network of the school and may be accessed by faculty, students and administrators. Preferably the DDP is accessed as a site on the school's internet site which enables access from on and off campus and inherently provides a graphical user interface that is easily understood by the users. Only authorized persons who have been assigned a user name and password may enter the DDP site. A student is able to review her performances and related assessments, but is prevented from accessing performance information of other students. However, a student in selected cases is able to evaluate another student's performance and her feedback may be inputted into the DDP, as will be described. Authorized faculty members and administrators may access performance information for all students.

[0023] An exemplary computer network 10 at an educational institution is depicted in FIG. 2. That computer network 10 comprises a server 6, on which the data and programs for the DDP are stored, and a plurality of personal computers 7 connected by a communication medium 8 to the server 6. Personal computers at remote locations, such as the homes of students, can access the DDP via an Internet connection 9.

[0024] The data relating to the DDP is stored in a number of tables in the memory of the network server and those tables are related, or linked, as shown in FIG. 3. A KEY PERFORMANCE DESIGN table 12 stores information regarding each significant, or key, performance that an instructor assigns in a particular course or that a student demonstrates in another context, such as by an internship or community service. That table contains a separate file for each key performance for the entire school and each file has several fields of data with the fields for one file shown in table 12. Every key performance design has an unique DESIGN ID which identifies it among all the other designs. The name of the design and an identification code for the faculty member or student who enters the design are stored. Information relating to the department of the school, the particular course, a specific section of that course are retained in this table 12. This information is utilized subsequently to sort and organize performances by specific departments, courses, and specific sections of a course. Because a given course may have several key performances during the semester, a field designated SEQUENCE identifies the order of those key performances of that course. For example, a particular performance may be the third one for a course or it may be a final project. The dates when the key performance design was created and when it will expire are also provided in the table. A title and a summary description of the performance also are retained in this table 12. A full description of the performance can be stored elsewhere on the server as a word processing document and the key performance design table 12 contains a file specification identifying the storage location of that document.

[0025] The key performance entries also contain a short textual summary of the criteria for evaluating the performance. A file specification provides a link to a word processor document that contains a full criteria description. The criteria identify the basis on which the performance is assessed and specifies elements that should be considered in the evaluation. Additional file specifications point to word processor documents which specify the criteria and the basis for the student to perform a self assessment and for faculty member use in providing feedback to the student. One should understand that the factors to be considered during self assessment by the student may focus on different aspects of the evaluation criteria than those considered by the faculty member. Hence, there are separate stimulus documents for prompting consideration of specific factors during each type of performance. Additional fields are provided to store file specifications of word processor documents which contain the feedback criteria for student peers or others to assess the student's performance. A field designated visibility is provided as a flag to indicate whether students may browse the key performance design or not. In some instances, a faculty member may not wish the students in a particular class to be able to see the key performance design information until the time of assessment. A status field is provided to indicate whether the key performance design has been completed or is still in a draft state.

[0026] Each key performance involves one or more of the abilities for which a student must demonstrate different levels of proficiency in order to advance and receive a degree. In that regard, the diagnostic digital portfolio contains a data table 24 that stores information regarding the abilities associated with each performance in the KEY PERFORMANCE DESIGN table 12. The records in the PERFORMANCE ABILITIES table 24 have a field that identifies the related performance by the associated DESIGN ID for the performance. Each such record also identifies a particular ability associated with the designated performance. A code that identifies the ability and its ability level corresponding to the matrix FIG. 1 are provided in the PERFORMANCE ABILITIES table 24. A particular ability may be divided into several sub-abilities, for example, the communication ability has constituent sub-abilities for writing, speech, listening, reading, and others. Not all of the sub-abilities may be associated with a particular performance. As a consequence, each record of the PERFORMANCE ABILITIES table 24 may designate a particular sub-ability. For a particular performance there may be multiple records entered into the PERFORMANCE ABILITIES table 24, one for each associated ability and sub-ability. The performance ability table 24 is linked to an ABILITIES table 26 that contains the ability code and the name of the ability.

[0027] The key performance design table 12 also is linked to a KEY PERFORMANCE table 14 which has a separate record for each key performance of each student. The KEY PERFORMANCE table 14 contains status information relating to the student key performances. Each record in this table 14 indicates whether an assessment has been submitted by each evaluator and whether the evaluation of that key performance is complete. The fields for one of these records is depicted in FIG. 3. Each performance is assigned a unique identification number in order to distinguish its record from all the other performance records in the data table 14. The ID number for the particular student being evaluated is entered into another field, along with the computer network user identification for the primary evaluator.

[0028] An identification is provided of the KEY PERFORMANCE DESIGN to which this particular key performance relates, thereby providing a link, represented by arrow 15, to a particular entry in the KEY PERFORMANCE DESIGN table 12. The date that the key performance was performed is specified. Fields also are provided in the KEY PERFORMANCE table 14 to indicate whether the student and the faculty member have completed their assessments of the performance and the date on which each assessment was entered into the DDP. An overall performance evaluation field indicates whether the performance has been completed satisfactorily (S), unsatisfactorily (U), or is still in progress (I). As an alternative to grades of satisfactorily or unsatisfactorily (pass/fail), traditional letter or numerical grade rankings may be used. Other fields are provided to indicate the nature of the performance, whether it is a “Best Work,” whether or not it is “Challenging,” and then when the entire performance evaluation has been completed. The key performance table 14 is linked to a faculty feedback table 16 and a student feedback table 18 in which data regarding the actual assessments of the performance are stored.

[0029] The FACULTY FEEDBACK table 16 contains a record for each performance evaluation stored in the diagnostic digital portfolio (DDP). This table 16 contains the Performance ID for the particular performance being evaluated, thereby providing a link to the associated record in the KEY PERFORMANCE table 14 to which the feedback relates. The FACULTY FEEDBACK table 16 has a field for an identification of the person making the evaluation and has a FEEDBACK field which contains the file specification of a word processor document that contains the actual feedback from the faculty member. The date on which the feedback was entered into the system also is provided. A SOURCE field indicates the origin of the feedback which is most cases will be the same individual as identified in the staff ID field. However, there may be situations where a generic staff ID is used and thus some identification of the individual beyond the staff ID must be provided. Another field in the FACULTY FEEDBACK table 16 stores an indication whether this is the primary feedback for the performance or a secondary feedback. For example, a teaching assistant, in addition to the course instructor, may evaluate the student's performance. Thus, the last field in the faculty feedback table provides a mechanism for indicating the type (primary or secondary) of that faculty member's feedback.

[0030] The student feedback table 18 contains a similar set of fields for information relating to the student's self assessment of her performance. In this table 18, the PERFORMANCE ID provides to link the associated record for this particular key performance in the KEY PERFORMANCE table.

[0031] An identification of the student is provided, as well as a field designated FEEDBACK which holds a file specification of the word processor document containing the actual assessment, providing a link to that document. The date on which the feedback was entered into the system is indicated. In addition, a field designated SELF is provided to denote whether this evaluation is from the student who conducted the performance or from a student peer who is evaluating another student's performance. The DDP accommodates external evaluations, such as those by members of the community who are not faculty or students. The source field indicates these kinds of evaluations.

[0032] The diagnostic digital portfolio also contains a FACULTY table 20 storing biographical information about each faculty member who may design or evaluate a key performance. Each record in that FACULTY table 20 contains an identification number of the person, his or her system name and full name, a faculty identification number, and information regarding the department of the school in which they are assigned. Additional information, such as email and phone numbers, may be provided.

[0033] The FACULTY data table 20 is linked to both the KEY PERFORMANCE DESIGN table and the FACULTY FEEDBACK table in order to provide the biographical information when the system accesses data in those other tables. The FACULTY data table may be linked to other tables of the system which refer to a particular faculty member.

[0034] Similarly, the STUDENT table 22 contains biographical information for those people. That information includes the student's network identification number, her network name, full name, college ID number, and residence address. Fields are included to store the student's email address and phone number. Additional information is stored in fields for a student advisor's name, the student's status with university (e.g., active, inactive, transferred, dismissed, etc.) and the number of credits she has received.

[0035] The STUDENT data table 22 is linked to both the KEY PERFORMANCE table 14 and the STUDENT FEEDBACK table 18 as those tables both contain information which identifies the student entering information in those tables and thus require student data table to be used to look up information such as the student's name.

[0036] It will be understood by those knowledgeable with databases and computer programming that additional tables may be provided to store other types of information which are linked to other tables or utilized by the DDP system.

[0037] The entry of data into the different tables 12-26 is best understood in the context of the preparation and completion of a performance as depicted by the flowchart of FIG. 4. The process begins with the faculty member accessing the DDP internet site which causes the DDP Faculty Home Page to be displayed on the computer screen. The DDP Faculty Home Page contains several buttons for selecting different functions of the DDP system. At this time the user selects the button associated with designing a key performance. That selection opens up a form having blank areas in which the faculty member enters information about the design of the key performance. That information corresponds to the fields of a record in the KEY PERFORMANCE DESIGN table 12. Thus at step 30 in FIG. 4 the faculty member designs a performance by filling out the key performance design form on her computer screen and then transferring the data to the DDP memory section of the network server.

[0038] Upon completing the key performance design form, the faculty member has the option of clicking the cursor on buttons which can save the design in draft form, activate the key performance design for use by the DDP, or delete the draft design. By saving the design in its draft form the user is able to work on the performance design at different points in time and even have others proof the design before activating it. The system stores the design with a draft status in the last field of the table record 12 shown in FIG. 3. Upon accessing the DDP, a faculty member is afforded the opportunity of reviewing all of their designs and selecting any one on which to work. When the design is completed, clicking on the ACTIVATE button changes the status to ACTIVE in the data file of the key performance design table 12.

[0039] The faculty member also is able to access and review all of his or her active designs or all of the active designs in the system regardless of the creating faculty member. At the expiration date of a given performance design its status is changed to ARCHIVE and the user also is able to select for review all his or her archived performance designs. By accessing these various categories of performance designs the faculty member is able to obtain information for use in creating a new performance design for a course.

[0040] After the key performance has been designed and activated in the DDP system, it can be assigned at step 32 to individual students either by a faculty member or by each student. At that time, the faculty member may enter the DDP system, select the particular active key performance design, and designate the students to whom the performance is assigned. Specifically, the faculty member can either enter the student's name or school ID number in an appropriate field. As each student is assigned the performance, a record is created in the KEY PERFORMANCE table 14.

[0041] Alternatively a key performance may be assigned by the students in response to a notice from an instructor. In this case each student in the class accesses the DDP and browses through the active Key Performance Designs that have been designated as visible to the students. When the student locates the designs for the Key Performance designated by the instructor, she designates that the key performance should be added to her list of work in progress.

[0042] Regardless of the manner in which a Key Performance was assigned to a given student, the student can view her uncompleted Key Performances via the DDP system. From the DDP Student Home Page, the student can select a button marked MY WORK and bring up a display of all her assignments that have not been fully completed. By clicking on the designation of an assigned key performance, the student is able to review the details of that assignment and the criteria against which her work will be evaluated.

[0043] Eventually the student completes the performance at step 34, such as submitting a paper to the instructor, making a presentation to the class, or whatever the performance may be. Thereafter at step 36, the student conducts a self assessment of her performance by accessing the MY WORK section in the DDP system and selecting the particular performance. The student then chooses the self assessment option in the display which brings up an assessment form with the relevant evaluation criteria. That form is generated from the data in the associated KEY PERFORMANCE DESIGN record in table 12. The student then completes the form by providing to all of the information requested. At the end of the form, the student then clicks on a button to submit the self assessment document.

[0044] The DDP responds to the self assessment submission by storing text of the self assessment in the word processor file area of the network server and by updating the contents of tables in the DDP system. Specifically, a new record is created in the STUDENT FEEDBACK table 18 identifying the related Key Performance, the file specification of the word processor document for the assessment, and the other information described previously. At that time, entries also are made in the KEY PERFORMANCE table 18 for that performance by that student, indicating that she submitted the self assessment and the date of that submission.

[0045] At step 38, if the Performance Design indicates that peer review also will be performed one or more additional students submit evaluations of this performance in a similar manner to that by which the self assessment was submitted. However, the criteria employed in a peer evaluation of another student's performance may be different than that used for the self assessment, depending upon the key performance design.

[0046] Periodically, each faculty member accesses the diagnostic digital portfolio system and reviews the performances which have been assigned to his or her students. Usually, the faculty member determines whether students have submitted their own self assessment at which point the faculty member may submit an evaluation. The student cannot observe the faculty member's evaluation before submitting a self review of her performance.

[0047] Once the student has submitted her self assessment, an indication is provided to the faculty member of that fact. This may be accomplished by placing an indicator adjacent to the student's name in the list of students to whom this performance has been assigned. Thus, when the faculty member accesses the DDP and looks at the information related to the key performance, the faculty member can determine which students have completed their self assessment and thus know that the faculty feedback can be prepared.

[0048] At the faculty member's pleasure, she accesses the feedback section of the DDP and select the particular Key Performance from the list of ones that she has assigned. On the display screen for that key performance screen that faculty member is able to pull down a list of the students in the class and select one to assess. The Key Performance information then is displayed and by further clicking on an appropriately labeled box the form with the faculty assessment criteria will appear.

[0049] The faculty member completes the form and submits it to the DDP. In response to that submission, a new record is created in the FACULTY FEEDBACK table 16 identifying the related Key Performance, the file specification of the word processor document for the faculty assessment, and the other information described previously. At that time, entries also are made in the KEY PERFORMANCE table 18 for that performance by that student, indicating that faculty member has submitted an assessment

[0050] Thereafter, at any time the student may access the data regarding this particular Key Performance and review the faculty member's feedback. The faculty member for the course or other faculty members may also access this student's key performance data at any time.

[0051] A student can review her key performances in the DDP by accessing the system and selecting a button labeled MY PORTFOLIO on the DDP student Home Page. This brings up a matrix similar to the one shown in FIG. 5, which has rows corresponding to each of the eight abilities and columns for the four levels for those abilities. Within each cell of the matrix is a listing of the courses which have been satisfactorily completed to fulfill the requirements for that ability at the associated level. For example, Assessment Center performance AC101 applies to Communication Level one requirements in the reading and writing sub-abilities.

[0052] The user is able to click on a key performance to display general information about that key performance. This forms a display similar to that shown for FIG. 6 which presents data from the KEY PERFORMANCE DESIGN table 12 maintained by the DDP. Specifically, the key performance (e.g. AC101), the title of the performance and its description and criteria summaries are displayed. Additional information is obtained from other tables in the DDP system. For example, in the lower right corner of the display, information is provided about the faculty feedback for this student named Sue Alverno. That feedback includes whether the overall performance was Satisfactory and whether each of the abilities associated with the performance was Satisfactorily completed. These individual abilities are reflected in the entries in the matrix of FIG. 5.

[0053] Students also are able to save the actual performances which involve written documents, such as spreadsheets or word processor documents, or audio visual materials. A data table in the DDP provides a record for each student which lists the Key Performances that she has stored in the system. It should be noted that a student will not store every key performance, but only the most significant ones which demonstrate her progress in the eight abilities. The record for the student in this table contains the file specification that indicates the location of word processor document, a spread sheet, or other type of text material stored on the network server. The DDP Student Home Page provides a button for accessing this listing of performance documents. By clicking on an item of that listing the actual document can be retrieved and displayed.

[0054] As with any complex software system, help screens and informational items may be provided for accessing by the users to assist in their utilization of the DDP system.

[0055] The foregoing description was primarily directed to a preferred embodiment of the invention. Although some attention was given to various alternatives within the scope of the invention, it is anticipated that one skilled in the art will likely realize additional alternatives that are now apparent from disclosure of embodiments of the invention. Accordingly, the scope of the invention should be determined from the following claims and not limited by the above disclosure.