Translation management system

[0002] This application claims benefit of Provisional Application No. 60/075,740, filed Feb. 23, 1998.

[0003] 1. Technical Field

[0004] The invention relates to the multilingual translation of documents in a computer environment. More particularly, the invention relates to the management of monolingual documents, data streams, non-text files, and databases to enable their translation into target languages in a computer environment.

[0005] 2. Description of the Prior Art

[0006] The surge of growth in Internet access across the world has created the need for multilingual Web sites. The majority of Web sites are created in English which creates a problem for non-English speaking Internet users. This also creates a problem for Web site managers that maintain Web sites that are, or attempt to be, multilingual. The Web site content must be constantly maintained to be up to date and synchronized in all of the languages supported by the Web site.

[0007] One approach is shown in PCT International Publication Number WO97/18516 of Flanagan et al., published on May 22, 1997. This application integrates a machine translator in the end user's Web browser. The user selects the desired target language and the Web document retrieved by the browser is then translated by the machine translator. Unfortunately, errors occur in the translation and the appropriate content is not consistently displayed to the user.

[0008] An alternative implementation in the above application places pre-translated Web pages on a Web server. The Web server stores all of the translated Web pages locally. The Web site visitor sends a request to the Web server for a page in a specific language. The Web server searches for the pre-translated page that has been stored locally and sends it to the Web visitor's Web browser. This creates a storage and file management problem on the Web server because of the duplication of each entire page of the Web site.

[0009] Some search engines on the Internet offer an option to translate the hits that come back from a search query into different languages. The user initiates the translation through a menu selection. The translation is performed automatically by machine.

[0010] Japanese Pat. No. 9-114852 is a method for a search unit which takes a search string in a second foreign language and translates it into a first foreign language. The translated search string is used to search the original document which is in the first foreign language.

[0011] Japanese Pat. No. 9-44502 is a method wherein a document in a first foreign language is translated into a second foreign language. The two documents are then displayed separately or together through a user interface.

[0012] European Pat. No. 0376741 B1 issued to Francisco on Oct. 18, 1995, is a method for displaying error messages on a document collating and envelope stuffing machine. Said error messages exist simultaneously in a plurality of languages and are displayed in the language specified by the user. This approach has storage and maintenance problems because each error message must be duplicated in its entirety for each language.

[0013] European Pat. Appl. No. EP0774722A2 of Microsoft Corporation published on May 21, 1997, is a method for an information retrieval system that separates the design and content components of a document page. Pages are created with controls that define areas for content to be inserted into said pages.

[0014] It would be advantageous to provide a translation management system that gives the user local control of the multilingual translation of electronic content. It would further be advantageous to provide a translation management system that allows the user to easily manage, schedule, and track translation resources and the content that are sent to the translation resources, thereby enhancing the maintainability of the system as a whole.

[0015] The invention provides a translation management system. The invention utilizes an intuitive user interface for managing document translation for multilingual Internet Web sites, documents, data streams, and non-text files, enabling the user to incrementally update the language content of a Web site or document and enhancing the maintainability and storage of multilingual electronic content.

[0016] A preferred embodiment of the invention automatically detects when a document, data stream, or non-text file in the master language has been updated and notifies the user which corresponding documents, data streams, or non-text files in the other languages require translation. The documents, data streams, or non-text files requiring translation are staged and dynamically routed and sequenced to individual translation resources where the actual translation is performed.

[0017] Management status, reporting, scheduling, and accounting information is sent to the user as the translation process ensues. The user is notified of the completion of translation and the invention coordinates the delivery of the translated documents, data streams, or non-text files back to the user's site for installation and optional review.

[0018] The invention makes a variety of translation resources instantly available to the user. These translation resources include both automated translation tools as well as human translators. The translation resources are connected to the invention using a flexible architecture that can be deployed on intranets as well as the Internet.

[0019] Other aspects and advantages of the invention will become apparent from the following detailed description in combination with the accompanying drawings, illustrating, by way of example, the principles of the invention.

[0020] FIG. 1 is a block schematic diagram of the major components of a preferred embodiment of the invention according to the invention;

[0021] FIG. 2 is a block schematic diagram of the placement of the Visitor module in a network environment according to the invention;

[0022] FIG. 3 is a block schematic diagram of a decision flow diagram of the Visitor module according to the invention;

[0023] FIG. 4 is a block schematic diagram of the major components of the Developer module according to the invention;

[0024] FIG. 5 is a block schematic diagram of the placement of the Developer module in a network environment according to the invention;

[0025] FIG. 6 is a block schematic diagram of the major components of the Workflow Manager module according to the invention;

[0026] FIG. 7 is a block schematic diagram of a decision flow diagram of the Manager's Console module according to the invention;

[0027] FIG. 8 is a block schematic diagram of the major components of the Manager's Console module according to the invention;

[0028] FIG. 9 is a block schematic diagram of the major components of the Adaptor segment of the Workflow Pipeline module according to the invention;

[0029] FIG. 10 is a block schematic diagram of a decision flow diagram of the Adaptor module according to the invention;

[0030] FIG. 11 is a block schematic diagram of the major components of the Adaptor module according to the invention; and

[0031] FIG. 12 is a block schematic diagram of the placement of the Workflow Manager module in a network environment according to the invention.

[0032] The invention is embodied in a translation management system for computer applications. A system according to the invention provides an intuitive user interface for managing document content translation for multilingual Internet Web sites, documents, data streams, and non-text files, enabling the user to incrementally update the language content of a Web site or document and automatically initiate the translation of the content into the corresponding target languages, keeping the multilingual content synchronized and enhancing its maintainability and storage. Many existing application programs use approaches that make it difficult for the user to manage the translation and synchronization of multilingual content.

[0033] A preferred embodiment of the invention provides a comprehensive suite of three modules that allow a user to build, operate and maintain a multilingual Web site easily and efficiently. The invention enhances an existing Web site by adding advanced multilingual content management and process control capabilities to the customer's Web server. It also uses standard and open interfaces thus enabling the invention to work with all Web servers on supported platforms.

[0034] Each module is easy to set up and does not require special modifications to the Web site. The invention provides an easy migration path from early-stage customers who are only interested in measuring overseas visitor traffic to more sophisticated customers who need to manage the rapid translation of changing content on large, distributed multilingual Web sites.

[0035] Referring to FIG. 1, the invention provides three components. The Visitor module 101, the Developer module, and the Workflow Manager module 103. Each module accesses one or more language and country databases and file systems 104. The Visitor 101, Developer 102, and Workflow Manager 103 modules can operate in the same environment or in a standalone mode.

[0036] The Visitor module greatly enhances the multilingual Web site visitor's experience by providing an automated and seamless way to serve content in the correct language. Web site publishers can serve all their multilingual content through a single point of entry using the Visitor module. Web site visitors will immediately understand the information they see when they enter a site because it will be instantly presented in their language and for their country.

[0037] With respect to FIG. 2, the Visitor module 202 works in tandem with a customer's existing Web server 203. It automatically determines the language and country of a Web site visitor and directs the Web server 203 to deliver the appropriate localized content contained in one or more country/language databases and/or file-based content in a file system 204 to the visitor's browser 201. Recently accessed localized content is placed into a Cache 206. The content is placed in the Cache 206 so that if a similar request comes in for a document in that language and for that country, then the cached version will be pushed out to the browser 201. This saves time and processor overhead for accessing the database and file system 204 to rebuild the requested content. In addition, Visitor 202 informs the browser 201 of the proper font and content encoding needed to display the selected language and enables the browser 201 to download the font using, for example, Bitstream's TrueDoc technology, if required.

[0038] The Visitor module intercepts input text that is submitted using an HTML form (e.g., a customer feedback form), and writes it into a form database 205, in a manner so that it is easily translated later via the Workflow Manager. Most Web sites have forms that allow site visitors to submit comments or request more information from the site host. This feedback might not be in English and would otherwise be incomprehensible to the recipient in a multilingual environment.

[0039] The form database 205 includes sufficient information to identify the country, language and encoding of the text to properly interpret it for subsequent translation. Furthermore, the invention includes a novel database viewer which allows the translated content to be viewed in the context of the form in which it was originally entered.

[0040] Referring to FIG. 3, the Visitor module determines the Web site visitor's language and country 301 from one or more of the following criteria:

[0041] Language preference setting in the browser

[0042] Cookie from previous visit to the Web site

[0043] Root domain of the visitor

[0044] Manual selection by visitor

[0045] Language content available on the server

[0046] Local database of countries and languages of the world

[0047] Local configuration files on the Web site

[0048] The Visitor module supports content in all languages and encodings, including Unicode. Manual language selection is easily implemented through the use of a special command set described later in this document thus eliminating the need for complex CGI scripts or interlinked pages. The required content for the Web site visitor's language and country is checked to see if has been previously requested and resident in the cache 302. If the content is in the cache, then it is delivered from the cache 303.

[0049] If the required content is not in the cache, then the Web server is notified of the appropriate localized content required 304. The appropriate content is then placed into the cache for future reference 305. The Web site visitor's country and language preferences are recorded in the server log 306. The enhanced Web server log gives the site manager a detailed breakdown of country and language for visitors to the Web site. Finally, the browser is notified of the proper display font and allowed to download the font 307.

[0050] Input text that is submitted using an HTML form is intercepted 308. The text is converted to an internal format 309 and is placed in the forms database 310 for later translation.

[0051] The Developer module enables the Web site developer to build a single ‘master’ site which is subsequently displayed in any number of languages, compared with conventional methods whereby the site is re-engineered for each language.

[0052] Referring to FIG. 4, the Developer module contains all of the features of the Visitor module 401 which records the Web site visitor's country and language preferences in the server log 404.

[0053] A toolkit 402 is provided which allows a master site to be built that is language and country-independent. The actual language content is placed in a language and country database and/or file system 403 where it is easily managed and maintained. When a visitor enters the site, the requested document is automatically served in the visitor's language and for the visitor's country by filling in a document template from the master site with the correct language and country content from one or more databases.

[0054] This structure makes the site more compact, reduces site resource requirements and lowers the cost of operating the site.

[0055] The cost of making a design change is also reduced as only the master site needs to be updated. This is in contrast to the more time-consuming and expensive approach of propagating a single change throughout all language versions of the site which is unnecessary with the invention.

[0056] Keeping the translated content in a database 403 also makes it extremely easy and cost-effective to add additional languages to the site. Adding a language simply involves translating the appropriate content and creating new entries in the database 403. The Developer module makes it unnecessary to replicate the entire site for each new language and country as would typically be the case.

[0057] The toolkit 402 offers the multilingual Web designer a great deal of flexibility. It allows a site to be built which combines content common to all languages with content that is specific to a particular language and country. For example, company-wide information which needs to be served in all languages is easily combined with regional-specific information that may only be required in a single language.

[0058] The invention provides special tags that are used to insert language or country-specific content into an HTML document. The tags are: Multi-country server-side includes (MCSSI); and Multi-language server-side includes (MLSSI). MCSSI allows locale-specific elements of an HTML document to be dynamically included as a function of the current region or country, while MLSSI allows localized elements of an HTML document to be included as a function of the current language.

[0059] The Developer module also supports the incremental construction of a multilingual Web site through a feature that makes it unnecessary to fully populate the site's language matrix completely with documents. If a requested document does not exist (i.e., it has not been translated into the requested language), the invention automatically retrieves the document in the next most appropriate language for the visitor, or the default language of the document, as specified by the Web site manager. Documents can exist on separate servers and/or databases.

[0060] The invention walks down a priority list of languages for a designated country. If a document in the language that is first on the priority list is not present on the server, it then looks for the next most appropriate language. The list is sorted according to the most prevalent language or dialect for a particular country. For example, the languages entries for Egypt are {arabic, french, english, berber}. A visitor from Egypt will be presented with a requested document in Arabic, if it is available. If it is not, the system will look for one in French, and so on.

[0061] When a visitor is presented with content in an alternate language, an informational text message can be included telling the visitor that the preferred language was not available. The informational text is supplied in the most recently valid language for the visitor's country and language and is embedded within Javascript code. The Javascript code is embedded within the HTML stream that is sent back to the server. The HTML “body” tag has an optional attribute where Javascript code can be inserted after the tag. The informational text contained within the Javascript code is in english and enclosed within the appropriate system tags described below, enabling it to be replaced with its translation in the most recently valid language of the visitor. The informational text has been pretranslated into all of the languages in the system database for the customer. This allows easy access to the appropriate translation for the informational text based on the most recently valid language for the visitor's country and language.

[0062] This allows a multilingual Web site to be built incrementally, such that only a portion of the documents on the site need be translated and documents not available in a requested language will be served in the next most appropriate language for a Web site visitor.

[0063] This also gives the customer the option of translating only a subset of the total content on the site. The Developer module controls all aspects of content navigation and delivery for the entire site so that a visitor will always be served in the most appropriate language, based on what language content is actually available on the server. The visitor will never see an error message (i.e. a “404 -document not found”) because the requested content is not available in the visitors language.

[0064] Regions are mapped and organized according to the Web site's needs. For example, a multi-user server can have a separate region definition for each user's site. The regions file organizes the regions, countries, and languages that the user is concerned with and is configured by the user. The user sets up a mapping, for example, one user can have information specific to Italy and another user can have information that is just concerned with the European region and is not concerned with Italy. This approach allows a hierarchical region-based lookup scheme. Content is hierarchically stored as country and language independent elements such that a one-to-many and many-to-one mapping exists between country and language. The advantage to this approach is that there is no content duplication; the user has only as many files in the system as he has unique content.

[0065] With respect to FIGS. 4 and 5, the toolkit 402 also features a novel mechanism to create localized content for specific geographic regions or countries by using a template-based approach to dynamically create documents tailored for a specific language or country. This feature makes it easy to create a true global site localized for each area of the world with the smallest achievable site footprint on the Web server 503.

[0066] A template contains placeholders for country and language-specific information that has been removed from a document. This information is dynamically inserted from a TermDB 508 (an external glossary), another template or document located in a database or file system 509, or provided automatically by the Developer module 502 when the composite document is presented to the browser 501.

[0067] The basic approach to creating a template has two steps. First, remove as much country and language-specific information as possible from a document. Then, replace this information with appropriate tags and commands specific to the invention and described later in this document.

[0068] The advantage to using templates is that a single document can easily support many languages and countries. For example, a single form or CGI-generated document can be constructed so that it will be automatically localized for different languages and locales. This significantly reduces the number of documents that have to be maintained on the site and makes it very easy to add new languages. It also allows a single update to a document to be immediately propagated to all languages and countries.

[0069] This technique allows country or regional content to be conveniently separated from the structure of the document, thereby making it easy to change a design often by updating a single file. This in turn eliminates the ripple effect often seen when a single change must propagate through all of the localized documents on the site. This allows the Web Server 503 to control the locale-independent templates 504, language-specific elements 505, country-specific elements 506, and static HTML content 507.

[0070] Visitors to a multilingual Web site may wish to enter information and feedback into a fill-out form in their local language to send back to the Web site manager. A mechanism is provided, and described above, which facilitates the acquisition and translation of this information. The Developer module captures and tags all such information in a sufficient manner so it can be subsequently translated and analyzed by the customer.

[0071] This approach allows a site to be built with minimal effort and cost, and allows the site to grow over time to meet the exact needs of the customer.

[0072] Referring to FIG. 6, the Workflow Manager module enables the Web site manager to orchestrate the creation of foreign language versions of the site and to automatically keep them synchronized and up-to-date, cost-effectively and with minimal effort. The Workflow Manager module provides a Manager's Console 601, access to a Workflow Pipeline 602, and customizable Translation Resources 603. Translation queues 604 hold the incoming translated documents and outgoing documents to be translated. Documents, data streams, and non-text content are stored in one or more language and country databases and/or file systems 605. Data streams and non-text content are considered documents in the following text.

[0073] A brief summary of the duties of the three primary components are:

[0074] the Manager's Console, which serves as the user interface

[0075] the Workflow Pipeline, or transport layer

[0076] a set of Translation Resources that perform the actual translation work

[0077] The Workflow Manager module is the core component of the invention. The module allows the Web site manager, with no prior linguistic experience, to effortlessly manage the traditionally complex process of translating and updating a multilingual Web site.

[0078] The Workflow Manager module provides a natural migration path beyond the Visitor and Developer modules. The Visitor and Developer modules provide a solution to efficiently serve and organize content on a multilingual Web site. The Workflow Manager completes the product family by enabling the language content on the site to be kept up-to-date, efficiently and cost-effectively.

[0079] The Web's very nature is dynamic. A monolingual Web site must be updated frequently to remain current. A multilingual Web site has an even more demanding problem of requiring all languages to be updated and synchronized simultaneously. Using traditional techniques, the cost and time required to perform this update and synchronization exceed the benefit and lifetime of the newly updated information to be served, effectively rendering the Web site useless. The Workflow Manager provides a solution to this paradox by reducing the cost and time requirement for language update and synchronization dramatically, thereby ensuring that information on the Web site is always current, regardless of language.

[0080] The Manager's Console is the user interface for the Workflow Manager and is the primary point of interaction for the Web site manager.

[0081] Referring to FIG. 7, the Manager's Console detects when a document in the master language has been updated 705. It then notifies the Web site manager which corresponding documents in the other languages require translation 706. The Manager's Console provides “one-touch” translation, whereby at the click of a button, documents from any source requiring translation are converted to the internal format 703, staged down the Workflow Pipeline 704 then dynamically routed and sequenced to the individual Translation Resources where the actual translation is performed. Documents may be Web based and non-Web based and may also contain non-text elements. The documents may contain or refer to additional dependent components such as graphics, audio, video, and other multi-media elements.

[0082] When the Web site manager initiates the translation process, the newly updated master language document and its constituent elements, together with associated control information will be converted by the Console to an internal format, one which is more suitable for information transport over the Workflow Pipeline 703.

[0083] The Manager's Console automatically controls the sequencing and selection of Translation Resources during workflow processing according to subject matter of the document to be processed, target language of the translation, quality level (whether draft-only or high quality is required) and other variables. The Web site manager can also individually specify the use of a specific set of Translation Resources.

[0084] The Manager's Console provides management status and reporting as the translation process ensues 707. It then automatically notifies the Web site manager of the document translation completion and coordinates the delivery of the translated documents back to the Web site for installation and optional review 708. The Web site manager can directly install the translated documents back onto the active area of the Web site at the click of a button, or can make them available for internal review within the organization.

[0085] Status information is presented through a highly usable interface that facilitates interaction and improves the productivity of the Web site manager. The Console interface is designed to allow translation and update of the multilingual Web site to be performed cost-effectively and with minimal effort by a non-specialist.

[0086] New languages are easily added to the site as well. The Web site manager simply selects the new language from a pulldown list 701, and the Manager's Console automatically initiates the translation of documents into the desired language, under the control of the Web site manager 702. The documents are converted to the internal format used by the invention 703 and staged down the Workflow Pipeline 704.

[0087] The Manager's Console also has a built-in access and version control system which allows it to be easily integrated with a third-party authoring or document management system on the Web site.

[0088] With respect to FIG. 8, a task view of the Manager's Console is shown. The Update Status module 801 updates the Web site manager of the document translation status. The Complete Document module 803 retrieves the translated document and its constituent or dependent components from the Translation queues 804 and updates the associated document status information. The document delivery to the Web site manager is then coordinated by the Coordinate Document Delivery module 802.

[0089] Documents that need to be translated are extracted from the language and country databases and file systems 808 and are sent to Translation Queues 804. The Project Analyzer module 806 receives project components, in the form of documents and other electronic content from the Translation Queues 804 and analyzes the project in sufficient detail to determine project cost and resource requirements.

[0090] The Project Analyzer module 806 waits until all elements of a project have been received. It then sends the Scheduler module 805 a description of the project and instructs the Scheduler module 805 to begin the project. Typically, the Project Analyzer module 806 works with the Accounting Manager module 809 to generate a project cost estimate prior to beginning the project.

[0091] Once the Scheduler module 805 receives the project information from the Project Analyzer module 806, it creates Work Packets, Job Tickets and a Project Schedule. A single project consists of a set of Work Packets and associated Job Tickets. Each Job Ticket contains a reference to a Translation Resource. A set of Job Tickets is associated with a single Work Packet and describes the activities to be performed by that Work Packet. The Scheduler module 805 generates a Schedule that is sent to the Manager's Console for review and validation. The documents and their constituent or dependent components are then scheduled and sent down to the Workflow Pipeline.

[0092] The Scheduler module 805 updates the Project Schedule 810 and notifies the Accounting Manager module 809 during the course of the project. This permits the Project Schedule 810 to be adjusted when there are changes in the availability of Translation Resources, or when manual changes are made to the project by an operator via the Manager's Console.

[0093] The Accounting Manager module 809 updates customer and Translation Resource accounting in the Accounting Database 811 as jobs complete. The Accounting Manager module 809 generates project cost estimates (based on input from the Project Analyzer module 806), handles customer billing, and is responsible for generating purchase orders for Translation Resources, when needed.

[0094] Completed documents and their constituent or dependent components that have been checked by the Web site manager are then installed in one or more language and country databases and/or file systems 808 by the Monitor Database Documents module 807.

[0095] The Workflow Pipeline is a transport layer that delivers documents and their constituent or dependent components requiring translation to the Translation Resources where the actual work is performed. The Workflow Pipeline is heavily instrumented so that status and tracking of ongoing work is instantly conveyed to the Web site manager on the Manager's Console.

[0096] The Workflow Pipeline uses a highly configurable architecture allowing a variety of Translation Resources to be instantly available to the Web site manager. These Translation Resources include both automated translation tools as well as human translators.

[0097] Translation Resources are connected to the Pipeline using an open Application Programming Interface (API) known as an Adaptor. Adaptors allow a variety of Translation Resources to be connected to the Pipeline, making it easy to tailor the Workflow Manager to meet a customer's requirements. The flexible Pipeline architecture can be deployed on intranets as well as the Internet.

[0098] Referring to FIG. 9, Generators 901 are provided that convert the internal format document that is used within the Pipeline to the appropriate format required by the Translation Resource. The Adaptors 902 invoke the specific generator for the appropriate Translation Resource 904. Each Adaptor accepts the internal format document, sends the document to the appropriate Generator 901 which converts the format to one that is acceptable to the Translation Resource 904 attached to the Adaptor 902. The Generator 901 sends the newly formatted document to the Adaptor 902 which in turn sends it to the Translation Resource 904. When the document has been translated by the Translation Resource 904, the Adaptor 902 accepts the document and its associated information, and invokes a Parser 903 which converts the document back into the internal format. The Parser 903 distributes the packet back up the Pipeline.

[0099] Parsers are also invoked whenever a document is extracted from the Translation queues and sent down the Pipeline. The Parser converts the document into the internal format. Generators are, in a similar manner, invoked whenever a translated document is posted back onto the Translation queues after completion of translation. The Generator converts the internal format document back to the document's original format (e.g., HTML). The document is then sent to the final destination.

[0100] With respect to FIG. 10, the Adaptor converts work packets to the appropriate Translation Resource format 1001. The packets are then routed to the appropriate Translation Resource 1002.

[0101] The Adaptor also receives packets from Translation resources 1003. The packet is converted back to the work packet format 1004 and the status and control information in the work packet is updated 1005. The Manager's Console is then notified of the work packet's availability 1006 in the translation queue.

[0102] This degree of flexibility allows new translation technologies and translation service providers to be quickly integrated into the workflow process. Additional resources are easily allocated to accommodate sites which require frequent update and synchronization.

[0103] Referring to FIG. 11, a task view of the Adaptor is shown. The Convert to Translation Resource module 1101 converts work packets to the appropriate Translation Resource format that is understood by the resource. The packets are then routed to the Translation Resource by the Route Packet module 1102.

[0104] Packets are received by the Receive Packet module 1103. The Convert/Update Packet module 1104 performs the packet conversion back to the work packet format and updates the packet's status and control information. It then notifies the Manager's Console that the work packet is available in the translation queue.

[0105] The Workflow Pipeline together with the Translation Resources allow the translation process to proceed rapidly and efficiently.

[0106] Translation Resources are centralized or distributed functional blocks which are connected to the Workflow Pipeline where the actual work, including translation, is performed. A typical workflow process will use one or more Translation Resources in a specified sequence to complete the required work.

[0107] A Translation Resource performs translation or other linguistic functions on a set of input documents and their constituent or dependent components and produces a set of output documents that have been transformed linguistically.

[0108] The Workflow Pipeline connects to any type of Translation Resource.

[0109] Four examples of Translation Resources are:

[0110] Machine translation systems

[0111] Translation memory systems

[0112] Human translators

[0113] Terminology management systems

[0114] Machine translation is software which provides fully automatic, draft-quality translations.

[0115] Translation memory systems provide a database of source/target translation pairs for a particular document. When an updated version of a document needs to be translated, the translation memory can be used to pre-translate the document with translations from the previous version of the document.

[0116] Human translators are linked to the Workflow Pipeline through a special Adaptor that connects to the Internet or a customer's intranet. This allows any translator, anywhere in the world, to join a virtual translation team and provide translation services to multilingual Web sites or any other translation application serviced by the invention.

[0117] Terminology management systems provide functions that are necessary to support the entire translation process, such as automated generation and retrieval of client-specific glossaries and lexicons. Both human-based as well as technology-based Translation Resources require access to appropriate glossaries and lexicons to ensure that linguistic transformations are performed in the correct terminological domain.

[0118] Technology-based Translation Resources are easily upgraded as their technology improves. For example, older machine translation or translation memory systems are replaced with newer systems simply by unplugging the old systems and plugging in the new ones through the Adaptors on the Workflow Pipeline.

[0119] The Manager's Console provides the Web site manager with fully automated management of the entire translation process. The following is a description of how the Web site manager interacts with the Manager's Console in the process of keeping a site up-to-date.

[0120] With respect to FIG. 12, when the site is current and all language content is up-to-date, the Manager's Console 1202 reports to the Web site manager 1201 that no action is required.

[0121] If a document in the master language is subsequently updated—perhaps out-of-date product information on the Web site is being updated—the Console 1202 will immediately alert the Web site manager 1201 that the corresponding foreign language versions of the document are out-of-date and need to be re-translated.

[0122] From this point, the Web site manager simply clicks on a button, using the Console's 1202 “one-touch” translation feature, to automatically initiate the re-translation and update of each document. No additional action is required from the Web site manager 1201 until the documents have been translated and are ready to be mounted back onto the site.

[0123] When the Web site manager 1201 initiates the translation process, the newly updated master language document together with associated control information will be converted by the Console 1202 to an internal format, one which is more suitable for information transport over the Workflow Pipeline 1206. The converted document together with relevant control and status information is known as a work packet. The Workflow Pipeline 1206 is responsible for sending and receiving work packets to an appropriate set of Translation Resources 1207, 1208, 1209 where they are processed.

[0124] The sequence in which the Translation Resources are engaged by the Pipeline is pre-determined according to:

[0125] Settings previously established by the Web site manager

[0126] Subject matter of the content

[0127] Target language of the translation

[0128] Availability of a particular Resource

[0129] When a work packet is dispatched to a Translation Resource 1207 for processing, the packet first flows from the translation queues 1205, through the Pipeline Adaptor for that Resource where it is converted from the Workflow's internal format into one that can be processed by that Resource.

[0130] Once the Translation Resource 1207 has finished its processing, the packet which contains the work completed by the Translation Resource 1207 flows back through the Adaptor where it is re-converted back into the Pipeline's internal format. The Adaptor is also responsible for analyzing the completion status of the Translation Resource and updating the packet's control and status information accordingly.

[0131] A simplified example of a typical workflow sequence is shown below.

[0132] 1. Machine Translation: automatically translate the untranslated (i.e. new) sections of the document.

[0133] 2. Translation Memory: pre-translate the document using saved source/target translation pairs from the previous version of the document.

[0134] 3. Human Translation: post-edit the sections of the document that were machine translated and review the entire document for quality.

[0135] 4. Terminology Management: obtain the correct lexicons and glossaries required to properly apply the other Translation Resources.

[0136] Since the Pipeline 1206 is heavily instrumented, the Console 1202 gives the Web Site manager 1201 continuous updates on the progress of the translation. The Console 1202 also uses status information to automatically provide dynamic dispatch and control decisions to achieve the fastest and most cost-effective turnaround time.

[0137] After completion of the workflow sequence, the translated documents are routed back to the Console 1202 where the Web site manager 1201 is notified of their completion. From there, the Web site manager 1201 can directly install the translated documents back onto the active area of the Web site 1203, file system (local or remote), or database (local or remote) at the click of a button, or can make them available for internal review within the organization. For example, when the translated documents are installed onto the active area of the Web site 1203, they are referenced by the appropriate language HTML commands 1210, 1211, 1212, 1213, 1214, 1215 inserted into the Web site pages and the translated content is removed from the translated documents and stored back in the database.

[0138] New languages are easily added to the Web site. The Web site manager simply needs to select the new language from a pulldown list along with the desired documents on the Manager's Console and the necessary steps will be automatically invoked to initiate the translation.

[0139] The invention provides the Web site manager with an effortless way of keeping the site up-to-date.

[0140] One skilled in the art will readily appreciate that, although the invention has been described in the context of Web sites, the same concept applies to other applications. For example, in a fax application, documents are scanned and sent through the invention for translation into selected languages. The translated documents are then automatically faxed to selected locations. Another example would be a universal translator where spoken words are converted into a data stream and sent through the invention for translation into designated languages. The translated data stream is converted back into audible speech and sent to the designated audiences.

[0141] The invention is scalable to provide multilingual presentation and management functions across multiple servers in a distributed environment. For example, some sites might have English content on one server, Japanese on another, and so on. Each server will have installed on its respective site, an instance of the invention communicating with other instances of the invention throughout the system. One server is designated the master, and the others are slaves for the purpose of managing content. The servers communicate over a dedicated interface allowing content to be managed in a distributed fashion. This configuration also supports mirrored sites across multiple servers.

[0142] One skilled in the art will readily appreciate that, although a single user system is specifically mentioned, the same concept applies to multi-user systems, for example, a multi-user configuration would be appropriate for an Internet Service Provider (ISP) that serves multiple customers, with and without multilingual sites, from a single instance of the invention.

[0143] The following is an excerpt of a user manual describing the Visitor and Developer modules for a preferred embodiment of the invention called “WebPlexer”:

[0144] The following sections are ordered beginning with basic functions and gradually progressing to those that are more advanced. Similar functions have been grouped together logically for easy reading.

[0145] WebPlexer has 3 network ports. These ports are used to communicate with the client, the Web server, and the console. The ports can be assigned to any available port number on your system and are specified in WebPlexer's WP_Config file.

[0146] The 3 ports are:

[0147] Internetport. This is the port WebPlexer uses to communicate with the client.

[0148] ServerPort. This is the port WebPlexer uses to communicate with the Web server.

[0149] ConsolePort. This is the port WebPlexer uses to communicate with a client that is being used for the console. Please see Section 3.10, “WebPlexer Console,” for more information on console operation.

[0150] Port numbers are assigned starting at 1. Lower numbers are reserved for dedicated processes like ftp(21), http(80), and telnet(23). The lower numbered ports are also privileged in that they require root-level authority to use. For testing, we recommend choosing an available port with a number of at least 3000.

[0151] 3.1.1 Operating Modes

[0152] WebPlexer's InternetPort and ServerPort are configured to support two modes of operation, ‘test mode’ and ‘production mode’.

[0153] 3.1.1.1 Test Mode

[0154] In test mode, a separate access path is created for clients to send requests to WebPlexer. Normal http requests to port 80 and the existing Web server are not affected. This permits off-line testing of WebPlexer functionality until the site has been validated.

[0155] To configure test mode:

[0156] 1. Set InternetPort to an unused port number. This will be used for all incoming WebPlexer requests.

[0157] 2. Set ServerPort to 80. This will allow WebPlexer to connect to your existing Web server.

[0158] To access the Web server via WebPlexer, you would type a URL of the form:

[0159] http://www.mysite.com:iiii/

[0160] where iiii is the InternetPort assigned above.

[0161] Normal requests of the form:

[0162] http://www.mysite.com/

[0163] will go directly to the Web server, bypassing WebPlexer.

[0164] You can make WebPlexer available to ordinary port 80 users, even in test mode, by including a special link on a page to send a request to WebPlexer's port.

[0165] For example,

[0166] <A HREF=“http://www.mysite.com:3080/webplexer/index.html”>WebPlexer test</a>

[0167] could be included on a site's home page.

[0168] 3.1.1.2 Production Mode

[0169] In production mode, WebPlexer is configured to process all http requests before they reach the Web server. This would normally be done once off-line testing was completed and the site was ready to go into production.

[0170] To configure production mode:

[0171] 1. Set InternetPort to 80. All http requests will be processed by WebPlexer. Note that this will typically mean that WebPlexer must run as a privileged process.

[0172] 2. Set ServerPort to an unused port. This port will be dedicated to WebPlexer <—> Web server communication.

[0173] 3. Reconfigure your Web server so it is connected to ServerPort. This is typically done by editing your server's httpd.conf file, but the configuration method is determined by your specific server, so please check the documentation that comes with your Web server.

[0174] Normal requests of the form:

[0175] http://www.mysite.com/

[0176] will then go to WebPlexer for initial processing.

[0177] You can still access the Web server directly, with a URL of the form:

[0178] http://www.mysite.com:ssss/

[0179] where ssss is the ServerPort assigned above.

[0180] 3.1.2 Setting InternetPort

[0181] InternetPort is set by editing the line:

[0182] InternetPort iiii

[0183] in the WP_Config file.

[0184] This setting can also be overridden by a command line flag. Please see Section 3.11, “Runtime Options,” for more information.

[0185] 3.1.3 Setting ServerPort

[0186] ServerPort is set by editing the line:

[0187] ServerPort ssss

[0188] in the WP_Config file.

[0189] This setting can also be overridden by a command line flag. Please see Section 3.11, “Runtime Options,” for more information.

[0190] 3.1.4 Setting ConsolePort

[0191] ConsolePort is set by editing the line:

[0192] ConsolePort ccc

[0193] in the WP_Config file.

[0194] This setting can also be overridden by a command line flag. Please see Section 3.11, “Runtime Options,” for more information.

[0195] WebPlexer supports 3 types of logs:

[0196] AccessLog

[0197] ErrorLog

[0198] InformationLog

[0199] The filenames of these logs are defined in the WP_Config file. Additional information on how to define the log files can be found in Chapter 4, “Contents of WebPlexer Files.”

[0200] NOTE: A fourth log, the Usage Log is generated by the updtusage tool that analyzes the Information Log. This tool will be included in a future release.

[0201] The function of each log is described below. A detailed specification can be found in Section 4.4, “Log Specifications.”

[0202] 3.2.1 AccessLog

[0203] The AccessLog records information about each request received by WebPlexer, such as date, time, and request type. WebPlexer command requests (WPCommands) are also logged.

[0204] NOTE: When WebPlexer is configured for “production mode” (see Section 3.1.1.2, “Production Mode”), WebPlexer's Access Log should be used in place of the Web server's access log for analysis and auditing purposes (the Web server's access log will show all requests originating from WebPlexer).

[0205] Examples of AccessLog entries are: pp93.satcom.net—[28/Jul/1996:23:37:23-0700] “GET/cnv.post/info.html HTTP/1.0” 302-pp93.satcom.net—[28/Jul/1996:23:37:24-0700] “GET/cnv.post/english-american_United-States/info.html HTTP/1.0”200 3372 pp93.satcom.net—[28/Jul/1996:23:37:29-0700] “GET/cnv.post/coming/flags/United-States.gif HTTP/1.0” 200 3594

[0206] 3.2.2 InformationLog

[0207] The InformationLog contains detailed information about WebPlexer's language selection processing for each request.

[0208] Examples of InformationLog entries are:

[0209] prune.loria.fr—[26/Jul/1996:05:44:43-0700] Country Selection: prune.loria.fr→France

[0210] prune.loria.fr—[26/Jul/1996:05:44:43-0700] Language Set: prune.loria.fr→{french-continental, french}

[0211] prune.loria.fr—[26/Jul/1996:05:44:43-0700] Language Selection: prune.loria.fr→French-continental

[0212] 3.2.3 ErrorLog

[0213] The ErrorLog shows any errors that occurred during WebPlexer operation, or during the processing of a request.

[0214] Examples of ErrorLog entries are:

[0215] [23/Jul/1996:03:35:40-0700] FATAL: Could not send request to HTTP server!!!

[0216] [23/Jul/1996:04:03:34-0700] Timed out when writing response to client

[0217] 3.2.4 UsageLog

[0218] The UsageLog contains a summary of country and language statistics. It is generated from the InformationLog using the usage log analysis tool. The UsageLog can be further processed using the visits log analysis tool. Please see section x.xx for a description of WebPlexer's log analysis tools.

[0219] Examples of UsageLog entries are:

[0220] [17/Oct/1996:13:40:11-0700] Language: German www07.btx.dtag.de

[0221] [17/Oct/1996:14:09:57-0700] Country: Argentina server1.sminter.com.ar

[0222] WebPlexer's Language Selection feature allows the most appropriate language of a requested document to be served to a Web browser.

[0223] “Most appropriate language” means a language that has been:

[0224] specified as a preference by the Web browser

[0225] automatically selected by WebPlexer from the country of origin

[0226] manually chosen by the user

[0227] specified by a cookie

[0228] These three language selection modes all work together seamlessly. They do not require the addition of any special CGI scripts or links to be added to your Web site.

[0229] The language selection algorithm requires that each language be assigned a unique directory in the document tree. All documents in that language are placed in the corresponding directory. The entire document tree then becomes a collection of parallel language directories.

[0230] For example, if user ‘Joe’ has english, french, and spanish documents on your site, the directory structure might look like this:

[0231] htdocs/

[0232] joe/

[0233] english/

[0234] index.html

[0235] info.html

[0236] french/

[0237] index.html

[0238] info.html

[0239] spanish/

[0240] index.html

[0241] info.html

[0242] WebPlexer's language selection algorithm works by inserting the correct language directory in the URL before sending the request on to the Web server.

[0243] For example, an incoming URL of:

[0244] http://www.lai.com/joe

[0245] would send a modified URL of:

[0246] http://www.lai.com/joe/french

[0247] to the Web server, if the browser's preferred language were French.

[0248] 3.3.1 The Document Matrix

[0249] A group of language directories and the documents they contain can be likened to a matrix. This matrix has one row for each language, and one column for each document in the collection. When expressed in this form, the collection is known as a document matrix.

[0250] In the example just given, the document matrix might look something like:

[0251] where the ‘X’ indicates the presence of a document in that language.

[0252] WebPlexer's exclusive document management feature known as “SPAMM” (Sparse Matrix Manager) does not require each document to be translated in every language, and therefore does not require every cell of the document matrix to be filled in.

[0253] In addition, SPAMM also permits templates to be placed only in the default language directory (see Section 3.3.2.3, “Specifying the Local Default Languages”) and eliminates the need for them to be instantiated in every directory.

[0254] SPAMM offers an incremental approach to building a multilingual Web site. Please see Section 3.9.2, “SPAMM (Sparse Matrix Manager),” for more information on this feature.

[0255] 3.3.2 Setting Up Language Selection

[0256] WebPlexer's Language Selection feature is enabled by editing a few lines in the WP_Map file, located in the main WebPlexer directory. Once the file has been set up, any of the language selection modes (browser-specified, automatic, manual) are available.

[0257] The information that needs to be entered in the WP_Map file is:

[0258] 1. The top-level URL of your site.

[0259] 2. The list of languages on your site and the names of the directories where the documents are located.

[0260] 3. The encodings in which the documents are represented (e.g. ISO-8859-1)

[0261] 4. Default directories in case the requested language is not found (usually English).

[0262] These four pieces of information together make up what is known as a “URL group” declaration.

[0263] Often a single server will serve multiple sites. You can easily have WebPlexer support multiple sites by creating a separate URL group for each site.

[0264] The next sections explain each component of the URL group declaration. Please refer to the example WP_Map file supplied with this release while reading the following sections.

[0265] 3.3.2.1 Specifying the Top-Level URL

[0266] The top-level URL is specified by adding lines of the form

[0267] URL::<path><path_type>

[0268] to the WP_Map file.

[0269] where

[0270] <path>

[0271] is the directory path of the top-level URL below the htdoc root.

[0272] <path-type>

[0273] is either ‘d’ or ‘f’ depending on whether the path is a directory or a file.

[0274] NOTE: In the current implementation, when a directory is specified, it is also necessary to specify the corresponding top-level file as well (usually index.html).

[0275] In the previous example, there would be two lines reading:

[0276] URL::/joe d

[0277] URL::/joe/index.html f

[0278] Note that this file (/joe/index.html) does not actually exist in the document hierarchy but should match the URL the user will type when entering the site.

[0279] If the selected language were French, the URL mappings would be:

[0280] http://www.lai.com/joe→http://www.lai.com/joe/french and

[0281] http://www.lai.com/joe/index.html→http://www.lai.com/joe/french/index.html

[0282] 3.3.2.2 Specifying the Language-to-Directory Map and Charset

[0283] The language-to-directory map and charset (document encoding) are specified by adding lines of the form:

[0284] <language>: <language_directory><charset>

[0285] immediately after the URL specification.

[0286] where

[0287] <language>

[0288] is the name of the language. Please see Appendix B., “List of Available Languages,” for a list of which languages are available for this release.

[0289] <language_directory>

[0290] is the name of the subdirectory containing documents of that language.

[0291] <charset>

[0292] is the encoding used for documents in that language (e.g. ISO-8859-1)

[0293] In the previous example, these lines would read:

[0294] english: english ISO-8859-1

[0295] french: french ISO-8859-1

[0296] spanish: spanish ISO-8859-1

[0297] 3.3.2.3 Specifying the Local Default Languages

[0298] The local default languages tell WebPlexer which language to use in case the requested language is not available on the server.

[0299] There are two local default languages that need to be specified. The first is called “others” and the second is simply called “default”.

[0300] The “others” language is chosen when the requested language is valid but does not exist on the server.

[0301] For example, if English, French, and Spanish documents exist on the server, but the requested language is Russian, WebPlexer will use the language defined as “others”.

[0302] The “default” language is chosen when WebPlexer cannot determine the language of a request and none has been specified by the browser. This can happen when the country of origin cannot be determined because the hostname is unavailable, for example, when the requester is behind a firewall.

[0303] These directories are specified by including the following lines in the WP_Map file:

[0304] others: <others_language><charset>

[0305] default: <default_language><charset>

[0306] It is usually the case that “others” and “default” are both set to English.

[0307] In the example above, these lines would read:

[0308] others: english ISO-8859-1

[0309] default: english ISO-8859-1

[0310] The next sections describe the language selection algorithms in more detail.

[0311] 3.3.3 Browser-Specified Language Selection

[0312] In browser-specified language selection, the browser sends its language preference along with the http request. The language is set through a preference or option setting in the browser. The preference is sent to WebPlexer in the “Accept-Language” header within the request message.

[0313] The Accept-Language header is described in the http specification. The specification allows the browser to send more than one language in the header. The group of languages is ordered with the most desirable language first.

[0314] WebPlexer uses the Accept-Language header to select the correct language. The header is only used when the request is to a URL that matches the top-level URL given in the WP_Map file (see Section 4.2.1, “Specifying the Top-Level URL”).

[0315] For example, if the top-level URL were /joe, and the browser preference were set to Spanish, a request to:

[0316] http://www.lai.com/joe/

[0317] would cause the URL to be mapped to:

[0318] http://www.lai.com/joe/spanish

[0319] and sent to the Web server.

[0320] If the browser has specified more than one language, WebPlexer goes through the languages in order and stop when it finds the first one that is present on the server. If none of the languages are found, WebPlexer uses the “others” language specified in the WP_Map file (see Section 4.2.2.1, “others:”)

[0321] Once the language has been determined, WebPlexer redirects the browser to the directory for that language. All accesses to relative links within the requested document stay within that directory.

[0322] Changing the browser's language preference does not have any effect until the top-level URL is again requested and the browser re-enters the site. The recommended way to change language within a language directory is by manual selection (see Section 3.3.5, “Manual Language Selection”).

[0323] Browser-specified language selection has the highest priority upon entry to the site.

[0324] 3.3.4 Automatic Language Selection

[0325] In automatic language selection, WebPlexer determines the language by first determining the country from the requester's domain, then looking up the language for that country in a database.

[0326] Automatic language selection is only active under the following two conditions.

[0327] 1. The request is to the top-level URL specified in the WP_Map file (see Section 3.3.2.1, “Specifying the Top-Level URL”).

[0328] 2. The browser has not set a preferred language (see Section 3.3.3, “Browser-Specified Language Selection”).

[0329] NOTE: If the browser has specified a language preference, it overrides any automatic language selection.

[0330] For example, if the top-level URL were /joe, and the host domain were eiffel.tower.fr, a request to:

[0331] http://www.lai.com/joe/

[0332] would cause the URL to be mapped to:

[0333] http://www.lai.com/joe/french

[0334] and sent to the Web server.

[0335] The country-to-language database contains a list of languages for each country. These languages are arranged in order of frequency from most-used to least-used.

[0336] This list of languages is processed in the same way as preferred languages are handled when received from a browser (see Section 3.3.3, “Browser-Specified Language Selection”). This means that the languages are examined until the first one present on the server is found.

[0337] For example, the languages of Switzerland are {german, french, italian, romansch} and will be searched in that order. The rules for selecting the local default languages are the same as for browser-specified languages, if the language cannot be found.

[0338] 3.3.5 Manual Language Selection

[0339] Manual language selection is effective at any time. Manual language selection overrides any automatic or browser-specified language.

[0340] Manual language selection is performed by issuing a WPCommand whose type is “language”. WPCommands are more fully described in Section 3.6.3.3, “WPCommands.” They result in the browser sending a request to WebPlexer of the form:

[0341] http://www.lai.com/joe/xxxx/WPCommand?language=yyyy

[0342] where xxxx is the current language (plus country) directory and yyyy is the desired language to switch to.

[0343] NOTE: language (plus country) means that the language directory is appended with the currently selected country (e.g. english_United-States)

[0344] Note that even with manual language selection, the languages and directories in the WP_Map file still need to be set up as in automatic language selection.

[0345] 3.3.6 Cookie-based Language Selection

[0346] 3.3.6.1 Using WPCommand?language

[0347] WPCommands are normally issued through hypertext links embedded in an HTML document. This statement illustrates a simple example,

[0348] <A HREF=“WPCommand?language=french”>French</A>

[0349] If the user is currently looking at an English document, clicking on the word ‘French’ will cause the browser to send WebPlexer a request for the URL

[0350] http://www.lai.com/joe/english_United-States/WPCommand?language=french

[0351] which will map to

[0352] http://www.lai.com/joe/french_United-States/xxx.html

[0353] where xxx.html is the current document (the one that contains the WPCommand link). Note that WebPlexer will return the browser to the original document in the new language, in this case, French.

[0354] The manual language selection feature is performed completely by WebPlexer, without the need for CGI scripts or complex interlinking of HTML documents. It merely requires the addition of a link in the HTML to send the WPCommand to WebPlexer for each desired language.

[0355] The WPCommand hypertext link can be used like any other HTML link, that is, it can be placed within a table, connected to a graphic or icon, or built into an imagemap.

[0356] 3.3.6.2 Using WPCommand?link

[0357] The method described above works well when the number of languages is small. However, as the number of languages on the server increases, there may not be enough space to have the links for all the languages appear on each page.

[0358] In this case, a generic mechanism exists for handling WPCommands that will move all of the language selection links off to a separate page. Once the links are moved, a hypertext link that points to this special language selection page is placed on each page of your site.

[0359] Because of the need to return to the originating page when the language is changed, WebPlexer requires that a special WPCommand be used to invoke the link to the change language page.

[0360] This WPCommand is of type ‘link’ and takes the name of the page to link to as an argument. Its external behavior is the same as if a direct HTML link were used, except it notifies WebPlexer that a change language (or change country) WPCommand is about to be issued.

[0361] This allows WebPlexer to return the browser to the originating page once the language has changed.

[0362] In the HTML, this link would be coded as

[0363] <A HREF=“WPCommand?link=langsel.html”>Change Language </A>

[0364] and would produce a request URL to WebPlexer of the form

[0365] http://www.lai.com/joe/english_United-States/WPCommand?link=langsel.html

[0366] where langsel.html is the language selection page.

[0367] 3.3.7 Hostname and IP Address Overrides

[0368] WebPlexer provides a method of optionally forcing the selection of specific language(s) for certain hostnames or ip addresses.

[0369] This feature overrides automatic language selection when the top-level URL specified in the WP_Map file is requested. A browser-specified language preference will still be used if specified.

[0370] There are two files used to specify automatic language selection overrides: HostNameMap and IPAddrMap, both of which are in WebPlexer's language subdirectory.

[0371] The format of these files is described in Section 4.3, “Map Overrides.”

[0372] 3.3.8 Setting the Global Default Language

[0373] The global default language is set in the WP_Map file with the directive:

[0374] defaultLanguage <language>

[0375] and is normally the same as the local default language specified in the URL group definition. Specifying the local default language is described in Section

[0376] 3.3.2.3, “Specifying the Local Default Languages.”

[0377] The global default language is used as the default for the console and for some WebPlexer initialization.

[0378] 3.3.9 Content Tagging and Support for Multilingual Browsers

[0379] Content tagging allows WebPlexer to inform the browser of the language and charset used to encode a document when the document is accessed. Some browsers, like Netscape Navigator, use this information to automatically switch to the correct font and encoding for a particular language.

[0380] Content tagging is always active whenever a document is fetched from a language directory that is specified in the WP Map file.

[0381] When a document is content tagged, the http Content-Language and Content-Type response headers are returned with the document to indicate the language and charset in use.

[0382] These fields are automatically updated by WebPlexer even when the language is changed by manual selection.

[0383] WebPlexer provides three ways by which the country of an incoming request is determined.

[0384] Automatic country selection

[0385] Manual country selection

[0386] Cookie-based country selection

[0387] These are described in detail below.

[0388] 3.4.1 Automatic Country Selection

[0389] With automatic country selection, WebPlexer determines the country from the requester's domain. If the domain cannot be determined (usually because the requester is behind a firewall), the country is set to the global default country (see Section 3.4.5, “Setting the Global Default Country”).

[0390] Automatic country selection is only active when the browser request is to the top-level URL specified in the WP_Map file. See Section 3.3.2.1, “Specifying the Top-Level URL,” for a detailed description of how to set the top-level URL. Once the country has been determined, its value is saved by appending it to the language directory in the mapped URL.

[0391] For example, if the top-level URL were /joe, and the host domain was eiffel.tower.fr, a request to:

[0392] http://www.lai.com/joe/

[0393] would cause the URL to be mapped to:

[0394] http://www.lai.com/joe/french_France/

[0395] By embedding the country name in the URL, WebPlexer can retain the country setting for the lifetime of this session.

[0396] 3.4.2 Manual Country Selection

[0397] Manual country selection is performed by issuing a WPCommand of type “country”. WPCommands are more fully described in Section 3.6.3.3, “WPCommands.” They result in the browser sending a request to WebPlexer of the form:

[0398] http://www.lai.com/joe/xxxx/WPCommand?country=yyyy

[0399] where xxxx is the current language directory appended to the current country, and yyyy is the desired country to switch to. Once the country changes, the language will also change to match the new country.

[0400] For example, if the current country is France and the current language is French, and the current document being viewed is info.html, to request that the country be changed to German, this request should be sent to WebPlexer:

[0401] http://www.lai.com/joe/french_France/WPCommand?country=germany

[0402] WebPlexer will respond with a new URL that looks like

[0403] http://www.lai.com/joe/german Germany/info.html

[0404] and which will fetch the German version of the info.html document.

[0405] 3.4.3 Cookie-based Country Selection

[0406] 3.4.4 Declaring Worldwide Regions

[0407] Regions are defined in the WP_Regions file. Regions allow groups of countries to be combined into geographical areas that are meaningful for the type of content that is on the site.

[0408] For example, if a company had one sales office in Europe, and another in Asia, and had developed Web site content that was specific to these markets, it might want to create two regions like this:

[0409] Sales_region1: France, Germany, Italy, Spain

[0410] Sales_region2: China, Hong-Kong, Japan

[0411] Regions can be either user-defined or pre-defined (default). User-defined regions are specified in the WP_Regions file (see Section 4.9, “WP_Regions”).

[0412] Pre-defined (default) regions are established by WebPlexer. Please see Appendix D., “List of WebPlexer Default Regions,” for more information.

[0413] Regions are used by the multi-country server-side include feature (see “Multi-Country Server-Side Includes (MCSSI)”). They allow specific content to be presented to users from countries in those regions when they enter the Web site.

[0414] 3.4.5 Setting the Global Default Country

[0415] The global default country is set in the WP_Map file with the directive:

[0416] defaultCountry <country>

[0417] where <country>is one of the valid countries given in Appendix C., “List of Available Countries.”

[0418] The global default country is used as the default for the console and for some additional WebPlexer functions.

[0419] WebPlexer has the unique ability to independently maintain both the country and the language associated with each request. This provides a great deal of flexibility in selectively targeting content to specific regions or countries of the world and, at the same time, making that content available in multiple languages.

[0420] It is important to understand how WebPlexer's country management and language management are different and how they are similar.

[0421] In the following description, the term “session” refers to a logical grouping of requests between browser and WebPlexer. A session begins with the browser making a request for the top-level URL. It ends with the final response being sent from WebPlexer to the browser. A session can have any number of request/response pairs between these two endpoints.

[0422] A session has a state associated with it. Included in this state are the country and language of the requester.

[0423] Once the country and language state have been determined from the initial request, they remain unchanged until the browser initiates an action like manually changing the country or language.

[0424] Unfortunately, the http protocol used for data transport on the Web is a stateless protocol. This means that the protocol does not allow state to be maintained from one request to another.

[0425] WebPlexer uses a unique persistence mechanism for retaining session state across multiple requests. This mechanism allows country and language information to be saved throughout the lifetime of a session.

[0426] WebPlexer uses a simple 4-step algorithm for managing country and language when processing a request:

[0427] 1. Determine country.

[0428] 2. Determine language from country.

[0429] 3. Select content for country.

[0430] 4. Modify content for language.

[0431] Once the country and language for a request have been established, they are available as state variables within the HTML and can be used for:

[0432] 1. Content selection based on country, region, or language.

[0433] 2. Word and phrase replacement out of an external glossary (this glossary is referred to as a TermDB).

[0434] 3. Setting language and country-specific “custom variables” such as country flag, language name, and country name.

[0435] 4. Sorting a list of items according to sort rules of the currently displayed language.

[0436] WebPlexer's 4-step algorithm for country and language management is described in more detail below.

[0437] 3.5.1 Determine Country

[0438] WebPlexer will first determine the country of the requester if it is not known (on an initial request), or when it is manually selected (through a WebPlexer change country command (see Section 3.4.2, “Manual Country Selection”)).

[0439] 3.5.2 Determine Language from Country

[0440] WebPlexer uses the country from the previous step to determine the slanguage of the requester. A list of languages for the selected country is obtained from a database and the first one that matches a language present on the server is selected.

[0441] For example, if the requestor's country were Switzerland, the requestor's language could be set to either german, french, italian, or romansch depending on which one was found first on the server.

[0442] The exceptions to this rule are:

[0443] 1. When the browser specifies the language explicitly in its preferences (see Section 3.3.3, “Browser-Specified Language Selection”).

[0444] 2. When the language is overridden through a HostName or IPAddress map (see Section 3.3.7, “Hostname and IP Address Overrides”).

[0445] 3. When the language is manually selected through a WebPlexer change language command (see Section 3.3.5, “Manual Language Selection”). Once country and language are determined, the following rules apply.

[0446] 3.5.3 Select Content for Country

[0447] WebPlexer uses the country to dynamically select content specific to that country. This allows the content to change based on the geographic location of the requester. The actual selection mechanism is controlled by the multi-country server-side include feature. This feature is more fully described in “Multi-Country Server-Side Includes (MCSSI).” Additional information can be found in Section 3.6.1, “Template Model,” which describes WebPlexer's template model.

[0448] 5 3.5.4 Modify Content for Language

[0449] WebPlexer presents the content in the language selected by the requestor.

[0450] There are 3 methods that WebPlexer uses to present content in the selected language:

[0451] 1. A static document that has been pre-translated is selected from the corresponding language directory. For example, if the language is French, the document will be retrieved from the french directory.

[0452] 2. Content that has been pre-translated is inserted into a template using the multi-language server-side include feature. This feature is more fully described in “Multi-Language Server-Side Includes (MLSSI).” Additional information can be found in Section 3.6.1, “Template Model,” which describes WebPlexer's template model.

[0453] 3. Words and sentences that are marked up with WebPlexer's WPReplace tags are dynamically “translated” into the requested language using an external TermDB. WPReplace tags and TermDBs are described in Section 3.6.3.1, “WPReplace,” and Section 3.6.2, “TermDBs (tM).”

[0454] The Multilingual ToolKit is a set of HTML extensions and templates that simplifies the design and construction of multi-language and multi-country Web pages.

[0455] The Multilingual ToolKit consists of several key WebPlexer resources:

[0456] 1. A template-based model of page construction.

[0457] 2. External, multilingual glossaries known as TermDBs.

[0458] 3. A set of HTML extensions to allow country and language settings to dynamically control a document's content.

[0459] 4. Built-in support for manually changing language and country, eliminating the need for special CGI scripts or complex interlinked pages.

[0460] 5. Custom variables to facilitate customization of a document according to the current language and country.

[0461] 6. Custom functions to change the appearance of a page according to language and country.

[0462] 7. Server-side includes to allow a document to be built out of language and country-specific component parts.

[0463] The following sections discuss the specific components of the Multilingual ToolKit in more detail.

[0464] 3.6.1 Template Model

[0465] WebPlexer's Multilingual ToolKit uses a template-based approach to dynamically create documents tailored for a specific language or country.

[0466] A template contains placeholders for country and language-specific information that has been removed from a document. This information is dynamically inserted from a TermDB, another template or document, or provided automatically by WebPlexer when the composite document is presented to the browser.

[0467] The basic approach to creating a template has two steps. First, remove as much country and language-specific information as possible from a document. Then, replace this information with appropriate WebPlexer tags and commands.

[0468] The advantage to using templates is that a single document can easily support many languages and countries.

[0469] For example, a single form or CGI-generated document can be constructed so that it will be automatically localized for different languages and locales.

[0470] This significantly reduces the number of documents that have to be maintained on the site and makes it very easy to add new languages. It also allows a single update to a document to be immediately propagated to all languages and countries.

[0471] This technique also allows country or regional content to be conveniently separated from the structure of the document, making it easy to change a design often by updating a single file. This eliminates the ripple effect often seen when a single change must propagate through all of the localized documents on the site.

[0472] 3.6.1.1 Template Types

[0473] WebPlexer supports three basic kinds of templates.

[0474] 1. Static (no template).

[0475] 2. File-based.

[0476] 3. Tag-based.

[0477] One document may contain all three template types. We call such a document a “composite” document, since it is made up of multiple elements.

[0478] Most documents will have a mixture of these three basic types. In the sample included with this release, the file ‘info.html’ is a combination of static and tag-based templates, whereas the template of ‘contact.html’ is completely tag-based.

[0479] Please look through the sample files in the htdocs directory of this release for examples showing the use of templates.

[0480] File-based templates require the server-side include feature. This feature is not available in the current release.

[0481] Tag-based templates are the most generic and allow a single document to be used across all languages and countries.

[0482] In general, the smaller the amount of static content in a template, the more generic the template is and the fewer the number of times it needs to be instantiated.

[0483] The next sections discuss the template types in more detail.

[0484] Static

[0485] Static templates are not really templates at all. They are simply collections of static text.

[0486] File-based

[0487] File-based templates allow external files containing localized content such as country, regional, or language-specific information to be inserted dynamically into a document. File-based templates are provided using the server-side include feature described in Section 3.6.3.5, “Server-Side Includes.”

[0488] Tag-based

[0489] Tag-based templates allow replacement of marked-up text with translations from an external TermDB according to the selected country or language. Tag-based templates are provided using the WebPlexer HTML extensions described in Section 3.6.3.1, “WPReplace.”

[0490] 3.6.2 TermDBs (tM)

[0491] A TermDB is an HTML representation of a multilingual glossary. The glossary is built in the form of an HTML table. Rows in the table correspond to entries in the glossary, be they words, phrases, or arbitrary strings. Columns in the table correspond to individual languages in the glossary such as English, French, or German.

[0492] One language in the TermDB is designated the primary language and is assigned to column 1 of the table. This primary language is used as the language of the key when looking up entries in the TermDB.

[0493] The TermDB is used together with the WPReplaceBegin and WPReplaceEnd tags. (See Section 3.6.3.1, “WPReplace”). The tags enclose text that will be replaced by a TermDB entry. The text must be in the primary language.

[0494] The following example shows how the word “hello” would be translated using the TermDB.

[0495] Sample HTML:

[0496] <!--WPReplaceBegin-->hello<!--WPReplaceEnd→>

[0497] Sample TermDB:

[0498] english hello

[0499] french bonjour

[0500] spanish hola

[0501] When WebPlexer accesses the TermDB, it first finds the word ‘hello’ by looking in row 1 (the primary language row) of the TermDB and finding the column that contains the desired term. Then it retrieves the term from the row of the table corresponding to the current language (row 2, in the case of French) in the column it identified as containing the desired term.

[0502] This release of WebPlexer comes with a standard TermDB that includes the names of all languages and countries translated into 25 languages.

[0503] WebPlexer supports the use of multiple TermDBs. Each top-level URL declared in the WP_Map file can be associated with a different TermDB, and multiple TermDBs can be defined for a given URL. This makes it possible to maintain several small TermDBs rather than one large one.

[0504] There are three types of TermDBs:

[0505] The System TermDB (included with the product)

[0506] Add-on TermDBs (available from Language Automation)

[0507] User-defined TermDBs (created with the tpbuilder utility, explained in the next section).

[0508] TermDBs, together with WebPlexer's WPReplace tag, allow multilingual terminology to be centralized. This has several advantages:

[0509] It makes it easy to add support for new languages without changing any of the documents on the site.

[0510] Changes to the form or structure of a document can be made immediately available in all languages. All languages can be updated simultaneously.

[0511] The number of documents that need to be maintained on the site is reduced. It is not necessary to have separate versions of a document for each language.

[0512] A single CGI script or form can be localized for multiple languages from a single copy of source code. No update is required when new languages are added.

[0513] TermDBs and the other resources of the Multilingual ToolKit can be used most effectively if you plan appropriately before you design your site. If you need assistance in building your multilingual site, please contact Language Automation and one of our site support engineers will be happy to assist you.

[0514] 3.6.2.1 Building User-defined TermDBs

[0515] User-defined TermDBs can be created and updated using the tpbuilder utility that comes with WebPlexer.

[0516] tpbuilder has the following syntax:

[0517] tpbuilder <controlFile><outputTermDB>or

[0518] tpbuilder <controlFile><inputTermDB><outputTermDB>

[0519] The first form is used when creating a new TermDB. The second form is used for updating an existing TermDB.

[0520] controlFile has the following syntax:

[0521] where

[0522] <TermDB title string>

[0523] is the title string in the generated HTML file.

[0524] <TermDB table caption>

[0525] is the caption of the generated table.

[0526] <TermDB text body>

[0527] is the string inserted before the actual table and is part of the text body.

[0528] Note that these strings are for informational and readability purposes only and are not used by WebPlexer.

[0529] A line of the form

[0530] <languageN><encodingN><input-fileN><comment>

[0531] specifies the target language, encoding and input file that contains the list of strings to be entered into the TermDB. The first line must specify the TermDB's master language. The master language is used to index into the TermDB using a key. The key is obtained from the string contained between <!--WPReplaceBegin-->and <!--WPReplaceEnd→>tags.

[0532] NOTE: the control file and input files must be in the same directory.

[0533] For example, a typical control file might look like this:

[0534] Joe's TermDB

[0535] This is the TermDB for Joe

[0536] Joe's TermDB

[0537] In this case, english is defined to be the master language of the TermDB and the list of index terms will be taken from the file laieng.txt. The corresponding translations into arabic and french will be read from the files laiarah.txt and laifre.txt, respectively.

[0538] The input files have the following syntax:

[0539] 1:<TermString1>

[0540] 2:<TermString2>

[0541] .

[0542] .

[0543] .

[0544] N:<TermStringN>

[0545] where <TermStringN>is a string to be entered into the TermDB for that particular language. Note that the numbers at the beginning of the line are arbitrary and are used to easily align the text across each of the input files.

[0546] For example, a typical set of input files might look like this:

[0547] English

[0548] 1:hello

[0549] 2:goodbye

[0550] French

[0551] 1:bonjour

[0552] 2:au revoir

[0553] Spanish

[0554] 1:hola

[0555] 2:adios

[0556] When tpbuilder is run, it will write status information to stdout.

[0557] Updating an Existing TermDB

[0558] An existing TermDB is updated using this form of the tpbuilder command:

[0559] tpbuilder <controlFile><inputTermDB><outputTermDB>

[0560] NOTE: The controlFile must contain all of the languages that were specified when the TermDB was initially created.

[0561] Generally speaking, there are three types of update operations that can be performed on a TermDB.

[0562] Adding new entries (and optionally their translations)

[0563] Replacing old entries (and optionally their translations)

[0564] Deleting entries

[0565] These operations are controlled through the contents of the individual input files specified in the controlFile and are described in more detail below.

[0566] Adding an Entry

[0567] To add a new entry, simply add a new TermString to the master language input file and optionally, corresponding TermStrings in the target language input files. If a matching translation is not found in a target language input file, an empty entry will be created and will be filled in later when the translation is available through the WebPlexer console's translation management feature. Please see section x.xx for further information on console operation.

[0568] If an empty TermDB entry is accessed during a lookup, the TermDB will return the value of the original master language string (i.e. the index).

[0569] NOTE: Prior to updating a TermDB, it is recommended that TermStrings that will not change be removed from the input files (i.e. input files should only specify actions to be performed during the update).

[0570] Replacing an Entry

[0571] To replace an entry, simply replace old translation(s) in the target language input files with new translations of the TermString. The original TermString must be specified in the master language input file.

[0572] If new translations are not available, simply put a blank line in the target language input file(s) with a number that matches the original TermString in the master language input file. The translations will be updated later through the WebPlexer console's translation management feature.

[0573] Deleting an Entry

[0574] To delete an entry, simply put a minus sign (‘−’) in front of the line in the master language input file corresponding to the entry to be deleted. The master language entry for that TermString along with all translations will be removed from the TermDB.

[0575] Including HTML Tags in a TermDB Entry

[0576] HTML tags can be included in TermStrings to allow complete HTML constructs to be embedded between WPReplace tags. This provides a great deal of freedom in content organization.

[0577] Embedded HTML tags must be preceded by a backslash (‘\’)

[0578] For example, the following is a valid TermString:

[0579] 25:<img src=∫balloon.gif∫\>

[0580] The backslash character can also be used as a valid character in the TermString but it must be preceded by another backslash.

[0581] 3.6.3 HTML Extensions

[0582] The Multilingual ToolKit's HTML extensions are a set of special-purpose commands and comment-embedded HTML tags. They enable WebPlexer to provide special country and language processing when serving a document.

[0583] The extensions consist of

[0584] 1. WPReplace for replacing a text string with its translation from a TermDB.

[0585] 2. WPSort for sorting lists according to the sort rules of the currently selected language.

[0586] 3. WPCommands for providing the change language/change country function without the need for CGI scripts or interlinked pages.

[0587] 4. CustomVars for dynamically inserting language name, country name, and country flag into the HTML.

[0588] 5. Server-side includes for building documents out of country and language-specific components.

[0589] Each of these is described in detail in the following sections.

[0590] 3.6.3.1 WPReplace

[0591] The WPReplace function replaces a string with its translation.

[0592] The string to be replaced and the currently selected language are used as search keys within a TermDB. The TermDB is accessed and returns the translation of the string in the specified language.

[0593] The WPReplace function is implemented with two tags:

[0594] <!--WPReplaceBegin→>

[0595] <!--WPReplaceEnd→>

[0596] The string to be replaced is enclosed between the tags:

[0597] <!--WPReplaceBegin-->String to be replaced

[0598] <!--WPReplaceEnd→>

[0599] The string must be in the primary language of the TermDB (see Section 3.6.2, “TermDBs (TM)”).

[0600] For example, if a TermDB contained these entries:

[0601] ..

[0602] ..

[0603] english hello

[0604] french bonjour

[0605] spanish hola

[0606] then, this HTML code

[0607] <!--WPReplaceBegin-->hello<!--WPReplaceEnd→>

[0608] would produce

[0609] bonjour

[0610] if the current language were French, and

[0611] hola

[0612] if the current language were Spanish.

[0613] Please note the following:

[0614] 1. If the string is not found in the TermDB, it is not replaced.

[0615] 2. The WPReplace commands are removed after replacement.

[0616] Please see the sample file ‘contact.html’ included with this release for an example of WPReplace.

[0617] Please see Section 4.2.3, “outbound:: Enabling the Output Manager,” for information on how to specify TermDBs.

[0618] 3.6.3.2 WPSort

[0619] The WPSort function sorts a list of items according to the sort rules for the current language

[0620] NOTE: this is restricted in the current release. Please see Section 1.3, “Known Bugs and Limitations of This Release.”

[0621] The text to be sorted is divided into a series of lines, each of which contains a key.

[0622] Line 1 (sort key=‘box’): < . . . box . . . >

[0623] Line 2 (sort key=‘ball’): < . . . ball . . . >

[0624] Line 3 (sort key=‘car’): < . . . car . . . >

[0625] After sorting, the lines will be arranged such that the keys will be in the correct sort order.

[0626] Line 1 (sort key=‘ball’): < . . . ball . . . >

[0627] Line 2 (sort key=‘box’): < . . . box . . . >

[0628] Line 3 (sort key=‘car’): < . . . car . . . >

[0629] The WPSort function is implemented with five tags:

[0630] <!--WPSortBegin→>

[0631] <!--WPSortEnd→>

[0632] <!--WPSortFieldBegin→>

[0633] <!--WPSortFieldEnd→>

[0634] <!--WPSortLineEnd→>

[0635] <!--WPSortBegin-->and <!--WPSortEnd-->surround the entire block of text to be sorted.

[0636] <!--WPSortFieldBegin-->and <!--WPSortFieldEnd-->surround the sort key on each line.

[0637] <!--WPSortLineEnd-->delimits adjacent lines.

[0638] NOTE: The first line must start with <!--WPSortBegin-->and the last line must end with <!--WPSortEnd→>.

[0639] WPReplace tags can be included within a sort field, and modify a sort key, if desired.

[0640] In the above example, the HTML would be coded as

[0641] <!--WPSortBegin→>

[0642] <<!--WPSortFieldBegin-->box

[0643] <!--WPSortFieldEnd-->. . . ><!--WPSortLineEnd→>

[0644] < . . . <!--WPSortFieldBegin-->ball

[0645] <!--WPSortFieldEnd-->. . . ><!--WPSortLineEnd→>

[0646] < . . . <!--WPSortFieldBegin-->car

[0647] <!--WPSortFieldEnd-->. . . >

[0648] <!--WPSortEnd→>

[0649] WPSort can be used to sort a list of languages or countries in local-language order.

[0650] For example, consider this bulleted list of languages:

[0651] <!--WPSortBegin→>

[0652] <LI><A HREF=“WPCommand?language=english”>

[0653] <!--WPSortFieldBegin→>

[0654] <!--WPReplaceBegin-->English<!--WPReplaceEnd→>

[0655] <!--WPSortFieldEnd--></A>[English]

[0656] <!--WPSortLineEnd→>

[0657] <LI><A HREF=“WPCommand?language=french”>

[0658] <!--WPSortFieldBegin→>

[0659] <!--WPReplaceBegin-->French<!--WPReplaceEnd→>

[0660] <!--WPSortFieldEnd--></A>[French]

[0661] <!--WPSortLineEnd→>

[0662] <LI><A H REF=“WPCommand?language=german”>

[0663] <!--WPSortFieldBegin→>

[0664] <!--WPReplaceBegin-->German<!--WPReplaceEnd→>

[0665] <!--WPSortFieldEnd--></A>[German]

[0666] <!--WPSortEnd→>

[0667] This code will allow the list items to be sorted in the correct order, according to the current language.

[0668] In the above example, the WPReplace tags are processed first, replacing the words “English”, “French”, and “German” with their translations. The translations will then be used as the sort keys.

[0669] All WPSort tags are removed before sending the result to the browser.

[0670] 3.6.3.3 WPCommands

[0671] WPCommands are special WebPlexer commands included in a document's HTML. They allow WebPlexer to manage country and language selection within a document.

[0672] WPCommands are coded as hypertext links in a document. When a link is clicked, the WPCommand is sent to WebPlexer in the request URL.

[0673] When WebPlexer receives the WPCommand, it performs some function and returns the result to the browser.

[0674] WPCommands have the form:

[0675] WPCommand?<command>=<arg>

[0676] where <command>is one of

[0677] 1. link

[0678] 2. country

[0679] 3. language

[0680] and <arg>is a function of <command>.

[0681] Each of these commands is described in more detail in the following sections. See also Section 3.3.5, “Manual Language Selection,” for a discussion of how WPCommands are used within the HTML.

[0682] Link

[0683] WPCommand?link=<link-target>

[0684] The link command makes a hypertext link to the URL specified by <link-target>. This command has the side effect of notifying WebPlexer that a hypertext link was executed. This information is used to save the location where the browser came from, so it can return after country or language selection.

[0685] When WebPlexer receives a link notification, it saves the current value of the referer URL in the http request for this session. After a change country or change language command is executed (see next sections), the saved referer URL is restored and the browser returns to the page from where the link was issued.

[0686] The link command is used when the change country or change language function is implemented on a separate page. This allows the browser to return to the originating page once the new country or language is selected. Please see Section 3.3.6.2, “Using WPCommand?link,” for additional information.

[0687] An example of the link command is included in the sample documents included with this release. The change country and change language functions are implemented in the files ‘countsel.html’ and ‘langsel.html’, respectively. These pages are reached by issuing a WPCommand?link=<link-target>from the index and form pages.

[0688] Country

[0689] WPCommand?country=<new-country>

[0690] The country command changes the current country to <new_country>.

[0691] The current language may also change depending on the session state. (See Section 3.5.2, “Determine Language from Country”).

[0692] Language

[0693] WPCommand?language=<new-language>

[0694] The language command changes the current language to <new_language>.

[0695] 3.6.3.4 Custom Variables

[0696] WebPlexer's custom variables are special placeholders that can be included in HTML code. They are automatically replaced by current country and language information when the document is served to the browser.

[0697] Custom variables begin with ‘&’ and must always be enclosed within

[0698] <!--WPReplaceBegin-->and <!--WPReplaceEnd-->tags.

[0699] The next sections describe the custom variables supported in this release. Please see the sample files (e.g. contact.html) for examples of their use.

[0700] &country

[0701] &country is a placeholder for the translated name of the currently selected country.

[0702] The name of the country is translated into the currently selected language.

[0703] For example, if the current country were “United-States” and the current language were Spanish, this sequence

[0704] <!--WPReplaceBegin-->&country<!--WPReplaceEnd→>

[0705] would be converted to

[0706] Estados-Unidos.

[0707] &language

[0708] &language is a placeholder for the translated name of the currently selected language.

[0709] The name of the language is translated into the currently selected language.

[0710] For example, if the current language were German, this sequence

[0711] <!--WPReplaceBegin-->&language<!--WPReplaceEnd→>

[0712] would be converted to

[0713] Deutsch.

[0714] &flag

[0715] &flag is a placeholder for the name of the gif file containing the flag of the currently selected country. The name is automatically prepended with ‘flag/’. The WebPlexer release includes gif images of flags from the countries of the world.

[0716] For example, if the current country were Germany, this sequence

[0717] <img src=“../image/<!--WPReplaceBegin-->&flag

[0718] <!--WPReplaceEnd-->” height=30>

[0719] would be converted to

[0720] <img src=“../image/flags/Germany.gif” height=30>

[0721] 3.6.3.5 Server-Side Includes

[0722] An ordinary server-side include is an HTML tag which causes an external file to be included within an HTML document. The file is included when the document is served to the browser.

[0723] WebPlexer's server-side include extends this capability by making the decision of what file to include based on the current country, region, or language. This enables a document to be built dynamically out of country and language-specific components.

[0724] WebPlexer supports two kinds of server-side includes:

[0725] 1. Multi-country server-side includes (MCSSI)

[0726] 2. Multi-language server-side includes (MLSSI)

[0727] MCSSI permits locale-specific elements of a document to be included as a function of the current region or country.

[0728] MLSSI permits localized elements of a document to be included as a function of the current language.

[0729] Multi-Country Server-Side Includes (MCSSI)

[0730] The MCSSI feature provides the ability to dynamically insert country and regional-specific content into the HTML stream.

[0731] MCSSI is invoked when this construct:

[0732] <!--#WP_include_MCSSI “file_name”→>

[0733] is detected in an HTML document being served to the browser.

[0734] When WebPlexer detects this command, it will insert the file

[0735] <head_of_mcssi_directory>/<country_or_region_directory>/<file_name>

[0736] in-line into the HTML stream.

[0737] Where:

[0738] head_of_mcssi_directory is the top-level directory of the MCSSI area on the server and is defined in the WP_Config file (see Section 4.1, “WP_Config,” for more information) and country_or_region is determined dynamically according to the algorithm described in “Determining the country_or_region_directory.”

[0739] The file which is included by MCSSI can itself contain additional WebPlexer tags (such as WPReplace and Custom Variables, see Section 3.6.3.1, “WPReplace,” and Section 3.6.3.4, “Custom Variables”). It can also contain additional MCSSI and MLSSI includes (see “Multi-Language Server-Side Includes (MLSSI)”).

[0740] By supporting WebPlexer tags such as WPReplace within the included MCSSI file, regional-specific content can be further localized according to language. This feature allows considerable flexibility in serving specific content for a region or country where more than one language is spoken.

[0741] Please note that language-dependent content is normally not included in MCSSI files and that template tags (see Section 3.6.1.1, “Template Types”) such as WPReplace (see Section 3.6.3.1, “WPReplace”) or MLSSI (see “Multi-Language Server-Side Includes (MLSSI)”) should be used instead.

[0742] Language-dependent content means any static text that is present in the file. Including language-dependent content within an MCSSI file is not recommended since an MCSSI file is included based on the setting of the current country. In other words, MCSSI files should contain only country or regional-specific content and be independent of language.

[0743] For example, if the current country of the browser is Switzerland, the user might be viewing the site in either French or German. If an MCSSI file for Switzerland is included that contains static text in French, the language will be wrong if the browser is requesting German.

[0744] This problem can be easily solved by replacing the static text in the MCSSI file with template tags, either by

[0745] 1. Placing the translations of the static text in a TermDB, and using WPReplace tags in the MCSSI file, or

[0746] 2. Placing each language version of the text in a separate file and including it with a single MLSSI tag within the MCSSI file.

[0747] Determining the country_or_region_directory

[0748] The country or region directory is determined dynamically from a combination of the following parameters:

[0749] the browser's currently selected country

[0750] the user-defined region in the WP_Regions file for the selected country

[0751] the WebPlexer default region for the selected country

[0752] where the file is located (i.e. whether it is in the primary, secondary,or tertiary directory as defined below).

[0753] The specific algorithm for determining the country_or_region directory from the above information is as follows.

[0754] 1. The browser's current country is obtained. See Section 3.5, “Country and Language Management,” for information on how this is done.

[0755] 2. The WP_Regions file is consulted to determine if this country is contained within a user-defined region. See Section 4.9, “WP_Regions,” for a definition of the syntax of the WP_Regions file.

[0756] 3. If there is no user-defined region for this country, the WebPlexer pre-defined region is used instead. See Appendix D., “List of WebPlexer Default Regions,” for a list of pre-defined regions.

[0757] 4. WebPlexer will look in 3 locations for the target MCSSI file. These 3 locations are known as the primary, secondary, and tertiary MCSSI directories, respectively.

[0758] These directories are consulted in order, one at a time until the requested file is found. The directories are defined below.

[0759] a Primary:

[0760] <head_of_mcssi_directory>/<current-country>/<mcssi_file>

[0761] Example:

[0762] Assume a statement of the form:

[0763] <!--#WP_include_MCSSI “xyz.html”→>

[0764] and

[0765] current country is Japan

[0766] head_of_mcssi_directory is <docroot_directory>/mcssi.

[0767] Then a primary search would look for the file in:

[0768] /usr/local/etc/httpd/htdocs/mcssi/Japan/xyz.html

[0769] b. Secondary:

[0770] If the file is not in the primary location, the secondary path will be checked:

[0771] <head_of_mcssi_directory>/<current_region>/<mcssi_file>

[0772] where current_region is either the user-defined or WebPlexer pre-defined (default) region for this country.

[0773] Example:

[0774] Assume a statement of the form:

[0775] <!--#WP_include_MCSSI “xyz.html”→>

[0776] and

[0777] current country is Japan

[0778] current_region is Pacific-Rim

[0779] head_of_mcssi_directory is <docroot_directory>/mcssi.

[0780] Then a secondary search would look for the file in:

[0781] /usr/local/etc/httpd/htdocs/mcssi/Pacific-Rim/xyz.html

[0782] c. Tertiary:

[0783] If the file is not in the secondary location, the tertiary path will be checked:

[0784] <head_of_mcssi_directory>/<mcssi_file>

[0785] 5. If the mcssi_file is not found in either the primary, secondary, or tertiary locations, the file is considered to be “not found” and the following message will be injected into the HTML stream in place of the MCSSI directive:

[0786] [an error occurred while processing MCSSI directive]

[0787] Multi-Language Server-Side Includes (MLSSI)

[0788] MLSSI allows the Web site designer to further customize an HTML document according to language, more generically than what can be provided by the WPReplace tags (see Section 3.6.3.1, “WPReplace”).

[0789] Like MCSSI, MLSSI allows a file to be included in-line by referencing it within an HTML document. However, the path to the file is determined by the currently active language, rather than by country or region.

[0790] The referenced file is pre-translated and placed in each of the language directories referenced in the WP_Map file (see Section 4.2, “WP_Map,” for a description of the WP_Map file). When the HTML document is presented to the browser, the file included by MLSSI is retrieved from the directory corresponding to the currently active language and inserted in-line into the HTML stream. This allows a single HTML document to be easily customized by including components that are specific to the selected language of the browser.

[0791] MLSSI is invoked when this construct:

[0792] <!--#WP include MLSSI “file name”→>

[0793] is detected in an HTML document being served to the browser.

[0794] When WebPlexer detects this command, it will insert the file

[0795] <language_directory>/<file_name>

[0796] in-line into the HTML stream.

[0797] Where:

[0798] language_directory is the directory specified in the WP_Map file and language is determined according to the rules outlined in Section 3.5, “Country and Language Management.”

[0799] The file which is included by MLSSI can itself contain additional WebPlexer tags and commands (such as WPReplace, see Section 3.6.3.1, “WPReplace”). It can also contain additional MLSSI and MCSSI includes.

[0800] If the MLSSI file_name is not found on the server, the following message will be injected into the HTML stream in place of the MLSSI directive:

[0801] [an error occurred while processing MCSSI directive]

[0802] The Multilingual Navigator is a collection of WebPlexer-enhanced HTML code which can be used to easily provide language and country selection for any document on the Web site. It provides complete language and country selection without the need for CGI scripts or complex interlinked pages.

[0803] The Multilingual Navigator is easily customizable to meet the specific navigation needs of your site.

[0804] The Multilingual Navigator is included with this release in a file called ‘mlnav.html’. It is also included on the index.html and contact.html pages in the sample directory.

[0805] WebPlexer's input manager allows forms-handling CGI scripts to be intercepted and user input extracted for later translation. It can be used to support the translation of customer feedback forms in a multilingual environment.

[0806] To use the input manager, the name of the CGI script to be intercepted is specified in WebPlexer's WP_Map file along with the location of a template file and an output (.prt—“pre-translated”) file.

[0807] There is one line at the end of the WP_Map file for each CGI script to be intercepted. The format of this line is:

[0808] inbound:: <cgi-name><template-file><.prt file>

[0809] Please see the sample WP_Map file included with this release for an example.

[0810] The template file tells the input manager which fields of the form should be extracted. When the CGI script is invoked, the input data from those fields is saved and appended to the output, or .prt (pre-translated) file. This file also contains the time stamp, current country and language, and domain of the requester.

[0811] 3.8.1 Template Definition

[0812] The input manager template file has the following format:

[0813] <FIELD-LIST>

[0814] <field1>::L

[0815] <field2>::L

[0816] .

[0817] .

[0818] .

[0819] <fieldn>::L

[0820] <LIST_END>

[0821] where <FIELD_LIST>and <LIST_END>are required tokens and the remaining lines list the field names used in the HTML form. The ‘::L’ at the end of each line is required.

[0822] For example, if an input field in an HTML form were coded as:

[0823] <INPUT TYPE=“text” NAME=“phone” SIZE=40 VALUE=“”>

[0824] the matching line in the template file would be

[0825] phone::L

[0826] Please see the files ‘contact.html’ and ‘contact.tmpl’ included in the sample directory of this release for additional examples.

[0827] 3.8.2 .prt File Description

[0828] The .prt file is the output file that contains data from the extracted fields specified in the template file.

[0829] Each entry of the file is preceded by the time stamp, current country and language, and host domain.

[0830] The .prt files can be queued by the console for later translation. Please see Section 3.10.10, “Maintaining Input Manager Database Files,” for more information.

[0831] WebPlexer's document manager manages the selection of documents to serve based on the current country and language. It requires that documents be placed in parallel language directories off of the Web server's htdoc root. It also requires that the location of language directories be specified in the WP_Map file.

[0832] The following sections provide additional information.

[0833] 3.9.1 Arrangement of Documents on the Server

[0834] The document manager requires that translated documents be arranged in parallel language directories off of the Web server's htdoc root. Please see Section 3.3, “Language Selection,” for detailed information on how to arrange the document hierarchy for the document manager.

[0835] 3.9.2 SPAMM (Sparse Matrix Manager)

[0836] WebPlexer's exclusive document management feature known as “SPAMM” (Sparse Matrix Manager) provides an incremental approach to building a multilingual Web site.

[0837] SPAMM allows the site administrator to translate only a portion of documents on the site, and into only those languages that are most relevant.

[0838] SPAMM is a feature that makes it unnecessary to fully populate the site's language matrix completely with documents.

[0839] If the set of documents on a multilingual Web site is visualized as a matrix, with each row representing a language and each column representing a document, SPAMM eliminates the need to populate every cell of the matrix.

[0840] When a document is requested in a particular language, WebPlexer will go to that cell of the matrix and try to retrieve the corresponding document.

[0841] If the document is not there (i.e. it has not been translated into the requested language), WebPlexer will automatically retrieve it from the default language directory specified by the “default::” declaration in the WP_Map file for this top-level URL (see Section 4.2, “WP_Map,” for an explanation of how to set up the default language directory).

[0842] SPAMM is completely transparent in operation. It does not require any special set-up or configuration. It also works with MLSSI so that if an included file cannot be found in the current language directory, it will look in the default directory instead.

[0843] Please note that even when SPAMM fetches a document from the default directory, WebPlexer still maintains the correct settings of country and language. This means that if a browser clicks on a link from within a document that was fetched from the default language directory (through SPAMM), WebPlexer will look for the new document in the currently active language directory, not in the default language directory.

[0844] 3.9.3 Specifying Language-to-Directory Mapping

[0845] The mapping of language to directory must be specified in the WP_Map file so the document manager can locate documents in the correct language.

[0846] Please see Section 3.3.2.2, “Specifying the Language-to-Directory Map and Charset,” for detailed information on how to specify this mapping.

[0847] 3.9.4 Specifying Charsets

[0848] Please see Section 3.3.2.2, “Specifying the Language-to-Directory Map and Charset,” for detailed information on how to specify the charset used to encode a document. This information is used by the document manager.

[0849] 3.9.5 Using Multi-Country Server-Side Includes (MCSSI)

[0850] MCSSI allows an HTML document to be easily customized with country and regional-specific content using a highly modular, “building-block” approach. This reduces the number of individual files that need to be kept on the server, lowering the cost and time required to maintain the site.

[0851] One application of MCSSI is to insert a targeted, country-specific banner at the top of an HTML page. It can also be used to easily tailor a marketing message for a specific country or region. For example, a travel company could use this feature to publish different airfares to various countries or regions. Or, tour packages or travel information could be presented only to those countries or regions where it is relevant.

[0852] The following is a simple example of an HTML document that shows how MCSSI is used.

[0853] <HTML>

[0854] <BODY>

[0855] <!--WPReplaceBegin-->Welcome to the ABC Travel Site!

[0856] <!--WPReplaceEnd→>

[0857] <!--#WP_include_MCSSI “airfares.html”→>

[0858] </BODY>

[0859] </HTML>

[0860] When this document is processed, WebPlexer will first determine the current language and replace the string “Welcome to the ABC Travel Site” with its translation out of the TermDB. Please see Section 3.6.3.1, “WPReplace,” for a description of WPReplace tags and how the replacement is done. Note that this is not part of the MCSSI operation but is included to make the example a bit more realistic.

[0861] Next, WebPlexer will process the MCSSI statement as follows:

[0862] 1. The currently selected country is determined. Please see Section 3.5, “Country and Language Management,” for an explanation of how this is done.

[0863] 2. The appropriate region that this country belongs to is determined using user-defined regions from the WP_Regions file and/or WebPlexer predefined (default) regions. Please see Section 4.9, “WP_Regions,” and “Multi-Country Server-Side Includes (MCSSI),” for more information about how regions are used.

[0864] 3. Based on the analysis of step 2, the target file, “airfares.html”, is fetched from the appropriate directory on the server. Please see “Determining the country_or_region_directory,” for a description of how this directory is determined.

[0865] 4. The target file, “airfares.html” is inserted in-line into the HTML stream. Any WebPlexer tags (e.g. WPReplace) that are included within the file are subsequently processed.

[0866] Note that the included MCSSI file can contain additional WebPlexer tags, including MLSSI and WPReplace tags. This allows the MCSSI file to be further customized for the current language, by replacing strings with their translations using the TermDB.

[0867] MCSSI is very easy to use by following the steps given below.

[0868] 1. Determine which countries or regions are important for your site, based on the country-specific or regional-specific content you wish to serve to visitors to your site.

[0869] 2. Group countries together into those regions of interest and edit the WP_Regions file to identify those regions to WebPlexer. Please see Section 4.9, “WP_Regions,” for information on the syntax of the WP_Regions file.

[0870] 3. Create country and region directories on your server as shown below, and place in those directories, any country-specific or regional-specific parts of a document you wish to have managed by MCSSI.

[0871] a. Place country-specific content in these directories:

[0872] <head_of_mcssi directory>/<country>/<mcssi_file>

[0873] where:

[0874] <head_of_mcssi_directory>

[0875] is the top-level directory of the MCSSI area on the server and is defined in the WP_Config file (see Section 4.1, “WP_Config,” for more information).

[0876] <country>

[0877] is the name of the desired country.

[0878] <mcssi_file>

[0879] is the target file to be included.

[0880] b. Place regional-specific content in these directories:

[0881] <head_of_mcssi_directory>/<region>/<mcssi_file>

[0882] where:

[0883] <head_of_mcssi_directory>

[0884] is the top-level directory of the MCSSI area on the server and is defined in the WP_Config file (see Section 4.1, “WP_Config,” for more information).

[0885] <region>

[0886] is the name of the desired region. This is either a user-defined region from the WP_Regions file, or a WebPlexer pre-defined region given in Appendix D., “List of WebPlexer Default Regions.”

[0887] <mcssi_file>

[0888] is the target file to be included.

[0889] c. Place generic content (all regions and countries) in this directory:

[0890] where:

[0891] <head_of_mcssi_directory>

[0892] is the top-level directory of the MCSSI area on the server and is defined in the WP_Config file (see Section 4.1, “WP_Config”).

[0893] <mcssi-file>

[0894] is the target file to be included.

[0895] 4. Include a statement in your HTML of the form:

[0896] <!--#WP_include_MCSSI “airfares.html”→>

[0897] where “airfares.html” is the name of the target MCSSI file.

[0898] 5. When WebPlexer detects the MCSSI include when processing the HTML, it will fetch the file from the appropriate directory (3a, b, or c from above—where ever it finds it first), and insert it in-line into the HTML stream.

[0899] 6. Any WPReplace or WebPlexer tags that are found in the inserted file will be resolved after the file is inserted.

[0900] Please note that language-dependent content is normally not included in MCSSI files and that template tags (see Section 3.6.1.1, “Template Types”) such as WPReplace (see Section 3.6.3.1, “WPReplace”) or MLSSI (see “Multi-Language Server-Side Includes (MLSSI)”) should be used instead.

[0901] Language-dependent content means any static text that is present in the file. Including language-dependent content within an MCSSI file is not recommended since an MCSSI file is included based on the setting of the current country. In other words, MCSSI files should contain only country or regional-specific content and be independent of language.

[0902] For example, if the current country of the browser is Switzerland, the user might be viewing the site in either French or German. If an MCSSI file for Switzerland is included that contains static text in French, the language will be wrong if the browser is requesting German.

[0903] This problem can be easily solved by replacing the static text in the MCSSI file with template tags, either by

[0904] 1. Placing the translations of the static text in a TermDB, and using WPReplace tags in the MCSSI file, or

[0905] 2. Placing each language version of the text in a separate file and including it with a single MLSSI tag within the MCSSI file.

[0906] 3.9.6 Using Multi-Language Server-Side Includes (MLSSI)

[0907] MLSSI is typically used to customize an HTML document by including HTML that might be too complex to be managed by WPReplace tags. For example, MLSSI can be used to include a language-specific heading at the top of a form constructed out of WPReplace tags.

[0908] Please see Section 3.6.3.1, “WPReplace,” for further information about WPReplace tags.

[0909] MLSSI also helps to separate the form of a Web site (the structure and layout of the site) from its content (the actual message to be displayed on the browser).

[0910] The design and layout of a multi-language site is usually very similar, and often invariant, across languages. MLSSI allows the language-dependent parts of the content to be isolated from the structural design making it easy to update the design of the site and have changes reflected automatically across all of the languages. Likewise, MLSSI allows language content to be easily updated and maintained without affecting the overall site design.

[0911] Another application of MLSSI is to include language-specific graphics or image maps within a template file.

[0912] For example, a form-based HTML document that is a pure template can be customized with a language-specific banner and graphic at the top of the file as follows:

[0913] <HTML>

[0914] <BODY>

[0915] <!--#WP_include_MLSSI “banner.html”→>

[0916] <INPUT NAME =“name” ><!--WPReplaceBegin-->name<!--WPReplaceEnd→>

[0917] <INPUT NAME =“address” ><!--WPReplaceBegin-->address<!--WPReplaceEnd→>

[0918] </BODY>

[0919] </HTML>

[0920] When this file is processed, the file “banner.html” will be fetched from the language directory specified in the WP_Map file (i.e. french/banner.html, german/banner.html, etc.)

[0921] The contents of the banner.html files might look something like:

[0922] french/banner.html:

[0923] <IMG SRC=“images/french/banner.gif” WIDTH=80HEIGHT=20>Bienvenue<br>

[0924] german/banner.html:

[0925] <IMG SRC=“images/german/banner.gif” WIDTH=10OHEIGHT=20>Wilikommen<br>

[0926] and so on.

[0927] Of course, the WPReplace strings (“name”, “address”) will also be replaced by their translations from the TermDB during processing of the file. See Section 3.6.3.1, “WPReplace,” for a description of how this is done.

[0928] As can be seen, MLSSI can be used to separate out the language-specific components of an HTML document making it easier to maintain the collection. In the previous example, if the structure of the HTML form needed to change, it would only require changing the template file, and not any of the language-specific files on the site.

[0929] MLSSI is very easy to use by following the steps given below.

[0930] 1. Place any language-specific parts of a document you wish to have managed by MLSSI in the respective language directories specified in the WP_Map file.

[0931] 2. Include a statement in your HTML of the form:

[0932] <!--#WP_include_MLSSI “component.html”→>where “component.html” is the name of the file created in step 1.

[0933] 3. When WebPlexer detects the MLSSI include when processing the HTML, it will fetch the file from the language directory, as defined in the WP_Map file, corresponding to the current language selection, and insert it in-line into the HTML stream.

[0934] 4. Any WPReplace or WebPlexer tags that are found in the inserted file will be resolved after the file is inserted.

[0935] 3.9.7 Using CGI Scripts

[0936] CGI scripts producing HTML output should be written to be as independent of language and country as possible. This means these scripts should use the HTML extensions provided by the Multilingual ToolKit. They should write out templates that do not contain language-specific or country-specific information. Having only one script to maintain means that changes can be made easily and efficiently.

[0937] When the HTML output of such a CGI script is presented to the browser, WebPlexer will fill in the template dynamically with current language and country information. Text that is marked-up with WPReplace tags will also be translated.

[0938] In order for WebPlexer to correctly process a CGI script, the filename of the script must be changed when it is referenced in an HTML document. The filename must not include the path to the cgi root, even though the script is actually located in that directory.

[0939] In addition, the cgi root must be specified in the WP_Config file (see Section 4.1.18, “cgiRoot”).

[0940] For example, suppose that the cgi root is “/cgi-bin”.

[0941] Normally, a reference to a CGI script called “script.cgi” would look like this:

[0942] <A HREF=“/cgi-bin/script.cgi”>CGI</A>

[0943] But with WebPlexer, the cgi root should be omitted:

[0944] <A HREF=“script.cgi”>CGI</A>

[0945] This will ensure correct country and language processing of the HTML.

[0946] 3.10 WebPlexer Console

[0947] 3.10.1 Setting the Console Port

[0948] 3.10.2 Setting the Console Password

[0949] 3.10.3 Invoking the Console

[0950] 3.10.4 Closing the Console

[0951] 3.10.5 Writing Console Templates

[0952] 3.10.6 Console Commands

[0953] 3.10.6.1 System Configuration

[0954] 3.10.6.2 URL Selection

[0955] 3.10.6.3 System Status

[0956] Status Annunciators

[0957] 3.10.6.4 Language Status

[0958] Status Annunciators

[0959] 3.10.6.5 Queue Status

[0960] 3.10.7 Translation Queues

[0961] 3.10.7.1 Enqueue

[0962] 3.10.7.2 Dequeue

[0963] 3.10.7.3 Maintaining the Translation Queues

[0964] 3.10.7.4 Initiating Off-line Translation Using qtrans

[0965] 3.10.8 Document Maintenance

[0966] 3.10.8.1 Including Document Types

[0967] 3.10.8.2 Including Specific Documents

[0968] 3.10.8.3 Excluding Documents

[0969] 3.10.9 Maintaining TermDBs

[0970] 3.10.10 Maintaining Input Manager Database Files

[0971] 3.10.1 1 Log Analysis

[0972] 3.10.1 1.1 Usage utility

[0973] 3.10.11.2 Visits utility

[0974] 3.11 Runtime Options

[0975] WebPlexer accepts a number of command line flags used for configuration and debugging. These are described below.

[0976] 3.11.1 Configuration Flags

[0977] -internetport port

[0978] -serverport port

[0979] -consoleport port

[0980] These flags override the InternetPort, ServerPort, and ConsolePort specified in the WP_Config file.

[0981] -alog file

[0982] -elog file

[0983] -ilog file

[0984] These flags override the AccessLog, ErrorLog, and InformationLog specified in the WP_Config file.

[0985] -c file

[0986] This flag specifies a configuration file to use other than the default WP_Config.

[0987] -version

[0988] This flag displays the current version of WebPlexer and libraries.

[0989] -restart

[0990] This flag forces WebPlexer to go through a restart sequence.

[0991] 3.11.2 Debugging Flags

[0992] -v

[0993] -vv

[0994] -debug

[0995] These flags cause WebPlexer to send debugging and informational messages to stderr. They should not be used in normal operation as they will cause some amount of performance degradation. Type ./webplexer -h for more information.

[0996] -t file

[0997] This flag uses the specified file to activate WebPlexer's test mode. The test file contains a list of up to 64 domain suffixes that will be consecutively “spoofed” for each access.

[0998] For example, if the file contained:

[0999] fr de it

[1000] the first access would appear to come from France, the next from Germany, and the last from Italy, then the cycle would repeat. Depending on the access pattern, and the contents of the document being retrieved (e.g. if there are embedded graphics), it may be necessary to repeat the same suffix more than once in the file to achieve the desired result.

[1001] -nobg

[1002] This flag tells WebPlexer to run in the foreground and not to fork a child process on startup.

[1003] -noredirection

[1004] This is a debugging flag and should not be used in normal operation.

[1005] -nosigs

[1006] This disables WebPlexer's exception handling. Normally, WebPlexer notifies the client when an exception occurs (if possible).

[1007] -noforking

[1008] This is another debugging flag that forces requests to be handled by the parent process.

[1009] -h, -help

[1010] These flags list and describe the help options.

[1011] The following sections give detailed specifications of all WebPlexer external files.

[1012] The WP_Config file contains site-specific information used to set up and configure WebPlexer.

[1013] The WP_Config file is similar in format to a “httpd.conf” file and consists of a series of key/value pairs, one per line.

[1014] A key is a non-blank string ending with one or more white spaces. Examples of keys are InternetPort and ServerPort.

[1015] A value is a parameter string and varies depending upon the key. Examples of values are the strings “80” and “logs/WPErrorLog”.

[1016] Keys are separated from values by one or more column delimiters (spaces or tabs). Any text on a line after a comment symbol (#) is ignored.

[1017] The key/values currently defined for WP_Config are listed in the sections below.

[1018] 4.1.1 InternetPort

[1019] Syntax:

[1020] InternetPort <port>

[1021] where <port>is the port on the Internet side that WebPlexer monitors for an http request. When WebPlexer is used in “production mode” (see Section 3.1.1.2, “Production Mode”), this value will be 80, the standard http port.

[1022] Example:

[1023] InternetPort 80

[1024] 4.1.2 ServerPort

[1025] Syntax:

[1026] ServerPort <port>

[1027] where <port>is the port used to communicate with the httpd server. It should be set to an unused port (e.g. 180) when WebPlexer is in “production mode” (see Section 3.1.1.2, “Production Mode”). It must match the port specified in the Web server's configuration file.

[1028] Example:

[1029] ServerPort 180

[1030] 4.1.3 ConsolePort

[1031] Syntax:

[1032] ConsolePort <port>

[1033] where <port>is the port used to communicate with the WebPlexer console.

[1034] Example:

[1035] ConsolePort 2800

[1036] 4.1.4 AccessLog

[1037] Syntax:

[1038] AccessLog <filename>

[1039] where <filename>is the name of WebPlexer's access log file. It can be relative to the WebPlexer directory or it can be a fully specified file name.

[1040] Example:

[1041] AccessLog logs/WPAccessLog

[1042] 4.1.5 ErrorLog

[1043] Syntax:

[1044] ErrorLog <filename>

[1045] where <filename>is the name of WebPlexer's error log file. It can be relative to the WebPlexer directory or it can be a fully specified file name.

[1046] Example:

[1047] ErrorLog logs/WPErrorLog

[1048] 4.1.6 InformationLog

[1049] Syntax:

[1050] InformationLog <filename>

[1051] where <filename>is the name of WebPlexer's information log file. It can be relative to the WebPlexer directory or it can be a fully specified file name.

[1052] Example:

[1053] InformationLog logs/WPInfoLog

[1054] 4.1.7 ServerName

[1055] Syntax:

[1056] ServerName <name>

[1057] where <name>is the DNS entry name of this server.

[1058] Example:

[1059] ServerName www.lai.com

[1060] 4.1.8 RemoteMaintenanceKey

[1061] Syntax:

[1062] RemoteMaintenanceKey <off I on>

[1063] This should be set to “off” in the current release.

[1064] 4.1.9 Console

[1065] Syntax:

[1066] Console <off I on>

[1067] Disables/enables the WebPlexer console.

[1068] 4.1.10 MasterLanguage

[1069] Syntax:

[1070] MasterLanguage <language>

[1071] Specifies the language to be used as the master or reference language for the console (i.e. documents in that language will be used as a reference when determining which languages are out of date).

[1072] Example:

[1073] MasterLanguage english

[1074] NOTE: The language must be one of those supported by WebPlexer (see Appendix B., “List of Available Languages”).

[1075] 4.1.11 ConsoleTextExtensions

[1076] Syntax:

[1077] ConsoleTextExtensions <ext1 ext2 . . . extN>

[1078] The extensions specified as extN indicate what filetypes will be under console maintenance functions.

[1079] Example:

[1080] ConsoleTextExtensions .html .htm

[1081] 4.1.12 ConsoleFileInclude

[1082] Syntax:

[1083] ConsoleFileInclude <file1 file2 . . . fileN>

[1084] Specifies individual files to be included under console maintenance functions beyond those included using the ConsoleTextExtensions directive.

[1085] Example:

[1086] ConsoleFileInclude faq.txt info.txt

[1087] 4.1.13 ConsoleFileExclude

[1088] Syntax:

[1089] ConsoleFileExclude <file1 file2 . . . fileN>

[1090] Specifies individual files to be excluded from console maintenance among the set that was included using the ConsoleTextExtensions directive.

[1091] Example:

[1092] ConsoleFileExclude dummy.html empty.html

[1093] 4.1.14 Enqueue

[1094] Syntax:

[1095] Enqueue <directory>

[1096] Specifies the location of the console's enqueue directory.

[1097] NOTE: This directory is relative to the DocumentRoot (see below)

[1098] Example:

[1099] Enqueue enq

[1100] 4.1.15 Dequeue

[1101] Syntax:

[1102] Dequeue <directory>

[1103] Specifies the location of the console's dequeue directory.

[1104] NOTE: This directory is relative to the DocumentRoot (see below)

[1105] Example:

[1106] Dequeue deq

[1107] 4.1.16 Plugins

[1108] PlugIns use the WebPlexer API. Information on PlugIns is not available in this release. However, the WP_Config file should contain the following two lines:

[1109] PlugIns MLFWI Early 5

[1110] Plugins MLFWO Late 5

[1111] 4.1.17 DocumentRoot

[1112] Syntax:

[1113] DocumentRoot <path>

[1114] where <path>is the path to the Web server's document root.

[1115] Example:

[1116] DocumentRoot /usr/local/etc/httpd/htdocs/

[1117] 4.1.18 cgiRoot

[1118] Syntax:

[1119] cgiRoot <path>

[1120] where <path>is the path to the Web server's cgi root, relative to the document root.

[1121] NOTE: <path>should not contain a trailing slash.

[1122] Example:

[1123] cgiRoot /cgi-bin

[1124] 4.1.19 MCSSIRoot

[1125] Syntax:

[1126] MCSSIRoot <path>

[1127] where <path>is the path to the head of MCSSI directory, relative to the document root.

[1128] Example:

[1129] MCSSIRoot/webplexer/MCSSI

[1130] The WP_Map file contains a list of top-level URLs that are subject to language selection. URLs are organized in URL groups, and as many groups as necessary can be specified. Each group contains the following information:

[1131] 1. The top-level URL of the group.

[1132] 2. The list of languages in the group and the names of the directories where the documents are located.

[1133] 3. The encodings in which the documents are represented (e.g. ISO-8859-1)

[1134] 4. Default directories in case the requested language is not found (usually English).

[1135] 5. TermDB(s) used when processing files for this group.

[1136] 6. Input Manager files (CGI script, templates, output files) for any forms contained within this group.

[1137] The format of the file is as shown below. Any text after a comment symbol (#) is ignored.

[1138] # URL Group 1 definition

[1139] URL::<path1><path_type>

[1140] URL::<path2><path_type>

[1141] .

[1142] .

[1143] .

[1144] URL::<pathN><path_type>

[1145] <language>:<language_directory><charset>

[1146] <language>:<language_directory><charset>

[1147] .

[1148] .

[1149] .

[1150] default:<default_language><charset># REQUIRED

[1151] others:<others_language><charset># REQUIRED

[1152] .

[1153] .

[1154] outbound:: ‘*’ <TermDB1.1>

[1155] outbound:: ‘*’ <TermDB1.2>

[1156] .

[1157] outbound:: ‘*’ <TermDB1.N>

[1158] .

[1159] .

[1160] inbound:: <cgi-script1.1><template-file1.1><.prt file1.1>

[1161] inbound:: <cgi-script1.2><template-file1.2><.prt file1.2>

[1162] .

[1163] inbound:: <cgi-script1.N><template-file1.N><.prt file1 .N>

[1164] # URL Group 2 definition

[1165] URL::<path1><path_type>

[1166] URL::<path2><path_type>

[1167] .

[1168] .

[1169] .

[1170] URL::<pathN><path_type>

[1171] <language>:<language_directory><charset>

[1172] <language>:<language_directory><charset>

[1173] .

[1174] .

[1175] .

[1176] default:<default_language><charset> #REQUIRED

[1177] others:<others_language><charset> #REQUIRED

[1178] outbound:: ‘*’ <TermDB2.1>

[1179] outbound:: ‘*’ <TermDB2.2>

[1180] .

[1181] outbound:: ‘*’ <TermDB2.N>

[1182] .

[1183] inbound:: <cgi-script2.1><template-file2.1><.prt file2.1>

[1184] inbound:: <cgi-script2.2><template-file2.2><.prt file2.2>

[1185] .

[1186] inbound:: <cgi-script2.N><template-file2.N><.prt file2.N>

[1187] # URL Group N definition

[1188] .

[1189] .

[1190] .

[1191] defaultCountry <country>

[1192] defaultLanguage <language>

[1193] 4.2.1 Specifying the Top-Level URL

[1194] Syntax:

[1195] URL::<path><path_type>

[1196] where

[1197] <path>

[1198] is the directory path of the top-level URL below the htdoc root.

[1199] <path_type>

[1200] is either ‘d’ or ‘f’ depending on whether the path is a directory or a file.

[1201] Examples of valid paths and path_types are:

[1202] /d

[1203] /user1 d

[1204] /user1/docs d

[1205] /user1/docs/info.html f

[1206] You can specify as many paths as needed within a URL group.

[1207] A directory specification only matches a request for that directory. It will not match any requests for files in that directory.

[1208] For example,

[1209] URL::/user1 d

[1210] would match

[1211] GET http://www.lai.com/user1

[1212] GET http://www.lai.com/user1/

[1213] but would NOT match

[1214] GET http://www.lai.com/user1/info.html

[1215] GET http://www.lai.com/user1/index.html

[1216] These would require additional path specifications

[1217] URL::/user1/info.html f

[1218] URL::/user1/index.html f

[1219] 4.2.2 Specifying Language/Directory/Charset

[1220] Syntax:

[1221] <language>: <language_directory><charset>

[1222] where

[1223] <language>

[1224] is the name of the language.

[1225] <language_directory>

[1226] is the name of the subdirectory containing documents of that language.

[1227] <charset>

[1228] is the encoding used for documents in that language (e.g. ISO-8859-1)

[1229] All the URLs in a group share a common language-to-directory map.

[1230] NOTE: <language>must be one of the available languages listed in Appendix B., “List of Available Languages.”

[1231] Examples of valid language to directory entries are:

[1232] french: french ISO-8859-1

[1233] english: eng_docs ISO-8859-1

[1234] catalan: spanish_dir ISO-8859-1

[1235] japanese: japanese ISO-2022-JP

[1236] There are two reserved <language>names that have special meaning. One is “default” and the other is “others”. Both of these must be specified in the directory map of each URL group definition. They are described in more detail below.

[1237] 4.2.2.1 others:

[1238] Syntax:

[1239] others: <others_language><charset>

[1240] The “others” language is used when the requested language is valid but does not exist on the server.

[1241] For example, if English, French, and Spanish documents exist on the server, but the requested language is Russian, WebPlexer will use the language defined as “others”.

[1242] It is usually the case that “others” is set to English.

[1243] Example:

[1244] others: english ISO-8859-1

[1245] 4.2.2.2 default:

[1246] Syntax:

[1247] default: <default_language><charset>

[1248] The “default” language is used when WebPlexer cannot determine the language of a request and none has been specified by the browser. This can happen when the country of origin cannot be determined because the hostname is unavailable or when the requester is behind a firewall.

[1249] It is usually the case that “default” is set to English.

[1250] Example:

[1251] default: english ISO-8859-1

[1252] 4.2.3 outbound:: Enabling the Output Manager

[1253] The output manager (i.e. Multilingual ToolKit) is enabled by specifying TermDB(s) in the WP_Map file for a given top-level URL.

[1254] All documents containing templates, tags, or other resources of the Multilingual ToolKit which require dynamic processing will be processed through the listed TermDBs for this URL group.

[1255] Syntax:

[1256] outbound:: ‘*’ <TermDBN>

[1257] where

[1258] <TermDBN>

[1259] is the name of a TermDB to be applied for this top-level URL.

[1260] NOTE: The TermDB should be in the main WebPlexer directory.

[1261] Examples:

[1262] outbound:: *master_termdb.html

[1263] outbound:: *user_termdb.html

[1264] 4.2.4 inbound:: Enabling the Input Manager

[1265] To use the input manager, the name of the CGI script to be intercepted is specified in the WP_Map file along with the location of a template file and an output (.prt) file for this URL group.

[1266] A separate inbound statement is required for each CGI script.

[1267] NOTE: Each inbound statement must specify a different .prt file.

[1268] The template file tells the input manager which fields of the form should be extracted. When the CGI script is invoked, the input data from those fields is saved and appended to the output, or .prt (pre-translated) file. This file also contains the time stamp, current country and language, and domain of the requester.

[1269] Syntax:

[1270] inbound:: <cgi-scriptN><template-fileN><.prt fileN>

[1271] where

[1272] <cgi-scriptN>

[1273] is the name of the CGI script to be intercepted.

[1274] <template-fileN>

[1275] is the name of the file defining the fields to be captured. Please see Section 3.8.1, “Template Definition,” for a definition of this file.

[1276] <.prt fileN>

[1277] is the name of the output (pre-translated) file. Please see Section 3.8.2, “.prt File Description,” for a definition of this file.

[1278] Examples:

[1279] inbound:: form.cgi form.tmpl form.prt

[1280] inbound:: contact.cgi contact.tmpl contact.prt

[1281] Also see the sample WP_Map file included with this release.

[1282] 4.2.5 Setting Global Defaults

[1283] Global defaults are declared at the end of the WP_Map file and apply to all URL groups.

[1284] 4.2.5.1 defaultCountry

[1285] Syntax:

[1286] defaultCountry <country>

[1287] where

[1288] <country>

[1289] is the country to be used as WebPlexer's global default country.

[1290] NOTE: The country must be a valid country from the country.list file.

[1291] Example:

[1292] defaultCountry United-States

[1293] 4.2.5.2 defaultLanguage

[1294] Syntax:

[1295] defaultLanguage <language>

[1296] where

[1297] <language>

[1298] is the language to be used as WebPlexer's global default language.

[1299] NOTE: The language must be a valid language from Appendix B., “List of Available Languages.”

[1300] Example:

[1301] defaultLanguage english

[1302] WebPlexer provides two files that can optionally force the selection of specific language(s) for certain hostnames or IP addresses when the top-level URL is accessed.

[1303] These files are HostNameMap and IPAddrMap, and are described below.

[1304] Both files are in WebPlexer's language subdirectory.

[1305] 4.3.1 HostName Map

[1306] Syntax:

[1307] <hostID1><language_set1>

[1308] <hostID2><language_set2>

[1309] .

[1310] .

[1311] .

[1312] <hostIDN><language_setN>

[1313] where

[1314] <hostID>

[1315] is a hostname

[1316] <language_set>

[1317] is an ordered list of one or more languages of the form

[1318] {<language1>, <language2>, . . . , <languageN>}

[1319] and languageN is preferred over languageN+1 for this hostID.

[1320] NOTE: The HostName Map file must be in WebPlexer's ‘language’ subdirectory.

[1321] Examples:

[1322] www.lai.com {english}

[1323] www.generichost.com {default}

[1324] 4.3.2 IPAddress Map

[1325] Syntax:

[1326] <ipID1><language_set1>

[1327] <ipID2><language_set2>

[1328] .

[1329] .

[1330] .

[1331] <ipIDN><language_setN>

[1332] where

[1333] <ipID>

[1334] is an IP address

[1335] <language_set>

[1336] is an ordered list of one or more languages of the form

[1337] {<language1>, <language2>, . . . , <languageN>}

[1338] and languageN is preferred over languageN+1 for this ipID.

[1339] NOTE: The IPAddress Map file must be in WebPlexer's ‘language’ subdirectory.

[1340] Example:

[1341] 192.42.172.9 {farsi, russian, english}

[1342] WebPlexer supports four log file types:

[1343] AccessLog

[1344] ErrorLog

[1345] InformationLog

[1346] UsageLog

[1347] The filenames corresponding to these log file types, except for the UsageLog, are defined in the WP_Config file (See Section 4.1.4, “AccessLog,” Section 4.1.6, “InformationLog,” and Section 4.1.5, “ErrorLog,” for additional information). The UsageLog is created and maintained by the usage script, described in Section 3.10.1 1. 1, “Usage utility.”

[1348] 4.4.1 AccessLog

[1349] The AccessLog records information about each request received by WebPlexer, such as date, time, and request type.

[1350] Each AccessLog entry conforms to the Common Logfile Format:

[1351] <remotehost><rfc931><authuser><[date]><“request”><status><bytes>

[1352] where the fields are defined as follows

[1353] <remotehost>

[1354] Remote hostname (or IP address if DNS hostname is not available).

[1355] <rfc931>

[1356] The remote logname of the user.

[1357] <authuser>

[1358] The username by which the user has authenticated himself.

[1359] <[date]>

[1360] Date and time of the request.

[1361] <“request”>

[1362] The request line exactly as it came from the client.

[1363] <status>

[1364] The HTTP status code returned to the client.

[1365] <bytes>

[1366] The content-length of the document transferred.

[1367] Examples:

[1368] pp93.satcom.net—[28/Jul/1996:23:37:23 -0700] “GET/cnv.post/info.html HTTP/1.0” 302-pp93.satcom.net—[28/Jul/1996:23:37:24 -0700] “GET/cnv.post/english-american_United-States/info.html HTTP/1.0” 200 3372

[1369] pp93.satcom. net—[28/Jul/1996:23:37:29 -0700] “GET/cnv.post/coming/flags/United-States.gif HTTP/1.0” 200 3594

[1370] NOTE: When WebPlexer is configured for “production mode” (see Section 3.1.1.2, “Production Mode”), WebPlexer's AccessLog should be used in place of the Web server's access log for auditing purposes. The Web server's access log will show all requests originating from WebPlexer.

[1371] 4.4.2 ErrorLog

[1372] The ErrorLog logs any errors that occurred during WebPlexer operation, or during the processing of a request.

[1373] The format of an ErrorLog entry is:

[1374] [date] <Error-Msg>

[1375] Examples:

[1376] [01/Aug/1996:21:02:23 -0700] Couldn't bind to socket for internet port 3080

[1377] 4.4.3 Information Log

[1378] The InformationLog records information about WebPlexer language and country selection. An entry is written whenever language or country changes, whether automatically or by manual selection.

[1379] The format of an InformationLog entry is:

[1380] <remotehost><rfc931><authuser><[date]><status-message>

[1381] where the fields are defined as follows

[1382] <remotehost>

[1383] Remote hostname (or IP address if DNS hostname is not available).

[1384] <rfc931>

[1385] The remote logname of the user.

[1386] <authuser>

[1387] The username by which the user has authenticated himself.

[1388] <[date]>

[1389] Date and time of the request.

[1390] <status-message>

[1391] Status information pertaining to country or language processing.

[1392] Examples of InformationLog entries are:

[1393] pc113.fiz-karlsruhe.de—[01/Aug/1996:01:55:13 -0700] Country Selection:

[1394] pc113.fiz-karlsruhe.de-->Germany

[1395] pc113.fiz-karlsruhe.de—[01/Aug/1996:01:55:13 -0700] Language Set:

[1396] pc113.fiz-karlsruhe.de-->{german}

[1397] pc113.fiz-karlsruhe.de—[01/Aug/1996:01:55:13 -0700] Language Selection:

[1398] pc113.fiz-karlsruhe.de-->German

[1399] 4.4.4 Usage Log

[1400] The UsageLog contains a summary of country and language statistics. It is generated from the InformationLog using the usage log analysis tool. The UsageLog can be further processed using the visits log analysis tool. Please see Section 3.10.11, “Log Analysis,” for a description of WebPlexer's log analysis tools.

[1401] The format of a UsageLog entry is:

[1402] <[date]><™Language:𝑓|™Country:𝑓><language|country><remotehost>

[1403] where the fields are defined as follows

[1404] <[date]>

[1405] Date and time of the request.

[1406] <language>

[1407] New language of this request (i.e. an entry is created when a visitor enters the site or changes language once on the site).

[1408] <country>

[1409] New country of this request (i.e. an entry is created when a visitor enters the site or changes country once on the site).

[1410] Examples of UsageLog entries are:

[1411] [17/Oct/1996:13:40:11 -0700] Language: German www07.btx.dtag.de

[1412] [17/Oct/1996:14:09:57 -0700] Country: Argentina server1.sminter.com.ar

[1413] The CountryMap contains a database of the world's languages and dialects. It is not human-readable.

[1414] 4.5.1 Creating a User-defined CountryMap

[1415] The country.list file contains a list of available countries. It is identical to the list included in Appendix C., “List of Available Countries.”

[1416] The WP_LangCode file contains the iso639 language map.

[1417] The WP_Languages file contains a list of available languages and dialects. It is identical to the list included in Appendix B., “List of Available Languages.”

[1418] The WP_Regions file consists of a list of names of user-defined regions, each one followed by a list of countries contained within that region.

[1419] Syntax:

[1420] # this is user-defined region1

[1421] <user-defined_region1>::

[1422] <country1>

[1423] <country2>

[1424] .

[1425] .

[1426] .

[1427] <countryN>

[1428] # this is user-defined region2

[1429] <user-defined_region2>::

[1430] <country1>

[1431] <country2>

[1432] .

[1433] .

[1434] .

[1435] <countryM>

[1436] Blank lines as well as lines beginning with ‘#’ are ignored. Tabs at the beginning of a line are also ignored.

[1437] Example:

[1438] sales_region 1::

[1439] Japan

[1440] China

[1441] Hong-Kong

[1442] sales_region2::

[1443] France

[1444] Great-Britain

[1445] Germany

[1446] Countries must be chosen from the list of valid countries for WebPlexer. See Appendix C., “List of Available Countries.”

[1447] The WP_Regions file contains only user-defined regions. WebPlexer also maintains a set of pre-defined (default) regions. These are listed in Appendix D., “List of WebPlexer Default Regions.”

[1448] “Determining the country_or_region_directory,” further describes the interaction and relationship between pre-defined (default) and user-defined regions.

[1449] The Master_termdb is included with the product and contains translations of the names of the world's countries and languages in the following languages and encodings:

[1450] Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the claims included below.