US 7,010,746 B2Grant
System and method for constraint-based document generation
Issue Date:2006-03-07
•1 Claims
•2 Drawing Sheets
Abstract
What is disclosed is a system and method for specifying a custom document as a constraint satisfaction problem (CSP) to create the specified document using existing constraint solving algorithms wherein the document, its content components, and its layout requirements as elements of a constraint satisfaction problem which when solved, results in an automated document layout for the set of content components. This enables an easier (i.e., automated) custom document creation process while providing a wider array of output documents.
Metadata
Assignee
- Xerox Corporation
Inventor
- Lisa S Purvis
Application Information
Application Number:US 10/202,275
Filing Date:2002-07-23
Priority Date:2002-07-23
Art Unit:2176
Classifications
IPC:
G06F3/00
Patent Drawings (2 sheets)
Description
Field of the Invention
[0001] The present invention is directed to systems and methods to find document components and assemble them into a custom document such as a variable data document and, in particular, those systems and methods which use constraint-based approaches wherein the document, its content, components, and its requirements are expressed as elements of a constraint satisfaction problem.
Background of the Invention
[0002] Custom documents are documents that are personalized or tailored in some way to the particular user of the document. Two growing applications of custom documents are in the domain of variable data printing, as well as in web personalization.
[0003] In traditional variable data applications the creation of a custom document is accomplished by an expert in graphic arts, databases, layout, document design, etc. This expert document creator develops an overall layout for the document that includes slots for the variable data. The creator also finds or creates appropriate content pieces, and specifies rules for how to fill in the variable slots with this content, or places the content in a database and then links the slots to particular fields in the database. The variable data application then creates a document for each customer by inserting the data for the customer into its linked slot. These types of templates are typically called “lick and stick”, because the template has “art holes” which are defined by the document creator, and then the variable data is placed into those art holes to form different instances of the document. The resulting set of documents is typically quite similar: each variable slot has one piece of content of about the same size and the general layout is the same for all instances, regardless of the available content pieces. Thus, the traditional ‘variable data template’ not only requires extensive time and expertise from the document creator, but it also does not respond dynamically to varying amounts or types of content pieces, or to restrictions imposed by the variable content.
[0004] In the domain of web documents and web personalization, the focus is often not on a dynamic document layout per se but rather on dynamic data. The assumption is that the document template is predetermined and the challenge is to find the right data at the right time. For instance, dynamic web documents are typically formed by embedding “scripts” (i.e., programs) into the HTML page. These scripts are then interpreted by the web server. This enables certain elements of the document (e.g., a weather reading) to be inserted or created at the time of document display. This approach can only produce documents according to the underlying program. In order to achieve different results, the program must be changed and thus this approach is not truly dynamic. In web personalization applications, much of the focus is on user profiling and understanding what e-commerce customers are doing online so that you can provide the appropriate information. Again, the focus is on finding the appropriate information, not on laying it out differently based on what is found. The techniques used to create dynamic web pages often involve writing specific scripts or programs that can be embedded in web pages and interpreted by the web server. This approach lacks in generality, introduces restrictive platform dependencies, and can be extremely difficult and expensive to maintain.
[0005] Traditional creation of custom documents such as variable data documents requires expertise in many areas such as graphic arts and databases and is a time consuming process. With the ever-increasing amount of information in the digital world and the amount of un-trained users producing documents, old publishing tools often prove cumbersome and demanding whereas present dynamic digital environments demand tools that can reproduce both the contents and the layout automatically tailored to personal needs and which can enable novices to easily create such documents.
[0006] In the realm of custom document creation, what is needed in the arts is a way to automatically select document content and lay it out into a final document, without relying on expert graphic artists, or using specialized scripts that are embedded in an HTML file.
[0007] Known methods for automated creation of documents have focused more on particular types of documents, and not on modeling the problem in a general way in order to address all types of documents. Existing work provides methods for creating diagrams (see Dengler, E. Friedell, M., Marks, J., Constraint-Driven Diagram Layout, Proceedings of the 1993 IEEE Symposium on Visual Languages, pages 330-335, Bergen, Norway, 1993), or multimedia presentations (see Rousseau, F., Garcia-Macias, A., Valdeni de Lima, J., and Duda, A., User Adaptable Multimedia Presentations for the WWW, Electronic Proceedings from the 8th International World Wide Web Conference, 1999), or flowcharts and yellow pages (see Graf, W. H., The Constraint-Based Layout Framework LayLab and Applications, Electronic Proceedings of the ACM Workshop on Effective Abstractions in Multimedia, 1995). Others have explored automating the process of web document layout (see Kroener, A., The Design Composer: Context-Based Automated Layout for the Internet, Proceedings of the AAAI Fall Symposium Series: Using Layout for the Generation, Understanding, or Retrieval of Documents, 1999). None of the existing automated approaches provide a formalized, constraint-satisfaction formulation of the document creation process, nor do they include the selection of content in their automated process. Furthermore, existing automated approaches concentrate solely on a single type of document.
[0008] What is needed in the arts is a way to eliminate some of the traditional limitations of custom document creation by providing a methodology that supports a flexible and efficient assembly of document components resulting in truly dynamic and personalized documents.
Summary of the Invention
[0009] What is disclosed is a system and method for specifying a custom document as a constraint satisfaction problem (CSP) to create the specified document using existing constraint solving algorithms wherein the document, its content components, and its layout requirements as elements of a constraint satisfaction problem which when solved, results in an automated document layout for the set of content components. This process enables an easier (i.e., automated) custom document creation process, while also providing a wider array of output documents than is possible today.
[0010] The document is expressed as a template made up of variables, values, and constraints. An instance of that template is then created, resulting in a set of content and a layout of that content, by using one of a set of well-known constraint satisfaction algorithms. The present invention described herein is the formulation of document creation as a constraint satisfaction problem and the application of existing constraint satisfaction algorithms to automatically create the desired output document, the combination of which is heretofore unknown in the arts. The present method of document composition comprises the steps of; specifying problem variables by identifying zero to n document areas which have associated constraints and at least 1 document parameter that can be changed; specifying value domains for each variable, by identifying either content pieces appropriate for the associated document area, or by identifying valid ranges for the associated document parameter; specifying layout and/or content constraints which further constrain the valid values of the document parameter and/or document area variables; optionally specifying dynamic constraints that are added/deleted as problem solving occurs; determining a layout for the document such that the values chosen for the document parameters and document areas pass all of the specified constraints according to one of a plurality of existing constraint satisfaction algorithms. Other objects, advantages, and salient features of the invention will become apparent from the detailed description which, taken in conjunction with the drawings, disclose the preferred embodiments of the invention.
Description of the Drawings
[0011] The preferred embodiment and other aspects of the invention will become apparent from the following detailed description when taken in conjunction with the accompanying drawings which are provided for the purpose of describing the invention and not for the limitation thereof, in which:
[0012] FIG. 1 illustrates a document template which specifies that there are two areas that should be filled with content: areaA and areaB, and which also specifies that the positions and sizes of areaA and areaB can be changed.
Detailed Description of the Invention
[0013] What is disclosed is a system and method for specifying a custom document as a constraint satisfaction problem (CSP) to create the specified document using existing constraint solving algorithms wherein the document, its content components, and its layout requirements as elements of a constraint satisfaction problem which when solved, results in an automated document layout for the set of content components. This process enables an easier (i.e., automated) custom document creation process, while also providing a wider array of output documents than is possible today.
[0014] The document is expressed as a template made up of variables, values, and constraints. An instance of that template is then created, resulting in a set of content and a layout of that content, by using one of a set of well-known constraint satisfaction algorithms e.g., backtracking, forward checking; as taught in Nadel, Tree Search and Arc Consistency in Constraint Satisfaction Algorithms, Search in Artificial Intelligence, edited by L. Kanal and V. Kumar, Springer-Verlag, 1988, 287-342. The present invention described herein is the formulation of document creation as a constraint satisfaction problem, and the application of existing constraint satisfaction algorithms to automatically create the desired output document, the combination of which is heretofore unknown in the arts.
[0015] In particular, a document template is represented as a constraint satisfaction problem (CSP), and therefore contains a set of variables, a value domain for each variable, and a set of constraints on those variables and values.
[0016] In this invention, the areas of the document to be filled with content are modeled as problem variables, as are any parameters of the document that can be changed. As an example, consider the document template shown in FIG. 1. The template specifies that there are two areas that should be filled with content: areaA and areaB. The template also specifies that the positions and sizes of areaA and areaB can be changed. Thus, the problem variables for this example are: areaA, areaB, areaA-topLeftX, areaA-topLeftY, areaB-topLeftX, areaB-topLeftY, areaA-width, areaA-height, areaB-width, areaB-height.
[0017] The constraint satisfaction problem formulation further specifies that each problem variable has a value domain consisting of the possible values to assign to that variable. This invention teaches that for variables that are document areas to be filled with content (e.g. areaA and areaB of FIG. 1), the value domains are the content pieces that are applicable to each area. For variables that are document parameters, the value domains are discretized ranges for those parameters, so that each potential value for the parameter appears in the value domain. The initial value domain is be set up to be some default range e.g., [1 . . . MAXINT]. For variables whose value domains are content pieces, the default domain is set up to be all possible content pieces in the associated content database, which is specified in the document template.
[0018] The constraints specify relationships between variables and/or values that must hold in order for the resulting document to be valid. Constraints may be unary (apply to one variable), binary (apply to two variables/values), and/or n-ary (apply to n values/variables), and in our invention are entered by the user as part of the document template. An example of a unary constraint in the document domain is: areaA must contain an image of a castle. An example of a binary constraint is: areaA-topLeftY+areaA-height<areaB-topLeftY. If we had another variable (areaC), an example of a 3-ary constraint is: areaA-width+areaB-width>areaC-width. In a variable data application of this invention (one of many possible applications), the constraints would also refer to customer attributes e.g., areaA must contain an image that is appropriate for customer1.age.
[0019] Any one of the known existing constraint solving algorithms for discrete constraint satisfaction problems is then applied to create the final output document. To follow through our example with one such existing algorithm, we consider a simple backtracking algorithm. Our example problem variables and value domains are:
[0020] AreaA {skier1.jpg, skier2.jpg, golf.jpg, snorkel.jpg, snorkel.txt, ski.txt, golf.txt}
[0021] AreaB {skier1.jpg, skier2.jpg, golf.jpg, snorkel.jpg, snorkel.txt, ski.txt, golf.txt}
[0022] AreaA-topLeftX {0 . . . 500}
[0023] AreaA-topLeftY {0 . . . 750}
[0024] AreaB-topLeftX {0 . . . 500}
[0025] AreaB-topLeftY {0 . . . 500}
[0026] AreaA-width {1 . . . 500}
[0027] AreaA-height {1 . . . 750}
[0028] AreaB-width {1 . . . 500}
[0029] AreaB-height {1 . . . 750}
The unary constraints for our example problem are:
[0030] C1: areaA must contain an image of a skier
[0031] C2: areaB must contain text about skiing.
We also have one binary constraint:
[0032] C3: areaA-topLeftY+areaA-height<areaB-topLeftY.
[0033] The constraint solver first achieves ‘node-consistency’, which runs the unary constraints C1 and C2 eragainst each variable's value domain, eliminating any values that don't satisfy the unary constraints. After node consistency, our problem variables and value domains are as follows (note that the value domains for areaA and areaB now have only the content pieces that meet the unary constraints):
[0034] areaA {skier1.jpg, skier2.jpg}
[0035] areaB {ski.txt}
[0036] areaA-topLeftX {0 . . . 500}
[0037] areaA-topLeftY {0 . . . 750}
[0038] areaB-topLeftX {0 . . . 500}
[0039] areaB-topLeftY {0 . . . 500}
[0040] areaA-width {1 . . . 500}
[0041] areaA-height {1 . . . 750}
[0042] areaB-width {1 . . . 500}
[0043] areaB-height {1 . . . 750}
[0044] The constraint solver then iteratively tries a value for each variable in succession, checking the binary (and any n-ary) constraints after each assignment, and backtracks to a different value if the assignment violates any constraints. Note that a constraint is considered to be trivially satisfied if any of its component values is not yet instantiated. Thus it isn't until areaA-topLeftY, areaA-height, AND areaB-topLeftY all have values that there is a possibility that C3 is violated.
[0045] Thus, a backtracking algorithm would iteratively assign:
[0046] areaA=skier1.jpg
[0047] areaB=ski.txt
[0048] areaA-topLeftX=0
[0049] areaA-topLeftY=0
[0050] areaB-topLeftX=0
[0051] areaB-topLeftY=0
[0052] areaA-width=1
[0053] areaA-height=1
[0054] at which point when it checks constraint C3, it will find that areaA-topLeftY (0)+areaA-height (1) is not less than areaB-topLeftY (0), and thus it will backtrack to choose the next possible value for areaA-height. Eventually it will backtrack to a point at which C3 is not violated, and thus will arrive at a valid solution. One can furthermore envision many more constraints that can be added to the problem to ensure a feasible looking output document (e.g., minimum widths for the content areas, etc.). The novelty of the invention is in the modeling of the document template as a constraint satisfaction problem, so that the document creator can specify any set of variables/values/constraints, and then a constraint solving algorithm can automatically create the final output document.
[0055] A further extension of the herein taught method for representing a document template as a constraint satisfaction problem and the method for creating the final document by applying a constraint solving algorithm is that the model can be further extended to encompass more complicated documents by using the well-known dynamic constraint satisfaction model and solving algorithms as taught in Mittal and Falkenhainer Dynamic Constraint Satisfaction Problems, Proceedings of AAAI 1990, pages 25-32. In dynamic constraint satisfaction, the set of variables, values, and constraints can change as the problem solving progresses. That is, given the current choice of values, other constraints might become active and/or certain variables and/or values might be added or deleted from the problem. In the context of automated document creation, this extension enables document content components to specify their own sets of constraints (e.g., “I'm an image, and my width cannot be less than 350 pixels”, or “I'm an image of a cigarette advertisement, and I cannot be placed on the same page as any content from the American Heart Association”). Similarly, dynamic constraints can be added to the problem that specify the use of other problem variables (e.g., “if we choose content for areaA whose width>300 pixels, then we must delete areaC from the problem”). These examples are meant to be illustrative and not comprehensive of all possible dynamic document constraints. In general, any constraint that involves adding/deleting document areas, or that is added/deleted as a consequence of content choices made during constraint solving is what we consider to be a part of the dynamic extension to our document constraint solving model.
[0056] We have furthermore implemented and herein describe our particular implementation (one of many possible implementations) of this dynamic document constraint solving model. We have combined the well-known backtracking algorithm with a linear constraint solver as taught in Badros G., Borning A., The Cassowary Linear Arithmetic Constraint Solving Algorithm: Interface and Implementation, University of Washington Technical Report, 98-06-04, 1998, to enable dynamic addition/deletion of constraints and problem variables during problem solving.
[0057] We model the document area variables (areaA, areaB) as variables for the backtracking algorithm, and further model the layout related variables (areaA-topLeftX, areaA-topLeftY, areaA-width, areaA-height, etc.) as sub-variables to the document area variables. As each top-level variable is instantiated with a value, we add its sub-variables to the linear constraint solver, and we also add any constraints associated with the chosen content value to the linear constraint solver. To continue our example, when we choose value skier1.jpg for areaA, skier1.jpg might have associated constraints that say it cannot be scaled, that its width and height must be its original width and height. Thus, we add the following constraints to the linear constraint solver:
[0058] C4: areaA-width=175
[0059] C5: areaA-height=255
[0060] Now in our particular implementation, we call on the linear constraint solver to determine the validity of the chosen value given these new additional layout constraints. The linear constraint solver processes these new constraints to see if values can be found for all of the other sub-variables (i.e. areaB-topLeftX, areaB-topLeftY, areaB-width, areaB-height) given this and any other already existing layout constraints. If so, then we continue with our backtracking. If not, then we remove constraints C4 and C5 from the linear constraint solver, and backtrack to a different value for areaA. The solver continues in this way, trying candidate content, adding/removing constraints, and checking the active constraints until all document areas are filled with content and the resulting document satisfies all active constraints.
[0061] The present invention has many advantages, for instance, the ability to specify what a document should look like but not have to specify exactly how such a layout should be accomplished relieves the document creator from having to evaluate different potential layouts and how each might satisfy or not satisfy certain document requirements. Also, it does not require the document creator to have prior knowledge of the document content at the very beginning of the document creation process thereby enabling the creation of more dynamic output documents.
[0062] The present system and method also advantageously provides the ability to automate content selection, which is typically not part of the document layout process, i.e., content pieces are selected that satisfy the rest of the document constraints, thereby allowing automatic decisions to be made about which content to include in the document. This is in contrast to the typical approach of forcing the document creator to select all included content before layout is begun. The present invention also advantageously enables one to define constraints on other aspects of the document such as, for example, the constraint of whatever goes in this area should be Times-Roman font-type of size between 8 and 12 or line spacing constraints between differing types of text. Furthermore, the extension of the representation to a dynamic constraint satisfaction model enables more complicated documents to be represented and automatically created. It advantageously provides the capability to specify constraints on content items themselves such as, for example, the constraint that content should not be scaled more than 50% or that content cannot be on the same page as a certain advertisement.
[0063] While the invention is described with reference to a particular embodiment, this particular embodiment is intended to be illustrative, not limiting. Various modifications may be made without departing from the spirit and scope of the invention as defined in the amended claims. Modifications and alterations will occur to others upon reading and understanding this specification; therefore, it is intended that all such modifications and alterations are included insofar as they come within the scope of the appended claims or equivalents thereof.
Claims
What is desired to be secured by United States Letters Patent is:
1. A system for constraint-based generation of documents comprising:
a processor; and
an input device to input user specified constraints;
said processor specifying, in accordance with the inputted user specified constraints, document area problem variables by identifying a plurality of document areas which have associated constraints;
said processor specifying, in accordance with the inputted user specified constraints, document parameter problem variables by identifying document parameters that can be changed;
said processor specifying document area value domains for each document area problem variable by identifying content pieces appropriate for the document areas having associated constraints;
said processor specifying document parameter value domains for each document parameter problem variable by identifying valid ranges for each document parameter;
said processor specifying a layout constraint which further constrains the value domains of the document parameter;
said processor specifying content constraints which further constrain the value domains of the document areas having associated constraints;
said processor determining a layout for a document such that the user specified constraints are satisfied; and
said processor generating a document in accordance with the determined layout that satisfies the user specified constraints.
Patent Citations (32)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US5181162(A) | 1993-01-01 | Smith et al. | Examiner |
| US5224206(A) | 1993-06-01 | Simoudis | Applicant |
| US5499366(A) | 1996-03-01 | Rosenberg et al. | Examiner |
| US5517621(A) | 1996-05-01 | Fukui et al. | Applicant |
| US5555362(A) | 1996-09-01 | Yamashita et al. | Examiner |
| US5848430(A) | 1998-12-01 | Takakura et al. | Examiner |
| US5943670(A) | 1999-08-01 | Prager | Applicant |
| US6014678(A) | 2000-01-01 | Inoue et al. | Applicant |
| US6023714(A) | 2000-02-01 | Hill et al. | Applicant |
| US6044384(A) | 2000-03-01 | Ishima et al. | Applicant |
| US6134563(A) | 2000-10-01 | Clancey et al. | Applicant |
| US6173286(B1) | 2001-01-01 | Guttman et al. | Applicant |
| US6212528(B1) | 2001-04-01 | Brophy et al. | Applicant |
| US6667750(B1) | 2003-12-01 | Halstead et al. | Examiner |
| US6668354(B1) | 2003-12-01 | Chen et al. | Applicant |
| US6826727(B1) | 2004-11-01 | Mohr et al. | Examiner |
| US2001/0051962(A1) | 2001-12-01 | Plotkin | Applicant |
| US2002/0111969(A1) | 2002-08-01 | Halstead, Jr. | Examiner |
| US2003/0163785(A1) | 2003-08-01 | Chao et al. | Examiner |
| US2003/0229845(A1) | 2003-12-01 | Salesin et al. | Examiner |
| US2004/0015782(A1) | 2004-01-01 | Day et al. | Examiner |
| US2004/0019847(A1) | 2004-01-01 | Purvis | Applicant |
| US2004/0019850(A1) | 2004-01-01 | Purvis | Applicant |
| US2004/0019851(A1) | 2004-01-01 | Purvis | Applicant |
| US2004/0019855(A1) | 2004-01-01 | Purvis | Applicant |
| US2004/0024613(A1) | 2004-02-01 | Harrington | Applicant |
| US2004/0025109(A1) | 2004-02-01 | Harrington | Applicant |
| US2004/0034613(A1) | 2004-02-01 | Purvis | Applicant |
| US2004/0205472(A1) | 2004-10-01 | Purvis | Applicant |
| US2004/0205493(A1) | 2004-10-01 | Simpson et al. | Examiner |
| US2004/0205588(A1) | 2004-10-01 | Purvis | Applicant |
| US2004/0205609(A1) | 2004-10-01 | Milton et al. | Applicant |
Non-Patent Literature (13)
- Furuta, Richard, et al, “Document Formatting Systems: Survey, Comcepts, and Issues”, ACM Computing Surveys (CSUR), vol. 14, Issue 3, Sep. 1982, pp. 417-472.Examiner
- Borning, Alan, et al, “Constraints For the Web”, Proceedings of the Fifth ACM International Conference on Multimedia, Nov. 1997, pp. 173-182.Examiner
- Bateman, John, et al, “Towards Constructive Text, Diagram, and Layout Generation for Information Presentation”, Computational Linguistics, vol. 27, Issue 3, Sep. 2001, pp. 409-449.Examiner
- Holzner, Steven, Inside XML, New Riders Publishing, Indianapolis, IN, © 2001, pp 68-73, 77-87, 402-412, 620-621, 626-643, 656-660 and 666-671.Applicant
- Bradley, Keith, et al., “Case-Based User Profiling for Content Personalisation”, Proc's of Int'l Conf on Hypermedia, Brusilovsky et al (eds), Adaptive Hyperm. & Adaptive Web-Based Systems Lecture Notes in CS, vol. 1892, Springer Verlag, © 2000 (11 pages).Applicant
- Villard, Lionel, et al., “An Incremental XSLT Transformation Processor for XML Document Manipulation”, WWW 2002, Honolulu, HI, May 7-11, 2002, pp. 474-485.Applicant
- Mendes, Emilia, et al. “A Comparison of Case-Based Reasoning Approaches to Web Hypermedia Project Cost Estimation”, WWW 2002, Honolulu, HI; May 7-11, 2002, pp: 272-280.Applicant
- Dengler, E. Friedell, M., Marks, J., Constraint-Driven Diagram Layout, Proceedings of the 1993 IEEE Symposium on Visual Languages, pp. 330-335, Bergen, Norway, 1993 (diagrams).Applicant
- Rousseau, F., Garcia-Macias, A., Valdeni de Lima, J., and Duda, A., User Adaptable Multimedia Presentations for the WWW, Electronic Proceedings for the 8th International World Wide Web Conference, 1999 (multimedia presentations).Applicant
- Graf, W. H., The Constraint-Based Layout Framework LayLab and Applications, Electronic Proceedings of the ACM Workshop on Effective Abstractions in Multimedia, 1995 (flowcharts and yellow pages).Applicant
- Kroener, A., The Design Composer: Context-Based Automated Layout for the Internet, Proceedings of the AAAI Fall Symposium Series: Using Layout for the Generation, Understanding, or Retrieval of Documents, 1999.Applicant
- Badros G., Borning A., The Cassowary Linear Arithmetic Constraint Solving Algorithm: Interface and Implementation, University of Washington Technical Report, Jun. 4, 1998.Applicant
- Purvis, Lisa, Document Assembly and Transformation: A Survey, XR&T/DITC/CADISYS/DI, Mar. 2001.Applicant