Directory system

[0001] The present invention relates to a directory system.

[0002] Electronic directories allow networked applications to access and manage items of information that represents entities such as people or items of equipment. The items of information are logically organized as a hierarchy of objects or entries in a directory information tree (DIT), where each object has a name (using naming attributes) and one or more other attributes. A directory service provides electronic access to one or more distributed DITs, allowing one or more objects to be accessed by a directory service user performing a search on an object's name or its other attributes, for example.

[0003] X.500, is the name given to an internationally agreed set of standards for electronic directory services, as defined by the International Standards Organisation (ISO) and the International Telecommunications Union (ITU). The X.500 directory service is defined in an abstract way and conceptually specifies a distributed object oriented system that has security and access control features. It provides a range of user services (such as Read, Search, List, Modify, Add, Remove and Modify Name operations) that enable a directory service to be distributed, replicated, managed and accessed. One of X.500's protocols used for directory access is known as Directory Access Protocol (DAP), and is defined in X.511. Lightweight Directory Access Protocol (LDAP) is based on DAP and is defined by the Internet Engineering Task Force (IETF) in RFC 2251. LDAP has allowed directory access technology to be easily incorporated into desktop client software.

[0004] Directory systems are used by many organizations to store information in relation to the their staff and the organisation's networked computer equipment (printers, routers). For example, a common use of LDAP is to store information representing users of a campus-wide network at a university. A user can log onto any workstation at the university using a single username-password pair that is verified by the workstation after accessing a networked LDAP directory service.

[0005] Although directory systems have been successfully applied to large organizations referred to as ‘Enterprises’ having up to ˜50,000 users, today's Internet-based electronic commerce business organisations may need to support much larger numbers (e.g., 30,000-200,000,000) of users. Such an ‘eBusiness’ could use directory systems to represent many aspects of the business organization, its customers, its services, its policies, its products (including catalogues, content systems and libraries) and its infrastructure components (network systems, mail servers) used to deliver services. In terms of information design, this type of directory virtualizes the organization (as much as possible) into directory system objects as identified entities. Such systems may be expected to hold five hundred million objects, with a schema configuration of hundreds of object classes and thousands of attribute types. In addition, such directory systems need to support the organization's respective security regimes (such as Public Key Infrastructure (PKI) and Access Controls as used for authorization) that are applied to protect ownership, provide information sharing rules and validate trusted identities.

[0006] The need to provide rapid, profiled and efficient updating, management and access to so many objects for so many users and services is beyond the capabilities of existing directory systems. Existing directories are engineered as distributed object-oriented databases that comply with the X.500 and LDAP standards. These directories incorporate an underlying storage system of their choosing, such as a Relational Database Management System (RDBMS) based on structured query language (SQL), or Object Oriented Databases.

[0007] SQL-based database products have significant drawbacks when ultimate performance is the objective. For example:

• • [0008] (i) The relational model tends to lead to complex objects being fragmented over multiple tables, reflecting “many to one” relationships of elements within a directory object class and the instantiations of that class within a named hierarchical structure.
  • [0009] (ii) SQL is an interpreted language, and this has a performance cost. For a conventional lower scale disk-based RDBMS, this cost is acceptable, as it is less than the cost of disk access when tables are cached, and the main optimization goal of conventional RDBMS products is to avoid unnecessary high rates of disk input/output operations.
  • [0010] (iii) SQL performance is determined by the effectiveness of the query optimizer in turning the SQL into an efficient database access route. Mature RDBMS products have extremely effective query optimizers, but their performance is not guaranteed, particularly when the directory may be subject to a wide range of random queries.
  • [0011] (iv) As the number of entries increases in an RDBMS-based directory system, so do the database table lengths (i.e., the number of rows in a table). This constrained way of extending and joining database tables increases information indexing and search times. A central problem is that RDBMS tables cannot be segmented laterally based on arbitrary object name values, naming contexts (what hierarchy the object might be in) or attribute types and values. This causes significant scaling and extensibility problems for very large directory systems.

[0012] Implementing a directory database using an object-based database management system would avoid some of these particular “SQL” problems. However, object databases do not scale well when searching for sparse attributes in complex directory trees or with large entries: object-based directory systems reach their storage, search access and indexing limits very quickly.

[0013] To provide high performance in a directory system, the DIT naming information, the attributes held within the directory entries, and the schema and access control information should be accessed as rapidly as possible. Existing directory systems use keys or indexes to access database data, and hash or B-Tree indexing systems are typical. For example, an RDBMS can use keyed columns in its database tables and build hash indexes for them.

[0014] One difficulty is that as RDBMS-based directory systems scale, the indexes need to be retuned or rebalanced. Alternatively, if every entry attribute type and value is indexed in the same way and at the same processing level, the system can become saturated with index processing and re-processing.

[0015] Moreover, when a new eBusiness user and a corresponding user-to-service relationship is added to a directory service, many directory objects and attributes have to be added in a sequential manner in existing directory systems. This makes the larger scale identity based eBusiness systems quite complex to build and creates a significant risk should the system fail during the user management update process.

[0016] Another difficulty facing Internet-based businesses today is the complexity and inefficiency of existing platforms. Internet based service platforms that provide dialup, email, instant messaging (IM) and web based services use a multitude of protocol and network interactions. The shortage of Internet Protocol addresses world wide, network bottlenecks and systems failures all suggest that simpler ways of building systems with fewer “networking” functions are needed.

[0017] Similarly, existing directory enabled systems use a considerable number of products and servers, a mixture of information storage paradigms, multiple protocols, and inconsistent data and user identity models to perform even the most basic of tasks in Internet service provisioning. Users require a service environment where they can access one or more on-line applications (e.g., email, IM, web, transactions), using one or more devices (e.g., phones, PCs, PDA, Kiosks and iDTV systems) using a one or more authentication and authorization tokens (e.g., passwords, coded cards, biometrics and signatures). Existing systems implement such interactions in a piecemeal fashion using files, databases, directories, and a range of application processes and decision points. Thus installing, deploying and growing such systems is exceedingly complex and constitutes a significant business risk due to poor scaling and reliability vulnerabilities.

[0018] It is desired to provide a directory system that alleviates one or more of the above difficulties, or at least provides a useful alternative.

[0019] In accordance with an exemplary embodiment of the present invention, there is provided a directory system for providing directory services in a communications network, the directory system including a plurality of memory segments for storing respective subsets of directory data for each directory object.

[0020] The present an exemplary embodiment of invention also provides a directory system for providing directory services in a communications network, the directory system including a plurality of directory information tree (DIT) segments for storing hierarchical structure data for directory objects, a plurality of object segments for storing management data for said directory objects, and a plurality of attribute segments for storing attribute data for said directory objects.

[0021] The present an exemplary embodiment of invention also provides a directory system for providing directory services in a communications network, the directory system including one or more messaging modules for providing transactional messaging services to users.

[0022] The present an exemplary embodiment of invention also provides a process for providing directory services in a communications network, including monitoring directory data stored in a plurality of memory segments; and redistributing at least a portion of said directory data in said plurality of memory segments based on said monitoring to improve performance of said directory services.

[0034] In a directory, the term ‘relative distinguished name’ (RDN) refers to the name of an attribute type-value pair at one level within the directory hierarchy, whereas the term ‘distinguished name’ (DN) refers to the sequence of RDNs from the root of the hierarchy to a specific entry and provides the unique name by which that entry can be identified. The term ‘content prefix distinguished name’ refers to the DN of the highest entry in a portion of a directory tree, for example, where that tree is a subordinate branch of a larger hierarchy.

[0035] The Context Prefix of a directory (DIT) namespace is the namespace of other directory entries in other directory systems that are superior in name space to this DIT. For example, if the context prefix of a DIT is {c=US, o=govt, ou=defense, ou=army}, then the entries of this prefix are in one or more other interconnected superior—distributed directories. The “Naming Context” of a DIT indicates the DIT's name space responsibility (i.e., what is at the end of and below the Context Prefix). (However, the context prefix of an entry refers to the naming components of that entry without the name of the actual entry (the entry's RDN).

[0036] Schema definitions define object class structures, specifying their name forms, what object classes they are derived from, what mandatory attributes they have and what optional attributes they have. Schema also defines the attributes as used in object class definitions re their types and syntaxes. Common schema (e.g., country, organization, organization unit, device, person, etc) is defined in X.520 and X.521.

[0037] A name binding rule is used in a directory to enforce structure on the DIT. For example, a name-binding rule might specify that an organization entry can only be placed under a country entry, and that a country entry can only be at the top of the DIT. Such a rule would prevent a directory user from placing countries under people entries or organisational entries. These rules are configured to enforce and maintain “DIT structure” or named object relationships.

[0038] As shown in FIG. 1, the functions performed by directory systems 100 to 106 can be classified into top end functions 108 and back end functions 110. A remote user 112 interacting with any one of the directory systems 100 to 106 via a communications network 114 such as the Internet can access X.500 and LDAP objects/entries of a directory via a distributed X.500/LDAP service 116. This user's view of the directory corresponds to the top end functions 108 of the directory systems 100 to 106. The directory is distributed with a number of object trees (known as Directory Information Trees or DITs) 118 to 126 that are stored and managed by respective directory systems 100 to 106, which may be distributed over a number of physical locations.

[0039] The back end functions 110 are those parts of the directory service that are hidden from the user 112. The back end functions 110 provide access to the X.500 and LDAP objects/entries that are stored in adaptive memory resident databases 126 to 132. A set of directory system modules 134 of each of the directory systems 100 to 106 manages and controls access to the memory resident databases 126 to 132. Portions of the data stored in each of the memory resident databases 126 to 132 can be stored or swapped out to standard disk arrays 136 when not in use and retrieved or swapped back from the disk arrays 136 on demand. The directory systems 100 to 106 are identical other than the content of their respective memory resident databases 126 to 132, and the entire directory can be accessed via any one of the directory systems 100 to 106 using chaining and referral processes to provide access to data stored in the databases of the other directory systems. For convenience, the following description is written from the perspective of a local directory system 100 that interacts with remote directory systems 102 to 106.

[0040] In contrast to existing directory systems that use generic relational databases or object databases, the directory system 100 uses an adaptive memory resident database 126 based on an extensible set of dedicated virtual memory “segments” for storing directory information and its associated configuration and management data. The virtual memory segments can be independently created, re-dimensioned or destroyed as required. In addition, the directory system 100 executes a directory adaptation process, as described below, that adjusts the data stored in these memory segments in order to improve performance of the directory system 100.

[0041] The directory system 100 is also based on a virtual machine environment and provides virtual machines that can operate on the local memory-resident database 126 or make requests through the distributed directory infrastructure to other memory-resident databases 128 or 132 hosted by the remote directory systems 102 to 106. The directory system 100 adaptively reconfigures the virtual memory segments based on usage patterns to reduce search and access times and improve performance of the system 100.

[0042] As shown in FIG. 2, the directory system modules 134 include four manager modules 200 to 206, and six types of virtual machine modules 208 to 218. The manager modules include a Virtual Machine Manager (VMM) 200, a Virtual Store Manager (VSM) 202, a Database Configuration Manager (DCM) 204, and a Virtual DIT Manager (VDM) 206. The virtual machine modules include Directory Service Operation Virtual Machines (OVMs) 208, Transactional Virtual Machines (TVMs) 210, Customised Virtual Machines (CVMs) 212, Distributed Directory Operation Virtual Machines (DVMs) 214, Replicated Directory Operation Virtual Machines (RVMs) 216, and Adaptive Replication Virtual Machines (ARVMs) 218. Each of the virtual machines 208 to 216 can be scheduled as a single instance or as multiple instances, particularly in a multi-processor environment where high throughput and concurrency are needed.

[0043] The memory resident database 126 uses virtual memory segments to store directory object/entry data and also to store data used to manage the directory. As shown in FIG. 3, a virtual memory segment 300 is typically divided into sub-segments 302 containing zero or more data sections referred to as cells 304 that are used to store directory entry or management data. The segment 300 and sub-segments 302 include respective header sections 306, 308 that contain metadata for characterising and managing the respective sub-segments 302 and the data stored in the corresponding cells 304, such as the type of data stored in the cells 304. Each sub-segment 302 may or may not contain sub-segment cells 304 depending on the sub-segment's purpose. The virtual memory segments are implemented as shared memory so that they can be accessed by any of the manager modules 200 to 206 and the virtual machines 208 to 218.

[0044] As shown in FIG. 2, the memory resident database 126 provides four types of virtual memory segments for storing and performing transactions on directory data, referred to as directory service segments 220 to 226, and two types of virtual memory segments 228, 230 for managing the directory service segments 220 to 226. The four directory service segment types include three segment types for storing directory entry/object data: attribute segments 220, object segments 222, and DIT segments 224. The attribute segments 220 are used to store attribute data for objects, while the object segments 222 store access control data and other management data for objects. The DIT segments 224 provide the hierarchical directory tree structure of the directory by storing pointers to object data contained in the attribute segments 220 and object segments 222. The segregation of directory data into these three distinct and independently manageable segment types, and the further segregation of object data by various characteristics within each segment type, as described below, provides numerous advantages, in particular by simplifying access and reducing the time needed to create, locate, retrieve, and/or update one or more directory entries. Moreover, the use of effectively one-dimensional virtual memory segments avoids the lateral and logical segmentation difficulties suffered by directory systems that use database tables. A fourth directory service segment type, transaction segments 226, is used to store transaction data for various phases of a transaction to prevent corruption of the directory in case of system failure during the transaction.

[0045] The management of the directory service segments 220 to 226 is facilitated by two types of management segments: adaptation segments 228 and virtual store segments 230. Adaptation segments 228 are used by the Virtual DIT Manager (VDM) 206 to adjust or adapt the arrangement of object data and management data stored in the directory service segments 220 to 226 to improve the performance of the directory system 100. For example, the VDM 206 can split a long attribute sub-segment storing many attributes into two or more shorter attribute sub-segments to facilitate parallel processing, as described below. The assignment of all segments as Virtual store segments 230 permits the VSM 202 to manage the storage and retrieval of data between the directory service segments 220 to 224 and the physical storage disk arrays 114.

[0046] The Virtual Machine Manager (VMM) 200 manages the creation, destruction, and scheduling of execution instances of the virtual machines 208 to 218, including error and event management. The VMM 200 provides event management and watchdog processes that ensure that virtual machine failures and timeouts do not lock directory resources, and audit and logging mechanisms.

[0047] The Virtual Store Manager (VSM) 202 manages the virtual memory segments and their mapping onto physical memory and disk storage, in addition to ‘Rolling In’ and ‘Rolling Out’ the virtual store segments onto backing store; i.e., storage media such as high-speed magnetic disks and CDs, in order to optimize use of the available physical memory. The VSM 202 creates, destroys and extends (i) the DIT segments 224 that are used to store the Directory Information Tree(s) (DITs), and (ii) the adaptation and transaction segments that contain the management data for these DITs.

[0048] The Database Configuration Manager (DCM) 204 configures the memory-resident database 126. In particular, changes to directory schema definitions, access control and user information are propagated into object segments 222 by the DCM 204. The DCM 204 also populates the adaptation segments 228 with the initial adaptation configuration parameters of the DIT, access control and attribute usage maps. These include parameters that determine when DIT and Attribute segment adaptation is performed. For example, they can indicate adaptation is to be performed daily at midnight; or to create additional attribute segments when attribute sub segments have greater than 1 million attributes (of the same type) and/or if some attributes are not searched for a specified period (e.g., 1 day) then place these attributes into another segment so they can be rolled out independently of the ones that do get accessed frequently.

[0049] The Virtual DIT Manager (VDM) 206 performs adaptive optimization of DIT segments 224, attribute segments 220, object segments 222, including sub-segments within those segments, as described below. This involves dynamically adapting DIT, attribute and object segments 220 to 224 of a DIT to suit the DIT organisation and the way that entry attributes are stored and retrieved. A DIT can be divided into many parts, and the VDM 206 can determine individual optimization and adaptation strategies for the DIT, attribute and objects segments 220 to 224 of each part. In particular, the VDM 206 adapts these segments 220 to 224 to improve performance of a directory as its DIT is modified or otherwise accessed. The VDM 206 dynamically re-profiles the DIT, attribute and object segments 220 to 224 by rearranging its grouping and search paths whilst the directory service is active, as described below. The VDM 206 uses its own adaptation segment 228 that contains sub-segments for storing DIT area maps, access control maps and attribute usage maps, as described below.

[0050] Nine Directory Service Operation Virtual Machines (OVMs) 208 provide respective multi-threaded X.500/LDAP directory service operations on the memory resident database 126, as defined by the X.500/LDAP directory standards. These operations are:

• • [0051] (i) Add Entry,
  • [0052] (ii) Read Entry,
  • [0053] (iii) Compare Entry,
  • [0054] (iv) Remove Entry,
  • [0055] (v) Modify Entry,
  • [0056] (vi) List (names),
  • [0057] (vii) Search,
  • [0058] (viii) ModifyRDN (modify name), and
  • [0059] (ix) Abandon.

[0060] The Distributed Directory Operation Virtual Machines (DVMs) 214 provide X.500 directory service transactional chaining protocol (DSP) service level operations on and between the local memory resident database 126 and remote memory resident databases 126 to 130. Nine types of DVM 214 are provided for respective directory operations as defined by the X.500 directory standards for DSP (i.e., X.518):

• • [0061] (i) Chained Add Entry,
  • [0062] (ii) Chained Read Entry,
  • [0063] (iii) Chained Compare Entry,
  • [0064] (iv) Chained Remove Entry,
  • [0065] (v) Chained Modify Entry,
  • [0066] (vi) Chained List (names),
  • [0067] (vii) Chained Search,
  • [0068] (viii) Chained ModifyRDN (modify name), and
  • [0069] (ix) Chained Abandon.

[0070] These DVMs 214 perform X.500 DSP directory operations and call OVMs 208 to perform any local directory operations on the memory resident database 126. The DVMs 214 handle the requests and responses of chained operations in a single-cast or multicast fashion, depending on the operation concerned. The DVMs 214 maintain distributed operation integrity through the use of the transaction segments 226 and the Transactional Virtual Machines (TVMs) 210, as described below.

[0071] Six Replicated Directory Operation Virtual Machines (RVMs) 216 provide respective X.500 DISP (replication/shadowing) service level operations on and between memory resident databases 126 to 132. These operations are defined by the X.500 directory standards for DISP (X.525) as:

• • [0072] (i) Create Operational Binding (Replication Agreement),
  • [0073] (ii) Modify Operational Binding (Replication Agreement),
  • [0074] (iii) Delete Operational Binding (Replication Agreement),
  • [0075] (iv) Coordinate Shadow Update (Supplier request),
  • [0076] (v) Shadow Update (Shadow transfer), and
  • [0077] (vi) Request Shadow Update (Consumer request).

[0078] The RVMs 216 perform the actual processing of the X.500 DISP directory operations and are described further below.

[0079] The Adaptive Replication Virtual Machine (ARVM) 218 provides an X.500 feature referred to as adaptive replication. On large-scale, multi-master and load-balanced systems, multi-party bilateral replication agreements are an operational overhead and add to the system complexity. The ARVM 218 performs DIT replication by replicating the attribute, DIT and object segments 220 to 224, sub-segments and cells in accordance with adaptive replication agreements and operational policies. Operational (replication) policies define whether a replication process is batch (process many entries at once) or incremental (when an entry is updated replicate it), the time window when replication can occur, the name space and the object types to be replicated—and whether this directory system is the master or the replica.

[0080] The Transactional Virtual Machines (TVMs) 210 provide integrity for transactional updates. In many directory systems, a power or other system failure may corrupt directories by leaving corrupt or partial entries. The TVMs 210 store transaction data in the cells of transaction segments 226 for all directory updates to manage updates and recovery situations when required. This allows directory data and its management segments to be updated and stored with both high performance and integrity.

[0081] Each TVM 210 determines the transaction phase as one of:

• • [0082] (i) The preparation phase—when the directory operation is started and its parameters are written to a Transaction Sub-segment cell by an OVM 208.
  • [0083] (ii) The examination phase—where the Transaction Sub-segment and cells are examined by the TVM 210 after a failure condition to see what update recovery process approaches should be taken.
  • [0084] (iii) The re-execution phase—where the transaction is re-executed by the TVM 210 from a full or partial recovery perspective.
  • [0085] (iv) The completion phase—when the transaction (or its recovery) is deemed complete.

[0086] Transactional information can be written directly into the transaction segment 226 and/or at the same time backed out to back up media/files depending on the setting of the integrity mode in the object sub-segment SYS cells, as described below.

[0087] The TVMs 210 are provided in different types for each transaction type, such as an X.500/LDAP directory service update (performed by an Add, Modify, Modify RDN or Remove OVM 208), the database adaptation and re-profiling process (performed by the VDM 206), a directory service replication procedure (performed by an RVM 216) or distributed operation (performed by a DVM 214). In addition, each type of transactional TVM 210 provides customized recovery procedures for the different transaction phases.

[0088] The Customised Virtual Machines (CVMs) 212 provide customized services that are extensions to the standard directory service model, including user and service identity management functions (multi object management), relational searching, user certificate PKI validation, and supporting the directory's transactional or instant messaging functions.

[0089] Specifically, 17 types of CVM are provided, one to support each of the following services within the directory system 100:

• • [0090] (i) CVM—Statistical Evaluation (CVM-SE);
  • [0091] (ii) CVM—Change Monitoring (CVM-CM);
  • [0092] (iii) CVM—Validate Certificates (CVM-VC);
  • [0093] (iv) CVM—Collective Attributes (CVM-CA);
  • [0094] (v) CVM—Multi-Object Management (CVM-MO);
  • [0095] (vi) CVM—Service Authorization (CVM-SA);
  • [0096] (vii) CVM—User Presence (CVM-UP);
  • [0097] (viii) CVM—Relational Searching (CVM-RS);
  • [0098] (ix) CVM—Directory Service Messaging (CVM-DM);
  • [0099] (x) CVM—Message Submission (CVM-EMS);
  • [0100] (xi) CVM—Message Management (CVM-EMM);
  • [0101] (xii) CVM—Message List expansion (CVM-EML);
  • [0102] (xiii) CVM—Message Retrieval (CVM-EMR);
  • [0103] (xiv) CVM—Address Book (CVM-EAB);
  • [0104] (xv) CVM—Mail Folder (CVM-EAF);
  • [0105] (xvi) CVM—Calendar/Diary (CVM-ECD); and
  • [0106] (xvii) CVM—Message Gateway (CVM-EMG).

[0107] In some cases, a CVM 212 has an active relationship with specific directory object classes such as Post Boxes (as used for transactional messaging), Users, Roles, Services and with Authentication and Authorization processing. Such ‘active’ CVMs monitor specific objects to either create system events, notify Users of directory entry changes, or maintain referential integrity between an entry and other directory entries (e.g., entries related to memberships and groups).

[0108] The structure and indexing of each virtual memory segment type will now be described.

Attribute Segments

[0109] Attribute Segments 220 contain one or more sub-segments, each containing data for one attribute type. Each attribute sub-segment contains zero or more cells for storing the details of each instance of that attribute type. Attribute Segments 220 contain all the attribute type and value information as well as their normalized forms and any control flags set to indicate that specialised processing is required.

[0110] The segregation of attributes by type permits single attribute types (or several attribute types) to be directly targeted in a Read/Search operation without needing to go through the entire set of directory entries and all of their attributes to find a specific attribute type(s) and values. In other words, the directory system 100 segregates attributes by type (for example, phone numbers, email addresses, etc are each stored together) within each sub-segment of an attribute segment 220 in order to allow high-speed attribute type level processing. Furthermore, if the number of attribute instances in a sub-segment exceeds a configurable threshold value then that sub-segment can be divided into two or more smaller sub-segments, allowing these to be processed in parallel.

[0111] Attribute types are also segregated by their naming role (distinguished and non distinguished) within the directory in order to improve navigation and entry selection processing. Specifically, attribute types are marked as either:

• • [0112] (i) Distinguished attributes (used for names (DNs/RDNs),
  • [0113] (ii) Aliased Distinguished Names, or
  • [0114] (iii) Non Naming Attributes.

[0115] Naming is a major property of directories and directories have functions like “navigation” and alias dereferencing that use DN structures—DNs are made of RDNs, and RDNs have one or more attributes that are tagged as “distinguished” attributes in that these attributes of an entry distinguish it from other entries in the same name space. (RDBs don't have these naming properties or functions). It is advantageous to process attributes (of any type) on the basis of whether they are distinguished or not. For example, a Navigation and Rename operation can only be performed on distinguished attributes, whereas a Modify operation will only work on non distinguished attributes.

[0116] Directory entry attributes are segregated by their directory information properties within their X.500/LDAP information contexts in order to improve directory information processing. Specifically, an attribute is identified as being one of the following attribute types:

• • [0117] (i) Collective Attributes,
  • [0118] (ii) Compound Attributes—such as X.509 certificates and certificate revocation lists (CRLs),
  • [0119] (iii) Attributes of Compound attributes,
  • [0120] (iv) X.500/LDAP operational attributes,
  • [0121] (v) User Operational (Presence) attributes,
  • [0122] (vi) Sponsoring Attributes, or
  • [0123] (vii) Normal attributes (where an attribute is none of the above).

[0124] The directory system 100 stores and processes all directory entry attributes including compound attributes (i.e., nested attributes defined within attributes, such as X.509 certificates and member lists), at the same processing level to optimise retrieval.

[0125] Each attribute value is stored with a normalized form of the value and a hash of the value in attribute sub-segment cells in order to support rapid value-based searching on entry attributes. A hash value of the Relative Distinguished Name (RDN) is stored as an RDN Identifier in each attribute sub-segment cell in order to support rapid entry level based searching and results collection.

[0126] Attribute Segments can be used in several ways, according to DIT optimization requirements as performed by an attribute segment adaptation process as shown in FIG. 7 and described below. For instance, Attribute Segments can be used in the following modes of operation:

• [0127] (i) Basic—There is one Attribute Segment that contains all the Sub-segments for all the attribute types held in the complete DIT. (This is the simplest form of DIT and Entry/Attribute configuration).
• [0128] (ii) Attribute Type Splitting—the attribute segment adaptation process can split up Attribute Sub Segments that have become very populated (long) because there are many instances of that attribute type in the DIT. This splitting enables parallel searching of Attribute Sub segments of a given type in addition to the parallel searching enabled by the segregation of attribute based on attribute type. The two or more resulting sub-segments for that attribute type can be stored in the same attribute segment or in two or more attribute segments.
• [0129] (iii) Object Class Splitting—There is more than one Attribute Segment where each segment contains Sub-segments that relate to particular object classes and their attributes held in the complete DIT. (This configuration is used where different object classes can be stored in distinct Attribute Segments. For example, X.500/LDAP Device object class attributes can be stored in one Attribute Segment, and X.500/LDAP Person object class attributes stored in another Attribute Segment).
• [0130] (iv) Complex/Distinct Attribute Splitting—There is more than one Attribute Segment and each contains one or more sub-segments for specific attribute(s) held in the complete DIT. (For example, the X.500/LDAP Organizational Attribute Sets or the X.509 certificates and CRLs can be held in distinct Attribute Segments.)
• [0131] (v) DIT Splitting—There is more than one Attribute Segment where each contains all the Sub-segments for all the attribute types held in a portion of the DIT. (This configuration is used where Search and Update rates may be high and higher concurrency is required.)
• [0132] (vi) Access Control Role Id Splitting (by Attribute Type)—This mode can be used if distinct Access Control Role Ids are associated with specific Attribute Types within an attribute segment. In this case the respective Role Ids can be stored in the headers of the specific Attribute Sub Segments they directly relate to.

[0133] Alternately, Attribute Segments can be used in any combination of the above modes. Such flexibility ensures shortest path accesses (Searches and Reads) can occur in a hierarchical or lateral manner over the DIT.

[0134] Each Attribute Segment Header contains management information about the complete segment. The Attribute Segment Header contents are:

• • [0135] (i) The segment type identifier (e.g., “Attribute Segment NNNN” or an object identifier (OID)).
  • [0136] (ii) The maximum size of the segment.
  • [0137] (iii) The number of attribute types being managed in this segment.
  • [0138] (iv) A Master/Replica indicator.
  • [0139] (v) The Attribute Segment mode (i.e., whether basic or adapted and, if adapted, whether the mode supports, Attribute, Object Class, Complex Attribute, or DIT Splitting).
  • [0140] (vi) Object Class identifier—used only if this attribute segment supports a single object class (through adaptation).
  • [0141] (vii) The existence or residence (file state) of the segment (being rolled in from backup storage or rolled out to backup storage by the VSM 202).
  • [0142] (viii) A file name for archive when the complete segment is archived.
  • [0143] (ix) A file name for restore when the complete segment is being restored.

[0144] Each Attribute Sub-segment Header contains management information about the way in which a single particular attribute type is managed. The Attribute Sub-segment Header contents are:

• • [0145] (i) The Sub-segment and attribute type identifier. (e.g. “Attribute Sub-Segment” TTTT, NN or an OID where TTTT=type, NN=number).
  • [0146] (ii) The maximum size of the sub-segment.
  • [0147] (iii) The number of attribute cells being managed in this sub-segment.
  • [0148] (iv) Attribute type control flags that indicate the Attribute cells contain:
    • [0149] (a) A complex type (CRL, Certificate or mail list)
    • [0150] (b) A component of a complex attribute (for example, a certificate Serial Number)
    • [0151] (c) Sponsoring attributes and can create multi object events.
    • [0152] (d) Distinguished attributes (the RDN or part of it).
    • [0153] (e) Aliased DN attributes.
    • [0154] (f) X.500/LDAP operational attributes.
    • [0155] (g) User Operational (presence attributes)
      • [0156] (i) Such as On-line, Location, Device Type, Profile, Security Level, Inform List.
    • [0157] (h) Collective attributes and will be applied to it and its subordinate entries.
  • [0158] (v) The attribute syntax (String, Binary, Numeric).
  • [0159] (vi) The attribute type AC Role Id—allowing access control information (ACI) roles to be placed at the attribute type level.
  • [0160] (vii) The attribute cell size.
  • [0161] (viii) The existence or residence (file state) of the sub-segment (being rolled in from backup storage or rolled out to backup storage by the VSM 202).
  • [0162] (ix) A file name for archive when cells of this attribute type are being archived.
  • [0163] (x) A file name for restore when cells of this attribute type are being restored.

[0164] Each attribute Sub-segment cell contains management information and the value of a single attribute of the corresponding entry as follows:

• • [0165] (i) The cell identifier. (e.g., “a cell number”)
  • [0166] (ii) The context prefix identifier of the object/entry with this attribute as taken from the DIT sub segment (i.e., the DN prefix of this of this (entry's) attribute).
  • [0167] (iii) The Relative Distinguished Name identifier of the entry with this attribute.
  • [0168] (iv) Attribute control flags:
    • [0169] (a) A value indicator (0-N) to indicate this is one value of a multi-valued attribute.
    • [0170] (b) An archive flag to indicate whether the attribute has been archived (because
    • [0171] (c) A flag to indicate if this attribute is associated with other attributes.
    • [0172] (d) A flag to indicate if this attribute is the sponsoring attribute associated with other attributes.
    • [0173] (e) A flag to indicate if this attribute is an alias.
  • [0174] (v) The length of the attribute.
  • [0175] (vi) The attribute value as given.
  • [0176] (vii) The attribute value normalized—white space and accented characters removed.
  • [0177] (viii) The hash of the attribute value—to assist Search processing.
  • [0178] (ix) A component indicator to indicate the attribute type is a component of another attribute type.
  • [0179] (x) A file name to indicate this attribute has been rolled out (archived) to backing store (used in the case of attributes where their values are very large binary objects).
  • [0180] (xi) A DN List that indicates the names of the objects that this associated attribute updates when modified.
    DIT Segments

[0181] The DIT Segments 224 contain one or more sub-segments that contain cells. DIT Sub-segments 224 contain references to the entries in a directory tree (by their name and context prefix), allowing very rapid directory entry navigation without having to find a specific entry in an hierarchical fashion through an entire set of directory entries and their naming attributes. The DIT Segments 224 stores a Context Prefix Distinguished Name in string form as well as a hash value generated from the Context Prefix DN. Both information items assist with DIT navigation far more efficiently than using full name value equivalence matching on complex DN—RDN hierarchical structures.

[0182] DIT sub-segments are created when leaf entries are added to leaf entries and removed when a parent entry no longer has any leaf objects below it. DIT sub-segments contain one or more cells that point to one or more Attribute Segments 220 and Object Segments 222. DIT sub-segment Cells can be copied from superior DIT sub-segment cells when leaf objects are added or they can be regenerated according to the access control rules for the new entry or the object class and schema of the new entry. One or more DIT sub-segment cells can index the same Attribute 220 and/or Object Segments 222.

[0183] DIT segments can be applied in several modes according to DIT optimization requirements. For instance, DIT sub-segments can be organized using a Group and Sequence Id, as described below, so that a group's sub-segments can identify to particular directory nodes and their subordinates in the DIT. DIT Segments can be used in the following modes:

• • [0184] Basic—There is one DIT Segment that contains all the DIT sub-segments with cells that point to all the Attribute Segments that hold the complete DIT. (This is the simplest form of DIT and Entry/Attribute configuration.)
  • [0185] DIT Organisation Splitting—There is more than one DIT Segment where each contains a number of DIT sub-segments with cells that point to the same Attribute Segments, which hold the complete DIT. (This configuration is applied where there are large numbers of entries held in subordinate name spaces in a particular DIT Segment and it is expedient (for navigation performance reasons) to split the DIT Segment into two or more distinct smaller DIT segments. This provides more entry points to the DIT(s) for navigation at the Segment level.
  • [0186] DIT Access Control Rule Splitting—There is more than one DIT Segment where each contains some sub-segments related to particular portions of the complete DIT (for example, a top level DIT with many Organizational Unit DITs). This configuration is used where there are many entries in the directory, but where distinct parts of the DIT contain distinct access control rules. An example is where one DIT Segment and its DIT Sub Segments are associated with the same access control Role Id, and another DIT Segment and its Sub Segments are associated with a different access control Role Id.
  • [0187] DIT Flat Space Splitting—This feature enables a flat namespace with large numbers of leaf entries to be split across a number of DIT Sub-Segments to improve performance. In this case splitting can be derived from entry counts held under one DIT segment or a namespace prefix mask that indicates the part of the internal name that is the full external name (e.g., the prefixes of 3 DIT sub segments might be “Parts1”, “Parts2”, and “Parts3” a prefix mask of “Parts” indicates that these sub-segments are to be internally and externally represented as a contiguous “flat space”. Where Flat Space Splitting is used, the Search VM of the OVMs 208 can create another execution instance of the Search VM so that Search concurrency can be achieved (where multi-processor hardware is used).

[0188] Alternately, DIT Segments can be applied in any combination of the above modes according to the DIT structure and responsibility of the database 126. Such flexibility ensures that fast access (Searches and Reads) can occur in a segmented manner over the DIT and complex access control rules are not tested for when they do not apply.

Attribute and DIT Adaptation Mode Combinations

[0189] Table 1 below defines the ways in which Attribute and DIT Segment modes of adaptation can be used in combination. Basic means there is one Attribute or DIT Segment only. The other modes use two or more Attribute or DIT segments.

[0190]

TABLE 1
Attribute Mode/		DIT	DIT- Access
DIT Mode	Basic	Organization	Control Rules	Flat
Basic	Allowed	Allowed	No	Allowed
Attribute Type	Allowed	Allowed	Allowed	Allowed
Split
Object Class	Allowed	Allowed	Allowed	Allowed
Split
Simple/Complex	Allowed	Allowed	Allowed	Allowed
Attribute Split
DIT Name Split	No	Allowed	Allowed	No
Access Control	Allowed	Allowed	Allowed	Allowed
Role
Id Splitting
(Attribute Type)

[0191] DIT sub-segment Cells point to Attribute Segments 220 (for attribute and entry information) and Object Segments 222 for Access Control and Schema management information. Different DIT sub-segments can point to different Attribute and Object Segment sets. Such partitioning enables additional levels of adaptability and efficiency in the instances where large numbers of object classes, attributes, Users and their access control information are dispersed asymmetrically throughout the directory service.

[0192] The DIT Segment Header contains the management information about the complete DIT Segment 224. The DIT Segment Header contents are:

• • [0193] (i) The segment type identifier (e.g. “DIT Segment NNNN” or an OID).
  • [0194] (ii) The maximum size of the segment.
  • [0195] (iii) The number of entries being managed in this segment.
  • [0196] (iv) The Context Prefix Identifier—stored as an object identifier. An object identifier (OID) is a set of numbers separated by periods (i.e., in ‘dot notation’) that indicates an object's position in a directory hierarchy. If an item in a directory system is given an OID, the OID provides rapid understanding of the item's position within a top down allocation scheme and the “tree depth” of the entity from the length of numbers used. For example, an entity with an OID of 1.7.4.88 is higher in the scheme than an entity at 1.7.5.35.99.222. OID numbers assigned to directory “names” facilitate rapid processing; for example, a request to retrieve all the entries below x.x.x.x indicates that the search starts at 4 levels down and all the entries returned will have at least 5 numbers.
  • [0197] (v) The DIT (Access Control) Identifier—a numeric value that uniquely identifies the parts of the DIT associated with this DIT segment, as described below. The Access Control evaluation process described below uses the DIT identifier to determine whether a user can access this part of the DIT.
  • [0198] (vi) The Access Control Role Id—if one Access Role Id is used throughout this DIT Segment. This is set through system configuration or via the DIT Adaptation modes.
  • [0199] (vii) The mode of this DIT (Master or Replica) and Read Only.
  • [0200] (viii) The existence or residence (file state) of the segment (being rolled in from backup storage or rolled out to back up storage by the VSM 202).
  • [0201] (ix) A file name for archive when the complete segment is archived.
  • [0202] (x) A file name for restore when the complete segment is being restored.

[0203] The DIT sub-segment Header contains management information for a particular DIT portion, as follows:

• • [0204] (i) The sub-segment type identifier (e.g., “DIT Sub-Segment NNNN” or an OID).
  • [0205] (ii) The sub-segment group—sequence Id. Used to group DIT sub-segments; for example, where Flat Name Space segmentation is applied.
  • [0206] (iii) The Access Control Role Id—present only if one Access Role Id is used. Otherwise, if multiple Role Ids exist for this DIT sub-segment, then they are determined by accessing the corresponding DIT sub-segment Cells and Object Segment Cells.
  • [0207] (iv) The Context Prefix of this DIT sub segment (including the external Context Prefix).
  • [0208] (v) The DIT (Access Control) Identifier—a numeric value that uniquely identifies the part of the DIT associated with this Sub Segment, as described below. The Access Control evaluation process described below uses the DIT identifier to determine whether a user can access this part of the DIT.
  • [0209] (vi) The Context Prefix Identifier, stored as an Object Identifier (OID).
  • [0210] (vii) The number of DIT cells being managed in this sub segment.
  • [0211] (viii) The Context Prefix DN, held in string form.
  • [0212] (ix) The Context Prefix DN hashed.
  • [0213] (x) The next RDN Id to be used when an entry is added to this Context Prefix.
  • [0214] (xi) Flat Space Identifier: 0=DIT area is not segmented, 1-N=DIT Flat Space portion
  • [0215] (xii) Maximum number of Flat Space Entries: When this value is reached, a new Flat Space DIT Sub-Segment is created.
  • [0216] (xiii) Flat Space Name Prefix: Set to the RDN prefix value of the entries that will be logically stored in Attribute Segments pointed to by this DIT sub-segment's cells.
  • [0217] (xiv) The existence or residence (file state) of the sub segment (being rolled in from backup storage or rolled out to backup storage by the VSM 202).
  • [0218] (xv) A file name for archive when these DIT cells are being archived.
  • [0219] (xvi) A file name for restore when these DIT cells are being restored.

[0220] Each DIT sub-segment Header is followed by one or more DIT sub-segment Cells that point to Object Segments 222 (for Access Control and schema information) and Attribute Segments 220 (for entry attribute information and values). The DIT sub-segment Cell contents are:

• • [0221] (i) The DIT cell identifier. (e.g. “DIT Cell” or an OID).
  • [0222] (ii) The Object Segment Identifier(s) that point to the object segments related to the attributes/entries contained in the Attribute Segment(s) identified below. The Object Segment(s) identified contains the schema and access control check information for these entries.
  • [0223] (iii) The Attribute Segment Identifier(s) that point to the Attribute Segment(s) where the attributes of this DIT portion are stored.
    Object Segments

[0224] Object Segments 222 contain one or more Sub-segments that contain cells. Object Sub-segments contain DIT schema definitions and access control information. As DIT Sub Segments are created (as a result of processing DIT configuration or DIT adaption rules), existing or new (adapted) Attribute and Object Segments are assigned to them. Attribute Segments hold the directory data for the DIT namespace. Object Segments store the schema and access control rules for the data in that DIT namespace. The DIT schema is a set of rules defining object classes, attribute type definitions, matching rules, and DIT structure rules that define where entries can be placed within the DIT. DIT update operations VMs (i.e., the Add, Remove, Modify and ModifyRDN OVMs of the OVMs 208) access Object Segments 222 to determine the directory's schema rules. DIT access (i.e., Read, Search, Compare, and List OVMs) and DIT update operations VMs access Object Segments 222 to determine access control information.

[0225] Object sub-segments are created and updated from configuration items (be they files or directory objects) by the DCM 204. Object sub-segments are sub typed as to their purpose and contain information cells where necessary.

[0226] Object Segments can be used in accordance with several modes according to DIT optimization requirements because particular aspects of DIT configuration may only apply to specific parts of a DIT. For instance, an Object Segment can contain all the sub-segment types (Object Class information, Name Binding, Access Controls and Knowledge information) necessary for the directory system 100 to operate. This configuration is referred to as an Object Segment basic mode of operation. Other Object Segment modes permit specific object class specifications, ACI and DIT structure rules (Name Bindings) to be assigned to distinct portions of the DIT so that simpler and faster checking can occur relative to that part of the DIT. This is done because, in large distributed systems, it is not efficient to always process large complex configuration files that cover the whole system for every type of access/update, when only a small portion of that configuration applies to specific parts of the system on specific access types. Hence an object segment adaptation process puts ACI, schema and name bindings in distinct Object Sub Segments and associates these with the corresponding DIT Sub Segments.

[0227] For example, if a subset of the object class and name binding rules only applies to distinct portions of the DIT because specialised object classes exist in that part of the DIT, then it is only those that should be tested when updates occur within that portion of the DIT.

[0228] In particular, if there is an access control rule that specifies particular Role Ids for a portion of the DIT, then only those Role Ids are inspected when that part of the DIT is accessed.

[0229] Object Segments can be organized according to the following modes:

• • [0230] Basic—There is one Object Segment that contains all the Object sub-segments and the cells necessary for the directory system 100 to operate as an X.500/LDAP directory service.
  • [0231] Name Binding Splitting—There is more than one Object Segment, where one Object Segment can contain in its Sub Segments all the commonly defined schema definitions and name binding rules. Other Object Segments and Sub Segments can contain distinct Name Binding rules for distinct parts of the DIT. This allows selected configuration elements to be assigned to those parts of the DITs that actually apply them, reducing processing overhead and improving the performance of the directory system.
  • [0232] Access Control Role Id Splitting (by name space)—There is more than one Object Segment where one Object Segment can contain the common Object sub-segments, and the other Object Segments contain sub-segments that store the distinct Access Control Role Ids (as per Name Bindings).
  • [0233] Multi Object Profile Splitting—There is more than one Object Segment, where one Object Segment contains all the common Object sub-segments and the other Object Segments contain sub-segments that hold distinct profiles for Multi Object updates. In single object updates, schema and name bindings are defined to ensure that objects are properly defined (schema check) when being added to the directory (as a single object) and to ensure the object is put in the right place in the hierarchy (DIT name binding check). Where multiple objects are added in an atomic way, a “multi object” schema check and a multi position naming check is performed. For performance and atomicity reasons, these checks are bundled into distinct Object segments for the update activity concerned. With multi object processes, a fast, concise “one test” is performed before attempting the update. If this information was not in one place, a multi object update would have to search through all the “common” configuration items that may be spread over a number of other object segments to determine what part of that configuration needed to be checked.

[0234] Alternatively, Object Segments can be applied in any combination of the above modes according to the DIT structure and responsibility of the directory system 100. Such flexibility ensures fast access (Searches and Reads) can occur in a segmented manner over the DIT and complex access control rules are not tested for when they do not apply.

[0235] An object segment header contains the following:

• • [0236] (i) The segment type identifier (e.g., “Object Segment NNNN” or an OID).
  • [0237] (ii) The maximum size of the segment.
  • [0238] (iii) The number of sub-segments being managed in this segment.
  • [0239] (iv) The Context Prefix of the DIT this Object Segment applies to (e.g., c=us, o=gov, ou=dod).
  • [0240] (v) The Context Prefix Identifier, held as an Object Identifier.
  • [0241] (vi) The mode of this Object Segment (Master or Replica) and Read Only (enabling directory service configuration information to be replicated).
  • [0242] (vii) The existence or residence (file state) of the sub segment (being rolled in from backup storage or rolled out to backup storage by the VSM 202).
  • [0243] (viii) A file name for archive when the complete segment is archived.
  • [0244] (ix) A file name for restore when the complete segment is being restored.

[0245] Each Object sub-segment Header contains management information about the cells that are subordinate to the header, as follows:

• • [0246] (i) The sub-segment type and sub type identifier. These identifiers indicate that the following cells contain one of:
    • [0247] (a) SYS—System Control information
    • [0248] (b) KI—Knowledge Information
    • [0249] (c) OC—Object Class Structure information
    • [0250] (d) ATT—Attribute Type Definitions
    • [0251] (e) NB—Name Binding Information
    • [0252] (f) US—User-Service Multiple Object Template, or
    • [0253] (g) AC—Access Control Information.
  • [0254] (ii) The maximum size of the sub segment.
  • [0255] (iii) The number of cells in this sub segment.
  • [0256] (iv) The Context Prefix that this Object Sub segment applies to (e.g., c=us, o=gov, ou=dod, ou=army). This is assigned by pre configuration or by the Object Segment adaptation process.
  • [0257] (v) The Context Prefix Identifier, held as an Object Identifier.
  • [0258] (vi) The existence or residence (file state) of the sub segment (being rolled in from backup storage or rolled out to backup storage by the VSM 202).
  • [0259] (vii) A file name for archive when this sub-segment Object is being archived.
  • [0260] (viii) A file name for restore when this sub-segment is being restored.

[0261] Object sub-segment Cells contain directory and system management information as shown above. The contents of the cells are dependent on the cell type. The Object sub-segment Cell contents are either the Object Segment Cell type identifier, or additional information, depending on the cell type. The various types of object sub-segment will now be described. A System Control Cell holds high-level flags used by the virtual machines 208 to 216, as follows:

• • [0262] (i) Integrity Mode—This indicates the integrity mode the directory system 100 is working in. The options are High Integrity Mode where all updates are journalized from the Transaction Segment 226 to backing store and Low Integrity Mode where all updates are just applied in line to the Transaction Segment only.
  • [0263] (ii) State—indicates:
    • [0264] (a) Activation state of the directory system 100, these can be off line, loading, suspended, active, or in error;
    • [0265] (b) DCM 204 is active (updating Access Control Information and User Ids); or
    • [0266] (c) VDM 206 is active (updating/re-profiling the DIT, Attribute and Object Segments).

[0267] The above State indicators are used by the VM scheduling processes to ensure that resources are locked when necessary and that no conflicts in the attribute, DIT and object segments reconfiguration processes occur. For example, to prevent an update operation being performed when an adaptation process is executing.

[0268] A Knowledge Information Cell provides the network or domain name address and namespace responsibilities of this directory system 100 and other associated directory systems 102 to 106. Note that each directory system 100 to 106 can be responsible for many Namespaces, as they can apply their processing to one or more DIT Segments. The Knowledge Information Cell contents are:

• • [0269] (i) Mode is set for master or replica;
  • [0270] (ii) Availability set to off line or on-line;
  • [0271] (iii) The Context Prefixes of the directory system 100;
  • [0272] (iv) The Context Prefix Identifiers, held as Object Identifiers;
  • [0273] (v) Domain name or IP address of the directory system 100 in the network;
  • [0274] (vi) Available VMs—indicates what capability this directory system 100 has including its management VMs;
  • [0275] (vii) Permissible VMs (Public)—indicates what non authenticated Users can do on this directory system 100;
  • [0276] (viii) Permissible VMs (Protected)—indicates what authenticated Users can do on this directory system 100; and
  • [0277] (ix) Adaptive Replica Management parameters—Refresh rate, Refresh time.

[0278] An Object Class Cell indicates the directory entry object class definitions as defined by the X.500/LDAP standards. The Object Class Cell contents are:

• • [0279] (i) Object Id is the standard identifier assigned to the object class.
  • [0280] (ii) Object Type is the natural register name of the object (for example. Person Object Class).
  • [0281] (iii) Must Contain attributes are the attributes that must always be present in an object if it is placed in the directory.
  • [0282] (iv) May Contain attributes are the attributes that may be present in an object if it is placed in the directory.

[0283] Object Class Control Flags indicate if this is a sponsoring object.

[0284] The Attribute Type/Syntax Cell indicates the directory entry attribute definitions as defined by the X.500/LDAP standards, as follows:

• • [0285] (i) Object Id is the standard identifier assigned to the attribute type.
  • [0286] (ii) Attribute Type Control Flags—indicate if this is a sponsoring attribute
  • [0287] (iii) Attribute Syntax defines if the attribute is String, Integer, Certificate,
  • [0288] (iv) Attribute Name is the standard name for the attribute
  • [0289] (v) Alternative (LDAP) names are the alias names for the attribute type.

[0290] The Name Binding Cell indicates the directory entry object class/hierarchy relationships definitions as defined by the X.500/LDAP standards. The Name Binding Cell contents are:

• • [0291] (i) Object Id is the standard identifier assigned to the name binding
  • [0292] (ii) Rule Id is a number assigned to this name binding
  • [0293] (iii) Superior Object Class is the object class of the superior object in this rule.
  • [0294] (iv) Subordinate Object Class is the object class of the subordinate object in this rule.
  • [0295] (v) Name attribute type is the attribute syntax (for example Country Name) that is used in this object class relationship.

[0296] The US (User-Service) Multiple Object Template Cell is used by the Multi-Object Management CVM. This cell is used to store the prefix names, object classes and attribute values that are applied when a User that has access to multiple services is created in the directory and that operation requires that multiple directory entries or a User group be created with its members as sponsored objects. The US (User-Service) Multiple Object Template Cell contents are:

• • [0297] (i) A User Id specifying who can use this template (for example, a Service Provisioning Application); and
  • [0298] (ii) Multiple entries of:
    • [0299] (a) The operation: Add or Modify entry.
    • [0300] (b) The attribute as taken from the primary entry being added that will be used for this entry's name if this entry is to be created. If the entry is to be modified (a member list) this is the name added to the list.
    • [0301] (c) The Context Prefix DN of where this object is placed or modified, as appropriate.
    • [0302] (d) The Object Class of the entry to be added or modified, as appropriate (e.g., a mail box, address book, vCard or Presence management object).
    • [0303] (e) The pre loaded attribute values for this object class.
    • [0304] (f) The access control Role Ids associated with the object.

[0305] Access Control Information (ACI) is logically defined in X.501 and the LDAP standards. These definitions are somewhat cryptic as they use Abstract Syntax Notation One (ASN.1) as their definition notation. Large directory systems use rules-based configuration (such as Administrator, Groups, Named Users, Directory Applications, Self Administration and Public User) placed above that of the X.501 definitions. These access control rules also define access policies on directory namespace, the operations permitted and what entries/attributes can be accessed.

[0306] The Object sub-segment(s) contain cells that store ACI used to determine a User's or a User group's permissions in the directory. This information is generated from a system level Rules based configuration. As it is important that ACI processing is both effective and efficient, DIT sub-segment Cells point to the Object Segments 222 for the purposes of associating specific schema management and ACI processing to specific parts of the DIT.

[0307] ACI related to a user's permissions in the directory system is stored in the Object Segment cells for performance reasons. Access control is managed through the use of Object sub-segment access control (AC) cells. An AC Cell contains the following data:

• • [0308] (i) A User Id, being the Distinguished Name (DN) assigned to a specific User or a group of users. A User Id is the DN of the user's entry in the directory that this AC cell relates to. For groups of users, it is the DN of an entry that is the immediate superior to that of the user's entries, or the DN of an entry that is an X.500/LDAP “group of names” object that contains the user's names (DNs) associated with this AC cell.
  • [0309] (ii) A Context Prefix Identifier; that is, the DIT namespace this User or User group has access permissions on.
  • [0310] (iii) The VM permissions of this User or User group, that is the ability to read or update.
  • [0311] (iv) The Role Id assigned to this User or User group, defining their capability on this part of the DIT.

[0312] As shown above, AC cells contain “Role Ids” for controlling access to all levels of the database 126 down to the attribute level of the DIT for specific users or user groups. AC cells provide the linkage between the Access Control rules (i.e., Role Ids) applied to portions of the directory name space (the DIT) and the Users or User groups that have access rights over that name space. Access controls can be applied in a singular or nested specification at many levels, e.g., applied to an entire directory, specific operations (updates, modifies), portions of name space, specific object classes and specific attribute types—and in combinations of these. The AC cells define a Context Prefix, permissions and role(s). One or more of these AC cells are applied by the DCM 204 to the database 126 when, for example, a User is created and that User requires access control rules (permissions) to be applied across the directory service. For instance, if a new user is added with the same permissions as one or more other users, then the new user can be added to a user group having those users as members. Alternatively, if the new user has very specific access control requirements, then one (or more) AC cells will be added to the Object Segment of the DIT segment(s) concerned, providing the new user's access permissions to the DIT(s). The coupling of user identifiers (or directory user names) and Role Ids in AC cells ensures that access control decisions can be made rapidly.

[0313] Directory services permit access to Public users; i.e., users that are not preconfigured in the directory with their own named entries and do not have explicit access control rules assigned to them. Explicit access control rules can only be assigned to users who have named directory entries and who authenticate with the directory by entering their name and password. Additionally, an IT system may have many servers of a specific type (e.g. web and mail servers) that accesses specific parts of the DIT for specific information (user password, mailbox details, forwarding address, etc). In both cases, optimisations can be made by assigning administrative Role Ids to those entities that update and manage the directory entry information and assigning a Role Id to the DIT Sub Segment for those users/servers who do not have Object Segment AC cells for this DIT namespace.

[0314] By storing access control Role Ids in DIT Sub segments, access control permissions or denial can be determined at the name space level (within a DIT namespace according to the DIT usage or the DIT and Attribute Segment adaptation mode combination). For example, where DIT and Attribute Segments are adapted for Object Class splitting, the DIT Access Control Role Id also grants or denies access to specific object classes and their attributes.

[0315] Access control processing can be further refined by assigning Role Ids to attribute types or groups of attribute types (attribute type groups) that require additional protection. Role Ids stored in Attribute Sub-segment headers can be used to allow or deny access for particular users or groups of users at the attribute type level (within a DIT namespace according to the DIT and Attribute Segment adaptation mode combination). For example, if Basic modes of Attribute and DIT adaptation are used, then a Role Id in an Attribute sub segment can indicate that a Private Telephone Number attribute type is for Read only access for the particular staff (of an organisation) that have access to this DIT namespace, but can only be updated by an administrator.

[0316] One or more of these Object Segment user AC cells can be used for a complete DIT or a portion of the DIT (as defined by DIT sub-segments). Role Ids can be defined as hierarchical or lateral in their operation across one or more DITs. For example, if there are types of users that have the same rights over a single DIT namespace (hierarchical)—e.g., they can read the organisation's “white pages” and modify their own telephone number they would all have the same Role Id. This role is a hierarchical role.

[0317] On the other hand, if users have diverse rights over multiple DITs (lateral operations) in that, for example, a user can modify their own private entry, update the corporate yellow pages for their group (in another name space), read the white pages services, and find all the assets (from the assets DIT) in their location, that would also constitute a “role”—a lateral role. A single access control Role Id can be applied to a DIT Sub-segment with that Role Id stored in the DIT Sub-segment header to improve performance. The directory system 100 assigns Role Ids based on how the DIT and its Users have been set up in its initial state and how it has been adapted, as shown in Table 2 below. For instance, if a single (larger) DIT segment is divided, through adaption, into two or more smaller DIT segments, then distinct Role Ids can be assigned to the DIT Sub-Segments within those Segments. Moreover, further optimisation can be achieved if, for example, a single Object Segment includes AC cells that relate to these DIT Segments, by dividing the Object Segment into two or more object segments if the access controls for each of the DIT segments are unique.

[0318] For practical reasons, there are very few cases where more than 30 “roles” for users are defined in a directory service. The numeric Role Ids used to indicate administrative superiority, namespace capability, object class capability and attribute type capability can be processed much faster than traditional “open ended” and nested definitions of permissions and denials for every conceivable case. In addition, the use of numeric Role Ids allows access control and security processing optimisations to be performed by trusted operating systems or dedicated hardware components, as described below, in order to meet high grade and trusted (military) directory service applications. AC cells allow one or more Role Ids to be assigned to a User or one Role Id to be applied to many users (user groups).

[0319] Examples of the use of roles within the directory system include assigning specific Role Ids to:

• • [0320] (i) an administrator—who has complete add, remove, read modify and modify name capabilities over all or a particular set of namespace(s);
  • [0321] (ii) a group of users who have rights such as add, remove, read and modify capability on a particular namespace;
  • [0322] (iii) an application that has rights to add particular objects in one name space, read on another namespace, and modification capability on their own management entries;
  • [0323] (iv) an individual user who has rights to read a particular namespace and a modification capability (on some attributes) in their own entry; and
  • [0324] (v) a public user (an anonymous user who does not authenticate with the directory) who has read access on certain attributes in a specific namespace (e.g., a white pages service).

[0325] Each Role-Id is a sequence of numeric permission category values, where each permission category value represents permissions in that permission category. Each bit of a permission category value represents a particular permission within the corresponding permission category. Thus a permission category value is determined by which permissions (i.e., bit fields) are set or cleared. It is therefore convenient to represent a Role-Id in ‘dot notation’, as a sequence of decimal permission category values separated by dots.

[0326] Five Role Id permission categories are defined, in the following order (from highest to lowest permission category): Admin Group (a), DIT identifier (d), Directory Operations (o), Attribute Group (g), and Attribute Type (t), although additional permission categories can be defined if desired. Thus an arbitrary Role Id can be represented in dot notation as a.d.o.g.t, where each permission category value a, d, o, g, and t is a decimal number. Within each permission category (i.e., numeric value of a, d, o, g, or t), a particular permission is specified as one of many possible bit values (i.e., powers of 2, or 1, 2, 4, . . . ).

[0327] The directory system 100 executes an access control process, as described below, to grant or deny access to a user in relation to a specific read or update operation on a specific entry within the directory. The granting or denial of access is based on a hierarchy of tests applied to the user's access request, starting by testing whether the user has the necessary high-level VM permissions to the directory (defined in the corresponding object segment's Knowledge Information cells and the corresponding DIT Segment Header) to perform the requested operation, and ending with testing whether the user can perform the requested operation on the corresponding attribute types. As part of these tests, an ACI validation process of the access control process is executed to grant or deny access based on comparison of a Role Id representing the user's requested operation, and a Role Id associated with the corresponding directory resource.

[0328] When a directory user requests that a directory operation be performed, the OVM corresponding to the desired operation (e.g., the Search OVM of the OVMs 208) is invoked and provided with the operation parameters that indicate what part or parts of the DIT namespace the operation wants to access and the attributes it wants to use. The OVM uses these parameters to generate a Role Id (the a, d, o, g and t parameters) representing the requested directory operation, as described below. This dynamically generated Role Id is referred to as the User's operation Role Id, and can be represented as Ua.Ud.Uo.Ug.Ut. The operation Role Id is generated as follows.

[0329] The Access Control cells for the requested DIT namespace are first inspected to determine whether this User does have access rights assigned to the namespace. If no AC cells for that User are found, the DIT access control Role Id is inspected to determine whether Public access permission is defined. Otherwise, access is denied and the operation is terminated. If this user (from the User Id) is found to have administrative rights defined in an Access Control cell, the User's Role Id sets the Administrator bit; i.e., Ua=1.

[0330] If administrative rights are not set in any Access Control cell, the DIT identifier component of each Role Id (from one or more DIT Segments and Sub Segments) for the DIT name space requested by the User's operation are used to generate the DIT identifier component of the User's operation Role Id. For instance, if a directory with the namespace dod has army (assigned a DIT identifier value of 0x1 in the DIT Sub Segment), navy (assigned a DIT identifier value of 0x2) and air force (assigned a DIT identifier value of 0x4) as subordinate name space and normal directory entries below them and the user has set a Search operation to start at the dod name space, then the corresponding DIT Identifiers are used to set the bit fields of the DIT part of User's Role Id (Ud). That is, Ud=army DIT identifier+navy DIT identifier+air force DIT identifier=0x1+0x2+0x4=0×7.

[0331] Finally, the Directory Operations (Uo), Attribute Group (Ug), and Attribute Type (Ut) permission category values are set to the predefined values representing the requested directory operation, and the attribute groups (if any) and attribute types requested for that operation.

[0332] The next step is to test the preconfigured permissions as defined by the Role Id(s) of that User as held in the Object Segment AC cells using the ACI validation process, as shown in FIG. 10. In the description below, the directory held Role Id(s) for that user as stored in the Access Control cells can be represented as Aa.Ad.Ao.Ag.At.

[0333] The ACI validation process begins by checking whether the administrator bit of the user's pre configured Role Id is set by determining whether Aa=0 at step 1002. If not, then a test is performed at step 1004 to determine whether Ad=0; i.e., whether the user's pre configured Role Id is equal to 1.0.x.x.x, where x represents any value. This Role Id value has special meaning in the directory service in that the user has overriding authority over the total DIT, allowing any operation on any attribute group or individual attribute type. Consequently, access is granted at step 1008.

[0334] Otherwise, if the test at step 1004 is answered in the negative, that is, the user's Role Id is equal to 1.Ad.x.x.x, with Ad≠0, then the user has overriding authority over the DITs specified by the bit value(s) of Ad with any operation on any attribute group or types. Accordingly, if the result of a bitwise AND operation (represented by the ampersand character “&”) on Ud and Ad is not zero, i.e., (Ud & Ad)≠0, then at step 1008 all operations are permitted on the DIT namespaces in question. Otherwise, access is denied (step 1012) and the process ends.

[0335] The above two conditions, where the user's Role Ids equal to 1.0.x.x.x or 1.Ad.x.x.x with Ud≠0, are special cases where any and all access is allowed to the entire or corresponding DIT namespace, respectively.

[0336] In any other case (for non administrative users), a hierarchy of tests are performed on the directory operation, attribute group, and attribute type permission categories. In essence, these tests allow or deny access to the DITs specified to perform the directory operations specified on the attribute groups and/or the attribute types specified.

[0337] Returning to step 1002, if the user's administrator bit in the Access Control cells was not set, then at step 1014 a test is performed to determine whether Ud & Ad=0. If so, then the user cannot access this DIT namespace, and access is denied at step 1012, thus ending the ACI validation process. Otherwise, the user has access to the DIT namespace, and the next step in the hierarchy is performed at step 1016.

[0338] Directory operations (Search, Read, Modify, Rename, etc) are assigned respective bit fields within the directory operation permission category o of a Role Id. At step 1016, a bitwise AND operation is performed on directory operation category permission value Ao from the Role Id in the Access Control cell (or DIT or Attribute Sub Segment if there is a Role Id present in these) and the bit value Uo of the operation being attempted by the user. A zero result indicates that permission is denied; a non-zero result enables the operation to be performed if the subsequent attribute tests pass.

[0339] Attribute Groups (such as organisational attribute sets or postal attribute sets) are assigned Role Id bit values (corresponding to the g value in a Role Id). Attribute Types (such as home address or private phone) are assigned Role Id bit values (corresponding to the t value of a Role Id). When a user has permission to enter the DIT namespace (step 1014) and perform the operation (Search, Read, Modify, Rename, etc) (step 1016), a test is performed to determine whether the two attribute Role Id fields (Ug and Ut) of the user's Role Id identify at least one attribute type that is also identified by the Role Ids in the Access Control cells (or the DIT or the Attribute Segments if they are present). That is, whether the at least one attribute is either a member of the one or more attribute group(s) identified by Ag or is one of the one or more attributes identified by At.

[0340] Specifically, these Role Id bit values are tested at step 1018 by performing bitwise AND operations. If (Ug & Ag)=0 AND (Ut & At)=0, then access is denied. Attributes from the attribute group or the individual attribute types can be accessed and processed (step 1020).

[0341] Any user (whether authenticated with the directory or not) that accesses the directory has no permissions at all, not even read access, unless a Role Id is assigned to that user or the user's type (e.g., a public user). For example, Public users (including users who have not been authenticated with the directory system 100) can be given access to a particular namespace by storing a Role Id in the corresponding DIT Sub Segment header.

Allocation—DITs

[0342] When the directory system is initialized and configured at start up, each namespace is assigned a DIT identifier value (d). For example, 5 DITs in a DIT Segment can be assigned DIT identifiers as follows:

[0343]

c = us, o = gov, ou = dod, ou = army	DIT Identifier = 0.1.0.0.0
c = us, o = gov, ou = dod, ou = navy	DIT Identifier = 0.2.0.0.0
c = us, o = gov, ou = dod, ou = airforce	DIT Identifier = 0.4.0.0.0
c = us, o = gov, ou = dod, ou = white pages	DIT Identifier = 0.8.0.0.0
c = us, o = gov, ou = dod, ou = secret	DIT Identifier = 0.16.0.0.0

Directory Operations Capability

[0344] When the directory system is initialized and configured at start up, Directory Operations permission category values (o) are assigned to directory operations as follows: Add=32, Remove=16, Modify=8, Modify Name=4, Self=2, Read=1.

Directory Attribute Type Group Capability

[0345] When the directory system is initialized and configured at start up, Attribute Group permission category values (g) are assigned to attribute groups. Attribute Group as: public set, white pages, specific group set, user self management, user management, application, PKI, etc.

[0346] For example: Application 1=128, Application 2=64, PKI=32, White Pages=16, Group Set=8, Public Set=4, Public White Pages=2, User Self Mgt=1.

[0347] In most cases, attributes can be grouped in a directory in a logical way that relates to their management and access. This is more efficient than storing single permissions for every attribute type for every user use case.

Directory Attribute Type Capability

[0348] When the directory system is initialized and configured at start up, Attribute Type (t) permission category values are assigned to specific attribute types. e.g., Private Address=4, Private Phone=2.

Examples of Role Ids at the DIT, Operations and Attribute Levels:

• • [0349] (i) If an administrator role has a role ID of 1.0.0.0.0, this is greater than 0.x.x.x.x (where “x” represents any value), and therefore the administrator has overriding power, and access is given to all DITs for all operations.
  • [0350] (ii) A Navy “Group” with a Role Id of 0.2.9.17.0 gets access to Navy (navy=0.2.0.0.0) for internal white pages attributes (17=16+1=WhitePages+UserSelfMgt) for reading and Self modification attributes (9=8+1=Modify+Read).
  • [0351] (iii) A User (e.g., the Chief of Defense) assigned a role ID of 0.15.3.12.0 can access (3=2+1=Self+Read) army, navy, airforce and public white pages DITs (15=8+4+2+1) and can also manage his own entry, but not see the secret DIT.
  • [0352] (iv) A User (e.g., an HR Manager) assigned a role ID of 0.15.96.12.3 can access army, navy, airforce, PKI attributes and white pages as well as manage all entries including private attributes, but cannot see the secret DIT.
  • [0353] (v) A public person with a Role Id of 0.8.1.2.0 cannot see army, navy or airforce DITs, and can only read “white pages” attributes from the white pages DIT.

[0354] The DCM 204 assigns DIT (Access Control) Identifiers to DITs (storing them in DIT Segment Headers and DIT Sub Segment Headers), and assigns Role Ids to users, based on administrator configuration information. For example, the command

• • [0355] SET DIT Role Id: {c=us, o=govt, ou=dod} 2
    can be entered by an administrator to set the DIT Identifier for the namespace {c=us, o=govt, ou=dod} (i.e., stored in the corresponding DIT segment and sub-segment headers) to be equal to 2. This DIT identifier can subsequently be used as the DIT identifier component (i.e., the DIT identifier permission category) of a Role Id; i.e., Role Id=x.2.x.x.x.

[0356] Similarly, the command

• • [0357] SET Attribute Type Group Role Id: telephone, postal, locality, email 8
    sets the Attribute Group permission category value (g) in the Role Ids assigned to the attribute types telephone, postal, locality, and email (i.e., the Role Ids stored in the corresponding headers of the corresponding attribute type sub-segments) to be equal to 8; i.e., Role Id=x.x.x.8.x.

[0358] A subsequent command can then be used to configure a user or groups of users with a Role Id on a particular DIT namespace. For example, the command

• • [0359] SET USER AC: {c=us, o=govt, ou=dod}\{c=us, o=govt, ou=dod, ou=army, ou=task force1} Role Id=0.2.9.8.0
    creates an Access Control cell in an object sub-segment associated with the namespace {c=us, o=govt, ou=dod} and stores a Role Id of 0.2.9.8.0 in the new AC cell for users in the organisational unit “task force1”, as specified. The DIT Identifier permission category value (d) for these users is thus set to 2 and a user group Id is created for users in the name space of {c=us, o=govt, ou=dod, ou=army, ou=task force} with the ability to perform particular operations (corresponding to the directory operations permissions (o) value of “9”) in this namespace on the attributes in attribute group 8, as defined by the “SET Attribute Type” command above.

[0360]

TABLE 2
DIT Structure/Use Profile	Role Ids relate to: (examples)
Basic -	An Attribute Segment can segregate sets of attribute types
Few Object Classes	with its Sub-segments (0.0.0.x.0). Within a Sub-segment, a
Few Roles	Role Id is stored for hierarchical use e.g. a Public Role Id, a
	Group Role Id and a Private Role Id. Each Role Id can access
	some or all Attribute Sub-segments (attribute types).
	Vertical Role Ids are applied by DIT permissions (0.x.0.0.0).
Mid -	Multiple Attribute Segments that segregate Object Classes
Many Object Classes with different	and their sets of attribute types - as completely different
attribute types	objects or even as per a basic object class and a refined
Few Roles	object class with their Sub-segments, e.g. a Public Role Id, a
	Group Role Id and a Private Role Id and each Role Id can
	access 1, 2 or 3 Attribute Segments and their Sub-segments
	respectively.
Major -	Attribute Segments are assigned as above, but Role Ids are
Many Object Classes with	assigned on a lateral or vertical basis. Lateral basis indicates
differing attribute types	distinctive name space grouping where vertical Role Ids
Many Normal Roles	indicate more or fewer permissions in the same namespace.
Worst Case	This level of sophistication means that some profiles use the
Many Object Classes with	a matrix of attribute types/object classes - permissions and
differing attribute type selections	denials and Role Ids would be assigned as a class that
Many Complex Roles	references a permissions/denials attribute type table.

[0361] This enables a graduated form of access control, which may be: (i) None, (ii) at the DIT level, or (iii) at the entry and attribute level, depending on the shape of the DIT and its Users. For example, hierarchical (i.e., vertical or “DIT area specified”) roles can be used to provide superior directory rights to administrators relative to public Users. This can be achieved by assigning a Role Id to administrators that is numerically greater than the Role Id assigned to a directory tree (as used by public Users).

[0362] Lateral roles can be defined to provide two individual Users or two groups of Users with unique rights over two separate parts of a directory. If a user's Role Id is equal to a Role Id assigned to a directory resource, access to the resource is granted.

[0363] Both hierarchical and lateral Role Ids are generated dynamically by the system DCM 204 from access control configuration information provided by a system administrator and stored in Object Segment AC cells. The majority of Users are typically configured for fast Role Id processing. Users can be assigned multiple Role Ids by including multiple AC cells in the Object Segment.

[0364] As described above, an AC cell contains a Role Id value that is generated by processing a role-based directory configuration specifying a user's access to the corresponding part (or parts) of the DIT. Each part of the DIT can have zero, one, or many Role Ids assigned to it. The default/access policy at this level (the most granular level in the directory service) is one of denial, with access allowed only by defining one or more Role Ids associated with the resource(s) in question.

[0365] Access Control information is pre-processed and applied using Role Ids to enable “quick decisions” at the time of a directory operation. This pre-processing approach provides a pseudo-indexing mechanism (of access control information and its decision processing).

[0366] Several stages of access control evaluation are performed, based on the granularity of the decisions being made. For example:

• • [0367] (i) Can I enter the system and perform this operation? Grant or deny access to the system
  • [0368] (ii) Can I go to a resource, the entry or entries? If so, then navigate to the resource required.
  • [0369] (iii) Can I perform the operation on the resource? Grant or deny the read or update operation on that entry or attribute.

[0370] The directory system 100 executes an access control process, as shown in FIG. 4, to grant or deny access to a user in relation to a specific read or update operation on a specific entry within the directory.

[0371] Once a User is authenticated at step 402 (or there are any changes to the access control rules), their access control data is generated by the DCM 204, if these are not already present.

[0372] At step 406, macro level tests are performed to determine whether the user can enter the system to perform the specified operation.

[0373] Specifically, the User Id is checked for the X.500/LDAP operation type (i.e., read or update) on the namespace (Context Prefixes) requested using the Object Segment—Knowledge Information cell and the VM capabilities defined within that. If the Context prefixes do not match and/or the VM capabilities of that Context Prefix do not support the operation, the operation is denied at step 408. Additionally, the DIT Segment mode stored in the corresponding DIT Segment Header defines whether the DIT is a Master or a Replica or is set to Read Only. If the User is attempting to do an update operation (Add, Modify) and the VM and DIT capabilities are set for Read operations only, the operation is denied at step 408.

[0374] In addition, if the operation is an update, DIT Object Class Schema and Name Binding tests can be performed on the operation using the Object Class schema definitions and Name Binding cells from the Object Segment.

[0375] If the user is granted access to enter the system 100 to perform the operation, then the process next determines whether the user can access the specific resource/entry. At step 410, the appropriate OVM evaluates the DIT Segments 224 and sub-segments until it arrives as the Context Prefix DN of the entry (or base object) in question. At step 412, the OVM uses the references in the DIT sub-segment to access the entry's attribute and access data stored in the referenced Attribute Segment 220 and Object Segment 222.

[0376] A user of the directory has an entry in the directory under their name (DN). This entry holds the user's details, including their password as well as other attributes the user can change with “Self Modification” rights. The name of this entry has a Context Prefix and an RDN (the actual DN of the user). For example, Context Prefix ID could represent {c=us, o=gov, ou=dod, ou=admin}, and the RDN could be cn=alan. If the name of the entry is equal to the name of the user (in this case, “alan”), then the user “alan” has rights to descend to that actual named entry to modify it.

[0377] At step 414, if Own Entry modification is being attempted, (update on an object name by a user with the same name), the Object Sub Segments and the AC Cells within the Object sub-segments are inspected to see if this User Id has a Context Prefix Id equal to or greater than that of the DIT Context Prefix Id (The User can descend the DIT past this point if they have a greater Context Prefix and Context Prefix Id capability.) If the user has passed the tests described above, the process next determines whether the user can perform the operation on that resource. At steps 418 and 420, the AC Cell is checked to see if this User is able to perform this operation at this depth of the directory tree, by checking the Role Ids associated with the user and the directory resource for permissions using the ACI validation process described above. If the Role Ids do not deny access, then the next test is to determine if the User has read or update permissions on the entry's attributes based on their type, including its name attributes. These tests use the Role Id(s) generated by the DCM 204, as described above. Users are assigned access control Role Ids according to their permissions defined in the high-level configuration parameters. For example, Role Ids can be classified as Public User, Own Entry management, Group or Administrator.

[0378] If the operation is an update operation (step 424), the DIT Object Class Schema and Name Binding tests are performed on the operation at step 426 using cells from the respective Object sub-segments. If access is granted at step 430, the operation is performed at step 430.

[0379] The access control process described above with reference to FIG. 4 relates to a directory operation on an entity. During a search process, it will be apparent that access control checks are performed on each DIT during navigation of the directory.

Transaction Segments

[0380] Each transaction segment comprises a Segment header, one or more sub-segment headers and one or more sub-segment cells. Transaction Segments contain the information necessary to keep the database 100 intact through VM processing and directory operation updates. Transaction sub-segments are sub-typed as to their purpose and contain information cells that relate to the transaction progress.

[0381] Transaction sub-segments are created from operations being performed on the directory by its Users through specific OVMs 208 and by the ARVM 218.

[0382] The Transaction Segment Header contains management information about a complete DIT segment. The Transaction Segment Header contents are:

• • [0383] (i) The segment type identifier.
  • [0384] (ii) The maximum size of the segment.
  • [0385] (iii) The number of sub-segments in this segment.
  • [0386] (iv) The Context Prefix of the DIT this Transaction Segment relates to (that is external to this DIT).
  • [0387] (v) The Context Prefix Identifier, held as an object identifier.
  • [0388] (vi) The mode of this Transaction Segment (Internal or External transaction management).
  • [0389] (vii) Exception Event Processor (for errors).
  • [0390] (viii) The existence or residence (file state) of the sub segment (being rolled in from backup storage or rolled out to backup storage by the VSM 202).
  • [0391] (ix) A file name for archive when the complete segment is archived.
  • [0392] (x) A file name for restore when the complete segment is being restored.

[0393] The Transaction Segment Sub Header contains the management information about the way in which a particular transaction is being protected. The Transaction sub-segment Header contents are:

• • [0394] (i) The sub-segment type identifier.
  • [0395] (ii) The sub-segment group—sequence Id:
    • [0396] (a) for the DVM 214 it is set as the Multicast Id (Distributed Operations)
    • [0397] (b) for the RVM 216 it is set to the Replication Transfer Id
  • [0398] (iii) Remote database Id RX if the transaction is initiated from a remote database (Such as a distributed basic or relational Search).
  • [0399] (iv) The Context Prefix of the DIT this transaction relates to.
  • [0400] (v) The Context Prefix Identifier as an object identifier.
  • [0401] (vi) Remote database Id TX if the transaction is sent to a remote database. (Such as a distributed or relational Search).
  • [0402] (vii) The Directory service operation as provided by the User (if OVM 208) or a VDM 206 identifier if this is an Adaptation process that is being performed.
  • [0403] (viii) The archived database DIT Segment information if this to be updated during the transaction.
  • [0404] (ix) The archived database (Attribute Segment Cells) entries as they existed before the update.

[0405] There are one or more Transaction sub-segment Cells within a Transaction sub-segment. They are used to record the progress of a transaction and its status. The Transaction sub-segment Header Cell contents are:

• • [0406] (i) The Transaction Cell Id relative to the Transaction sub-segment Id.
  • [0407] (ii) The transaction progress:
    • [0408] (a) Not Started
    • [0409] (b) DIT segments updated
    • [0410] (c) Attribute Segments updated
    • [0411] (d) Remote transaction sent—waiting reply
    • [0412] (e) Remote transaction received—scheduling
    • [0413] (f) Awaiting group response of distributed operations
    • [0414] (g) Transaction discarded
    • [0415] (h) Transaction in roll back
    • [0416] (i) Transaction failed, or
    • [0417] (j) Transaction completed
  • [0418] (iii) The date/time this cell was created.

[0419] Transaction sub-segments and cells are created on directory updates by particular OVMs 208 and the Adaptation processes, the VDM (Virtual DIT Manager) 206 and the database object configuration process, the DCM (Database Configuration Manager) 204. As defined above TVMs 210 have a number of process phases on which they operate:

• • [0420] (i) There is the preparation phase, when the directory operation and its parameters are written to a Transaction sub-segment and its cells.
  • [0421] (ii) There is the examination phase, where the Transaction sub-segment and its cells are examined after a failure condition to see what recovery process approaches should be taken.
  • [0422] (iii) There is the re execution phase, where the transaction is re-executed from a full or partial perspective.
  • [0423] (iv) There is the completion phase, when the transaction (or its recovery) is deemed complete.

[0424] Every instance of a new sub-segment created in the preparation phase carries with it a cell that indicates the date/time and the progress set to “Not Started”. It remains in this state until the update is completed successfully, at which this time the Transaction sub-segment and its cell(s) are destroyed. In the other TVM states (recovery), the sub-segment is updated with new progress state cells. The use of Transaction sub-segments is described further in Table 3 below:

[0425]

TABLE 3
Responsible VM	Condition	sub-segment Contents
OVM - Add Entry (a)	Add a leaf entry with other	OVM Parameters
	siblings -
OVM - Add Entry (b)	Add a leaf entry on an existing leaf	OVM Parameters
	entry (no siblings) -	New DIT sub-segment
OVM - ModifyRDN	Modify the distinguished attributes	OVM Parameters
	of a leaf entry.	Old Distinguished (naming)
		Attribute(s), Old RDN Id
OVM - Modify Entry	Modify the non-distinguished	OVM Parameters
	attributes of an entry.	Old Non -distinguished
		Attributes
OVM - Remove Entry (a)	Remove a leaf entry where	OVM Parameters
	siblings exist
OVM - Remove Entry (b)	Remove a leaf entry where no	OVM Parameters
	siblings exist	Old DIT sub-segment and cells
DVM - Distributed	Sending distributed operations or	DVM X.500 DSP request
Operations (DSP or local	Segment information to remote	information or Segment, sub-
mode) TX	systems.	segment and/or cell
		information. One cell for each
		request (multicast requests)
DVM - Distributed	Receiving distributed operations	DVM X.500 DSP request
Operations (DSP or segment	or Segment information from	information or Segment, sub-
mode) RX	remote systems.	segment and/or cell
		information
RVM - Replicated	Sending replicated information or	RVM X.500 DISP request
Operations (DSP or segment	Segment information to remote	information or Segment, sub-
mode) TX	systems.	segment and/or cell
		information
ARVM - Adaptive	Receiving replicated Segment	Segment, sub-segment and/or
Replicated Operations	information from remote systems	cell information
(segment mode) RX	as the result of adaptive
	processing.
ARVM - Replicated	Sending Segment information to	Segment, sub-segment and/or
Operations (segment mode)	remote systems as the result of	cell information
TX	adaptive processing.
RVM - Replicated	Receiving replicated information	RVM X.500 DISP request
Operations (DSP or AMRD	or Segment information from	information or Segment, sub-
mode) RX	remote systems.	segment and/or cell
		information
VDM - re profiling the	Re profile the Attribute, DIT and	Old sub-segment information
segments	Object Segments	Proposed sub-segment
		information
VCM - configure the Object	Update schema, User and Access	Old and proposed Object Sub
segments	Control information	segments and cells
		Old and proposed DIT sub-
		segment cells if required
CVM-MO - Multi Object	Add or Remove Multi Objects	Add - CVM-MO
Update		parameters/objects.
		Remove - Existing objects as
		per Object Sub segment
		template.

Adaptation

[0426] As described above, the directory system 100 executes a directory adaptation process, as shown in FIG. 6, to optimize the organization of the logical directory information for Search and retrieval processes (e.g., Searching on specific name spaces, object classes or attribute types) and to split up long linear data tables into two or more shorter segments to reduce search times by allowing parallel processing.

[0427] By re-organising directory information, Search processes can make big decisions early (e.g., which Attribute Segments to look through) as well as take advantage of multi-processor hardware by executing “syntax” aware multi-threaded information evaluation and retrieval processes on distinct Attribute Segments or Sub Segments. In the Search evaluation process, different search processes are assigned to distinct attribute Sub Segments based on the attribute type and syntax. If an Attribute Segment has been adapted to store distinct attribute types/syntaxes (such as PKI based attributes), then specific search processes can be assigned at that level.

Adaptation, Attribute Syntax and Search Optimisation

[0428] Directory attributes have syntaxes such as Case Ignore String and Certificate, etc.

[0429] A process to evaluate a string for equality or a predetermined value is different from a certificate matching process that may be looking for an embedded attribute type such as the certificate serial number (which just happens to be an integer) or the Issuer DN. It follows that specific Search processes can be assigned at the Attribute Sub Segment level and Attribute Segment level. For example:

• • [0430] (i) A Case Ignore String Match process can be assigned to an Attribute Sub Segment that stores email addresses;
  • [0431] (ii) A Case Exact Match can be assigned to an Attribute Sub Segment that stores a Clear text Password;
  • [0432] (iii) A Certificate Component Match process can be assigned to an Adapted Attribute Segment that stores X.509 certificates as well as its component attributes held in distinct Sub Segments according to the component attribute's type; and
  • [0433] (iv) A PKI management process can be assigned to an Adapted Attribute Segment that stores CA, Certificate, Attribute Certificate and Certificate Revocation List attributes.

[0434] As shown in FIG. 6, the directory adaptation process includes two distinct adaptation (sub-)processes: (i) a segment adaptation process (steps 602 to 61) executed by the VDM 206 that restructures the storage of data in the Attribute, DIT and Object Segments of a directory; and (ii) a replication adaptation process 612 that is executed if the directory system 100, as a local machine to its users, has the authority and capability of retrieving directory information from other (typically remote) directory systems 102 to 106 (i.e., other name spaces). This replication adaptation process 612 is executed by the ARVM 218 in order to ensure that such information is stored locally to improve system performance (this is also referred to as adaptive replication).

[0435] As shown in FIGS. 6 to 9, the segment adaptation process (steps 602 to 61) can be divided into three constituent (sub-) processes: an attribute segment adaptation process 606, a DIT segment adaptation process 608, and an object segment adaptation process 610 for reorganising the attribute, DIT and object segments of a directory, respectively.

[0436] The attribute segment adaptation process 606 and the DIT segment adaptation process 608 are used to divide or split a large amount of data stored in a single segment or sub-segment into two or more smaller segments or sub-segments in order to facilitate parallel processing. Consequently, these two adaptation processes are executed only if parallel processing hardware is available (e.g., multiple processors, such as multiple CPUs and/or one or more CPUs and one or more dedicated segment processors).

[0437] In contrast, the object segment adaptation process 610 and the replication adaptation process 612 are used to co-locate and cache selected data to improve performance, but not specifically for the purposes of parallel processing. Consequently, these two adaptation processes are executed regardless of whether parallel processing hardware is available or not.

[0438] All of the adaptation processes are controlled through the local Adaptation Segments 228.

[0439] When the Attribute 220, DIT 224 and Object 222 Segments are adapted, all writes/updates to the directory are buffered.

Adaptation Segments

[0440] Adaptation Segments 228 comprise a Segment header, a sub-segment header and cells.

[0441] Adaptation Segments 228 contain the information necessary to keep the database 126 and its information (Attribute, DIT and Object Segments 220 to 224) organized for optimum use, performance and scale. The information that is written to the Adaptation Sub Segment cells is stored in five types of “usage maps”. There are distinct Adaptation Sub-segments for each map type. These usage maps are used to profile and re-profile the DIT, Object and Attribute Segments 220 to 224 and sub-segments. Each cell represents a snapshot of directory usage at a point in time, and the Adaptation Sub Segment contains historical maps that indicate how the directory has been used and adapted.

[0442] Adaptation sub-segments are created by the VDM 206 and are updated by the operations being performed on the directory by specific OVMs 208.

[0443] Adaptive Replication information is held within an Adaptation sub-segment as a series of distinct Map cells. This replication Map information is generated by the ARVM 218 and processed with other map information to determine the outcomes required of the adaptive replication process. Adaptive Replication can be performed when the local database 126 is entitled to copy information from remote databases 128 to 132 if there is a local demand for such information.

Segment Adaptation Modes

[0444] The adaptation modes for the Segments are:

Attribute Segments 220:

• • [0445] (i) Basic;
  • [0446] (ii) Attribute Type Splitting;
  • [0447] (iii) Object Class Splitting;
  • [0448] (iv) Complex/Distinct Attribute Splitting;
  • [0449] (v) DIT Splitting, and
  • [0450] (vi) Access Control Role Id Splitting (by Attribute Type).
    DIT Segments 224:
  • [0451] (i) Basic;
  • [0452] (ii) DIT Organisation Splitting;
  • [0453] (iii) DIT Access Control Rule Splitting; and
  • [0454] (iv) DIT Flat Space Splitting.
    Object Segments 222:
  • [0455] (i) Basic;
  • [0456] (ii) Name Binding Splitting;
  • [0457] (iii) Access Control Role Id Splitting (by name space); and
  • [0458] (iv) Multi Object Profile Splitting.
    Segment Adaptation Processing

[0459] The segment adaptation process is executed by the VDM 206 when the directory system 100 starts up and initialises its Attribute, DIT and Object Segments from backing store and using its configuration files. At this time the VDM 206 adapts the Segments based on the configuration, the state of the existing segments and directory entry information available on start up. The segment adaptation process is also executed after startup, when a specific threshold is reached, or at a periodic interval specified in the configuration files.

Attribute Segment and Sub Segment Adaptation

[0460] As shown in FIG. 7, the Attribute segment adaptation process determines the number of attribute instances in each Attribute Sub Segment at step 702, and if the configured threshold value is exceeded, attribute type splitting is performed at step 706 by creating one or more new attribute sub-segments for the same attribute type and dividing the attribute instances between the original sub-segment and the new sub-segment(s). Alternatively, attribute type splitting can be initiated by when the search times for one or more attribute types exceed a threshold value configured by an administrator.

[0461] The next steps are performed if the number of entries held in an Attribute Segment (determined at step 708) has reached a configured threshold at step 710.

[0462] If so, then if it is determined at step 712 that directory service operations are targeting specific object classes (by monitoring directory operation statistics), then object class splitting is performed at step 714 by moving the targeted object class instances and their sets of attributes) into one or more new attribute Segments.

[0463] Otherwise, if it is determined at step 716 that directory service operations are targeting specific attribute types within object classes (by monitoring directory operation statistics), then Complex/Distinct Attribute Splitting is performed at step 718 by moving the targeted attribute types within the targeted object classes into one or more new attribute Segments.

[0464] Otherwise, if it is determined at step 720 that directory service operations are targeting specific DIT portions (by monitoring directory operation statistics), then DIT Splitting is performed at step 722 by moving the corresponding attribute Segments or Sub-segments into a new attribute Segment and creating a new DIT segment reference for the new attribute Segment.

[0465] The attribute segment adaptation process [correct?] yes is different to the other adaptation processes in that it takes all the distinguished attributes of the same type (the entry naming attributes) from one attribute sub segment based on their DIT context prefix (their namespace), and moves some or all of these into one or more new sub segments.

[0466] Returning to step 720, if it is not the case that specific parts of the namespace are being targeted, then if, at step 724, the VDM 206 determines that the entry ACI (the Role Id) validation process for specific attribute types only needs a limited set from the complete set, then the ACI check can be optimised by storing the Role Ids that relate to a particular attribute type with the attribute sub-segment(s) for that attribute type. This Access Control Role Id Splitting (Attribute Type) is performed at step 726.

[0467] As shown in FIG. 8, the DIT adaptation process begins at step 802 by determining the number of entries there are in a DIT segment at each level. If it is determined at step 804 that at least one of the pre-configured maximum values for depth and entry count is exceeded, then a new DIT sub-segment is created at step 806, and the entries are divided between the new and old DIT sub-segments at step 808.

[0468] For DIT Access Control splitting, at step 810 the VDM 206 monitors the access control Role Ids assigned to users and the access control Role Ids assigned to DIT Sub Segments. In large scale directories, DITs (specific name spaces) are generally designed and protected for particular functions such as user authorisation, vCard management, network service configurations, etc. In terms of navigation and access control processing performance, a smaller number of macro level access control checks can be performed more rapidly than fine-grained access control checks. Consequently, where particular DIT Segments have very specific access control profiles for a limited set of user access control profiles, then performance can be improved by assigning distinct DIT segments to such Role Ids. Access Control Role Ids in DIT Sub Segment cells provide the primary point of protection and permission on that DIT. Accordingly, if it is determined at step 812 that a subset of Role Ids is associated with a subset of the directory namespace, then at step 814 a new DIT segment is created for the namespace subset, and the subset of Role Ids is stored in the new DIT Sub segments within that Segment. This completes the DIT adaptation process.

[0469] DIT Organisation Splitting and DIT Access Control Splitting modes of adaptation are not mutually exclusive. Combinations of these adaptation situations can exist within the directory.

[0470] For DIT Flat Space splitting, the same steps in 802 to 804 are performed. The VDM 206 evaluates the number of entries under a DIT Sub Segment at step 802. If this number exceeds a pre-configured threshold value at step 804, new DIT Sub Segments are formed at step 806 with the same DIT Context Prefix and DIT Sub Segment header sub-segment group—the sequence Id in the DIT sub-segment header is set to indicate that this DIT Sub Segment belongs to a group for the same name space.

[0471] The Object Segment adaptation process creates new Object Segments and Sub Segments. New Object Segments are created in the adaptation process to hold schema, name binding rules and access control Sub Segments specific to one or more DIT segments. If a new Object Segment already exists because of earlier adaption, then only (one or more) Sub Segments are created.

[0472] As shown in FIG. 9, the object segment adaptation process begins at step 902 by determining whether a limited subset of schema and name binding rules are used when updating any of the DIT sub-segments. If so, then a new object segment and its sub-segments are created for the selected DIT sub-segment at step 904, and the subset of schema and name binding rules identified at step 902 is stored in the new Object segment/sub segments. This improves performance of the directory system by avoiding the need to process irrelevant schema and name binding rules when updating directory entries.

[0473] At step 906, the VDM 206, determine, at step 908, whether any Role Ids are only associated with a subset of the directory namespace. If so, then at step 910 one or more new Access Control Segments and/or Sub Segments are created for each namespace subset and the corresponding Role Ids are stored in the new segments and/or sub-segments.

[0474] At step 912, the VDM 206 monitors the associations between Role Ids are attribute types. If it is determined at step 914 that one or more Role Ids are only being applied to one or more specific attribute types, then at step 916 these Role Ids are stored in the corresponding attribute Sub Segment(s) for those one or more specific attribute types.

[0475] Where an Object Segment contains Multi Object update profiles, related name bindings, schema and access control information are checked during update processes. It is more efficient to package these details into a new Object Segment so that the whole process for Multi Object update is self-contained. This Object Segment/Sub Segment creation process is performed at start up time from information contained in the existing Object Sub Segments, using steps analogous to steps 902 to 910.

Adaptation Modes, Configuration and Events

[0476] Table 4 describes each of the adaptation modes listed above in the context of the database being in the basic mode of operation with pre-configured parameters, an adaptation event and an exit state (being the adapted mode produced by the segment adaptation process).

[0477]

TABLE 4
	Parameters Pre-Configured or
Segment Mode	generated by the Segment
on Exit	Adaptation VM	Event and Outcome
Attribute
Segments 220
Attribute Type	Maximum number of attribute	When the threshold is reached,
Splitting	cells in an attribute Sub Segment.	create new Attribute Sub Segment
	The attribute segment adaptation	(step 706). If an Attribute Sub
	process determines that number	Segment becomes empty (all
	of attribute cells in an attribute	attributes are deleted), remove the
	sub-segment (steps 702 to 704).	sub segment.
Object Class	Maximum Object Class counts	When the number of instances of
Splitting	per DIT namespace e.g.:	an object class reaches the
	1. Person 10,000,	threshold for that object class (and
	2. Device 10,000,	extra cpu resources are available),
	3. Document 10,000	a new Attribute Segment is created
	It has been determined by the	for that object class (steps 704,
	segment adaptation process that	706) and the old Attribute
	the number of distinct of object	Segment stores the other object
	classes and/or entry counts has	classes that have not yet reached
	reached a threshold value.	their thresholds. In the new
	Note: Where it is realised that	Attribute Segment, the Attribute
	performance can be improved by	Segment Id is marked to indicate
	initially starting with different	is the segment is adapted for this
	object classes (and their	object class. If all the instances of
	attributes) in different Attribute	this object class are subsequently
	Segments, the parameters above	deleted from this Attribute
	should be set to 1.	Segment, the Segment is deleted.
Complex/	This adaptation occurs based on	When the number of object class
Distinct	the number of Object Classes (of	instances that contain distinct
Attribute	the same type) in the system or a	attribute sets (e.g., basic and
Splitting	portion of the DIT where the	complex attributes) reaches the
	operations on such objects are	threshold for that object class (and
	biased toward particular attributes	extra cpu resources are available),
	of that object class (the common	a new Attribute Segment is created
	or compound attributes).	for the more frequently accessed
	Attributes Type sets within an	attribute types of that object class
	Object Class:	and marked as adapted. All the
	Certificate Authority {	existing attribute sub segments of
	Attr Segment 1 {	that type (and its component
	Name	attributes - as marked in the
	Organizational Attribute Set }	Attribute Sub Segments) are
	Attr Segment 2 {	moved from the original Attribute
	Certificate(s)	Segment to the new one. Any
	CRL(s)	subsequent search for (e.g.)
	ARLs (s) } }	attributes within a certificate is
	The attribute segment adaptation	directed to this Attribute Segment.
	process determines that searches	If all the instances of these
	are occurring on specific attribute	attribute types are deleted from
	types that can be either complex	this new Attribute Segment, the
	and compound attribute types or	Segment is deleted.
	simple attribute types (step 716).
	Note: different attribute syntaxes
	require different search
	evaluation processes. For
	instance, a process to evaluate the
	values for equivalence in a phone
	number attribute is much simpler
	than the process that has to
	evaluate the complex name forms
	in member lists or the values
	embedded within a X.509
	certificate.
DIT Splitting	Maximum number of Accesses	When the number of searches per
	per period over a given name	period on a name space reaches
	space; adaptation for example:	the threshold for that name space
	DIT {c = us, o = govt, ou = dod}	(steps 708 and 710), (and extra cpu
	10,000 per minute.	resources are available), a new
	Attr Segment 1 {	Attribute Segment is created for a
	c = us, o = govt, ou = dod, ou = army}	portion of the DIT (e.g., {c = us,
	Attr Segment 2 {	o = govt, ou = dod, ou = army}). All
	c = us, o = govt, ou = dod, ou = navy}	the existing attribute sub segments
	}	are scanned for their Prefix Ids and
	The attribute segment adaptation	where they match the split DIT
	process determines that a DIT	prefix, they are copied to the new
	with one Attribute Segment is	Attribute Segment and Sub
	being accessed at high rates in a	Segments. The respective DIT Sub
	random manner and the results	Segments are then updated to point
	are derived from dispersed entries	to the new Attribute Segment. If
	that span the namespace This is	all the entries are deleted from this
	determined from the Search	Attribute Segment the Segment is
	parameters (Base Object (Name))	also deleted.
	provided in a User's request that
	indicates where in the DIT to start
	a Search from. If two or more
	distinct Base Object (Names) are
	applied by directory users, then
	two or more distinct Attribute
	Segments can be created for them
	(and assigned to distinct DIT Sub
	Segments (And spare cpu
	resources are available.)
Access Control	The object segment adaptation	The ACI check can be optimised
Role Id Splitting	process determines (at step 916)	by storing the Role Ids that relate
(by attribute	that the entry ACI (the Role Id)	to a particular attribute type with
type)	validation process for specific	the attribute sub-segment(s) for
	attribute types only needs a	that attribute type (step 918).
	limited set from the complete set.
DIT Segments
224
DIT	DIT Segment assignment has	When the entry count and DIT
Organisation	been pre-configured for different	navigation depth of the DIT
Splitting	namespaces e.g.:	Segment has reach its boundaries,
	DIT Seg 1: c = us, o = govt,	the DIT Segment is split into two
	ou = dod, ou = army	or more DIT Segments.
	DIT Seg 2: c = us, o = govt,	If all the entries are subsequently
	ou = dod, ou = navy	deleted from the new (or old) DIT
	The segment adaptation process	Segment the Segment is deleted.
	determines that large numbers of
	entries and high DIT levels exist
	in one DIT Segment and that
	navigation times are excessive.
DIT Access	Access Control Rules (the Role	When the DIT Sub Segments
Control Rule	Ids) have been pre configured for	within a DIT Segment relate to
Splitting	different namespaces under a DIT	distinct access control rules, and
	Segment, e.g.:	those DIT Sub Segments can be
	c = us, o = govt, ou = dod, ou = army	partitioned into discrete name
	Role Id: 00.03.2.32.00	space, then a new DIT Segment is
	c = us, o = govt, ou = dod, ou = navy	created (step 814). The access
	Role Id: 00.02.2.32.00}	control Role Id for that Segment is
	The DIT segment adaptation	stored in the Segment Header.
	process determines (steps 810,	If all the entries are subsequently
	812) that the Access Control	deleted from the new (or old) DIT
	Rules (the Role Ids) over a period	Segment, the Segment is deleted.
	of time (as Access Control Rules
	can be changed dynamically)
	have polarised onto different
	name spaces in the same DIT
	segment.
DIT Flat Space	Flat Name Space rules have been	Where a large flat name space
Splitting	pre configured. e.g.:	occurs, it can be accommodated by
	Flat Name Space Entries (see DIT	assigning multiple Attribute Sub
	Sub Segment): entry count	Segments for a particular attribute
	(10,000) Suffix ( “group1”)	type (e.g., naming attributes) or by
	The DIT segment adaptation	establishing multiple DIT Sub
	process determines that the	Segments for respective portions
	subordinate entry count under a	of the Flat Space. In the DIT Sub
	particular entry has exceeded the	Segment, the Flat Space Identifier
	lateral usability (The Flat Space	is set to the Flat Space portion
	Entries) (steps 802, 804), a new	number.
	DIT Sub Segment is created (step	If all the entries are subsequently
	806), and a subset of the entries	deleted from the new (or old) DIT
	moved to this new sub-segment	Sub Segment, the Sub Segment is
	(step 808).	also deleted.
Object
Segments 222
Name Binding	Where the Object Sub Segment(s)	Where a DIT has reached a stable
Splitting	for Name Bindings have been pre	state and only leaf entries are
	configured with DIT prefixes to	being added (ie, the DIT Context
	which they relate. e.g.:	Prefix has remained stable and
	Name Binding Sub Segment	typical leaf entries (Subscribers,
	Prefix = c = us, o = govt, ou = dod,	vCards, device configuration
	ou = army	objects, etc) are just being added,
	The object segment adaptation	modified or removed), then the
	process determines that the entry	name binding check is made more
	update validation process for a	efficient by creating a new Object
	specific DIT name space (that	Segment and/or Sub Segment for
	inspects name binding rules) only	the Name Bindings in that Context
	needs a limited set from the	Prefix (step 904) and storing the
	complete set (step 902). (e.g.,	appropriate subset of name
	once a DIT has been established	binding rules in the new object
	at the top level, then generally a	sub-segment (step 906).
	system populates that with person
	entries under an ou entry. It
	follows that a prefix as
	exemplified above only needs two
	name binding rules to check: ou
	to people, and ou to role.
Access Control	Where the Object Sub Segment(s)	Where a DIT has reached a stable
Role Id Splitting	for Access Control Rules have	state and only leaf entries are
(by namespace)	been pre-configured with DIT	being added, read and deleted,
	prefixes to which they relate, e.g.:	then the ACI check is optimised
	ACI Sub Segment Prefix = c = us,	by creating a new Object Segment
	o = govt, ou = dod, ou = army	and/or Sub Segment for the Role
	The object segment adaptation	Ids in that Context Prefix (step
	process determines (at step 910)	912).
	that the entry ACI (the Role Id)
	validation process for specific
	DIT name space (that inspects the
	ACI Role Ids) only needs a
	limited set from the complete set.
	(e.g., once a DIT has been
	established at the top level, then
	generally a specific application
	populates that, and specific users
	(roles) read that information.
Multi Object	Where an Object Segment	A new Object Segment is created
Profile Splitting	contains Multi Object update	with the schema, name bindings
	profiles, related name bindings,	and access control rules for the
	schema and access control	Multi Object update process.
	information are checked during
	startup. It is more efficient to
	package these details in one
	Object Segment so that the whole
	process for Multi Object update is
	self contained.
	Object Segments are configured
	normally.
	The object segment adaptation
	process determines that Multi
	Object updates will be performed.
	It constructs a separate Object
	Segment for this process.

Adaptation Maps

[0478] The segment adaptation process inspects configuration parameters, the Object, DIT and Attribute Segments 220 to 224, and the directory's Management Information Base (MIB) periodically to gain statistical information about numbers of operations, the duration of operations and their performance (each OVM 208 can record its activity and performance). From this information, the VDM 206 updates map information in the adaptation segment 228 at step 602, based on the existing map information and the nature of the changes, as described below.

[0479] The map information is stored in five distinct Adaptation Sub Segments or maps. Each cell associated with these Sub Segments provides a record (a history) of what the directory has been doing. There is a DIT adaptation map, an access control adaptation map, an entry adaptation map, a usage adaptation map, and an adaptive replication map. These five maps store parameters that are processed by the VDM 206 during an assessment phase that can be executed periodically or triggered by an event (such as the segment adaptation process recognising that the number of cells in an Attribute Sub Segment has reached the pre-configured threshold maximum number of cells to create a new Attribute Sub Segment Cell, as described above). At this point the segment adaptation process determines whether the database segment layouts can be adjusted and optimized. As described above, this processing takes into account (i) the DIT structure and population at various depths of the DIT; (ii) the access control decisions being placed on those structures; (iii) the types of entries and attributes those structures contain; and (iv) the read and update operations being performed on them.

[0480] In order to describe the adaptation processing in further detail, the contents of the five maps are described below.

[0481] The DIT Adaptation Map contains:

• • [0482] (i) Context prefixes; The Context Prefix “length” indicates depth of the DIT because it shows how many levels of names (RDNs) are in the Prefix. e.g., the context prefix {c=us, o=gov, ou=dod, ou=admin}indicates the DIT is 4 deep to its leaf entries and 5 deep with leaf entries;
  • [0483] (ii) The Spread of this DIT in terms of its subordinate entries; i.e., how many entries are stored under this prefix at respective levels. (e.g., L-1=10 entries, L-2=100 entries, L-3=300,000 entries.
  • [0484] (iii) The number of entries below this Context Prefix;
  • [0485] (iv) The Types of Object Classes below this Context Prefix; and
  • [0486] (v) The Date/Time this cell was created.

[0487] The segment adaptation process can use the DIT adaptation map as follows:

[0488] Where DITs are building in “spread” or “depth”, new DIT Segments and/or sub-segments can be established. Where DITs are growing in entry count, the segment adaptation process can instigate Flat Space management or the generation of multiple Attribute Sub Segments for the same attribute type.

[0489] The Access Control Adaptation Map contains:

• • [0490] (i) A Context Prefix within the database 126;
  • [0491] (ii) Access control sub-segment entries at each subordinate level to Context Prefix;
  • [0492] (iii) The Quantity of Role Ids below this Context Prefix; and
  • [0493] (iv) The Date/Time this cell was created.

[0494] Where DIT Sub Segments (and its subordinates by Prefix) have a discernable single Access Control Role Id, the segment adaptation process creates a new DIT Segment for that Context Prefix.

[0495] The Entry Adaptation Map contains:

• • [0496] (i) A Context Prefix within this database 126;
  • [0497] (ii) The number of subordinate access control sub-segment cells (Role Ids) used under this context prefix;
  • [0498] (iii) The number of different Attribute Types below this Context Prefix;
  • [0499] (iv) A list of the Attribute Types below this Context Prefix; and
  • [0500] (v) The Date/Time this cell was created.

[0501] Where DIT Prefixes are showing large numbers of attribute types, the segment adaptation process examines the object class structures that use such attributes. From this information, the segment adaptation process can create new attribute Segments based on Object Class splitting, Complex Attribute splitting, according to the Access Control rules in place.

[0502] The Usage Adaptation Map contains:

• • [0503] (i) A Context Prefix within this database;
  • [0504] (ii) The quantity (over time) of directory operations relating to Reads and Updates on its Attribute sub-segments;
  • [0505] (iii) The number of different Attribute Types below this Context Prefix; and
  • [0506] (iv) The Date/Time this cell was created.

[0507] Typically, a large scale ebusiness directory stores a very large number of attributes that are only used very occasionally. In this case high usage attributes are given higher precedence in the directory service processes. That is, where read and update operations are occurring on specific attributes (i.e., on a subset of Attribute Sub Segments), then the less often accessed attribute Sub Segments can be moved to another Attribute Segment. This allows OVMs 208 to access all their relevant attributes in one Attribute Segment, avoiding the need to inspect all Attribute Sub Segments (that were originally in that Segment) and for the VSM 202 to roll the less frequently accessed Attribute Segments onto backing store (i.e., the disk arrays 136) if memory resources are limited. The associated DIT Segment, Sub Segment and cell(s) for a portion of the DIT allow multiple Attribute Segments to be associated with a DIT Sub Segment. This enables parts of the objects in the DIT to be made “resident” in memory for immediate processing whilst other, less frequently accessed parts of those objects are stored on backing store.

[0508] The segment adaptation process examines the four maps (and their histories) as described above to determine whether the data stored in attribute 220, Directory Information Tree (DIT) 224 and/or object segments 222 and sub-segments needs to be redistributed or reallocated (thus changing the “shape and extent” of the modified segments).

[0509] The Adaptive Replication Map contains:

• • [0510] (i) A Context Prefix of the database being chained to (frequently);
  • [0511] (ii) The Spread of directory operations relating to Reads and Updates on its Attribute sub-segments;
  • [0512] (iii) The number of Entries and distinct Attribute Types below its Context prefix that are being accessed; and
  • [0513] (iv) The Date/Time this cell was created.

[0514] Where operations are being chained to other directory databases 128 to 132 and those entries are being passed back to the local directory system 100 for the local client 112, the replication adaptation process can create replica DIT and Attribute Segments (for read only access) through the ARVMs 218 of the remote directory systems 102 to 106. Where updates occur on a master, the remote ARVM 218 passes these to the replica.

Historical Map Processing.

[0515] Periodically, the maps are inspected to determine whether patterns of adaptation are occurring. Such patterns may reflect adaptation in particular directions, including:

• • [0516] (i) The directory is increasing in size re its numbers of entries;
  • [0517] (ii) The directory is increasing in size re its numbers of entries with the same object classes;
  • [0518] (iii) The same types of directory accesses are increasing;
  • [0519] (iv) The complexity and distribution of directory accesses are increasing;
  • [0520] (v) The directory increases in size and then decreases its size in rapid cycles; or
  • [0521] (vi) Particular attributes get accessed/modified at specific times of the day.

[0522] The outcomes of such analysis are used to choose a specific mode or modes of adaptation or to postpone or avoid adaptation altogether. For example, flat space adaptation will occur if the number of entries under one DIT prefix has increased above the pre configured threshold. On the other hand, adaptation is postponed if user access rates are increasing and becoming more complex, or if the directory size (the number of entries) increases and decreases in cycles over short periods.

[0523] In general, alternative embodiments can be devised in which the adaptation processes can be implemented, applied and used in stages. The adaptive replication VM 218, and the VDM 206 can adapt the Attribute Segment(s) 220, the DIT Segments 224 and the Object Segments 222 individually or collectively. The modes of adaptation that can be applied to these directory service segments 220 to 224 are identified in Table 5 below.

[0524]

TABLE 5
Attribute Segment	DIT Segment	Object Segment
Basic	Basic	Basic
Attribute Type Splitting	DIT Organization	Name Binding Splitting
	Splitting
Object Class Splitting	DIT Access Control	Access Control Role Id
	Rule Splitting	Splitting (name space)
Complex/Distinct	DIT Flat Space	Multi Object Profile
Attribute Splitting	Splitting	Splitting
DIT Splitting
Access Control Role Id
Splitting (Attribute
Type)

[0525] Each Adaptation Segment Header contains management information for the complete segment, as follows:

• • [0526] (i) The segment identifier. (e.g. Adaptation Segment NNNN or an OID)
  • [0527] (ii) The maximum size of the segment.
  • [0528] (iii) The number of sub-segments being managed in this segment.
  • [0529] (iv) The DIT Segment Ids that this Adaptation Segment 228 relates to (may be more than one entry).
  • [0530] (v) The DIT sub-segment Ids that this Adaptation Segment 228 relates to.
  • [0531] (vi) The Attribute Segment Id that this Adaptation Segment 228 relates to (may be more than one entry).
  • [0532] (vii) The Attribute sub-segment Ids that this Adaptation Segment 228 relates to.
  • [0533] (viii) The Object Segment Ids that this Adaptation Segment 228 relates to (may be more than one entry).
  • [0534] (ix) The Object sub-segment Ids that this Adaptation Segment 228 relates to
  • [0535] (x) The Date/Time this Segment was created.
  • [0536] (xi) The Date/Time this Segment was used for Adaptation.
  • [0537] (xii) The mode of this Adaptation Segment (Master or Replica).
  • [0538] (xiii) Exception Event Processor (for errors and recovery).
  • [0539] (xiv) The existence or residence (file state) of the segment (being rolled in or out by VSM 202).
  • [0540] (xv) A file name for archive when the complete segment is archived.
  • [0541] (xvi) A file name for restore when the complete segment is being restored.

[0542] Each Adaptation Segment Sub Header contains management information for a particular aspect of the Adaptation being processed, assessed and actioned. The Adaptation sub-segment Header contents are:

• • [0543] (i) The sub-segment type identifier (Map types or History types).
  • [0544] (ii) A Group Sequence Identifier (Map Set Id).
  • [0545] (iii) The Context Prefix of this DIT sub segment (including the external Context Prefix).
  • [0546] (iv) The Context Prefix Identifier as an object identifier.
  • [0547] (v) Constraint Parameters, the constraints in which re adjustments can take place.
  • [0548] (vi) The existence or residence (file state) of the segment (being rolled in or out by VSM 202).
  • [0549] (vii) A file name for archive when the complete segment is archived.
  • [0550] (viii) A file name for restore when the complete segment is being restored.

[0551] There are one or more Adaptation Segment Cells in each Adaptation sub-segment. They are used to record the History and Maps associated with the adaptation processes. The Adaptation sub-segment Cell contents are:

• • [0552] (i) The Adaptation Cell Id—a sequence number that is managed in the context of its Adaptation sub-segment; and
  • [0553] (ii) The Adaptation Map cell parameters for the corresponding Map type, as described above.
    Directory Service Operations Virtual Machines

[0554] As described above, the Directory Service Operation Virtual Machines (OVMs) 208 provide standard directory service operations by interfacing with the virtual memory segments of the database 126. Each of the nine OVMs 208 is described below.

[0555] The OVM-Read Entry VM performs the LDAP/X.500 Read Entry (or Single Entry Search) service that selects all or a selected set of attributes from an entry based on an Entry Information Selection (EIS) specification. This process is performed as follows:

• • [0556] (i) Perform database VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0557] (ii) If Namespace requirements indicate one or more external databases then initiate distributed operations via the DVM 214.
  • [0558] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to read an entry or its attributes in this part of the DIT.
  • [0559] (iv) Navigate to the target entry by taking these steps:
    • [0560] (a) If there is more than one DIT Segment entry, then select the DIT Segment where the prefix matches the higher order components of the DN of the entry to be read. Extract the DIT Segment Context Prefix Id.
    • [0561] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same RDN components of the DN to that of the Object Name or is the same except for the last RDN (as this will indicate a leaf entry is being Read).
    • [0562] (c) Extract from the DIT cell the pointers to the Attribute Segment (s) for this context prefix and initiate a VSM transfer if not all the Attribute Segments/types are in memory.
    • [0563] (d) Extract the Attribute Segment cell pointers for the distinguished attributes of this new entry's RDN types. Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0564] (e) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of the Read entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name).
    • [0565] (f) If Alias (name) is detected in Attribute sub-segment and de-referencing is permitted, repeat navigation process, discarding duplicate target entry names.
  • [0566] (v) Access the other Attribute sub-segment cells for all attributes or the types of attributes specified in the EIS that have the same Context Prefix Id and RDN Id.
  • [0567] (vi) Access the Attribute Segment and sub-segments to see if collective attributes exist for this entry.
  • [0568] (vii) Send the attributes and positive response to directory User.

[0569] The OVM-Compare VM performs the LDAP/X.500 Compare service that tests all or a selected set of attributes from an entry, using a Purported Attribute Value Assertion (P-AVA) specification. This process is performed as follows:

• • [0570] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0571] (ii) If Namespace requirements indicate one or more external databases 126 to 130 then initiate distributed operations via DVM. 214
  • [0572] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to read/compare an entry or its attributes in this part of the DIT.
  • [0573] (iv) Navigate to the target entry by taking these steps:
    • [0574] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the entry to be read. Extract the DIT Segment Context Prefix Id.
    • [0575] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same RDN components of the DN to that of the Object Name or is the same except for the last RDN (as this will indicate a leaf entry is being Compared).
    • [0576] (c) Extract from the DIT cell the pointers to the Attribute Segment for this context prefix and initiate a VSM transfer if not all the Attribute Segments/types are in memory.
    • [0577] (d) Extract the Attribute Segment cell pointers for the distinguished attributes of this new entry's RDN types. Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0578] (e) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of the Compare entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name).
    • [0579] (f) If Alias (name) is detected in Attribute sub-segment and dereferencing is permitted, repeat navigation process, discarding duplicate target entry names.
  • [0580] (v) Access the other Attribute sub-segment cells for all attributes or the types of attributes specified in the P-AVA that has the same Context Prefix Id and RDN Id.
  • [0581] (vi) Test these attribute types/values against the P-AVA (e.g., determine whether a telephone number is present, or whether a telephone number is equal to 03 9123 4567, etc.)
  • [0582] (vii) If all attributes are not tested, Access the Attribute Segment and sub-segments to see if collective attributes exist for this entry. Apply tests if so.
  • [0583] (viii) Send a positive or negative response to directory User accordingly.

[0584] The OVM-Search VM performs the LDAP/X.500 Search (one or more multiple entries) service that selects all or a selected set of attributes from entries based on a Filter specification. This process is the most complex of operations on a directory in that a Filter Specification can be nested with logical operators, (ANDs, ORs or NOTs) and as well they may be multiple alias entries that point to a single entry. This process is performed as follows. (For brevity, details that are not specific to the architecture of the directory system 100 have been omitted.)

• • [0585] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0586] (ii) If Namespace requirements indicate one or more external database 102 to 106 then initiate distributed operations via DVM 214.
  • [0587] (iii) Test for presence of specialised Filter—Matching Rule Identifiers of:
    • [0588] (a) Validate—call CVM-Validate Certificate (CVM-VC)
    • [0589] (b) Relational—call CVM-Relational Searching (CVM-RS)
    • [0590] (c) Authorise—call CVM-Service Authorization (CVM-SA)
    • [0591] (d) Statistics—call CVM-Statistical Evaluation (CVM-SE)
    • [0592] (e) MailBox—call CVM-MailBox (CVM-MB)
    • [0593] (f) MessagePad—call CVM-MessagePad (CVM-MP)
    • [0594] (g) AddressBook—call CVM-AddressBook (CVM-AB)
  • [0595] (iv) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to search entries and their attributes in this part of the DIT.
  • [0596] (v) Navigate to the base object entry by taking these steps:
    • [0597] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the entry to be read. Extract the DIT Segment Context Prefix Id.
    • [0598] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same as RDN components of the DN to that of the Base Object DN or is the same except for the last RDN (as this will indicate a leaf entry is being Searched).
    • [0599] (c) Select from the DIT Sub Segment cells the Attribute Segments assigned to this DIT.
    • [0600] (d) Check the EIS and filter attribute types against the Attribute Segments (and their types) and initiate a VSM transfer if not all Attribute types are present in memory.
    • [0601] (c) Using the Search Subset parameter value (one level, whole subtree) select from the DIT sub-segment entries the Context Prefix Ids where the context prefix has the more RDN components to that of those matched the Base Object DN (as this will enable subtrees of the Base Object to be Searched).
    • [0602] (d) Where the Base Object DN indicates a single leaf object Search, extract the Attribute Segment cell pointers for the distinguished attributes of this entry's RDN types. Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0603] (e) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of the Read entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name).
    • [0604] (f) If Alias (name) is detected in Attribute sub-segment and dereferencing is permitted, repeat navigation process—discarding duplicate target entry names.
  • [0605] (vi) Form a list of Context Prefix Ids and Attribute sub-segments that contain the attribute types as specified by the Filter and EIS parameters.
  • [0606] (vii) Initiate Search Sub processing (multiple threads) for each Attribute Sub Segment if multiple processors are available (or initiate Attribute processing by one or more dedicated hardware components, as described below, if present).
  • [0607] (viii) Access the required Attribute sub-segment cells for all attributes or the types of attributes specified in the Filter and EIS that have the same Context Prefix Id
  • [0608] (ix) If all attributes requested are not satisfied, Access the Attribute Segment and sub-segments to see if collective attributes exist for this entry. Apply Search tests if so.
  • [0609] (x) Send the attributes and positive response to directory User.

[0610] The OVM-List VM performs the LDAP/X.500 List (or Single Level Search) service that selects the subordinate naming attributes of an entry. This process is performed as follows:

• • [0611] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0612] (ii) If Namespace requirements indicate one or more external databases 126 to 130 then initiate distributed operations via DVM. 214
  • [0613] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to list an entry's naming attributes in this part of the DIT.
  • [0614] (iv) Navigate to the target entry by taking these steps:
    • [0615] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the entry to be read. Extract the DIT Segment Context Prefix Id.
    • [0616] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same as RDN components of the DN to that of the Object Name or is the same except for the last RDN (as this will indicate a leaf entry is being Listed).
    • [0617] (c) If the List is being performed on a leaf entry, exit and return a negative response.
    • [0618] (d) Extract from the DIT cell the pointers to the Attribute Segment for this context prefix and initiate a VSM transfer if not all the Attribute Segments/types are in memory.
    • [0619] (e) Extract the Attribute Segment cell pointers for the distinguished attributes of this new entry's RDN types. Examine the Attribute sub-segments for “distinguished” attribute sub-segments.
    • [0620] (f) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of the List entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name).
  • [0621] (v) Access the other distinguished Attribute sub-segment cells for all attributes that have the same Context Prefix Id.
  • [0622] (vi) Send the attributes and positive response to directory User.

[0623] The OVM-Add Entry performs the LDAP/X.500 Add Entry service of adding a leaf entry to the directory structure. This process is performed by this OVM on the database 126 as follows. (For brevity, details that are not specific to the architecture of the directory system 100 have been omitted.)

• • [0624] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0625] (ii) If Namespace requirements indicate one or more external databases 126 to 130 then initiate distributed operations via DVM 214.
  • [0626] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to add an entry in this part of the DIT.
  • [0627] (iv) Test the Object Class and Attribute Types being added using the Object Segment information (OC-Tests).
  • [0628] (v) Test the DIT structure rules using the Object Segment information (DIT-Tests).
  • [0629] (vi) Write the Add Entry to a Transaction sub-segment and cell with date/time.
  • [0630] (vii) Navigate to the parent entry by taking these steps:
    • [0631] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the new entry. Extract the DIT Segment Context Prefix Id.
    • [0632] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same RDN components of the DN to that of the Object Name or is the same except for the last RDN (as this will indicate a leaf entry is being Added to a leaf entry).
    • [0633] (c) Extract from the DIT cell the pointers to the Attribute Segment for this Context Prefix and initiate a VSM transfer if not all the Attribute Segments/types are in memory
    • [0634] (d) Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0635] (e) If the DN—Context Prefix match determined the last two RDNs did not exist in the DIT sub-segment, then a new leaf entry is being added to an existing leaf entry. Set “Add First Leaf Entry”
    • [0636] (f) If “Add First Leaf Entry” is not set, test the Attribute sub-segment cells for the distinguished attribute types/values to be that of this entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name/duplication).
    • [0637] (g) If “Add First Leaf Entry” is set, test the Attribute sub-segment cells for the distinguished attribute types/values to be that of this entry's (RDN minus 1)-types/values. (test for existence of the parent entry).
    • [0638] (h) If Alias (name) is detected in Attribute sub-segment and dereferencing is permitted, update the DN to the target.
  • [0639] (viii) Extract the last RDN types and values of the entry to be added.
  • [0640] (ix) If “Add First Leaf Entry” is set, add a new DIT sub-segment and Cell to the DIT Segment with the new Context Prefix and Context Prefix Id—(generally this will use the same Attribute Segment(s) as the parent entry. Set the RDN Id to 1 in this sub-segment.
  • [0641] (x) Add the distinguished attributes to the distinguished Attribute sub-segment cells. Set Context Prefix Id and the RDN Id and Control Flags accordingly.
  • [0642] (xi) Add the other attributes as cells to the Attribute sub-segments for this type. Set Context Prefix Id and RDN Id and Control Flags in these cells accordingly.
  • [0643] (xii) Add the operational attributes as cells to the Attribute sub-segments for these types. Set Context Prefix Id and RDN Id and Control Flags in these cells accordingly
  • [0644] (xiii) (RDN Id is calculated from the RDN Id field in the DIT sub-segment)
  • [0645] (xiv) If this new entry creates a parent entry without a DIT sub-segment entry update the DIT Segment with the parents Context Prefix.
  • [0646] (xv) If Integrity mode equals Low Integrity Mode—Remove Add Entry from the Transaction Segment cell. Send positive response to directory User.
  • [0647] (xvi) If Integrity mode equals High Integrity Mode—Initiate disk update. Set Transaction RIRO state to RO. Await termination
  • [0648] (xvii) On termination clear Transaction Segment and cell entry. Send positive response to directory User.

[0649] (xviii) If the Segments being updated indicate they belong to a master database and the Object sub-segment Knowledge Information indicates that replicas of the same Context Prefix exist, schedule replication (incremental or batch) through the RVM 216 and the Transaction Segment.

[0650] Note: in the event of power failure the Transaction sub-segments are examined and recovery action taken if required.

[0651] The OVM-Modify RDN VM performs the LDAP/X.500 Modify RDN service of changing a leaf entry's name. This process is performed by this OVM on the database 126 as follows. (For brevity, details that are not specific to the architecture of the directory system 100 have been omitted.)

• • [0652] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0653] (ii) If Namespace requirements indicate one or more external databases 126 to 130 then initiate distributed operations via the DVM 214.
  • [0654] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to modify the name of an entry in this part of the DIT.
  • [0655] (iv) Test the DIT structure rules Name Bindings using the Object Segment information (DIT-Tests) to see if the new name form is appropriate.
  • [0656] (v) Write the Modify RDN to a Transaction sub-segment and cell with date/time.
  • [0657] (vi) Navigate to the parent entry by taking these steps:
    • [0658] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the new entry. Extract the DIT Segment Context Prefix Id.
    • [0659] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix is the same except for the last RDN (as this will indicate a leaf entry is having its name modified).
    • [0660] (c) Extract from the DIT cell the pointers to the Attribute Segment for this Context Prefix and initiate a VSM transfer if not all the Attribute Segments/Types are in memory.
    • [0661] (d) Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0662] (e) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of requested entry's RDN types/values (test for the existence of the original name).
    • [0663] (f) If an Alias (name) is detected in Attribute sub-segment and de-referencing is permitted, update the DN to the target.
    • [0664] (g) Test the Attribute sub-segment cells distinguished attribute types/values against that of the entry's new RDN types/values—in case an attempt is being made to change to the name to that of an existing entry.
  • [0665] (vii) Add the new distinguished attributes to the distinguished Attribute sub-segment cells. Set Context Prefix Id and the RDN Id and Control Flags accordingly.
  • [0666] (viii) If Integrity mode equals Low Integrity Mode—Remove Add Entry from the Transaction Segment cell. Send positive response to directory User.
  • [0667] (ix) If Integrity mode equals High Integrity Mode—Initiate disk update. Set Transaction RIRO state to RO. Await termination.
  • [0668] (x) On termination clear Transaction Segment and cell entry. Send positive response to directory User.
  • [0669] (xi) If the Segments being updated indicate they belong to a master database 126 and the Object sub-segment Knowledge Information indicates that replicas of the same Context Prefix exist, schedule replication (incremental or batch) through the RVM 216 and the Transaction Segment.

[0670] Note: in the event of power failure the Transaction sub-segments are examined and recovery action taken if required.

[0671] The OVM-Modify Entry VM performs the LDAP/X.500 Modify Entry service of modifying the attributes or their values of an entry. This process is performed by this OVM on the database 126 as follows. (For brevity, details that are not specific to the architecture of the directory system 100 have been omitted.)

• • [0672] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0673] (ii) If Namespace requirements indicate one or more external databases 126 to 130 then initiate distributed operations via the DVM 214.
  • [0674] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to modify an entry in this part of the DIT.
  • [0675] (iv) Test the Object Class and Attribute Types being modified using the Object Segment information (OC-Tests).
  • [0676] (v) Write the Modify Entry to a Transaction Segment cell with date/time.
  • [0677] (vi) Navigate to the entry by taking these steps:
    • [0678] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the new entry. Extract the DIT Segment Context Prefix Id.
    • [0679] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same RDN components of the DN to that of the Object Name or is the same except for the last RDN (as this will indicate a leaf entry is being Modified).
    • [0680] (c) Extract from the DIT cell the pointers to the Attribute Segment for this context prefix and initiate a VSM transfer if not all the Attribute Segments/types are in memory.
    • [0681] (d) Extract the Attribute Segment Cell pointers for the distinguished attributes of this new entry's RDN types. Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0682] (e) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of the Read entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name).
    • [0683] (f) If Alias (name) is detected in Attribute sub-segment and dereferencing is permitted, update the DN to the target.
  • [0684] (vii) Extract the last RDN types and values of the entry to be added.
  • [0685] (viii) Access the Attribute sub-segment (non distinguished) cells.
  • [0686] (ix) Add the distinguished attributes to the distinguished Attribute sub-segment cells. Set Context Prefix Id and the RDN Id and Control Flags accordingly.
  • [0687] (x) Add the other attributes as cells to the Attribute sub-segments for this type. Set Context Prefix Id and RDN Id and Control Flags in these cells accordingly.
  • [0688] (xi) Add the operational attributes as cells to the Attribute sub-segments for these types. Set Context Prefix Id and RDN Id and Control Flags in these cells accordingly
  • [0689] (xii) (RDN Id is calculated from the RDN Id field in the DIT sub-segment)
  • [0690] (xiii) If this new entry creates a parent entry without a DIT sub-segment entry update the DIT Segment with the parents Context Prefix.
  • [0691] (xiv) If Integrity mode equals Low Integrity Mode—Remove Add Entry from the Transaction Segment cell. Send positive response to directory User.
  • [0692] (xv) If Integrity mode equals High Integrity Mode—Initiate disk update. Set Transaction RIRO state to RO. Await termination
  • [0693] (xvi) On termination clear Transaction sub-segment and cell entry. Send positive response to directory User.
  • [0694] (xvii) If the Segments being updated indicate they belong to a master database and the Object sub-segment Knowledge Information indicates that replicas of the same Context Prefix exist, schedule replication (incremental or batch) through the RVM 216 and the Transaction Segment.

[0695] Note: in the event of power failure the Transaction sub-segments are examined and recovery action taken if required.

[0696] The OVM-Remove Entry VM performs the LDAP/X.500 Remove Entry service that removes a leaf entry from the directory structure. This process is performed by this OVM on the database 126 as follows. (For brevity, details that are not specific to the architecture of the directory system 100 have been omitted.)

• • [0697] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the System and User capabilities test for this operation.
  • [0698] (ii) If Namespace requirements indicate one or more external databases 126 to 130 then initiate distributed operations via DVM 214.
  • [0699] (iii) Test the access controls Role Ids here and through the navigation process using the Object Segment AC cells information or the DIT Sub Segment Role Ids (AC-Tests) to see if this User is permitted to remove an entry in this part of the DIT.
  • [0700] (iv) Write the Remove Entry to a Transaction Segment cell with date/time.
  • [0701] (v) Navigate to the actual entry by taking these steps:
    • [0702] (a) If there is more than one DIT Segment entry then select the DIT Segment where the prefix matches the higher order components of the DN of the new entry. Extract the DIT Segment Context Prefix Id.
    • [0703] (b) Select from the DIT sub-segment entries the Context Prefix Id where the context prefix has the same RDN components of the DN to that of the Object Name or is the same except for the last RDN (as this will indicate a leaf entry is being Removed).
    • [0704] (c) Extract from the DIT cell the pointers to the Attribute Segment for this context prefix and initiate a VSM transfer if not all the Attribute Segments/types are in memory.
    • [0705] (d) Examine the Attribute sub-segment for “distinguished” attribute sub-segments.
    • [0706] (e) Test the Attribute sub-segment cells for the distinguished attribute types/values to be that of this entry's RDN types/values. Extract from the cell the Context Prefix Id and the RDN Id. (test for existence of the name)
    • [0707] (f) If Alias (name) is detected in Attribute sub-segment and dereferencing is permitted, update the DN to the target.
  • [0708] (vi) If this entry has no sibling nodes copy its DIT sub-segment parent entry and its cells to a Transaction Segment cell and set “Last Leaf Entry”.
  • [0709] (vii) If this entry contains Active Attributes, copy the event(s) to a Transaction Segment cell.
  • [0710] (viii) If this entry has alias entries and the “alias integrity” feature is on, copy this action to a Transaction Segment cell.
  • [0711] (ix) Remove the DIT sub-segment and its cells for this entry
  • [0712] (x) Record the Context prefix Id and RDN Id then remove the distinguished attribute cells from the Attribute sub-segment
  • [0713] (xi) Where the Context Prefix Id and RDN Id match, remove the other attribute cells from the Attribute sub-segments of this type.
  • [0714] (xii) If Last Leaf Node set remove the DIT sub-segment of the parent entry.
  • [0715] (xiii) If Integrity mode equals Low Integrity Mode—Remove Add Entry from the Transaction Segment cell. Send positive response to directory User.
  • [0716] (xiv) If Integrity mode equals High Integrity Mode—Initiate disk update. Set Transaction RIRO state to RO. Await termination
  • [0717] (xv) On termination clear Transaction Segment and cell entry. Send positive response to directory User.

[0718] Note: in the event of power failure the Transaction sub-segments are examined and recovery action taken if required.

[0719] The OVM-Abandon VM performs the LDAP/X.500 Abandon (Search) service that stops outstanding operations. This process is performed by this OVM on the MRD as follows. (For brevity, details that are not specific to the architecture of the directory system 100 have been omitted.)

• • [0720] (i) Perform VM Operation type and Name space tests using the DIT Segment and the Object sub-segment Knowledge Information Cells. Perform the User capabilities test for this operation.
  • [0721] (ii) If Namespace requirements indicate one or more external databases 128 to 132 then initiate distributed operations via the DVM 214.
  • [0722] (iii) The Search Operation referenced if in progress is halted and its partial results are discarded.
  • [0723] (iv) Send a negative response to directory User for the Search—Abandoned.
  • [0724] (v) Send positive response to directory User for the Abandon.
    Distributed Directory Operations and Replicated Directory Operations

[0725] The directory system 100 supports the X.500 Distributed and Replicated modes of operation through the Distributed Directory Operation VMs (DVMs) 214 and the Replicated Directory Operation VMs (RVMs) 216.

[0726] Distributed Directory Operation VMs (DVMs) 214 provide the X.500 Directory System Protocol (DSP) service level operations on and between the databases 126 to 134. These operations are defined by the X.500 directory standards for DSP, namely X.518.

[0727] Distributed operations occur when a User asks the database 100 to perform operations (using OVMs 208) on namespace where that namespace is known to be outside the scope of the namespace managed by the local database 126. X.518 reflects the LDAP and X.511 (DAP) defined operations but additional parameters are added or modified such as base object and trace information in order that distributed operations can be processed correctly.

[0728] The OVMs 208 call the DVM 214 when detecting that the namespace of the request is external to the local database 126's namespace by testing the Context Prefix information in the DIT Segment(s) and the Knowledge Information in Object sub-segment cells. The DVM 214 sets in motion the construction of the DSP request and records that in a Transaction Segment 226 cell (as a DVM-TX transaction).

[0729] A single request from a User such as a Search or a List can be actioned across multiple databases 126 to 134 at the same time. Therefore if the operation is a Search or a List and it implicates multiple databases 126 to 134, the DVM 214 records this so that it can coordinate the asymmetric responses that it may receive from the local database 126 and the remote databases 128 to 132. Where multiple remote databases 128 to 132 are used, a Transaction Segment cell is created for each DSP request issued.

[0730] Where the operation is a Search or a List and it implicates multiple databases, the DVM 214 records this in a dedicated Transaction sub-segment, so that it can coordinate the asymmetric responses that it may receive from the local databases 126 and the remote databases 128 to 132. When a response is received its content is stored in the operation's Transaction sub-segment as a cell. Where this response is not the last response expected, the cell is flagged with “waiting group response”. When the last response is received the results are collated and the initiating OVM is called to send the response and terminate the entire operation.

[0731] Replicated Directory Operation VMs (RVMs) 216 provide the X.500 DISP (replication/shadowing) service level operations on and between databases 126 to 134. These operations are defined by the X.500 directory standards for Directory Information Shadowing Protocol (DISP), namely X.525.

[0732] Replication agreements defining the DIT area to shadow and the nature of the update (incremental or batch) are defined between directory servers on a bilateral basis.

[0733] The RVMs 216 perform the actual processing of the X.500 DISP directory operations and can operate in two modes:

• • [0734] (i) External DISP Mode—used where the interface to other parts of the directory service are using the X.500 DISP standards for replication; and
  • [0735] (ii) Segment Transfer Mode—used where the interface to other parts of the directory service can use segment-based information transfer formats for replication. These information transfer formats are as per the virtual storage Segment, Sub-segment and cell definitions described above. For example, this type of transfer is used where ARVMs 218 in each directory system 100 to 108 coordinate the transfer of DIT and Attribute Segments based on high statistics of local users continually accessing the remote systems 102 to 108 for entries.

[0736] The RVMs 216 can perform replication at the segment, sub-segment and cell levels. The RVMs 216 call OVMs 208 to perform local replication update operations if External DISP Mode is used. The RVMs 216 maintain replication integrity through the use of transaction segments 226.

[0737] Static replication is initiated when the scheduling of replication agreements is actioned or there is a change in the master information that is then transferred to the replica(s).

[0738] During update operations, the OVMs concerned test the modes of the Segments used and determine if they are set as masters. The Attribute, DIT and Object Segments have flags to indicate if they are masters or replicas. In the case of being a master, the Object Segment Knowledge Information is inspected for a database with the same prefix and its Mode set to “Replica”. If this is the case the RVM 216 is called to schedule the replication as a transaction using the Transaction Segment cells.

[0739] Adaptive replication is triggered by the ARVM 218 when it schedules its analysis of the Adaptation maps and finds that the Attribute, DIT and Object Segments should be copied to or from the other databases 126 to 130. Configuration parameters are used to control the clearing or refreshing of the entries created through the adaptive replication process.

Customised Virtual Machines—CVMS

[0740] Customised Virtual Machines (CVMs) 212 provide the following services:

• • [0741] (i) CVM—Statistical Evaluation (CVM-SE)
  • [0742] (ii) CVM—Change Monitoring (CVM-CM)
  • [0743] (iii) CVM—Validate Certificate (CVM-VC)
  • [0744] (iv) CVM—Collective Attributes (CVM-CA
  • [0745] (v) CVM—Multi-Object Management (CVM-MO)
  • [0746] (vi) CVM—Service Authorization (CVM-SA)
  • [0747] (vii) CVM—User Presence (CVM-UP)
  • [0748] (viii) CVM—Relational Searching (CVM-RS)
  • [0749] (ix) CVM—Directory Service Messaging (CVM-DM)
  • [0750] (x) CVM—Message Submission (CVM-EMS)
  • [0751] (xi) CVM—Message Management (CVM-EMM)
  • [0752] (xii) CVM—Message List expansion (CVM-EML)
  • [0753] (xiii) CVM—Message Retrieval (CVM-EMR)
  • [0754] (xiv) CVM—Address Book (CVM-EAB)
  • [0755] (xv) CVM—Mail Folder (CVM-EAF)
  • [0756] (xvi) CVM—Calendar/Diary (CVM-ECD)
  • [0757] (xvii) CVM—Message Gateway (CVM-EMG)
    CVM-Statistical Evaluation (CVM-SE)

[0758] The CVM-Statistical Evaluation (CVM-SE) provides statistical output of the directory entries in terms of their population, numbers, attribute types and keywords. This VM is initiated by any of the X.500/LDAP retrieval operations of Search that applies a Filter specification containing the entry types or attribute types/values to be counted and a Matching Rule Identifier equal to “Statistics”. This VM maintains DIT statistics such as cell and entry counts as stored in the Segments. Each operation in a OVM 208 records its statistics in the Directory Management Information Base as defined in RFC 1567 Directory Monitoring MIB. This includes information such as summary statistics on the directory accesses, operations, and errors, and summary statistics on the entries held by the directory service.

[0759] In addition the CVM-SE generates statistics indicating how the Object, DIT and Attribute Segments 220 to 224 are organized, their configuration and any adaptation maps that affected their configuration.

CVM-Change Monitoring (CVM-CM)

[0760] LDAP standards define a “Persistent” Search in that the User may register to be notified by the directory service if a specific entry/entries are updated. This VM is initiated by any of the X.500/LDAP Search operations that have their control attribute set for persistent Search.

CVM-Collective Attributes (CVM-CA)

[0761] Collective attributes are defined in X.500. They are defined as the common attributes that apply to a complete directory subtree that contains one or more directory information entries. For optimization, collective attributes are stored once at the “top of the subtree” to which they apply, but returned to the directory User as if they were stored within every entry or entries being returned. Collective attributes in the directory system 100 are explicitly flagged within an Attribute sub-segment. The CVM-CA is invoked when the Read, Compare or Search OVMs encounter a collective attribute in their DIT-Attribute Sub Segments.

CVM-Validation—Certificate Paths (CVM-VC)

[0762] X.509/PKI standards describe that a number of chained certificates can reflect a chain of trust from the User (the End Entity) to the Issuer of the certificate(s). Currently X.509 certificate paths are validated by the directory service application (email applications) that is external to the directory service by making a succession of LDAP Searches on the directory service. This makes the application of PKI very complex and results in significant network traffic and directory intensive Search operations.

[0763] For example, if there are three certificates in the trust path to verify a digital signature of a User and there are ten recipients of the signed message, then the LDAP directory traffic for validation will be 30 accesses or possibly even more if there are certificate revocation lists to be processed. In addition, a tremendous amount of data is transferred out of the directory to the end User and the message's recipient email applications. If the directory was trusted, a simple “Yes or No” about the validity of the certificate path is all that is required.

[0764] Directory services that support the USA standard SDN.705 (from the National Security Agency NSA—US) for strong authentication and signed operations of X.500 protocols, actually validate the certificate paths for the signatures on the protocols. In this case the directory is validating its own users signatures. It is not validating an external user signature for another external user.

[0765] The directory service itself is capable of providing this verification action internally when presented with the End Entity certificate. It instantiates this (trusted) VM and validates the certificate path accordingly. The CVM-VC is initiated by an X.500/LDAP Search operation or that contains a Filter specification containing the certificate to be validated and Matching Rule Identifier equal to “Validate”. The Search operation Filter may also contain the ROOT certificate upon which the certificate path will be tested. Where the ROOT certificate is not provided in the Search request, the Search response will return the ROOT certificate that the original End Entity certificate validated against. Where the validation process finds a CRL in the path that invalidates the certificate, the response will identify the CA's identity and indicate the revocation reason as provided in the CRL.

[0766] This certificate validation function can also be initiated through a directory service Online Certificate Protocol (OCSP) gateway in that an OCSP request will be asking the recipient if the certificate presented is valid. The directory system 100 can perform this function because currently OCSP servers will make a series of calls on the directory to read and validate the certificate chain.

[0767] Where the OCSP interface is incorporated, the directory system 100 will have to be trusted and capable of digitally signing its responses to the OCSP requests.

CVM-Multi-Object Management (CVM-MO)

[0768] The X.500/LDAP directory standards development process tried to address this issue. It did so by dictating “structure” around a set of objects that required such objects to be tightly connected by names or by using a superior object that “contained “subordinate” objects. This approach was seen as too simplistic and constrained to solve the real problem of managing related objects in a directory service that support on-line User to (multiple) Service relationships in an agile service environment.

[0769] The directory system 100 provides a User to Service object management (creation, modification and deletion service) when pre-configured with such details and rules. This CVM is controlled through directory object templates held in Object Segments Cells. These templates stipulate that if a specific User object is created, other objects that relate to that User such as mail boxes, presence management attributes, memberships, vcard information and service bundles are also created or modified across the DIT and their attributes populated with the predefined attribute values of that User. The primary object is referred to as the sponsoring object and its counterparts are referred to as the sponsored objects. In addition where such attributes are modified (the sponsoring attributes), its counterparts (sponsored attributes) are also modified. Where sponsoring attributes are updated after the User creation event, the sponsored ones are also updated.

[0770] The CVM-MO service supports an online, personalised User and their Service environment. This service can also be used for membership lists and group management functions. For example, a mail box, vCard, presence management object and some default service objects (email and IM) can be created automatically when a user object is created.

[0771] This VM is initiated by any of the X.500/LDAP update operations of Add, Remove, Modify or ModifyRDN that target the sponsoring object of an object set.

[0772] The CVM-MO requires that “sponsor and sponsored” objects and attributes are used within the directory service. The profiles for these objects are defined through the use of Object sub-segment cells of the US type.

CVM User Presence (CVM-UP)

[0773] When a User is authorised to access a directory (service authorisation), validated on the directory or an application authorises on behalf of the User, or the user is using the transactional messaging contexts of the directory, the directory updates the user's Presence object's User Operational attributes (e.g., On-line, Location, Device Type, Profile, Security Level, Inform List) in the Sub Segment that is assigned for this type of attribute to indicate that a User is present on the directory system 100 or its attached applications. In addition this update can be used to initiate events, and or to inform others of this event.

[0774] The CVM-UP writes the relevant attributes to the User's own entry User Operational attributes using the connect credentials (name and DN) as well as other attributes that can be derived from, the Users entry or the application which has permission to update the Users Operational attributes—for example. Location, Device Type, Security level.

[0775] This feature is ideal for Instant Messaging, and other mobile services. User Operational (Presence attributes) are defined as: On-line, Location, Device Type, Profile, Security Level, Inform List,

[0776] The nature of the object classes and attributes held in the directory will be aligned to that as defined in RFC 2778 and 2779, the RFCs for Presence Protocols and Instant Messaging. Such Object Classes will support the Presence information and its lodgement and broadcast and the Instant Messaging mail boxes. The directory system 100 also supports the objects and attributes that relate to the XML data defined in the XMPP standards as and when they are finalized.

[0777] The directory system 100 also supports an IM protocol gateway that converts the Presence and Instant Messaging protocols into Directory Updates or Reads.

CVM Service Authorization (CVM-SA)

[0778] The directory system 100 can be used to provide a service environment where users can access one or more on-line applications (email, IM, web, transactions), through one or more devices (phones, PCs, PDA, Kiosks and iDTV systems) using a one or more authentication and authorization tokens (passwords, coded cards, biometrics and signatures).

[0779] The directory system 100 provides a “service selection and profile” determination function if the User to Service (and authorization) environment is known. When a User is added to the directory, their service profiles can be added at the same time (as described above in relation to the Multi Object Management CVM) and, with this knowledge, the directory system 100 can perform composite authorization verification and service profile selection, all at the same time.

[0780] For instance, if a directory-enabled web application could “Search” a User's environment objects with a device type and authentication token (a name/password) as the multi object filter, and the response given (on success) was one or more permissible services for that User, there would be only a small number of directory interactions and only a small degree of explicit directory object knowledge required by the web based application.

[0781] The CVM-SA is initiated by an X.500/LDAP Search operation that contains a Filter specification containing the service requirements, the authorization token (to be validated), optional device properties, and a Matching Rule Identifier equal to “Authorise”.

CVM—Relational Searching (CVM-RS)

[0782] An object based directory is not well suited to manage relationships between objects that may exist in a single DIT or dispersed DITs. To overcome this problem, the CVM-RS provides searching facilities which are native to relational databases, but reengineered to suit the characteristics of X.500/LDAP directory operations. This provides a convergence of the two types of information management paradigms; that is, traditional RDBMS and object based X.500/LDAP directories.

[0783] A difficulty referred to as the split service provider problem occurs when a subscriber gets mobile phone services from company X and Internet services from Company Y. Company X and Y have agreed to interconnect their directory services but have dedicated entries for their customers. In prior art directory systems, if someone wants to find all the available details for the subscriber, then two or more directory operations are performed, and a directory result joining process is used to combine the results.

[0784] Where objects placed in the directory represent a user, the user's service profiles, address books, memberships, their presence and their own content, then the business level requirement for “a single customer view” is satisfied if all objects related to a user are seen collectively. This can only be achieved through relational searching features.

[0785] Relational Searching CVMs are activated when a directory User's search request contains filter items containing the Matching Rule filters and identifiers for relational searching.

[0786] Matching Rules are a standard X.500 mechanism for invoking specialised information matching policies. In this case they provide the capability for a directory User to examine the local and distributed databases 126 to 134 in sequence as one atomic operation, returning the relational results back to the directory User.

[0787] The following logical specification describes a convention for specifying a search-filter that permits relationships between objects/subtrees to be followed in a multi-step sequence without the directory User client having to perform iterative searching. This reduces both client application workload and network overhead and allows for optimization of the search-filter process in the directory system.

[0788] The Relational Searching CVMs use an X.500/LDAP directory search filter containing the following logical operators:

• • [0789] (i) Source—Specifies the attribute type and value that is obtained from the source subtree object used to populate the relational filter attribute used in the next search sequence.
  • [0790] (ii) Relate—Specifies the target object class and attribute Id on which the Source Value will be applied.
  • [0791] (iii) Base Object—Specifies the base DN (as per X.500) of a relational search filter. (Optional)
  • [0792] (iv) Scope—Specifies the scope of a relational search filter (as per X.500). (Optional)

[0793] For example, the following LDAP search filter finds all the users that are on line (present) in the locality of a particular user.:

Step 1

• [0794] (base DN=Subscriber name)
• [0795] Source=Locality, Location=“value” (of the respective subscriber)
• [0796] Relate=Location, Object Class=Subscriber (to other subscriber entries in step 2)
  Step 2
• [0797] (base DN=Subscriber DIT)
• [0798] Source=Subscriber Name (s) (where location value is the same as the initial subscriber)
• [0799] Relate=Subscriber Names, Object Class=Presence
  Step 3
• [0800] (base DN=Presence DIT)
• [0801] Source=Presence Status=TRUE (for the Subscriber names in the same location)
• [0802] Relate=Subscriber Names (return those names who are on line)

[0803] The Relational Searching VM is initiated by any of the X.500/LDAP Search operations that contain a Filter specification containing the service and authorization token to be validated and a Matching Rule Identifier equal to “Relational”.

USER—Service Management—Collective Functionality

[0804] The directory system 100 provides an identity-based service set with its CVM processing for:

• • [0805] (i) CVM—Multi-Object Management
  • [0806] (ii) CVM—Service Authorization
  • [0807] (iii) CVM—User Presence
  • [0808] (iv) CVM—Relational Searching
  • [0809] (v) CVM—Validation—Certificate Paths (CVM-VC)

[0810] These CVMs should be viewed as a collection of processes that enable (mobile) identity based eBusiness services to be implemented efficiently and with high integrity and flexibility, as follows:

• • [0811] (i) Multi Object management allows a User and their environment to be created and managed easily,
  • [0812] (ii) Service Authorization allows a User to rapidly enter the service environment according to access device types and security token usage,
  • [0813] (iii) User Presence allows the system to know of and profile the User on demand as well as inform other Users about their presence.
  • [0814] (iv) Distributed relational searching enables the Service managers to obtain the business level intelligence from the directory about their Users and services.
  • [0815] (v) Validation of Certificate Paths enables signed message recipients to validate the originator with one action (not three, four or five).
    Directory Service Messaging (DSM)

[0816] The directory system 100 incorporates transactional messaging functionality, including instant messaging, into its directory service infrastructure. Transactional messaging differs from traditional (non-transactional) messaging (e.g., standard email) in that the delivery of messages is transactional in an atomic, end-to-end sense, rather than a store-and-forward sense.

[0817] Existing computer desktop environments can provide email, instant messaging and directory services. All three are quite distinct in terms of their User interfaces, protocols and storage approaches (e.g., mail boxes and desktop folders). In local desktop environments, users can drag and drop items into folders and desktop objects. The directory system 100 provides an environment where these objects are represented in a large scale, external directory service in which items can be dragged and dropped explicitly using a mouse or implicitly by sending a “message”.

[0818] With a directory-based approach to messaging and transaction systems, a directory object is used as a mailbox that can be updated using a modify attribute command with a message that may contain a document (the attachment). In contrast with existing messaging systems that transfer messages on a store-and-forward basis, the directory system 100 provides message transfer and delivery processes that are transaction based and effectively instant. A property of users interacting with transactional or instant messaging is that a user's presence can be detected. Presence is assigned to directory operations that are associated with User authentication and the transmission and reception (accessing a mailbox or note pad object).

[0819] In the description below, the term “Directory Supported Messaging” (DSM) is used to describe traditional messaging systems such as SMTP, MIME and X.400 directory based messaging. This type of system is where the messaging functions (the store-and-forward components of the system) are seen as the primary elements and the directory service is provided to authenticate Users, support address books and message route configuration data.

[0820] The term Directory Service Messaging/Transactions (DSM/T) is used to describe the transactional and instant messaging processes used by the directory system 100.

[0821] DSM/T provide many advantages over existing DSM systems, including:

• • [0822] (i) Consistency in “drag and drop” operations for eBusiness Users.
  • [0823] (ii) Fewer communications and network protocols within and between the systems therefore reduced configuration effort and higher resultant throughputs
  • [0824] (iii) A simpler yet more effective infrastructure, which is name and identity based, distributed and object oriented.
  • [0825] (iv) A single point of log-on can protect a much wider set of integrated services and support presence.
  • [0826] (v) Consistency in terms of dealing with User privileges and access profiles over the system services and its information content such as address books and post boxes.
  • [0827] (vi) The replacement of the messaging store-and-forward concepts to on-line presence based transactional and instant messaging operations.
  • [0828] (vii) Less complexity in the User client software (thinner clients) and therefore less support costs.
  • [0829] (viii) A single mailbox concept that can have (typed) partitions for: documents, messages, faxes and voice, instant messages.
  • [0830] (ix) Recipient capability to bar message originators through the use of the directory service access controls and directory resident member lists.
  • [0831] (x) Directory entries that represent User groups, distribution lists, common security profiles and content evaluation rules.

[0832] In order to process email and messages within a directory service, Message Store Management (delete a message), Message Submission, Message List Expansion, Message Retrieval and Message Delivery functions are provided by message processing VMs of the customised VMs 212, as follows:

• • [0833] (i) A Message Submission VM, (CVM-EMS),
  • [0834] (ii) A Message Management VM, (CVM-EMM),
  • [0835] (iii) A Message List expansion VM, (CVM-EML), and
  • [0836] (iv) Message Retrieval VM, (CVM-EMR).

[0837] The directory operations performed by these VMs are shown in Table 6 below.

[0838]

TABLE 6
CVM
Operation	Directory Operation	Comment
Message	Modify Entry to add an	Entry Object Class is MailBox -
Sub-	attribute value of an	Attribute will be Message
mission	“OutBox”
Message	Modify Entry to remove an	Entry Object Class is MailBox -
Store Mgt	attribute value of an Inbox	Attribute will be Message + Id
Message	Read/Search Entry	Entry Object Class is MailBox -
Retrieval		Attribute will be Messages or
		Message + Id
		Similar to the POP3 approach of
		examining an Inbox.
Message	Modify Entry to add an	Entry Object Class is
List	attribute value (the	Distribution List - Attribute will
Expansion	message) to the Distribution	be Message
	List

[0839] Message Ancillary VMs are used to manage Address Books, customizing message folders (mail boxes) and managing Calendar/Diary functions, as follows:

• • [0840] (i) CVM-EAB (Address Book);
  • [0841] (ii) CVM-EMF (Mail Folder); and
  • [0842] (iii) CVM-ECD (Calendar/Diary).

[0843] The directory operations performed by these VMs are shown in Table 7 below.

[0844]

TABLE 7
	Directory
CVM Operation	Operation	Comment
Address Book	Modify Entry to	Each entry in the address
Mgt	add, modify or	book is a composite attribute
	remove attribute	containing the named entity's
	values	details
Mail Box Mgt	Modify mailbox	as per existing email systems
	naming attributes
Calendar/Diary	Modify Entry to	as per existing email systems
	add, modify or
	remove attribute
	values

[0845] Message Gateway VMs are used to interface the directory to existing messaging systems to enable transition and inter-working. Gateway VMs are used to provide HTTP, SOAP, SMTP, SMIME, and POP3/IMAP protocols/services.

[0846] For example, a POP3 Gateway VM acts as a client's “INBOX” system, and performs the directory actions listed in Table 8.

[0847]

TABLE 8
POP3
Command	Directory Action	Comment
STATus	Read Mail box stats	Lists number of message attributes
LIST	Read a message header
RETRieve	Reads a message entry
DELete	Deletes a message entry
NOOP	Nothing
RSET	Resets Delete Flags	clears the mail box delete message
		flags
QUIT	Updates the mailbox	according to STATE (previous
	or quits the session	commands) deletes the messages
		marked for deletion or
		disconnects from the gateway
TOP	Reads a message header
UIDL	Reads a message with a
	Unique Id
USER	Authenticates User
	Name
PASS	Authenticates Password
APOP	Duplex Connection
	Authentication

[0848] This entity is represented in the directory system as an entry with an object class of, for example, “InBox—xxx” and any update on this entry (which is a message for the client) is stored. The POP3 protocol gateway manages this mail box directory entry when a client process accesses it via the POP3 protocols.

[0849] The directory system 100 supports the entities and attributes shown in Table 9.

[0850]

TABLE 9
Object
Classes	Attributes	Comment
Mail Box	Mail Box Owner (DN)	The mailbox o/c class can invoke
	Mail Box Owners	special processing in that its
	email address(es)	message attribute is processed for:
	Mailbox Name (eg	(i) Message delivery mailbox
	Inbox)	selection
	Mailbox mode (In ,	(ii) Delivery notification
	Out or store)	requirements
	Mail Box Size	(iii) Out of Office message
	Out of Office Message	responses
	Barred Subjects	(iv) Deleted message removal
	Barred Content	(v) Unread messages
	Barred Originators	(vi) Barred items
	Mail Box Entry (s)	(vii) Limits being reached
Message	Message List Owner
Distribution	(DN)
List	List's Owners email
	address(es)
	List Identifier
	Message Non
	Response - Policy
	re notify
	list/originator
	Barred Subjects
	Barred Content
	Barred Message
	Originators
	Permitted Message -
	Originators
	Member Entries (s)
Address	Address Book Owner
Book	(DN)
	Multiple entries of
	Name
	Email
	Phone
	Fax
	Web
	Business Category,
Calendar/	Date/Time/Event/
Diary	Duration,
	entries
Instant	Friend List/Rosters
Messaging	Friend Status
Pad	Banners
	Date/Time Created
	Line Number

[0851] Submitted or ‘posted’ messages are represented in the directory system 100 as an entry with an object class of, for example, “Out Box—xxx”, where xxx represents a date, such as “July 2003”, for example, and any update on this entry (which is a message for transfer, can be validated and sent by the local directory system 100. This object class is monitored by the CVM-EMS so that event signals can be sent to the mailbox owner if notification of submission (and transfer) is required.

[0852] Message distribution lists are represented in the directory system 100 as entries with an object class of “MailList-xxxx” and any update on this entry (which is a message for expansion and onward delivery), is be validated, responded to (back to the originator), and the message expanded and stored for onward processing. This is either by the directory system 100 itself as transactional mail to other mail lists, or recipient objects, or sent via its mail gateways VMs.

[0853] A user's mailbox is represented in the directory system as an entry with an object class of “In Box—xxx” and any update on this entry (which is a message for expansion and delivery in the local system, is validated, responded to (back to the originator) if required, and the message expanded and stored in the local directory system. This object class is monitored by the CVM-EMR so that event signals can be sent to the mailbox owner through Persistent Search operations or LDAP event systems.

[0854] The above features can deal with facilities such as:

• • [0855] (i) multi-casting messages to a group of directory names,
  • [0856] (ii) holding messages for receipt—if the User is deemed off line.
  • [0857] (iii) providing an alternative delivery method to the User's entry or target entry protection.
  • [0858] (iv) message vetting (time of day or subject).

[0859] The directory system 100 described above can be implemented as application software modules for execution by a standard computer system executing a standard operating system, such as Linux. Alternatively, one or more modules of the directory system 100 can be implemented as part of an operating system (as drivers through APIs) executed by the standard computer system in order to improve the performance of the directory system 100. However, it will be apparent to those skilled in the art that one or more of the processes executed by the directory system 100 can be implemented as dedicated hardware components such as application-specific integrated circuits.

[0860] In an alternative embodiment, the directory system 100 can be implemented using a computer system having at least one dedicated ASIC referred to as an attribute segment processors (ASP) 500, as shown in FIG. 5. One or more ASPs 500 can be provided on a PCI card for insertion into a standard IA-32 or IA-64 computer system. The ASP 500 includes input registers 502, output registers 504, attribute sub-segment memory 506, filter and entry information selection processing logic 510, attribute cell, ACI Role Id and filter evaluation units 512, ASP command and (input/output) control logic 514.

[0861] The input and output registers 502, 504, each include status, configuration and directory parameter registers, as described in Table 10 below. The attribute cell and filter evaluation units 512 determine whether cell values are present, less than, equal to, greater than, a prefix or substring of the evaluation criteria presented.

[0862] The ASP 500 can be loaded with up to 32 Attribute sub-segments and up to 4 MB of cell data (e.g., 1024 data cells of up to 4096 bytes per cell). However, the number of Attribute sub-segments and bytes of cell data per ASP 500 is a fabrication issue and can be modified as desired, subject to design rule constraints.

[0863] The ASP 500 has a control interface (not shown) through which the higher level software and its driver passes data and commands, via the PCI bus of the host computer system. The ASP 500 has a status and response interface (not shown) through which the higher level software and its driver can read the ASP's command results. The ASP 500 can load and dump its Attribute sub-segments and Cells from or to the hosting machine's memory hardware via its PCI bus. Where directory read or search operations are performed, the ASP 500 marks the cells that comply with the read/search criteria. The ASP 500 then writes this to its output memory area.

[0864] The ASP control interface is represented in Table 10:

[0865]

TABLE 10
Directory	ASP
Operation	Command	Notes/Parameters
ASP Management	INITIALISE	Resets all registers, Clears all Attribute sub-
		segments and their Cells
ASP Management	INITIALISE	Clears the specified Attribute sub-segment and
	SUB	its Cells
	SEGMENT
ASP Management	RESET -	Resets all registers or the defined register. See
	REGISTERS	sub Table below
ASP Management	STOP	Stop any existing commands and unmark any
		marked cells
Order Preparation	LOAD	See sub Table below
	REGISTER
Order Completion	READ	See sub Table below
	STATUS
sub-segment	HASH	Cycles the Cells of the specified Attribute sub-
Management		segment and generates the value hash fields.
sub-segment	SET BASE	Cycles the Cells of the specified Attribute sub-
Management	and DUMP	segment and where they are marked - dumps
		them to the memory area defined.
Add Entry/Attribute	ADD CELL	Writes a new cell to the specified Attribute sub-
		segment.
		If the RDNId specified already exists and the
		attribute type is distinguished a naming error is
		indicated in the status register.
Remove	REMOVE	Deletes a cell (or cells of multi valued attributes)
Entry/Attribute	CELL	(as specified by the RDNId) from the specified
		Attribute sub-segment.
		If the RDNId specified does not exist an attribute
		error is indicated in the status register.
Read Entry/Attribute	READ CELL	Reads the cells (as specified by the RDNId) from
		all Attribute sub-segments that comply with the
		Entry Information Selection criteria
Compare	MODIFY	Compares the cells (as specified by the RDNId)
Entry/Attribute	CELL	from all Attribute sub-segments with that of the
		Comparison criteria
Modify	MODIFY	Modifies a cell (as specified by the RDNId) in
Entry/Attribute	CELL	the specified Attribute sub-segment.
		If the RDNId specified does not exist an attribute
		error is indicated in the status register
Modify Name	MODIFY -	Modifies a cell's RDNId (as specified by the
Entry/Attribute	RDNID	RDNId) in the specified Attribute sub-segment
		If the new RDNId specified already exists and
		the attribute type is distinguished a naming error
		is indicated in the status register.
Copy Attribute	COPY	Creates a new cell from the RDNId specified to
	ATTRIBUTE	that of the “new RDNId” in the specified
		Attribute sub-segment
		If the new RDNId specified already exists and
		the attribute type is distinguished a naming error
		is indicated in the status register.
Search	SEARCH	Cycle all Attribute Segments and mark the cells
		that comply with the Search Filter and Entry
		Information Selection Criteria

[0866] The ASP RESET, LOAD REGISTER and READ STATUS commands operate on a range of hardware registers within the ASP 500. Table 11 defines these registers:

[0867]

TABLE 11
Register Name	Comments
ATTSUBSEG(s)	Sets the Attribute sub-segment for the operation
	(initialize, modify or delete)
RDNId	Sets the RDN Id all operations (except Search)
Filter	Sets the X.500/LDAP Filter registers. These have
	specific groups for AND, OR and NOT functions.
EIS	Sets the X.500/LDAP Entry Information Selection
	parameters into the ASP as used on Read and Search
Assigned ACI	Sets the ACI Role Ids for the Attribute Sub Segment
User ACI	Sets the ACI Role Ids for the User
VALUE	Sets the attribute value for write/modify type
	operations
VALUE NORM	Sets the normalized version attribute value for
	write/modify type operations
MEMORY BASE	Sets the memory base used for Dump, Load operations
	and writing marked cells into memory.
STATUS	Read by the software to determine process outcomes
CONFIG	Read by the software to see how many:
	Attribute sub-segments are supported,
	Cells per Attribute sub-segment are supported and
	their size,
	Attribute sub-segments are free,
	Cells per Attribute sub-segment are free and;
	to see what attribute types are supported by the
	Attribute sub-segments being used.
	Written by the software to set:
	The number of Cells per Attribute sub-segment
	The size of Cells per Attribute sub-segment

[0868] The ASP Status register is set on completion of its command processing. There are general status indicators and X.500/LDAP operation indicators, as shown in Table 12.

[0869]

TABLE 12
Status Register Item	Comments
Busy/Completed	The ASP has its operation in progress or has
	completed,
ASP Error	eg memory errors, internal malfunction, etc
Cell Flags	Set to the flags of the cell being read
Naming Error	Named item exists on add entry or modify RDN
Attribute Error	Named item does not exist on modify or remove
File Flag set	Set when the value of an attribute being read
	is not in the ASP - as it has been backed out
	to the machines file system
Entries Found	Set to the number of entries if entries have
	been marked as meeting the selection criteria

[0870] Many modifications will be apparent to those skilled in the art without departing from the scope of the present invention as herein described with reference to the accompanying drawings.