System and method for creating, searching and executing artificial intelligence agents

Application Programming Interface (API)

[0001] APIs are mechanisms that enable two or more computer programs to communicate with each other using a set of definitions and protocols. An API provider makes an API available on a server that is accessible from an endpoint. There are many API protocols, the most popular being Hyper Text Transfer Protocol (HTTP), Representation State Transfer (REST), GraphQL, Simple Object Access Protocol, Remote Procedure Call and gRPC.

[0002] An API proxy server sits between a client and an API server, providing an access point to the API server with additional functionality such as security, caching, or rate limiting. It can also be used to expose a different interface to the API by changing the protocol, adding or removing headers and parameters.

[0003] An API can be public, which is available from anywhere on the internet, or private, which is available only from within an intranet.

Artificial Intelligence (AI) Models and Techniques

[0004] AI models are trained on a set of data to recognize certain patterns.

[0005] Large Language Models (LLM) are deep learning models that are trained on vast amounts of data. A pretrained LLM is an LLM that has already been trained on large amounts of data and which is available to ask questions.

[0006] LLM fine-tuning involves adapting a pre-trained model to a specific task. Specifically, the LLM is partially retrained using input-output pairs of representative examples of the desired behavior.

[0007] Some LLMs like ChatGPT and Anthropic can be called only through an interface over the internet while others like Facebook Llama can be hosted and used from within other systems.

[0008] Retrieval-Augmented Generation (RAG) enables language models to incorporate relevant information beyond their training data by prompting them with real-time information from APIs, traditional IT systems, vector database or a full text search engine.

[0009] GraphRAG is a RAG, where the retrieval is done from a knowledge graph stored in a graph database.

[0010] Retrieval-Interleaved Generation (RIG) is a variation of RAG in which calls to LLMs and traditional IT systems are interleaved.

[0011] AI based systems can also use LLMs to query information from traditional IT systems and get a specific task done. Text2SQL is a technology that converts user query in plain English to Structured Query Language (SQL).

Artificial Intelligence Agent (Agent)

[0012] An agent is an application that uses an LLM to decide the flow and output of the application. An agent typically makes one or more calls to LLMs, APIs and other systems to answer a question or perform a task and would be prepared to answer follow-up questions or perform additional tasks. RAGs, GraphRAGs, RIGs, Text2SQL can all be considered as subsets of agents.

Natural Language Processing (NLP)

[0013] NLP is a machine learning technology that gives computers the ability to interpret, manipulate, and comprehend human language.

[0014] Named-entity recognition is a field of NLP that locates named entities mentioned in a text into pre-defined entity types. Common entity types recognized are person, location, organization, event, numerical and temporal. Within each entity, there can be several sub-entities like email, Uniform Resource Locator (URL), city, state, quantities, monetary values, percentages and others.

[0015] Relation Extraction is the task of extracting semantic relationships between two or more entities in a text.

[0016] Both LLMs and NLP libraries can perform named-entity recognition and relationship extraction. Apache OpenNLP and Stanford NLP are popular NLP libraries.

[0017] Stop words are commonly used in NLP to filter out some words from processing.

AI Model Files

[0018] A deep learning model file typically contains information about the network architecture, including the number of layers, neurons per layer, activation functions, weights for each connection between neurons, biases, and sometimes additional metadata like training parameters and performance metrics, all structured based on the chosen deep learning framework and often saved in a specific file extension like .h5, which is a Keras file, .pb, which is a TensorFlow file, or .pt, which is a PyTorch file, depending on the framework; essentially capturing the learned state of the model after training. Once a deep learning model is trained, the model file can be uploaded to another computer and be used in an application to make predictions on new data.

Other Products and Platforms

[0019] LangChain is a software framework that helps facilitate the integration of large language models into applications.

[0020] No-code LLM app builders allow creating LLM applications by dragging and dropping Graphical User Interface (GUI) elements. Flowwise AI, Langflow and Dify are three such products.

[0021] The Hugging Face Hub is a platform with over 900,000 AI models publicly available, on an online platform.

[0022] Github is a version control system that allows developers to store, share, and collaborate on the code they create.

[0023] Splunk is a software platform that helps users collect, analyze, and visualize large amounts of machine-generated log data.

[0024] MySQL is a relational database management system. Cloud-based, relational database services with MySQL are available from Google cloud and Amazon web service.

[0025] Amazon Web Service (AWS) S3 and Google cloud storage are popular cloud-based object storage services. An object storage accepts a key and a value. The key is the name of the object. The value is the content of what is being stored, which can be in text or binary format.

[0026] An object cache server stores small objects in memory. Popular caching products are Memcached and Redis. AWS offers a cloud caching service called ElastiCache.

[0027] Nosql databases are non-relational databases that store data in a flexible, non-tabular format. Cassandra, HBase and MongoDB are popular nosql databases. AWS DynamoDB is a nosql cloud service.

[0028] Key-value storage, also known as a key-value database, is a database that links named keys to values of any type. AWS DynamoDB can be used as a key-value storage service.

[0029] IP Location database service is offered by multiple companies including Google maps platform and ip2location.com. Using these products, an IP address, zip code, city name or place of interest can be mapped to a location, including city, state, country and local time.

Full Text Search Service

[0030] Two of the most popular full text search engines are Elasticsearch and Apache Solr.

[0031] These products provide ways to split text into tokens, add the tokens to a full text search index and expose efficient ways to search the index. Common search ranking algorithms are TF-IDF and BM25.

[0032] Many vendors offer cloud-based services for full text search including AWS Opensearch.

Vector Database Service

[0033] A vector database stores text as vector embeddings and allows one to search for text based on similarity. Words, phrases, sentences and paragraphs can be mapped to N dimensional vectors and stored in the database. The process of representing text as an N-dimensional vector, is called vectorization.

[0034] An embedding model is used to convert text into an N dimensional vector. The distance between two vectors gives the similarity in meaning between the texts they represent. Common ways to measure vector distances are Euclidean distance, Cosine similarity, Dot product similarity, Manhattan distance and Hamming Similarity.

[0035] A vector database service can be created by hosting vector databases like Milvus or interfacing with a cloud vector database service like Pinecone.

AI Model Hosting Service

[0036] There are several vendors that offer a cloud service to host an AI model and provide an endpoint to which clients can send an input and receive an output. Some of them are Hugging Face, Google Colab and Amazon SageMaker.

Graph Database Service

[0037] A knowledge graph is an organized representation of real-world entities and their relationships using nodes and relationships. It is typically stored in a graph database. Entities in a knowledge graph can represent objects, events, situations, or concepts. Neo4j is a popular graph database that can be hosted. Neo4j AuroDB is a hosted graph database service on the cloud.

Namespace

[0038] Namespace is a string with dots separating hierarchical levels, that serves as a label to group and uniquely identify a collection of names or entities within a specific context, ensuring there are no naming conflicts across different scopes.

Formatted String Literals (f-Strings)

[0039] F-strings are part of the Python programming language. The concept also exists in other languages like Dart and Java 21. An f-string is a sequence of characters which can contain one or more placeholders. Placeholders start and end with a pair of delimiters and contain an expression in between. Expressions are programming code that are evaluated at program runtime.

Machine-Readable, Textual Format

[0040] Machine-readable textual formats are structured data formats that computers can automatically read and process, while also being easy for a human to read, write and understand. Some examples of machine-readable textual formats include: JavaScript object notation (JSON), eXtensible Markup Language and Yet Another Markup Language.

Universally Unique Identifier (UUID)

[0041] A UUID, or Universally Unique Identifier, is a 128-bit number that is used to identify information in computer systems. When generated according to the standard methods, UUIDs are, for practical purposes, unique.

Search Query

[0042] A search query contains one or more words that are keywords, operators, a phrase, a full sentence, a question or a directive to perform an action. Typically, they are entered into a textbox on a GUI or uttered verbally to a device.

[0043] A machine-readable, textual format to define an agent has many advantages, including being implementation technology agnostic. Such a textual format should support common needs of an agent like calling an LLM, calling an API, control the flow of the program, managing variables, handling errors, techniques for follow up queries, raising events and saving and restoring the state of a running agent.

[0044] An end user would not be aware of all the agents that are available for different purposes. They would typically enter a search query and expect an answer, or an action taken. There is a need to have a central computing platform that can store all the agents, match a user's search query to the right agent, execute that agent and provide an answer or perform an action. A technique to describe an agent to a search service is needed so that a search query can be matched to an agent.

[0045] A central computing platform as described above can only consume systems and APIs that are either public or private within that central platform. An organization with private systems and APIs on its own intranet cannot use agents that run on the above central platform. Such an organization would need a client component to the central computing platform which can execute agents within its intranet and consume the private systems and APIs that are available only within its intranet.

[0046] This invention addresses all the above needs by combining concepts from agents, enterprise search and computer design and architecture that includes a central part and a client part. The fields of this invention are AI, agents, enterprise search, computing platforms and computer design and architecture.

[0047] A computing platform is defined using a central system domain and distributed client modules. The system domain contains a data storage service, a first LLM service, a search engine module, authentication module, admin module, creation module and an execution module.

[0048] Admin users and first users register and login to the system domain using the authentication module. An admin user takes ownership of a namespace and associates an authorization key with the namespace, using the admin module. Admin users also approve first users in the namespace.

[0049] First users use the creation module to create agent specifications in that namespace and store them in the data storage service. First users also create agent descriptors, which describe agents to the search engine module. Agent specifications and agent descriptors are specified in a machine-readable, textual format.

[0050] Client modules are started anywhere on the internet. Each client module is associated with an authorization key, which associates it with a namespace. Each client module can search and execute agents only in the namespace it is associated with.

[0051] A second user sends a search query to a client module, which sends a search request to the execution module. Using the search engine module, the search query is matched to one or more agent names and returned. The client module then retrieves a matched agent's specification from the execution module, creates an agent instance based on the specification and executes the agent instance, which produces a response to the search query or performs an action.

System Domain, Client Module and Users

[0059] System domain is an internet domain in which this invention maintains its state. The system domain runs the following services and modules:

• • [0060] 1. A data storage service
  • [0061] 2. A first LLM service
  • [0062] 3. Authentication module
  • [0063] 4. Admin module
  • [0064] 5. Search engine module
  • [0065] 6. Creation module
  • [0066] 7. Execution module

[0067] There would be many client modules and they would be implemented in many languages and platforms. They can be started anywhere on the internet. Each client module communicates with the execution module in the system domain, identifying its namespace with an authorization key. Multiple client modules can run with the same authorization key.

[0068] Admin users login to the system domain, take ownership of a namespace and approve first users in that namespace. First users login to the system domain, create an agent specification in their namespace and describe them to the search engine module using an agent descriptor. Second users send a search query to a client module and receive a response or an action taken on their behalf.

Agent Specification

[0069] An agent specification defines an agent in a machine-readable, textual format. This has the following advantages:

• • [0070] 1. Client modules can be implemented in any technology or platform
  • [0071] 2. Agent specification be added to a version control system
  • [0072] 3. Agent specification can be validated automatically
  • [0073] 4. Implementing an approval process for agents is easier as many approval criteria can be automated

[0074] A client module creates an instance of an agent based on an agent specification and executes it. The client module also creates an object in memory called an agent state object for each agent instance it executes, which maintains the state of the agent instance that is being executed.

[0075] An agent specification includes the following fields:

• • [0076] 1. Agent name
  • [0077] 2. Agent title
  • [0078] 3. An ordered list of task units. Each unit task unit contains
    • [0079] a. Task name
    • [0080] b. An ordered list of command units. A command unit specifies a command that should be run using a command type and a command spec unit. Command spec units can have different fields based on the command type. A command can be one of the following types:
      • [0081] i. apiClient—Calls an API
      • [0082] ii. systemLLM—Calls an LLM supported by the system domain
      • [0083] iii. systemFunction—Calls a function defined in the system domain
      • [0084] iv. systemQuery—Queries a file uploaded with the upload module
      • [0085] v. userFunction—Calls a function defined in language packages included with the client module.
      • [0086] vi. execTask—Executes another task
      • [0087] vii. conditional—Executes a list of commands based on a condition or in a loop
      • [0088] viii. saveState—Saves a current agent state object or a variable within it
      • [0089] ix. restoreState—restores a previously stored agent state object or a variable from within it
      • [0090] x. codeBlock—Executes a block of code
      • [0091] xi. followupInputs—Stores information needed to answer further questions.
      • [0092] xii. raiseEvent—Raise an event that would be saved in the system domain
      • [0093] xiii. removeVariables—remove variables from an agent state object before it is returned to a caller

[0094] FIG. 1A illustrates an agent specification using a UML class diagram. An agent includes a list of command units. A command unit can include a variable unit, that includes an ordered list of variables. Each variable specifies a variable name and an f-string. By default, a variable is assumed to be of String type. If a variable is not of string type, it can additionally specify a type, which can be one of JSON, HTML, CSV, Protocol buffers, or binary.

[0095] A command unit can also specify a list of alternate commands, which are executed only if there is an error in executing the command.

[0096] A field named customFields can be included at the agent level, task unit level or command unit level, to include namespace specific fields.

[0097] Each command can include a responseName field and a responseType field. If the command returns a value, it is stored in a variable of name specified in the responseName field in the agent state object. The type specified in the responseType field is used to store the response with the right type. Some of the values for the responseType field can be String, JSON, HTML, CSV, Protocol buffers, or binary.

[0098] The following paragraphs discuss each field in the agent specification in more detail.

Agent Name

[0099] An agent name can have only letters, numbers, and hyphens. Second users can bookmark agents and call them directly. Such agent URLs will have the agent name as part of the URL.

Agent Title

[0100] The agent title field is specified as an f-string. This is to generate the title dynamically for each search query. The agent title field can be used to show a friendly link to an agent on a second user GUI.

Task Unit

[0101] A task unit has a task name that is unique within an agent specification. This allows a task unit to call another task unit by name. A Client module executes an instance of an agent by executing a task that occurs first in its task list. Other tasks in the agent are executed only when they are called from a previous task.

Variable Units

[0102] A variable unit specifies a list of variables and their values as f-strings. Each f-string is evaluated and assigned to the corresponding variable in the order they appear in the list.

[0103] A variable is stored in the agent state object. It is created when it is first assigned to. Variables that occur later in the agent specification can refer to variables set earlier in their f-strings.

[0104] Variable units are executed after the command.

Alternate Commands

[0105] A command can fail for at least two reasons:

• • [0106] 1. An apiClient's call to an API server can fail due to network, authorization or quota reached issue.
  • [0107] 2. An exception can be raised in a codeBlock command or an f-string.
  • [0108] 3. An agent runs longer than a maximum time set for that namespace.

[0109] Commands in the alternate command list are executed if a command fails.

customFields

[0110] customFields can be included at the agent level, task unit level or command unit level.

[0111] Subfields in customFields are used by client modules to enhance the functionality of an agent. For example, an agent can set a custom field that specifies a name of a GUI template in that namespace. Second user GUI can download that template and use that template to render an agent state object in the second user GUI.

[0112] By default, subfields within customFields are not checked when the creation module validates an agent. An admin user can add validation rules to be applied within customFields.

Commands in an Agent

[0113] FIG. 1B illustrates all the commands in UML. Each command can be implemented as a class that derives from a command spec class using object-oriented design techniques. The following paragraphs describe each command.

apiClient Command

[0114] An apiClient command makes an API call to a server. It can specify the following fields:

• • [0115] 1. protocolType field defines the protocol used to communicate with the API server. Some of the possible values are REST, SOAP, HTTP, graphQL and gRPC. This field will default to REST.
  • [0116] 2. protocolSubtype field specifies one of the many ways in which the protocol might be used. For REST protocol, the subtype might be GET or POST. This field will default to GET.
  • [0117] 3. endpoint field is the URL where the API service is available. Endpoints are specified as f-strings. This is to facilitate:
    • [0118] a. HTTP GET fields that should be included with the endpoint
    • [0119] b. Endpoints that are discovered dynamically from responses to previous API calls.
  • [0120] 4. headers field specify name-value pairs that are sent as headers in an API request. This can include request types, response types and authorization.
  • [0121] 5. parameters field specify name-value pairs that are sent as POST parameters to the API server.
  • [0122] 6. responseType defaults to JSON.

[0123] A first user can use an API proxy service to hide all the API details and expose a simple REST interface, which can be called from the apiClient command. Function module provides an API proxy service which is explained later.

[0124] A first user can also use a secret vault service to hide API authorization keys and not expose them in the agent specification.

systemLLM Command

[0125] systemLLM calls a LLM in the system domain with an input and receives an output from the LLM. These calls are sent to the execution module.

[0126] A name field in this command specifies the model name of the LLM. Params field species the parameters to the LLM. responseType defaults to JSON.

systemFunction Command

[0127] systemFunction executes a function defined in the system domain. These calls are sent to the execution module.

[0128] There are two kinds of functions that can be executed:

• • [0129] 1. Global functions that are available to all namespaces
  • [0130] 2. Namespace specific functions are functions that are available only within a namespace. These are just API proxies defined in the system domain, by a first user.

[0131] Examples of global functions are:

• • [0132] 1. getLocation—Get location details including a city name, latitude/longitude, zipcode and others from various parameters including an IP address or name of a place of interest, airport code etc.
  • [0133] 2. getQueryDetails—Get details of the search query including entity types and values, classifications, filters and patterns matched.

[0134] A name field in this command specifies the name of a function. Params field species the parameters to the function. responseType defaults to JSON.

systemQuery Command

[0135] systemQuery runs a query against a file uploaded from the creation module. These calls are sent to the execution module.

[0136] A first user can upload files to the system domain using the creation module and any agent running in the same namespace as the first user can use this command to query the files. The type of files determines the syntax of a query that can be used in this command.

[0137] This command can also be used to query files uploaded by the system domain as explained in the creation module section.

[0138] A name field in this command specifies the name of an upload. Params field species the parameters to the system domain function. responseType defaults to JSON.

userFunction

[0139] A client module can include many programming language packages for the execution environment of an agent. Functions in such a language package can be called using a standard interface from the agent.

[0140] By default, a client module would include database drivers and other common interface packages and define functions to call various kinds of databases like relational, nosql and bigdata. These can be used to send queries to private systems in the environment that a client module runs.

[0141] A name field in this command specifies the name of a user function. Params field species the parameters to the user function. responseType defaults to JSON.

execTask Command

[0142] An execTask command executes another task by name.

[0143] taskName field specifies the name of the other task. This can be blank to left blank to end the execution of the agent.

[0144] endThis field specifies whether the current task should end or continue executing after executing the other task.

Conditional Command

[0145] A conditional command executes one of two command lists based on the result of evaluating a condition.

[0146] ‘If’ field specifies a condition to be evaluated. It is specified as an f-string, which when evaluated and converted to a boolean value, should be either true or false.

[0147] ‘then’ field specifies a list of commands to execute, if the ‘if’ condition is evaluated to true.

[0148] ‘else’ field specifies a list of commands to execute, if the ‘if’ condition is evaluated to false.

[0149] Not specifying ‘then’ or ‘else’ block is the same as specifying a list with no commands.

[0150] If an additional field ‘loop’ is specified and set to true, the command behaves like a ‘while’ loop in programming languages. The command goes through a repeated process of checking the ‘if’ condition and if that evaluates to true, executing the conditions in the ‘then’ block. When the ‘if’ field is evaluates to false, the ‘else’ block is executed and the command ends.

saveState and restoreState Commands

[0151] saveState stores a current agent state object in the system domain. This command calls the execution module which in turn calls the data storage service.

[0152] saveState creates an object name by generating a UUID and appending the client module's namespace to it. It then saves the object name to a variable ‘savedState’ in the agent state object, serializes the agent state object and stores the serialized agent state object by calling the execution module, which in turn calls the object storage service. The reason to store the object name in the object itself is to reuse the same name in a subsequent call.

[0153] Instead of saving the entire agent state object, saveState can store just a variable in the agent state object. The name of the variable to store can be specified in a field called ‘storeObjectName’.

[0154] restoreState restores an agent state object by calling the execution module, which in turn calls the object storage service. A key to the stored location is passed as an input to the agent. The received object is deserialized and then a current agent state object is updated with values from the stored state.

[0155] Instead of restoring the entire state object, restoreState can restore just a variable. The name of the variable to restore can be specified in a field called ‘restoreObjectName’.

[0156] Typically, the same agent stores an agent state object and restores the agent state object to answer a follow up query from a second user, to answer follow up questions. But saveState and restoreState can also be used to share information between one agent and another.

codeBlock Command

[0157] codeBlock command executes a block of code. The code should be in a programming language that is supported by the client module. Language field specifies the language of the code block.

[0158] codeBlock is specified as a f-string. The f-string is first evaluated into a string and that string is executed as code. This allows the code to be generated dynamically from the output of a previous systemLLM call.

followupInputs Command

[0159] followupInputs specify a list of additional inputs needed from a second user to answer a follow up question. For each additional input, a ‘hint’ field can be specified, which a GUI can show to a second user, to solicit an input value. The GUI is expected to send a subsequent call to execute the same agent with a value for each additional input.

[0160] A common pattern in agents is a chat-like interface where a second user can ask follow-up questions on the same search query. In that case, an agent can set the search query field name as the additional input required. A second user GUI can show the current answer to the user and also show a widget for a follow up query.

[0161] This command can also be used to get an additional input with which an agent can provide a more useful answer to the search query. The additional input can be from a list of values, a text field or a file upload.

[0162] followupInputs can specify validation criteria for each additional input. Second user GUI can use the validation criteria to reject values that a second user enters. Validation criteria can be:

• • [0163] 1. A list of allowed values for an input value
    • [0164] a. Second user GUI can use this to show a drop down widget.
  • [0165] 2. A regular expression pattern that an input value should match.
  • [0166] 3. Length of a text field
  • [0167] 4. The size of a file that can be uploaded
  • [0168] 5. The type of a file that can be uploaded
    raiseEvent Command

[0169] raiseEvent command send an event to the execution module in the system domain to record an event. The command takes an eventName and a list parameter names and values.

removeVariables

[0170] removeVariables would typically called before ending execution of an agent to clean up the agent state object.

[0171] keepOnly is a field that specifies a list of variables that should be kept. All the other variables in the agent state object will be removed.

[0172] removeList is a field that specifies a list of variables that should be removed.

[0173] Either keeyOnly or removeList can be used to clean up the agent state object.

Agent Descriptor

[0174] An agent descriptor describes an agent, so that it can be matched to a search query entered by a second user.

[0175] Like an agent specification, an agent descriptor is also specified in a machine-readable, textual format. This has the following advantages as mentioned for an agent specification.

[0176] An agent descriptor contains the following fields:

• • [0177] 1. Agent name
  • [0178] 2. Description
  • [0179] 3. A list of filters
  • [0180] 4. A list of classes in one or more classification schemes
  • [0181] 5. A list of keywords
  • [0182] 6. A list of names of named lists
  • [0183] 7. A list of patterns
  • [0184] 8. A list of named entities
  • [0185] 9. A list of example queries
    Agent Name

[0186] Name of the agent that is described by this agent descriptor.

[0187] A first user uses this field to describe an agent in one or more paragraphs. Classifications, entities, keywords and example queries can be generated from this field using the first LLM service.

Filters

[0188] A filter is a name and a list of possible values. Filters can be global to all agents, or they can be specific to agents in a namespace.

[0189] The system domain will define global filters related to second user location, second user device and search query attributes. Location related filters can be country, city and local time. Search query related filters can be the language the query is in, whether the query type is informational, transactional or navigational or whether the query is location based.

[0190] For filters defined by the system domain, the filter value need not be sent directly by the client module. It can be inferred from one of the inputs that the client module sends. For example, when a client module sends an IP address in the input, the execution module will map it to a location including country, city and local time and apply location-based filters. Similarly, the execution module would categorize the search query's language, type and whether it is location based using the search engine module and apply search query related filters.

[0191] First users can define filters that are specific to a namespace. For such filters, the client module would send the filter's name-value as an input, along with the search query. The execution module will select a subset of agents associated with that namespace based on filter values set in the agent descriptor for each agent.

Classifications

[0192] The system domain can define multiple classification schemes like Cooperative Patent Classification and Library of Congress Classification. It can also define its own classification schemes, like a scheme called Current, that includes classes like News, Politics, Business, Health, Entertainment, Style, Travel, Sports, Science and Weather.

[0193] When a first user adds an agent descriptor to the index, the creation module will query the first LLM service to suggest classes in each classification scheme. The first user can select one or more of the suggested classes or add others.

[0194] A first user can add a classification scheme specific to a namespace and agents in the namespace can use that classification scheme. The first user would additionally specify an input prompt to the first LLM service or an API that can be called to find the class of a search query in the classification scheme.

Keywords

[0195] Keywords are commonly used in full text search engines. Unlike filters and classification schemes, keywords are not defined by the system domain. First users can associate any keyword with an agent. Stop words defined by the system domain will be excluded from being included as a keyword.

[0196] Just like classifications, the creation module will suggest keywords to a first user based on the description, using the first LLM service.

Named Lists

[0197] A named list is a list of strings with a name. A first user can create a named list from the first user GUI. These names can be of people, places, organizations, concepts etc. The named lists field in an agent descriptor can then refer to all the values in that list by just specifying the list name. The system domain can provide many such lists corresponding to popular people, places, company names etc. The system domain provided lists can be referred by any agent descriptor, whereas a first user created list can be referred only by an agent descriptor in their namespace.

[0198] To illustrate, a first user from namespace1 can upload a list of all National Football League team names and call it nfl.namespace1 and can refer to the entire list in the keywords as ‘nfl.namespace1’ from the named lists field.

Patterns

[0199] A pattern is any word that cannot be found in a language dictionary. It can be made up of a combination of letters, numbers and punctuation. IP addresses, email addresses, International Standard Book Number, US postal service tracking number are all examples of patterns. Patterns are represented by regular expressions in an agent descriptor.

Finding Pattens in a Search Query

[0200] It would be too expensive to match every word found in a search query against thousands of regular expressions defined by agents. A more practical approach is to drastically reduce the number of possible regular expression matches by storing certain meta data about each pattern defined by agents.

[0201] This invention uses the following method to find agents that match a pattern in the search query:

• • [0202] 1. For each pattern in each agent, save the minimum and maximum number of letters, numbers and each punctuation character that is allowed in the pattern.
  • [0203] 2. Find a word in the search query that is not part of the language dictionary.
  • [0204] 3. Count the number of letters, numbers, and the number of each punctuation character in the word.
  • [0205] 4. Find all the patterns that match the count of letters, numbers and punctuations that are within the numbers found in the word found.
  • [0206] 5. The above steps will reduce the number of patterns to a small number.
  • [0207] 6. For each pattern found in (4) match the regular expression for that pattern to the word found and check whether the word still matches.

[0208] The following table illustrates a few common patterns, and the metadata that would be associated with them.

[0209]

	Regular expression	Letters	Numbers	Hyphens	Dots	At @
IP address	(\d{1,3}\.){3}\d{1,3}	0	4 to 12	0	3	0
Email address	{circumflex over ( )}\S+@\S+\.\S+$	3 to 192	0 to 190	0 to 190	1	1
ISBN	\d+\−\d+\−\d+\−\d+,\d+	0	13	4	0	0
Postal service	[A-Z]{2}\d{9}US	4	9	0	0	0
Flight number	[A-Z]{2}\d{1,4}	2	1 to 4	0	0	0

Named-Entities

[0210] This field specifies named entities found in the search query. A named entity consists of an entity type and an entity value.

[0211] For each entity type, this field specifies:

• • [0212] 1. Minimum and maximum number of times the entity type can be present in a search query. If these fields are unspecified, then any number of this entity type can be present in the search query, including zero.
  • [0213] 2. One or more classification of the entity value for which this agent is applicable. If classifications are not specified, then this entity would match all values for that entity.
    • [0214] a. For example, an agent related to sports players records might look for person entities in the search query and only if the person found can be classified as a sports player by the search engine module.

[0215] Named entities can also specify one or more entity relationships between one entity and another. The search engine module can use the actual relationship found in a search query to find an agent that specify the same relationship between entities.

[0216] Example queries are examples of possible search queries from a second user that should be matched to an agent. Like classifications and keywords, the creation module can generate a list of example queries based on the description field using the first LLM service. A first user can select one or more of the generated queries or add their own.

[0217] An example query can also be expressed in a generalized form by removing stopwords, replacing entities into entity types and question words into a placeholder. The motivation to do this is to make it easier to match search queries that ask a similar question about different entities. For example, an agent related to records of sports players can express an example query as “How many Wimbledons did Federer win?”. The generalized form of this example query would be “[QuestionWords] [event] [person] [action]?”, if ‘did’ is a stopword. Subsequently, when a second user issues a search query for “How many US Open did Nadal lose?”, it would get translated to “How many [event] [person] [action]” and would match the above agent.

[0218] When example queries are expressed in a generalized form, a search query also needs to be converted to its general form and matched against example queries. Prophetically, the search engine module might find a better match with generalized queries.

[0219] FIG. 2A shows the components of the computing platform of this invention. Some of the modules also have sections that specialize in certain areas of the module. The following table summarizes the components, their sections, if they have any, and their relationships. Each component is described in the paragraphs following the table.

[0220]

Component	Section	Used by
System domain
Data storage service		Authentication module
		Admin module
		Creation module
		Execution module
First LLM service		Creation module
		Execution module
Search engine module
	Indexer section	Creation module
	Search section	Execution module
Authentication module		Admin GUI
		First user GUI
Admin module		Admin GUI
Creation module
	Specification section	First user GUI
	Descriptor section	First user GUI
	Upload section	First user GUI
Execution module
	Searcher section	Client module
	Fetch agent section	Client module
	Function section	Client module
	Event section	Client module
	Subscription section	Client module
Admin GUI		Admin user
First user GUI		First user
Client module		Second user GUI
Second user GUI		Second user

Data Storage Service

[0221] Data storage service provides a key-value storage service, an object storage service and a database service. This can be implemented by hosting such products or interfacing with cloud-based technologies mentioned in the prior art section.

First LLM Service

[0222] The first LLM service is an interface to hosted LLMs in the system domain and LLMs available on the internet. Facebook Llama is an example of an LLM that can be hosted. ChatGPT and Anthropic are examples of LLMs that can be called over the internet.

[0223] The first LLM service will receive a model name, input and a list of parameters to the model. Parameters are model specific name-value pairs. Based on the model name, the first LLM service will select a hosted model in the system or send a call to a model's API on the internet. It will include the input and the parameters in a request to the model, receive an output from the model and return the output to the caller.

Authentication Module

[0224] The system domain would include an authentication module to authenticate admin users and first users. Admin users and first users register and authenticate with the system domain using website registration and authentication mechanisms.

[0225] The authentication module would also provide integration to single sign-on systems of organizations, so that admin users and first users can register and login to the system domain using organization specific authentication mechanisms.

[0226] Implementing an authentication mechanism for a website or webservice, including integrating it with organizations' single sign-on systems are well known to a person of skill in the art.

Admin Module and Admin GUI

[0227] Admin users use the admin module from the admin GUI. The admin module provides these functions for admin users:

• • [0228] 1. Assume ownership of a unique namespace
  • [0229] 2. Generate an authorization key for the namespace
  • [0230] 3. Authorize first users to be associated with the namespace.
  • [0231] 4. Set up an approval process for first users to create new agents or make changes to agent specifications and agent descriptors in the namespace. The system domain would provide an approval process that can be customized to each namespace that needs it.
  • [0232] 5. Setup additional rules for validation of agent specifications and agent descriptors in the namespace
  • [0233] 6. Setup a namespace to support subscriptions
  • [0234] 7. Set up default values for parameters in search calls to the execution module
    • [0235] a. An admin user can set up default values for search calls to the execution module and give it a name. A client module would send a customization name and receive customized search results based on the customized name. The kinds of customizations are mentioned in the searcher section in the execution module.

[0236] An admin user takes ownership of a namespace by requesting for it. The namespace cannot be owned by another admin user.

[0237] If a namespace matches or ends with an existing internet root domain name, the admin module may require verification of the ownership of the internet root domain by the admin user. Once an admin user takes ownership of a namespace, they may automatically own all the sublevels within that namespace. That is, if an admin user owns example.com, they will automatically own any namespace like first.example.com.

[0238] An authorization key for a namespace is a UUID generated by the admin module. The admin module stores the association between the authorization key and the namespace in the data storage service.

[0239] Once an admin user generates an authorization key for a namespace, they can approve a first user to be associated with that namespace. They can also assign the privileges of administering the namespace to the first user.

Search Engine Module

[0240] Search engine module is internal to the system domain. It is used by the creation module to index agent descriptors and execution module to run a search.

[0241] The search engine module includes the following services:

• • [0242] 1. Full text search service
  • [0243] 2. Vector database service
  • [0244] 3. Second LLM service
    • [0245] a. Second LLM service is different from the first LLM service in the following ways:

[0246]

First LLM service	Second LLM service
Provides LLM services to agents	Provide LLM service to the search
running on client module	engine module
Represents many LLM models	A single LLM model
Read only models	This model can be fine-tuned

• • [0247] 4. Embeddings model service
    • [0248] a. LLMs typically also provide an interface to an embeddings model service. This service can be implemented by interfacing with the embeddings model service of the second LLM service.
  • [0249] 5. Relational database service

[0250] Each of the above services can be implemented using techniques mentioned in the prior art section. The search engine module will also include an NLP library, which can be called though its interface.

[0251] The search engine module includes an indexer section and a searcher section. The following paragraphs go through one way of matching a search query to one or more agents. A person of skill in the art will use different search algorithms established in the field of enterprise search to enhance and alter these methods.

Indexer Section

[0252] Indexer section indexes an agent descriptor and maps it to an agent name in the search engine module.

[0253] The following table shows which fields are included in each part of the search engine:

[0254]

		Vectorized and		Included
		included in	Used for fine-	in
	Included	Vector	tuning in the	relational
	in full text	database	second LLM	database
	search service	service	service	service
Namespace	Yes	No	No	Yes
Description	Yes	Yes	Yes	No
Filters	Yes	No	No	Yes
Classifications	Yes	No	No	No
Keywords	Yes	No	Nc	No
Named lists	Yes	No	No	Yes
Patterns	No	No	No	Yes
Entities	Yes	No	No	Yes
Example queries	No	Yes	Yes	No

[0255] For the fields included in the full text search engine, the following schema can be used:

[0256]

		Remove			Single
	Tokenized	Stopwords	Lemmatized	Synonyms	valued
Namespace	no	No	no	No	yes
Description	yes	Yes	yes	Yes	yes
Filters	no	No	no	No	no
Classifications	no	No	no	No	no
Keywords	no	No	yes	Yes	no
Named lists	yes	No	no	No	no
Entities	no	No	no	No	no

[0257] Namespace is included in the search engine although it is not part of the agent descriptor. Agent title is also stored in the full text search engine service and relational database service, but not indexed. To facilitate searches In the full text search index, each namespace can be indexed to a different collection or shard. Alternately, namespace can just be a field in the full text search index and example queries can be filtered using that field. Vector databases support the concept of namespaces natively. In the relational database, namespace can be included in a column and results can be filtered based on that column.

[0258] Description and example queries fields will be vectorized using the embeddings model service and then included in the vector database service in a namespace that matches the agent's namespace.

[0259] Second LLM service can be fine-tuned with example queries as input and agent name as output. Second LLM service can also be fine-tuned with description as input and agent name as output.

[0260] Keywords will be indexed in the full text search service. They can also be included in the relational database.

[0261] Named lists can be indexed in two different ways. It can be included as a multivalued field in the full text search service after tokenization. This would help matching a search for “Cowboys next game” with an agent that supports searches on NFL teams. Named lists can also be indexed in the relational database by including namespace, agent name and each name in the named list in three columns and doing a regular SQL query on them.

[0262] Patterns metadata are included in the relation database to reduce the number of patterns to match for as explained in the section on pattens.

[0263] Filters and classifications are name-value pairs where one name can have multiple values. They can be stored in the search engine module with multivalued fields and in the relational database with multiple entries, one per each name-value combination.

[0264] Entities can also be stored both in the relational database and the full text search service.

Searcher Section

[0265] Searcher section receives a namespace and a search request from the execution module and returns a list of agent names and titles. A search request contains a search query and a list of inputs, where each input is a name-value pair. The search request is processed with at least the following steps:

• • [0266] 1. Filter the search to the subset of agents that match the namespace.
  • [0267] 2. Determine values for system domain filters like location and search query attributes based on the search query and input values. Select agents that match filters in the inputs. Use the second LLM service in the search engine module to determine whether the search query is based on a location and filter agents that specify location-based filter to be true.
  • [0268] 3. Classify the search query using either the NLP library or the second LLM service in the search engine, under each classification scheme. Select agents that match the classes specified for those classification schemes.
  • [0269] 4. Fing patterns in the search query by looking for words that are not in a language dictionary. Select agents that match those patterns, as explained in the patterns section.
  • [0270] 5. Find all the named entities in the search query and their entity types and the entity relationships using the NLP library or the second LLM service in the search engine module. For each entity type found, find the classification of the entity value. Select agents that specify the entity types found in the search query and their classification. Give higher scores to agents that specify the same relationship between entities as found in the search query.
  • [0271] 6. Select all the agents that specify words from the search query in the keywords field using the full text search service in the search engine module. Assign scores based on number of keywords matched.
  • [0272] 7. Select all the agents that specify words from the search query as a value in named lists using the full text search in the search engine module. Assign scores based on matching the entire value in the named list.
  • [0273] 8. Look up the vectorized search query in the vector database service in the search engine module and find nearest matched agents. Assign scores by normalized vector distances.
  • [0274] 9. From the fine-tuned LLM model in the search engine module, input the search query and get agent names as output.
  • [0275] 10. Assign weights to each of the methods above. Weights are floating point numbers between 0 and 1.
  • [0276] 11. Assign a net relevancy score to each matched agent by multiplying the scores assigned in each method by the weight assigned in the previous step.
  • [0277] 12. Order agents in decreasing net relevancy scores.
  • [0278] 13. Select the top agents, based on how many matched agents were asked to be returned.
  • [0279] 14. For selected agents
    • [0280] a. Fetch titles from the full text search service or the database service
    • [0281] b. Evaluate each title as an f-string
    • [0282] c. Use second LLM service in the search engine module to convert evaluated titles to be grammatically correct
    • [0283] d. Include selected agent names and titles from (c) above in a search results object.
  • [0284] 15. Return the search results object.

[0285] The search request can also include several ways to customize the search logic described above and the information returned from the search call. Some of the customizations can be:

• • [0286] 1. Specify the weights mentioned in step 10 to calculate net relevancy.
  • [0287] 2. Specify the number of agent names to be returned.
  • [0288] 3. Specify that agent specifications also be included for one or more matched agent names.

[0289] An admin user can setup search customizations for a namespace and the execution module can include them in search requests to the search engine module.

Special Words in the Search Query

[0290] The client module can also process the search query to take specific actions.

[0291] For example, ‘@agent <first_agent_name> What events are happening around me?’ can execute an agent name first_agent_name with the search query “What events are happening around me?”.

Creation Module

[0292] Creation module is used by first users from the first user GUI. It consists of the following sections:

• • [0293] 1. Specification section
  • [0294] 2. Descriptor section
  • [0295] 3. Upload section
    Specification Section

[0296] Specification section creates and deletes agent specifications. First users create agents from the first user GUI by entering an agent specification. First user GUI will also provide a graphical interface to create an agent specification. The specification section validates the agent specification and if the validation succeeds, stores the agent specification in the data storage service, along with the namespace.

[0297] An admin user can define additional validation rules for agents in a namespace, over what the specification section performs by default. They can prevent usage of certain commands in the agent, like codeBlock. They can also specify rules on processing custom fields in the agent specification. The creation module would run additional validation specified for each namespace.

[0298] An admin user can set an approval process for a namespace that requires the admin user to approve new agent specifications and changes to existing agent specifications in the namespace. The process can also require approval to create or change agent descriptors. The approval process would be similar to app approval process on mobile platforms like Google Play Store and App Stores.

[0299] A first user can also delete an agent specification and the corresponding agent descriptor.

Descriptor Section

[0300] Descriptor section creates and deletes agent descriptors. An agent can be searched in the search engine module only if an agent descriptor is created for it. An agent can be created and tested before creating an agent descriptor.

[0301] This section validates an agent descriptor and if the validation succeeds sends it to the search engine module to index the descriptor.

[0302] As with specifications, an admin user can set up a namespace to require approvals, before a new agent descriptor is indexed or to make changes to existing agent descriptors.

[0303] When an agent descriptor is created, it sends a request to the indexer section of the search engine module to index the agent descriptor.

Upload Section

[0304] Upload section processes upload of files to the system domain. First users can upload files to the system domain and then query them from an agent using the systemQuery command. The first user would specify an upload type for each file. Each upload type will be supported by a different cloud service. Such cloud services are explained in the prior art section. The upload section can implement this functionality by interfacing with the right cloud service for each type. Uploaded files would be associated with the namespace of the first user and can be queried only by client modules associated with that namespace. This can be implemented by saving the association between the uploaded file names and namespace in the data store service or by enforcing uploaded file names to have a suffix matching the namespace.

[0305] Some of the upload types are:

• • [0306] 1. AI model files
  • [0307] 2. A full text search engine
  • [0308] 3. A vector database
  • [0309] 4. A graph database
  • [0310] 5. A relational database
  • [0311] 6. A text file

[0312] For (2), (3), (4) and (5), the upload section would create a new, private search engine or database using the cloud service when a first file is uploaded. For (2), (3) and (4) JSON files can be used. For (5) .db files can be used with SQLite or CSV files can be used with MySQL. More than one file can be uploaded to the same search engine or database. For each type, the underlying cloud service would determine what file formats are supported for that service.

[0313] The system domain itself can upload files of all the above types and make them available to all the namespaces.

First User GUI

[0314] First user GUI is used by first users and it interacts with the creation module. It provides a development environment to create agents and agent descriptors.

[0315] The first user GUI will provide at least the following functionalities:

• • [0316] 1. Provide an interactive process to create an agent specification
  • [0317] 2. Provide an interactive process to create an agent descriptor
    • [0318] a. This will interface with the search engine module to suggest values for other fields in the agent descriptor based on the description field.
  • [0319] 3. Check syntactic or functional errors in the agent specification
  • [0320] 4. Provide an interface to upload files to the system domain
  • [0321] 5. Provide an interface to upload named lists to the system domain
  • [0322] 6. Execute an agent with each example query specified in the agent descriptor to show statistics on agent's runtime.
  • [0323] 7. Show statistics on events generated by running agents.
  • [0324] 8. Provide an interface to set up a proxy API call. This can be called from a systemFunction in an agent in the first user's namespace.

[0325] First user GUIs can run from within an organization and integrate with single sign-on systems in the organization, which would integrate with the authentication module in the system domain.

Execution Module

[0326] Execution module is the point of contact for all the client modules and handles all their requests. It contains these sections:

• • [0327] 1. Searcher section
  • [0328] 2. Fetch agent section
  • [0329] 3. Function section
  • [0330] 4. Events section
  • [0331] 5. Subscription section
    Searcher Section

[0332] Searcher section forwards search requests to the search engine module and responds with the results returned by the search engine module.

Fetch Agent Section

[0333] Fetch agent section receives a request from a client module to fetch an agent specification, reads the agent specification from the data storage and returns it to the client module.

[0334] This section will check that the namespace of the agent requested is the same as the namespace of the client module making the request.

Function Section

[0335] Function section handles requests from three commands:

• • [0336] 1. systemLLM command
    • [0337] a. This translates to a call to the first LLM service. These calls will be available from any namespace.
  • [0338] 2. systemFunction command
    • [0339] a. There are two kinds of system functions.
      • [0340] i. Global functions are functions or APIs implemented in the system domain. These functions will be available from any namespace.
      • [0341] ii. Namespace specific functions are API proxy calls to public APIs. A first user can set up an API proxy call in the system domain from the first user GUI. This API can be called from the systemFunction command by any agent in the first user's namespace. The purpose of using an API proxy in the system domain is to hide the authorization details from being exposed in the agent specification.
  • [0342] 3. systemQuery command
    • [0343] a. These calls would translate to corresponding calls to each cloud service supported in the system domain. The request will be processed only if the file being queried is associated with the namespace of the client module making the request.
      Events Section

[0344] Events section handles requests from the raiseEvent command. It receives an event name and a list of name-value parameters. It stores the agent name, event name, IP address of the client module that made the request, a timestamp and the name-value parameters to the data storage service. Additionally, it also logs them to a logging service.

[0345] Admin users and first users can set themselves up to be notified when an event occurs in their namespace or agent.

[0346] The system domain will define certain events that are common to all agents. For example, an event called exceptionRaised can specify that executing an event raised an exception in a client module. Another can be maximumRuntimeExceeded event, which can be raised when an agent runs beyond a maximum time set for that namespace.

[0347] Client modules can send an agentExecuted event every time they execute an agent, with time taken for the execution as a name-value parameter.

[0348] Client modules can also use events to track feedbacks from second users for an agent. A second user GUI can show a feedback GUI widget, which a second user can use to provide feedback on the agent. Client module can record the feedback as an event. A namespace administrator can adjust the relevancy score of agents based on feedbacks.

[0349] First user GUI will provide ways to summarize event data in a namespace similar to popular tools like Splunk.

Subscription Section

[0350] A subscription section provides an interface to support subscriptions for a namespace.

[0351] An admin user can set up a namespace to support subscriptions in the admin module and set a subscription price and frequency. This namespace would then let first users mark an agent as a premium agent, which can be executed only when a second user buys a subscription to that namespace. They can also create a free version and a premium version of an agent.

[0352] The subscription section would accept subscription payments for a second user identifier using a payment gateway and would store the namespace and the second user identifier in the data storage service. A second user GUI would interact with this section to accept subscription payments seamlessly from a second user.

[0353] In each call to the execution module, client module would include a second user identifier as an input. The execution module would look up the second user identifier in the data storage and if there is an associated subscription, the request would be categorized as a premium request.

[0354] For a search agents request, the search engine module would:

• • [0355] 1. Include a variable in the result called isPremiumUser and set it to true or false
  • [0356] 2. If a request is premium, all agents would be searched.
  • [0357] 3. If a request is not premium, two searches will be made—a first search for free agents and a second search for premium agents. Results from both the first search and second search would be included in separate lists in the search results. This allows a client module to execute a free agent but show the titles of premium agents that can offer better search results in the second user GUI.

[0358] For a request to retrieve an agent, if a request is not premium and the request is for a premium agent, the request will fail.

[0359] An agent can also check ‘isPremiumUser’ variable in the agent object and execute different task units based on that.

Client Module

[0360] A client module

• • [0361] 1. Can be an independent application, be part of another application or can be built into a mobile app
  • [0362] 2. Runs anywhere on the internet
  • [0363] 3. Communicates with the execution module in the system domain
  • [0364] 4. Has access to an authorization key for a namespace
  • [0365] 5. Includes the authorization key in each request it sends to the execution module. The execution module associates a namespace with the request based on the authorization key
  • [0366] 6. Exposes a REST interface which can be consumed by a second user GUI

[0367] Many client modules would run on the internet, each representing a namespace. Multiple client modules can represent the same namespace.

[0368] The client module typically receives search requests from a second user through the second user GUI. A search request consists of:

• • [0369] 1. A search query
  • [0370] 2. One or more inputs, which are name-value pairs
  • [0371] 3. One or more input files uploaded by a second user to the client module

[0372] The client module takes the following steps when it receives a search request:

• • [0373] 1. Send a request to the execution module to search for agents with the authorization key, search query and inputs
  • [0374] 2. Receive a list of agent names and titles from the execution module
  • [0375] 3. Select the agent name that occurs first in the agent list.
  • [0376] 4. Send a retrieve agent request, authorization key and the selected agent name to the execution module and receive the agent specification of the selected agent.
  • [0377] 5. Create an agent instance using the selected agent specification
  • [0378] 6. Create an object to represent the state of the selected agent called the agent state object.
    • [0379] a. An agent state object can be implemented using an associative array data structure.
    • [0380] b. There can be several instances of the same agent running in the client module at the same time. Each instance will have its own agent state object.
  • [0381] 7. Add the search query, input name-value pairs, input files, list of agent names and titles to the agent state object. This will be the initial state of the agent state object when the agent execution starts.
  • [0382] 8. Execute the task unit that occurs first in the list of task units in the agent. To execute a task unit:
    • [0383] a. Execute each command unit in the order they are specified in the task unit.
    • [0384] b. Each command is executed as explained in the section describing that command unit in this document.
    • [0385] c. If a command calls another task, execute that task.
    • [0386] d. Execution ends when there are no more commands to execute in the previous step.
  • [0387] 9. Executing an agent can result in:
    • [0388] a. Agent completing successfully
      • [0389] i. Send an agentCompleted event to the execution module
    • [0390] b. Agent throws an exception
      • [0391] i. Send an agentException event to the execution module
      • [0392] ii. Execute the next agent in the list.
    • [0393] c. Agent runs for too long
      • [0394] i. Client module would check the runtime of an agent after executing each command and compare it to a maximum configured. If the runtime exceeds the maximum, it will stop executing the agent.
      • [0395] ii. Send a maximumRuntimeExceeded event to the execution module. This can include time taken by every command in that execution.
      • [0396] iii. Execute the next agent in the list
  • [0397] 10. Once execution ends, return the agent state object to the caller in JSON format.

[0398] In processing a search query, a client module can choose not to execute an agent automatically. It can instead show a list of agent titles on the second user GUI and let a second user choose one or more agents to run.

Second User GUI

[0399] Second user GUI is specific to a client module. It is used by second users. At a minimum, the second user GUI takes a search query from a second user and shows the result from executing an agent.

[0400] Second user GUI can be customized to:

• • [0401] 1. Show a list of evaluated titles of matched agents to a search query and let a second user pick an agent to execute.
  • [0402] 2. Execute multiple agents and show results from each one on the GUI

[0403] A second user GUI can be a web page, an independent application or part of a mobile app.

[0404] Every agent in a namespace can be accessed with a URL relative to a client module. A second user can bookmark an agent from the second user GUI and execute a specific agent. They can also use call an agent's URL from a script or a program.

Multiple Client Modules with the Same Authorization Key

[0405] An organization can run several client modules, each customized to a different environment, like Java or Python, or customized with different language packages, or running in different areas of a corporate intranet with access to different systems and private APIs.

[0406] With this setup, a central client module can run search queries and from the search results, forward an agent name to the right client module for execution. A second user GUI connected to the central client module will be able to engage all the agents in that namespace, although each agent requires a different environment to execute.

Publishing an Authorization Key for a Namespace

[0407] An admin user can publish the authorization key of a namespace publicly. This allows anyone to run a client module with that authorization key.

[0408] With this setup, agents can be created which access to a system or function available only from a second user's machine. This can be a useful technique to run customer support agents or software tools.

[0409] For example, an agent can respond to search queries like “Are there single point of failures in my Solr cloud running at http://localhost:1100”. This agent would make a systemLLM call to generate the code that can query a Solr cloud, pass the generated code to a codeBlock command and execute the code to produce an answer. The URL http://localhost:1100 is accessible only from a second user's machine.

[0410] FIG. 1A describes an agent using a class in an object oriented design in a UML class diagram. An agent specification described earlier can be converted from JSON to the Agent class shown in this figure using standard JSON to object libraries like Google Gson. The arrows indicate one-to-one and one-to-many relationships. Each command in an agent specification described earlier is shown as a value in the CommandType enumeration.

[0411] FIG. 1B shows each command in the agent specification as a derived class of CommandSpec in an object oriented design in a UML class diagram.

[0412] FIG. 2A illustrates the components of the invention. The boundaries of the system domain are marked with a thick outline. The four bottom row modules, Authentication module, Admin module, creation module and execution modules, do not communicate directly with each other and they can be accessed from the internet. Data storage service, first LLM service and the search engine module cannot be accessed outside of system domain. First LLM service is read-only. Data storage service is read and written from all the four bottom row modules. Creation module indexes into the search engine module and execution module searches from it. The direction of arrows represent the flow of data between modules.

[0413] Three client modules are shown each with a second user GUI. Each client module and second user GUI may have customizations specific to their own namespaces.

[0414] The lines and arrows flowing out of the system domain indicate a majority communication flow between modules and GUIs. Client modules communicate only with the execution module and this is shown by lines diverging from the execution module, through the internet. Admin GUI communicates with the admin module and first user GUI communicates with the creation module.

[0415] Authentication module is used by both admin GUI and creation GUI, but this is not shown in the picture, as it just implements standard website functionality.

[0416] FIG. 3A illustrates the flow of information and actions that take place when a second user enters a search query. The modules and GUIs are shown on the left column and the action that takes place inside the module or GUI is shown on the right column.

[0417] FIG. 4A illustrates a first embodiment of the invention where a first organization owns a first namespace and provides enterprise solutions within its corporate intranet called org 1. The boundaries of org 1 are shown with a thick outline. For clarity, only the three modules of the system domain are shown, as these are the ones that communicate with org 1.

[0418] Admin GUI and first user GUI are shown inside org 1 to indicate that they use a corporate single sign-on mechanism to login to the system domain and to indicate that admin users and first users would be first organization personnel.

[0419] A fourth client module runs inside org 1 and connects with the execution module. Agents in the first namespace run in the fourth client module. They can access private APIs and corporate Information Technology (IT) systems that are available only within org 1. They can also access public APIs on the internet. A second user GUI 4 shown connected to the fourth client module is available only from within org 1, as second users would also be first organization personnel.

[0420] FIGS. 5A and 5B Illustrate a second embodiment of the invention where a second organization owns a second namespace and provides internet search and E-commerce services. The second organization controls an intranet called org 2 and runs a fifth client module inside org 2. It offers a free service which executes a subset of agents in the second namespace and a subscription-based service, which executes any agent in the second namespace.

[0421] Second organization personnel can become first users in the second namespace, as shown by first user GUI inside org 2. Second organization also authorizes non-personnel on the internet to be first users in the second namespace. This is indicated by two first user GUI boxes outside org 2.

[0422] Personnel from the second organization will approve all the agent specifications and agent descriptors that are created in the second namespace. This is indicated by an admin GUI inside org 2.

[0423] All the agents are executed in the fifth client module, which runs inside org 2. This allows the second organization to make private APIs and private IT systems available to agents in the second namespace.

[0424] Three second user GUI 5s are shown to indicate that anyone on the internet can run search queries in the second namespace. Second user GUI 5 can be an internet browser like Google Chrome, second user GUI 6 can be a GUI inside a mobile app and second user GUI 7 can be inside an app in a wearable device.

[0425] FIG. 5B illustrates second user GUI 5 when a search query is executed. A second user entered the query ‘When can I go for a walk today?’ and got an answer. There are three premium agents shown below the answer that are available only with a subscription. A textbox is also shown for a follow up question. The mechanics of an agent that can provide this answer is illustrated in the ‘illustration of an agent’ section.

[0426] FIG. 6A Illustrates a third embodiment of the invention in which a third organization owns namespace 3 and provides mobile app services. The third organization controls an intranet called org 3. The third organization sells an app that is packaged with a sixth client module.

[0427] The mobile app has three versions, mobile app 1, mobile app 2 and mobile app 3 for App Store, Play store and a watch platform. All three versions of the sixth client module include the same authorization key, which is associated with the third namespace. All three versions of the mobile app support search queries entered both textually in the app and verbally to the mobile device.

[0428] Just like the second organization, the third organization allows both third organization personnel and others to be first users, but third organization personnel approve all the agent specifications and descriptors.

[0429] Each sixth client module converts an agent specification into an agent instance in that native platform. Sixth client module 1 will convert an agent specification into an agent instance on iOS, sixth client module 2 into an agent instance on Android and sixth module 3 into an agent instance on the watch platform. This allows a single agent specification to execute on all the three platforms.

Illustration of an Agent

[0430] The following paragraphs illustrate a first agent using agent specification, agent descriptor and an agent state object.

[0431] These illustrations use $(and) as delimiters for expressions in f-strings. Python language is used in the following illustrations with dot notation to denote lookups in a dictionary. That is, if an associative array ‘foo’ contains an object ‘bar’, instead of accessing the object as foo[“bar” ], the illustrations use foo.bar. Dot notation can be achieved in Python using third party libraries.

[0432] The first agent does the following:

• • [0433] 1. Determine the location of a second user from an IP address in the input.
  • [0434] 2. Use the US government weather service to find the weather in that location.
  • [0435] 3. Pass the weather information and the location from (2) to an LLM as a prompt
  • [0436] 4. Get an answer from the LLM and return it to the caller.
  • [0437] 5. If there is a follow up question, send that question to the LLM along with the previous question and answer and get a new answer.

[0438] The first agent defines three task units called firstTask, coreTasks and llmTask.

[0439] firstTask has a single conditional command. The conditional command checks whether the first agent is answering a first question or a follow up question. If it is a follow up question, it uses restoreState command to restore the state of the first agent from a previous state, that is, when it answered the first question. It then calls llmTask with the question and answer from the previous state. If the first agent does not have a previously saved state, firstTask calls coreTasks.

[0440] The first command in coreTasks calls a systemFunction called ‘getLocation’ with the ip address in the inputs as the parameter. The second command in coreTasks calls US weather service API using location information and also specifies an alternate command is the call to the weather API fails. A third command makes another API call to the weather service to get the forecast information. Next, a codeBlock command constructs an AI prompt from the weather forecast, including. The last command calls the llmTask.

[0441] The first command in the llmTask calls a LLM with a prompt. The prompt is created by combining a previous state, if that exists, with the current search query. The command stores the response in a variable called agentResponse. The second command saves the current state of the first agent to prepare for a follow-up question. The third command defines the follow up question as an additional input.

[0442]

First agent specification:
{
″agentName″: ″DailyActivity″,
″agentTitle″: ″Answer to your question related to $(activity) in $(location.city)″,
″taskUnits″: [
{
″taskName″: ″firstTask″,
″commandUnits″: [
{
″commandType″: ″conditional″,
″commandSpec″: {
″if″: ″$(exists(agentState.inputs.savedState)″,
″then″: [
{
″commandType″: ″restoreState″,
″commandSpec″: {
}
},
{
″commandType″: ″execTask″,
″commandSpec″: {
″taskName″: ″llmTask″,
″endThis″: true
}
}
],
″else″: [
{
″commandType″: ″execTask″,
″commandSpec″: {
″taskName″: ″coreTasks″,
″endThis″: true
}
}
]
}
}
]
},
{
″taskName″: ″coreTasks″,
″commandUnits″: [
{
″commandType″: ″systemFunction″,
″commandSpec″: {
″name″: ″getLocation″,
″params″: {
″ip″: ″$(ip)″
},
″responseKey″: ″location″
}
},
{
″commandType″: ″apiClient″,
″commandSpec″: {
″endpoint″: ″https://api.weather.gov/points/$(location.latitude,location.longitude)″,
″responseKey″: ″pathToWeather″
},
″alternateCommands″: [
{
″commandType″: ″execTask″,
″commandSpec″: {
″taskName″: ″″
″endThis″: true
},
″variableUnit″: [
{
″agentResponse″: ″Temporary error in executing this agent″
}
]
}
]
},
{
″commandType″: ″apiClient″,
″commandSpec″: {
″endpoint″: ″$(agentState.PathToWeather.properties.forecastHourly)″,
″responseKey″: ″hourlyWeather″
},
″variableUnit″: [
{
″llmQuestion″: ″″
},
{
″agentResponse″: ″″
}
]
},
{
″commandType″: ″codeBlock″,
″language″: ″python″,
″code″: ″for period in
agentState.hourlyWeather[′properties′][′periods′].items( ):\\n\\tagentState.llmQuestion =
agentState.llmQuestion + f′Weather {period.number} hours from now is {period.temperature}F
and {period.shortForecast}.′″
},
{
″commandType″: ″execTask″,
″taskName″: ″llmTask″
}
]
},
{
″taskName″: ″llmTask″,
″commandUnits″: [
{
″commandType″: ″systemLLM″,
″commandSpec″: {
″name″: ″llama_3_2″,
″parameters″: {
″modelInput″: ″$(agentState.llmQuestion) $(agentState.agentInput.SearchQuery)″,
″temperature″: 0.7
},
″responseKey″: ″agentResponse″
},
″variableUnit″: [
{
″llmQuestion″: ″$(agentState.llmQuestion) $(agentState.agentInput.SearchQuery)
$(agentState.agentResponse)″
}
]
},
{
″commandType″: ″saveState″,
″storageKeyName″: ″saveStateKey″
},
{
″commandType″: ″followupInputs″,
″commandSpec″: {
″inputs″: {
″searchQuery″: {
″hint″: ″Do you have a follow up question?″
},
″savedState″: {
″value″: ″$agentState.saveStateKey″
}
}
}
}
]
}
]
}

Illustration of Agent Descriptor

[0443] An agent descriptor for the first agent is illustrated here using the JSON format.

• • [0444] 1. agentName below has the same value as the first agent specification above and refers to the first agent of this descriptor.
  • [0445] 2. Description is a text field that describes the first agent.
  • [0446] 3. Filter fields consist of a name and a list of values. First agent is matched only for search queries in the English language, from the US, location based, from the ‘prod’ environment and in a healthy state.
  • [0447] 4. Classes are specified for the ‘Current’ classification explained earlier. The classes can be generated from the description field using the first LLM service. First agent is applicable only for search queries that are weather or health related.
  • [0448] 5. A list of keywords is specified next. If the search query has one of these keywords, the first agent gets a higher relevancy score.
  • [0449] 6. There are no namedLists for the first agent.
  • [0450] 7. Search queries for the first agent will not contain any patterns.
  • [0451] 8. namedEntities field is specified with one named entity of ‘activity’. Min specifies that it should be present at least once in the search query. Further, the value of the ‘activity’ in the search query should fall under the classification of ‘outdoor’.
  • [0452] 9. A list of example queries is specified, which can be generated from the description using the second LLM service.
    First Agent Descriptor:
  • [0453] {
    • [0454] “agentName”: “DailyActivity”,
    • [0455] “description”: “This agent looks at today's weather in your location and answers a weather related question.”,
    • [0456] “filters”: {
      • [0457] “query_language”: [
        • [0458] “English”
      • [0459] ],
      • [0460] “location_countryCode”: [
        • [0461] “US”
      • [0462] ],
      • [0463] “location_based”: [
        • [0464] “true”
      • [0465] ],
      • [0466] “environment”: [
        • [0467] “test”
      • [0468] ],
      • [0469] “state”: [
        • [0470] “healthy”
      • [0471] ]
    • [0472] },
    • [0473] “classifications”: {
      • [0474] “current”: [
        • [0475] “weather”,
        • [0476] “health”
      • [0477] ]
    • [0478] },
    • [0479] “keywords”: [
      • [0480] “exercise”,
      • [0481] “fitness”,
      • [0482] “today”,
      • [0483] “walk”,
      • [0484] “run”,
      • [0485] “bike”,
      • [0486] “swim”
    • [0487] ],
    • [0488] “namedLists”: [ ],
    • [0489] “patterns”: [ ],
    • [0490] “namedEntities”: {
      • [0491] “activity”: {
        • [0492] “min”: 1,
        • [0493] “classification”: [
        • [0494]  “outdoor”
        • [0495] ]
      • [0496] }
    • [0497] },
    • [0498] “queries”: [
      • [0499] “When would be an ideal time for a walk today?”,
      • [0500] “Is it a good time to mow my lawn?”
    • [0501] ]
  • [0502] }
    Illustration of an Agent State Object

[0503] The following paragraphs illustrate an agent state object, when an instance of the first agent is executed in a client module.

[0504] When an agent state object for the first agent is initialized by a client module, it will have the following information. This represents the inputs, search query and the response from the search engine module.

• • [0505] agentState={
    • [0506] “agentName”: “DailyActivity”,
    • [0507] “SearchQuery”: “When should I go for a walk today?”,
    • [0508] “ip”: “173.73.20.1”,
    • [0509] “isPremiumUser”: false,
    • [0510] “premiumAgents”: [
      • [0511] {“agent2”: “Weather forecast in Washington, D.C.” },
      • [0512] {“agent3”: “Events happening now in Washington D.C.” },
      • [0513] {“agent4”: “Best walking paths in Washington D.C.” }
    • [0514] ]
  • [0515] }

[0516] The first agent first calls the system function getLocation. After this call, the agent state object will have an additional variable:

• • [0517] agentState[“location” ]={
    • [0518] “country_code”: “US”,
    • [0519] “city”: “Washington, D.C.”
    • [0520] “zip_code”: “20001”,
    • [0521] “latitude”: 38.89539,
    • [0522] “longitude”: −77.03948
  • [0523] }

[0524] Next the first agent executes an apiClient command. The variable pathToWeather will be set from the response from the US weather API. The actual US weather API on the internet has more fields, but only a few fields are shown here for clarity.

• • [0525] agentState[“pathToWeather” ]={
    • [0526] “properties”: {
      • [0527] “forecast”: “https://api.weather.gov/gridpoints/LWX/96,71/forecast”,
      • [0528] “forecastHourly”: “https://api.weather.gov/gridpoints/LWX/96,71/forecast/hourly”,
      • [0529] “forecastGridData”: “https://api.weather.gov/gridpoints/LWX/96,71”,
      • [0530] “observationStations”: “https://api.weather.gov/gridpoints/LWX/96,71/stations”
    • [0531] }
  • [0532] }

[0533] After calling the second apiClient command in the coreTasks unit, the hourlyWeather variable would be set from the response from the second call to the weather API. Once again, only relevant fields from the response are shown here.

• • [0534] agentState[“hourlyWeather” ]={
    • [0535] “properties”: {
      • [0536] “periods”: [
        • [0537] {
        • [0538]  “number”: 1,
        • [0539]  “temperature”: 65,
        • [0540]  “shortForecast”: “Scattered Showers And Thunderstorms”
        • [0541] {,
        • [0542] }
        • [0543]  “number”: 2,
        • [0544]  “temperature”: 68,
        • [0545]  “shortForecast”: “Slight Chance Rain Showers”
        • [0546] {,
        • [0547] }
        • [0548]  “number”: 3,
        • [0549]  “temperature”: 72,
        • [0550]  “shortForecast”: “Patchy fog”
        • [0551] {,
        • [0552] {
        • [0553]  “number”: 4,
        • [0554]  “temperature”: 75,
        • [0555]  “shortForecast”: “Clear skies”
        • [0556] }
      • [0557] ]
    • [0558] }
  • [0559] }

[0560] The first command in llmTask calls a hosted AI model called llama_3_2. The following is an illustration of a response from a LLM:

• • [0561] agentState[“agentResponse” ]=“Based on the forecast:
    • [0562] **65° F., Scattered Showers and Thunderstorms**: Not ideal, due to rain and storms.
    • [0563] **68° F., Slight Chance Rain Showers**: There's a slight chance of rain, but still not optimal.
    • [0564] **72° F., Patchy Fog**: This is close to your ideal temperature, though fog might reduce visibility.
    • [0565] **75° F., Clear Skies**: Warmer than your preferred temperature, but the clear skies make it a great option.
    • [0566] Since your ideal temperature for a walk is 72° F., I suggest going during the **72° F.** hour with patchy fog. It's the closest to your preferred conditions, and while there might be some fog, it's a reasonable trade-off compared to thunderstorms or warmer temperatures later.”

[0567] The last command in llmTask sets additional inputs with a hint.

[0568] The agentResponse variable is sent back to the second user GUI.