Skip to main content

Gen AI Management

With Teneo Gen AI Management, users can build, manage, and visualize Gen AI agents that are capable of performing numerous runtime tasks. This framework enables users to craft underlying Gen AI prompts through guided UIs, modular implementation, and with full platform support, thus streamlining development, enhancing collaboration, and delivering secure, resilient, and brand-aligned AI interactions.

The sections on this page provide information about the various aspects of Gen AI Management in Teneo:

Concept​

Gen AI Management is native in Teneo and facilitates the implementation of Gen AI driven runtime Tasks with designated guided UIs and bring Gen AI usage across a solution to life.

Users can easily build, manage, and modify Generative AI agents in a modular way, implementing Gen AI driven tasks through specific guided UI processes, where the system is in charge of composing the final prompt, ensuring it is secure, complete, robust, functional, and on brand.

Users can create Personalities and Goals to perform specific Gen AI Tasks. This way, it is possible to:

With the Generative AI features, the workload can be split between members of a team based on their roles/responsibilities to reduce the cognitive load and make the development more efficient. The users can connect to the Gen AI service of their choice and multiple connections can be defined for different purposes.

The Gen AI Overview graphically visualizes all use of Gen AI throughout the solution, as well as allowing for easier troubleshooting, maintenance, and navigation help. The Overview can be used to explore usage across the solution, to validate that the correct personalities or handlers are in use for each goal and to see in which flows Gen AI is in use. The Gen AI Overview is also where you can see the final prompt that is to be sent to the Gen AI system for a given Task.

tip

The user has full freedom when defining Personalities, Goals and Tasks, but recommendations of what should be included in each section to compose the final prompt are given in tooltips in Teneo Studio and in the UI section of this page.

Personalities​

A custom Personality interacts with the end-user by giving output in a flexible and imaginative way based on the specifications in the Personality, Goal, and Output sections.

The System Personality, available out-of-the-box, is designed to boost efficiency and perform various tasks that are not shown to the end-user. You can annotate a user input to be used in a Match or perform an Action with the help of the System Personality.

note

Users can create as many Custom Personalities as they need for their various use cases.
There is only ever 1 System Personality.
Any Personality can be associated with as many Goals as needed.

Custom Personality​

A Personality defines the tone, style, and behavioral traits of a Gen AI agent. It acts as the narrative layer that shapes how the AI communicates with end-users and provides guidelines on how to best reflect the company brand. Personalities are especially important when the Gen AI prompt is exposed to end users, such as in customer support or brand engagement scenarios. For tasks not involving generating text to be displayed directly to the user, consider using the System Personality instead.

Personalities can be associated with one or more Goals, ensuring consistency across tasks and easier maintenance. It is also possible to adjust the agent's tone of voice, with parameters Creativity, Formality, Friendliness, and Verbosity, and set a brand identity. See Personalities tab for more information on the sections.

For a practical use case, please see Rephrasing Output.

System Personality​

The System personality comes pre-configured, out-of-the-box with Teneo and is specifically optimized to perform tasks that require the Gen AI model to act as a system rather than taking on a persona. The system agent is your go-to personality for any tasks where the result from the Gen AI model isn't exposed to the end user in form of a response, but rather for tasks that result in commands to be used internally in Teneo, such as:

  • Intent classification
  • Entity extraction
  • Summarization
  • Sentiment analysis
  • Input annotation

Unlike user-facing Personalities, the System agent is intended to operate silently in the background, executing logic that supports the broader conversational experience.

For practical use cases, please see Intent Classification and Dialogue Summarization.

note

As the System agent isn't designed to output text directly to the end-user, it does not have the parameters Creativity, Formality, Friendliness, and Verbosity for modifying the tone of voice.

Goals​

A Goal should outline the overall purpose or job that a given agent must achieve and what knowledge or scope is required. The Goal will specify which Personality it is associated with, what Handler to use, and the required format of the result from the Gen AI model. A Goal is associated with one Personality, and its overall purpose can be achieved by one or multiple Tasks.

Goals ensure:

  • Consistency: The same logic and tone are applied across different use cases.
  • Reusability: A single Goal can be used for multiple Tasks, reducing duplication and improving maintainability.
  • Modularity: You can swap out Handlers or Personalities without rewriting the entire setup.
info

A Goal can only be associated with one Handler at a time. If the developer wants to compare the results for two different Gen AI systems, a new identical Goal could be created, using a different Handler.

Changing from one Gen AI system to another can affect the output as much as changing prompt or the tone of voice settings; testing different systems or models is highly encouraged.

A Goal assigned to the System Personality could for example look like one of the following:

Your goal is to summarize dialogues. Please give a summary of the available conversation.
Classify an input from a user into predefined categories.

A custom Personality could, for example, be associated with one of the following Goals:

  Your goal is to rephrase the default response of a chatbot given information in a user input.
  The rephrased answer should contain the same information of the default response, but be rephrased according to the user’s personal information.
  Where relevant, information about the user included in their input should be incorporated into the rephrased answer, e.g.:
  - by using their name
  - by referencing their location
  - by referencing other relevant information in their input
Your goal is to report to the user that you don't have information about the request or that you don't understand the user's question.

A Goal can be as short or long as needed, but it is recommended to keep it as concise as possible. Please see Best Practices for more tips.

Tasks​

Tasks define the specific event that should happen for the Gen AI use case. A Task is carried out based on a previously defined Goal and Personality that it has been assigned and the final prompt that will be sent to the Gen AI System combines all the provided information and prompts from a Personality, a Goal and a Task.

info

The prompts for any Task support string interpolation (with the syntax ${variable}), making it possible to embed any Groovy such as Global and Flow Variables or Engine access methods to create a dynamic prompt.

Annotators​

Gen AI Annotators will annotate the input with the result of a Gen AI prompt. This annotation can then be used for matching in a Gen AI Match on a Trigger or Transition.

The annotation will be named the same as the Gen AI Annotator document (normalised to be unique and valid for TLML), and with a suffix .GEN_AI and it holds a variable named value, which will contain the result of the Gen AI prompt. It is of course also possible to refer to a Gen AI annotation from TLML syntax, just like with any other Annotation. Gen AI Annotators use Deferred Annotations "behind the scenes" and as such are guaranteed to only be called when required, which optimizes the impact of Gen AI usage.

Output​

Gen AI Output generates and displays a response to the end-user based on a prompt. It replaces traditional output nodes in Flows where dynamic, AI-generated responses are preferred. A Flow can have a combination of Gen AI Output nodes and "traditional", static Output nodes to create a hybrid implementation.

Action​

The Gen AI Action executes a prompt and stores the result of the prompt in a variable for further use in the dialogue. An Action can be used for internal logic and tasks, such as summarizing a dialogue, detect and extract entities from an input etc.

Handlers​

A Handler is responsible for calling the Gen AI Model API and understanding the response. The Handler is implemented as a Groovy script which is able to make the required API calls and convert the response into the appropriate format for Teneo to understand.

It is strongly recommended to use a Secret script and Secrets for the actual HTTP requests to the Gen AI API in order to keep any API keys, tokens or locations protected within the solution - the Handler should call this Secret implementation.

The Handler script receives two variables (prompt & temperature) and is responsible for calling the AI Service (via a secret script) with these parameters and returning the result. The Handler script must set the variable chunker before completing. This Chunker is an object which provides appropriately sized chunks to the solution for output/handling - "Appropriate size" for the chunks depends on the output channel, but paragraphs work well for most scenarios.

In/OutParameterTypeDescription
InputpromptStringComplete prompt, constructed from the information provided in the specific Personality, Goal and Task.
InputtemperaturefloatDerived from the creativity, defined in the Personality configuration and is a float value between 0 and 1.
OutputchunkerChunkerMust be populated before the end of the handler script. Provides the solution with access to the prompt response (in chunks). see Chunker specifications.

The UI provides a simple template of the handler script as a starting point of it's implementation, and it can be adapted so that the user can connect to a Gen AI service of choice. For an example on how to connect to a deployment in Azure Cloud, read more at Configure the handler

note

Although supported, it is not recommend to use DeepSeek models due to data privacy concerns.

Chunker specifications​

The return from the Handler method must be a class that supports stream responses buffered in an internal queue, with partial responses represented as Chunk objects. The specification for this class is defined here - the implementation is case specific and becomes part of the solution.

Methods

Method NameReturn TypeDescription
getAggregatedChunk()ChunkReturns a chunk with the response text. The consumer waits to all the partial responses, chunks, and aggregates the result in a single chunk.
getNextChunk()ChunkReturns the next chunk with the next partial response - used when streaming. The last partial response must be a chunk with an empty response, content = ““, and isFinal = true.

Chunk specifications​

The Chunk class represents the actual returned chunks. This is a simple data holder (record) class and can be shared by all Chunker implementations. To do this define the Chunk class in an On Solution Loaded Global script.

Field NameReturn TypeDescription
contentStringResponse text
isFinalbooleanWe’ve receive the last part of the response
isErrorbooleanThe consumer has finished unexpectedly. Check the errorMessage field to get more details.
errorMessageStringError message if necessary

Security​

The Prompt editor includes strict security measures to ensure the responsible and secure use of LLMs in your System Agents and Personalities. These measures provide protection against jailbreaking and attacks, but also ensure alignment with company policies and ethical standards.

  • Malicious user inputs are detected to prevent prompt injection, abuse, or manipulation of the model's behavior.
  • Avoid misuse of personal information if provided in the prompt.
  • Prevent the LLM requesting sensitive data or recommending dangerous actions.

Additional security measures may be available through your Cloud provider; please contact them for more information.

info

The security measures have been designed to be non-disruptive and to protect against potential issues. We recommend you keep them on, especially when handling user-sourced dynamic information.

RAG scenarios for example are especially susceptible as there is access to more information - and more user input involved in the prompting. RAG implementations hand over large portions of user input directly to Gen AI to extract the information and provide an answer to the end user.

Overview​

With the Gen AI Overview, the user can see all usages of Gen AI features throughout the solution and how they work together to deliver the desired functionality. The Overview can be used to explore usage across the solution, to validate the correct Personalities or Handlers are in use for each Goal and to see in which Flows Gen AI is in use. The Overview is updated in real time, so you always see when you create a new element and what elements are related to each other. The Gen AI overview is also where you can see what the final prompt sent to the Gen AI system for any given Task.

How To​

Configure the Handler​

note

This guide assumes that the user has a running deployment in Azure cloud. It also assumes the user has sufficient user permissions to work with Secrets and secret scripts.

Below guide provides and example for how to connect to a deployment in the Azure cloud and how to make it accessible for a Gen AI Handler.

First of all, under the Account, create the following Secrets and populate them with the values for your deployment in Azure:

SecretDescription
openai_resource_nameResource name
openai_deployment_idDeployment name
openai_api_keyPrivate key
openai_api_version *API version

*Even though the API version is not a confidential parameter, it is stored as another secret to tie it to the deployment.

The scripts available for download are the following:

Solution Loaded scripts

  • Chunk A simple declaration of the chunk object with its required parameters
  • Open AI SSE Chunking Consumer A template implementation of a Chunker consumer
  • Open AI Prompt Handler A template with the OpenAI API standards

Secret Script

  • OpenAI Connection Handler A secret script with the http connection to an OpenAI endpoint

Handler Script

  • GenAI Handler openAI Populates the chunker with the provided parameters

To set up a handler, follow these steps:

  1. In your solution, navigate to Global Scripts, create the above three Solution loaded scripts and paste as their script the equivalent script available from this Download.

  2. The execution ordering of the scripts is important, The Chunk record is used by the SSE Chunking Consumer, and the SSE Chunking Consumer must be declared before the Prompt Handler. With Script ordering, ensure that the Chunk consumer script is ordered above the Prompt Handler.

    order of scripts

  3. Add a new Secret script OpenAI Connection Handler and paste as its script the equivalent script available from this Download.

  4. Navigate to the Handlers tab in the Gen AI Manager. Add a new Handler named OpenAI Handler and replace the default script with the script available from this Download.

  5. Once all documents are created and saved, you can test and verify your new Handler, for example with one of the use cases from the Practical Examples.

UI​

In Teneo Studio Desktop, you can access the Gen AI Manager from the top ribbon bar:

ribbon bar

The Gen AI Manager in Teneo Studio Desktop has two tabs:

Gen AI Overview​

When clicking the Gen AI Manager, the Gen AI Overview opens. The overview helps the developer to visualize the use of Personalities, Handlers, Goals, and Tasks in the solution. It is also possible to see which Flows has Gen AI Tasks.

gen ai overview web

In the example above, the Personality Serious is used with the handler GPT Input Prompt to perform the Goal Complaint Handling which uses an annotator Sentiment Analysis to annotate user input. This annotator is used by a Gen AI Match in the flows Complaint - Friendly and Complaint - Angry.

When selecting a Task, the user can also see the final prompt that is to be sent to the Gen AI system for that given Task.

Personalities tab​

When clicking the Gen AI tab, the Personalities section opens:

desktop personalities

SectionDescription
SystemTeneo's out-of-the-box System Personality.
Edit optionsSave or Cancel when editing, or Edit and Delete when viewing. You can also view the History and Set Stable, Include/Exclude from branching.
NameAdd a Name that helps to identify the Personality.
LanguageThe language used for an agent is the solution language.
OutlineGive an Outline which defines a role, background and/or core traits for the agent. You can use the default Outline as a starting point or provide your own e.g. You are a helpful AI assistant designed to provide accurate responses to user queries.
InstructionsProvide any instructions that should be followed when interacting with the end users. You can use the pre-defined instructions, adapt them or create your own instructions.
VerbosityControls how wordy or concise your agent should reply.
CreativityControls the amount of freedom the agent has when crafting an answer. A higher value would make an answer more likely, whereas a lower value would make the answer more controlled.
FriendlinessControls how polite or friendly your agent should act.
FormalityControls how formal or informal your agent should act.
SecurityToggle security settings on or off. It is recommended to keep the security settings on.

Goals tab​

When selecting Goals, the user finds the following view:

goals desktop

SectionDescription
Edit optionsSave or Cancel when editing, or Edit and Delete when viewing. You can also view the History and Set Stable, Include/Exclude from branching.
NameGive a Name to the Goal for easy identification.
PersonalitySelect the associated Personality. The system is selected by default unless the solution contains at least one Personality. In that case, the first Personality in the list is selected by default.
OutlineOutline what the overall purpose and knowledge/scope for the Goal is by stating what the Gen AI model should achieve, e.g. Your goal is to answer questions about incidents. or You have knowledge about returns and complains or You are an entity extractor
HandlerSelect the associated Handler.
FormatDescribe in what format the results from the Gen AI model should be given. You can use the default format or provide your own, e.g. Max three short sentences or Give the output as a JSON. For better precision, you can also provide some examples of the desired format.
info

By default, if the Solution has at least one Personality, the Goal will be associated with the first Personality of the list. If no Personalities are defined, the Goal will be associated to the System by default.

Annotators tab​

When selecting Annotators, the user finds the following view:

annotators desktop

SectionDescription
Edit optionsSave or Cancel when editing, or Edit and Delete when viewing. You can also view the History and Set Stable.
NameGive the Annotator a meaningful name for easy identification.
DescriptionThe description is an internal text field that can be used for comments among solution developers, for example, for describing the purpose of the Annotator.
GoalSelect the associated Goal.
Annotation nameWhen saved, the annotation name is generated automatically by Teneo, based on the name of the Annotator document. The Annotation carries the suffix .GEN.AI for easy identification.
PromptThe exact instruction or question to get a specific result from a Gen AI Model. This field supports string interpolation (with the syntax ${variable}), making it possible to embed any Groovy such as Global and Flow Variables or engine access methods to create a dynamic prompt.

Handlers tab​

When selecting Handlers, the user finds the following view:

handler desktop

SectionComment
Edit optionsSave or Cancel when editing, or Edit and Delete when viewing. You can also view the History and Set Stable, Include/Exclude from branching.
NameGive a name that helps the user to identify the Gen AI Handler and its use within the Solution.
DescriptionThe description is an internal text field that can be used for comments among solution developers, for example, for describing the purpose of the Handler.
ScriptA Groovy script which is able to make the required API calls and convert the response into the appropriate format for Teneo to understand. When creating a new Handler, a template script is provided, which the developer can use as a starting point. See Configure the Handler and Gen AI Handler script for more information.

Gen AI Flow nodes​

Gen AI Action and Output Nodes can be added to the Flow in the ribbon bar.

d_add_genai_node

Practical Examples​

Rephrasing Output​

For this example, we will be defining a custom Personality for a fictional restaurant named TableTop Bistro. Our use case is to let a Gen AI rephrase our Flow's answer texts, making it more personal by taking into account relevant information from the user input.

Setup​

First, let's set up and configure the documents needed.

info

The example assumes that you already have configured a Handler.

We need to add a new Personality:

  1. Give it a name such as TableTop Bistro.
  2. Replace the default Outline with the following:
You are a virtual assistant working for "TableTop Bistro", a cozy neighborhood restaurant that offers a chatbot to help customers with menu inquiries and reservation support.

  1. You can use the default instructions and perhaps expand them with:

    - Provide clear and concise information about the menu and services.

  2. Optionally, use the sliders to adjust the tone of voice of your agent.

Next, let's add the Goal:

  1. Add a new Goal and name it Rephrase response.

  2. Assign the Personality you created above.

  3. Paste in the following outline:

    Your goal is to rephrase the default response of a chatbot given information in a user input.
    The rephrased answer should contain the same information of the default response, but be rephrased according to the user’s personal information.
    Where relevant, information about the user included in their input should be incorporated into the rephrased answer, e.g.:
    - by using their name
    - by referencing their location

  4. Assign it your Handler of choice.

  5. For this goal, it makes sense to keep the default format Always give the answers as plain text.

Create your Flow​

Let's create a Flow where we want to customize the bot's answer.

  1. Create a new Flow.
  2. Add a Match to trigger on relevant inputs. In our example, we will be using this Flow to respond to requests about vegetarian food options at the restaurant, so for this simple example let's add a Language Object Match with the value VEGETARIAN.ADJ.LEX.
  3. Next, replace the default Output node with a Gen AI Output node.
  4. Assign to it the Goal you just created, and add the following prompt:
Rephrase the output "Yes, we offer several vegetarian dishes on our menu." given the user input ${_.userInputText}.

Now, head over to Tryout and test an input that will trigger the Flow, for example: Hi, I'm Sarah from Austin. Do you guys have vegetarian options?. The output given should be rephrased by the Gen AI system based on the input given.

Intent Classification​

For this example, we will use Gen AI for classifying the users intents into one of several predefined categories relevant to a simple banking use case.

Setup​

First, let's set up and configure the documents needed.

info

The example assumes that you already have configured a Handler.

Intent classification is an internal task, and for this, we will use the System agent, but we will need to define a generic intent classification Goal:

  1. Add a new Goal and name it Intent Classification and assign it the System Personality.
  2. As an Outline, add something like: Classify an input from a user into predefined categories.
  3. Assign it your Handler of choice.
  4. For the matching process, we need the Gen AI system to respond only with the name of the detected intent and to achie this, we replace the default format with something like A string consisting of the relevant intent name in CAPITAL letters.

Next, we will need to add an Annotator, where we tell the Gen AI system our possible intent that we want the user input classified as:

  1. Add a new Annotator named Banking intent classification and assign it the previously created Goal.

  2. Paste in the following prompt:

    You only have one job. Based on the following intents and their descriptions, Please classify this text: ${_.userInputText} as one of them. Answer only with the intent name.

    Intent name: TRANSFER_MONEY
    Description: The user wants to send or transfer money to friends or family.

    Intent name: CANCEL_CREDITCARD
    Description: The user needs to cancel their credit card because it is lost or stolen.

    Intent name: OPEN_ACCOUNT
    Description: The user wants to open a new bank account such as a savings account or checking account.

    Intent name: ONLINE_BANKING
    Description: The user has questions or issues related to online banking.

    If the input doesn't correspond to any of the Intent, just answer UNKNOWN.

Create your Flows​

To test our simple intent classification, we need to add some flows that trigger when a given intent is detected. Lets start with the flow that should trigger when the user intent is to transfer money.

  1. Create a new Flow and add to its trigger a Gen AI Match.
  2. Select the Annotator Banking intent classification and type as the match value TRANSFER_MONEY.
  3. Write an answer text that should be returned when the user intent is to transfer money.
  4. Test your flow in tryout and ensure it triggers on inputs such as "I want to transfer 50€ to Anna"
  5. Carry on and create flows following this same approach for all of your intents.

Dialogue Summarization​

In this example, we will use Gen AI to get a summary of a dialogue between a bot and a user. This could be be very useful for example when handing over a conversation to a live agent, giving the agent a brief overview of what the bot and the end user has discussed so far.

Setup​

First, let's set up and configure the documents needed.

info

The example assumes that you already have configured a Handler.

Summarizing a dialog is an internal task and we will use the System agent.

Next, we will need to define a Goal:

  1. Add a new Goal called Dialogue Summary and assign it the System Personality.
  2. Add as the Outline something like: Your task is to summarize dialogues. Please give a summary of the available conversation.
  3. Assign it your Handler of choice.
  4. We are interested in a brief summary, so we can replace the default format with the text: Stick to max 120 characters maximum and focus on the user's inputs in this summary.

Create your Flow​

  1. To test the summary Action, create a new Flow and add a match, for example a Language object match with the valueSUMMARIZE.VB.LEX
  2. Add a flow variable summary to the flow
  3. Insert a Gen AI Action node directly after the Raise node, but before the Output node.
  4. Assign your previously created Goal to the Action and make sure the result from the Gen AI is saved to the flow variable.
  5. Add the following text to the prompt:
Please give a summary of this available conversation:
     ${  _.dialogHistory.collectMany {
        ["user: ${it.userInputText\}", "bot: ${it.answerText\}"]
         \}.join(System.lineSeparator())
      }

In a real scenario, we could for example pass on the content of the variable summary in an integration that hands over the conversation to a live agent, but in our case, we just print it to ensure our Action did its job. To do this, we add a new output node direct after the Gen AI Action node in the flow, with the answer text:

This is the result of the Gen AI prompt: ${summary}

Now, head over to Tryout and carry out a simple conversation with your bot. To get the summary, input something like: Can you please summarize the dialog? The answer given should provide the summary.

Tips and Tricks​

Best practices​

Below are some best practices that apply for creating the most optimal outlines, instructions, and prompts in your Gen AI systems.

1. Be Clear and Specific

Avoid ambiguity. The more precise your prompt, the better the results.

  • Instead of: Generate a short text.
  • Try: Generate an output that has max 120 characters.

2. Give Examples

Giving examples of the result you expect helps to guide the model’s logic. You may want to do this when performing intent classification, sentiment analysis, or any other task where the user input should be analyzed and classified.

  • Example:
Goal: Intent Classification
Task: Classify user input into one of the following intents:

BookFlight
CheckWeather
PlayMusic

Example responses:
Input: "I want to fly to Paris next week."
Your response: BookFlight

Input: "Is it going to rain in Barcelona tomorrow?"
Your response: CheckWeather

Input: "Play some jazz music."
Your response: PlayMusic

3. Say What to Avoid

Including instructions on what kind of information or language to avoid helps keep the response aligned with your intent.

  • Example: Help customers with requests related to insurances. Avoid technical jargon.

4. Avoid Contradicting Information

Keep in mind that the final prompt consists of reusable blocks defined in the Personality, Goal, and Flow node, as well as the tone-of-voice settings. Please make sure that all elements align in order to get the desired results.

  • Instead of: Selecting an informal personality in the tone-of-voice settings and defining a formal tone in the outline.
  • Try: Selecting an informal personality in the tone-of-voice settings and omitting any references to tone in the outline.

Hybrid implementation​

Creating Gen AI agents that perform Tasks in Teneo is easy and straight forward and all Tasks can seamlessly co-exist in conjunction with other Teneo features and functionalities, allowing you to create robust and reliable implementations:

  • In Teneo, you can combine any type of Match to create Hybrid Matching. For example, combining a Gen AI Match with a TLML Syntax Match brings together the fluency and expressiveness of Gen AI with the precision and control of TLML—allowing you to carefully define and monitor the scenarios in which Gen AI should be called.
  • For cases when the outcome generated by the Gen AI is not as desired, it is always possible to resort to a static Output and Gen AI Outputs and static Outputs can co-exist side by side in any Flow.
  • Gen AI Outputs and static Outputs can also be combined into one response, allowing only part of the response to be generated, to give a more controlled, hybrid response to the end user.
  • By letting the Gen AI act "behind the scenes" with Gen AI Actions that provides logic to the dialog, the responses given to the end users can still be fully controlled, static responses.
  • It is possible to call the Gen AI System only under a certain condition, for example, to retrieve entities from the user input only if they have not already been found in the conversation.

Download​

Download all template scripts for configuring the Handler here:

Download ZIP