Skip to main content

Tryout

In Teneo Studio's Tryout, the developer can test and debug their solution by testing different inputs and seeing how the solution responds.

The sections on this page provide information about the various aspects of Tryout:

Concept​

Tryout provides the developer with extensive tools to test and debug a solution before going live with changes. Here, the developer can instantly check if a certain input triggers the desired Flow and if the implementation of a Flow dialogue behaves as intended, e.g. when the Flow contains many nodes and transitions.

With the Tryout connector, developers can also connect any external user interface directly to Tryout of a running solution. In this way, solutions can be tested via text or voice from outside of Studio in a quick and straight forward way, without having to publish a solution.

Tryout is available in both Studio Web and Studio Desktop, and Studio Desktop also includes Advanced Tryout features.

Simple Tryout allows the user to quickly test an input and ensure that the expected answer is given, allowing them to check that the behavior of a Flow dialogue is as desired.

Advanced Tryout also allows the user to test an input and ensure that the expected answer is given, but with detailed information of the Flow dialogue, such as input annotations, variables, and how the Flow was raised.

Test inputs​

Before using Tryout:

  • Save the relevant Flow or items you were working on.
  • If Tryout shows a Restart now or Reload now option, make sure to do so.
  • If Tryout shows the message model training started, wait for it to show the message model training completed.
  • Be sure you are testing with the right input.

Follow these steps to try out the solution:

  • In the Home tab of Teneo Studio, ensure that the simple Tryout is open in the right side of the window. If not, click the Tryout tab to open it.
  • Now, imagine an end-user is interacting with the Conversational AI application; type a user input in the text box and hit Enter or click Send to see which answer is returned. If the expected answer is returned for the input, it means that the solution is working as expected.

When right-clicking in the input/output field, the user is presented with the Replay option, which can be used to automatically send inputs to the Tryout again to re-test them.

tip

Clicking a previously given input in the Dialog view will populate the text box with the same input again to easily re-test it.

Add parameters​

Parameters can be used by a frontend when the user wants to send information to Engine that is not (part of the) user input text. Such parameters can be simulated by adding them in Tryout by clicking Add Parameter.

Input parameters can be fetched and processed in Global Scripts in Teneo Studio and can, for example, be extracted from the request sent to Engine with their values stored in Global Variables.

All parameters are considered to be of type 'string'. To add a parameter, fill out the name and the value. These will be sent to Teneo for that active transaction.

Classification​

The standard Intent Classification implies that intent annotations from Predict are created only on demand when references to class annotations are detected by Engine during the input matching process in either Class Matches, Syntax Matches, or in solution scripting.

For testing purposes, sometimes it is useful to be able to see intent annotations for all tested inputs. The developer can therefore change the classification setting to Forced which ensures intent classifications are always created during the processing of an input.

Errors and warnings​

In both Tryout views, messages and warnings can be displayed. In simple Tryout, this information is displayed as a brief message which can be clicked to open the Messages section of Advanced Tryout to quickly draw the user's attention to these, and more extended information is available in the Messages section of the Advanced Tryout window.

Error messages displayed include:

  • Errors when starting the Tryout Engine (i.e. HTTP errors when communicating with the server or failing initialization of Engine)
  • Missing/incorrect Language Object/Entity references
  • Disabled Language Objects/Entities used in TLML syntaxes or as entry values
  • Missing Gen AI document references
  • Undeclared Variables
  • Script errors in Flows

Other relevant messages are also displayed in the Tryout, e.g. information related to solution loading or if the user has a running Tryout open in another tab.

Read more on the Troubleshooting page.

ML model messages​

Learn intent model messages

Teneo Learn sends information related to the intent model training directly to Tryout via different messages, such as Model update in progress or Model update completed. The messages are meant to help the developer understand the status of the intent model and are purely informative.

When the Model update completed message is displayed, the developer can test the latest Learn intent model in Tryout.

Learn more about Intent Classification in Teneo here.

Multiple users

The machine learning model is available for the user whose save action initiated the training; other users that are editing the same solution continue using the old version until their Tryout is manually reloaded (A restart will not trigger a model update). Again, the Model update completed message is used to indicate that the model is ready for use. New users connecting to a solution get the latest model trained for that solution.

Reload now​

Restart starts a new conversation or session which can be useful, for example, when testing Flows with variables as it resets their values to their initial state.

The Reload option, on the other hand, recreates the Tryout engine instance which is, for example, useful when collaborating with other users on the same solution as this reloads the engine and applies the latest updates from collaborators as well. As this option will reload the entire solution, this includes ending all active Flows, but also re-setting Global Variables and scripted contexts.

Please note that although one user might see a collaborator's recent changes in a document (i.e. in a Flow), the Tryout engine instance is not updated automatically, and Reload is needed for those changes to become available in the Tryout.

Sometimes, a Reload now message is displayed above the text field of either of the Tryout views; this occurs when a change in the solution requires a reload of the Tryout Engine, e.g. after performing changes to Global documents or File resources. An Engine Reload Required message is also displayed in the lower right-hand side of the Teneo Studio window.

desktop_reload-now

Tryout connector​

The Tryout connector makes it possible to connect any external end user interface directly to Tryout of a running solution. This allows solution developers to test against an unpublished solution, using either text or voice from an external UI, such as Teneo Web Widget or any voice interface. The connection is using the TIE API, which is built on top of the Published Solution API, and without using web sessions.

This feature enables a quick and straightforward way to test, demo, and/or troubleshoot while developing and to make changes on the fly that are updated instantly without having to re-publish the solution after every change. With the advanced Tryout of Teneo Studio Desktop, the developers can track the processing path of each transaction as it unfolds, gaining full visibility into what happens at every step, for example what flow was raised, what scripts were executed, what input annotations were created, the state of variables etc. A special focus advantage is also on developing and troubleshooting for voice interfaces, as this feature allows the developer to instantly see in text what a voice input was interpreted as by the STT.

The external connection will be made to the existing Tryout session, meaning that the state of the Tryout session will be seen in the externally connected session. For example, if Tryout is reloaded with the stable solution version, or a session is restarted, then this is what will be exposed by the external connection.

Per-input state defined in the Tryout UI (eg. input parameters or forced annotations) will only apply to inputs originating from Tryout UI, not inputs originating from the external connection. Engine streaming is not currently supported by Tryout and is therefore not supported by the Tryout connector.

Read more on how to set up the connector here.

In Studio Desktop, the icon and color of the user input in the Tryout window will vary depending on whether the user input is coming from Tryout or an external UI.

tryout-in-teneo-vs-external_d

How To​

Use the Tryout Connector​

Tryout can easily be connected to an external user interface with the Connection URL, which can be found in Tryout.

note

The Tryout Connector does not support streaming/SSE.

  1. In an open solution, open the Advanced Tryout window.
  2. Click on the Connection URL button in the ribbon bar. This will copy the URL to your clipboard.

connection_url-d

In the external end user interface, simply replace the link to the published engine URL with the Tryout connection URL. If using Teneo Web Widget, the method teneoWebWidget.setEngineUrl can be used, as this method allows to temporary replace the current engine URL with the one provided. For more details, see Teneo Web Widget - Configuration.

While interacting with the external UI, the exact same dialog can now be followed in parallel in Tryout of the running solution. A new user session can be restarted both from Tryout or from the external UI, and any configurations or settings from Tryout, such as testing a stable version or re-playing inputs, can be applied while testing.

note

Input parameters defined in Tryout or Forced annotations will only apply to inputs originating from the Tryout UI.

Export a Tryout session​

When right-clicking in the Tryout window, the developer is presented with different export options depending on where the user clicked. Usually the options include the possibility to copy or open a summary, open the chat log or full log, and to Clear All information from the Tryout.

In the Dialog area, if right-click is activated in the fields of an input/output, the developer has the option to Open Chat Log either As Text or In CSV format. Selecting one will open the available chat log in either a text editor or Excel.

Note that the top ribbon of the Tryout window also provides an option to Export Engine XML.

Access Path data via scripting​

The data displayed in the Path section of Tryout can be accessed via scripting to perform certain actions and send/receive messages from the Teneo Engine. The main entry point is engineAccess which provides access to the state and functionalities of the Teneo Engine, including dialogue history and input annotations.

The following methods, provided by the engineAccess (and its alias _), gives access to the processing path programmatically:

  • _.getProcessingPath() This method can be used to obtain the path information of the current user input turn.
  • _.getDialogHistory()[<history index>].getPath() This method can be used to obtain the path information of earlier user input turns that can be picked from the session history.

For more information, please see the Engine Scripting API.

UI​

Simple Tryout​

note

The simple Tryout is only visible when the Advanced Tryout window is closed.

Teneo Studio Desktop offers two different views of the Tryout: a simple Tryout available in the right-hand side of the Teneo Studio window, and an Advanced Tryout available via the Tryout button in the top ribbon or via the Advanced button in the simple Tryout view.

desktop-simple-tryout

When inputs and outputs are tested in Tryout, icons are displayed below the answer text with basic information of how the Tryout Engine handled the input. The icons are clickable and takes the user to the relevant (Flow) node when clicked.

adv-tryout-icons

The above image contains the following icons:

  • Trigger represents the Intent Trigger that fired when the user input was sent to Engine.
  • Raise Flow illustrates the raise Flow event (when entering the Flow).
  • Output represents the Output node from which the answer text came.
  • Drop Flow illustrates the drop Flow event (when leaving the Flow).
  • URL is displayed only when the answer contains a URL (clicking the icon will navigate to the URL).

Advanced Tryout​

note

Advanced Tryout is available in Studio Desktop.

To access the Tryout window, click the Tryout button in the top ribbon of Teneo Studio, or the Advanced... button in the simple Tryout panel.

note

Opening the Tryout window automatically closes the simple Tryout panel; any data (tested inputs and outputs) available in the simple Tryout is automatically populated into the Tryout window at opening - and vice versa when closing the Tryout window.

advanced-tryout

Note that, in the lower left-hand corner of the window, the status bar provides information on which classifier (i.e. Learn or CLU) is currently used for the stable and the latest solution versions. Read more about the Status Bar here.

Top ribbon

The top ribbon of the Tryout window offers the following options:

  • Close the Tryout window
  • Restart to start a new conversation (session); this is useful if testing a Flow with Global variables, as the restart starts a new session and resets the value of all Global variables to their initial values.
    • Clicking the arrow in the lower part of the Restart button gives you the option to Reload Engine: this recreates the Tryout Engine instance; this is particularly useful when collaborating in Teneo Studio, as the reload applies changes performed by other users. Pressing Alt + Restart will also re-synchronize the Tryout Engine with the solution knowledge base.
  • Export Engine XML exports the Engine XML.
  • Replay an input or several inputs by selecting one of the inputs in the Tryout dialog view and clicking Replay.
  • Clear All to clear previous dialogues and messages from the Tryout view.
  • Stable Version: Latest/Stable provides the option to test either the latest versions of all documents in a solution or the stable versions of documents in a solution; learn more about versioning in Teneo here.
  • Classification: Standard/Forced allows you to switch between the standard deferred input classification or force immediate input classification; learn more here.

Detailed information

The right side of the Advanced Tryout window provides the developer with a lot of detailed information related to the solution and the tested inputs/outputs; this information will be described in the following sections.

tryout_path_icons

Input​

In the Input section of the Tryout window, the developer can find information related to the test input.

adv-input

The provided details - related to the input - are divided into different sections in the view:

  • The top of the section displays the given input.
  • Classifications details information about the top predicted Class Intents. When an intent model is available in the solution, Teneo Predict may generate up to 5 top intent annotations for the classes with the highest confidence scores and the Classifications section provides information about the predictions related to the tested input. Read more about the annotations generated by Predict in Intent Classification and Predict Input Processor.
note

Although the classification is displayed in the Input section, the triggered Flow trigger might not contain the displayed Class as a Match. The information related to the actual triggered trigger is displayed under the Path.

  • Annotations details the produced annotations. Here the user can find information about the created annotations for a given user input, i.e. the annotations created by Teneo Predict but also annotations produced by other Input Processors or by the Teneo Engine (such as System and Standard annotations), such as POS tags and NER tags. Custom, project-specific annotations, such as Deferred Annotations or Gen AI Annotators, are also displayed. Hovering over an annotation provides further information, such as detailed descriptions or information related to variables and values. Exporting a Tryout session as text or CSV includes the information related to annotations.
  • Processing details information about e.g. initialized variables and changed session variables, Input Processor results, as well as information related to Global Scripts in the solution.

Path​

The Path section provides information about the different items that crossed Engine's path while processing a request.

adv-path

The Path can contain one or more of the below events or information types, depending on the path taken from the user input to the final output; learn more here.

The elements displayed under Path are expandable (clicking the arrow icon in the right side of the view) and allow users to navigate directly to the location in the solution by clicking the Id.

  • Blue colored edge: Trigger Matches and Trigger elements
    • Deferred Annotation Displays details about any requested Deferred Annotations.
    • Match (requirements) Specifies which Match(es) was matched by the given user input, e.g. Gen AI, Class, Context, TLML Syntax, Language Object, etc. Read more about the different Match types here.
    • Trigger Displays details related to the triggered Intent Trigger.
    • Gen AI Annotator Provides information about any requested Gen AI Annotators.
  • Yellow colored edge: Transition Matches and Transition elements
    • Match (requirements) Specifies which Matches were matched by the given user input. Read more about the different Match types here.
    • Transition Details information related to the Transition such as the Name and vertex ID, or whether the Transition is input consuming.
  • Green colored edge: Other elements that occur within the Flow context
    • Raise Flow Provides details of the Flow the triggered Intent Trigger belongs to.
    • Raise Gen AI Output Provides details of the Gen AI Output being called.
    • Junction Provides details of Junction's Id.
    • Output Specifies details related to the Output node, such as Revisits, Emotion, Answer ID, URL, and Answer Text.
    • Gen AI Action Provides details on the Gen AI Action node, such as changed session variables and Id.
    • Drop Flow Specifies details related to the Flow being dropped, including if the Flow was aborted or stuck.
    • Drop Gen AI Output Specifies details related to the Gen AI Output being dropped.

Output​

The Output section gives information related to the Output, such as the answer text, emotion, and, when applicable, the URL and/or Output parameters.

State​

Information related to the state of Global Variables is displayed here. In this view the user will find any Watched Variables in the top of the list for easier control of these selected variables. To add a Global Variable to the list of Watched Variables, simply click the Eye icon next to the name of the Variable in the list. If no variables are currently followed, the Watched Variables section is not shown. Note that you can watch Flow Variables here as well.

6-with-watched-variable-v2

Messages​

The Messages section provides different information related to the Engine status depending on which interaction in the Dialog view is currently selected; selecting an Online message will, for example, display information related to:

  • Solution parsing
  • Engine initialization
  • The solution's machine learning model (model status: update in progress, update completed, update failed)
  • Statistics, such as, number of Flows, Language Objects, and Entities in the solution and any assigned lexical resource
  • Available script engines (such as Groovy Scripting Engine) and the default scripting language
  • Error and warning messages

While a user is working in the solution, information about created and updated Flows, Language Objects, and Entities is also displayed as well as any errors detected during development. See the Troubleshooting Tryout page for more information.

Selecting a tested input/output will present the user with information related to the response time, input text and, if available, any input parameters.