Class Management

The Class Manager allows developers to build an Intent Classifier by adding, editing, and removing Classes.

The sections on this page provide information about the various aspects of the Class Manager:

• Concept
• How To
• UI
• Practical Examples

Concept

The Class Manager allows developers to modify Classes used to build an Intent Classifier with Teneo Learn. Each Class is made up of a Class Name, training data, and, optionally, test data. The training data includes common utterances within a specific User Intent, and they are used to automatically train a Class with Teneo Learn. Classes are then used to match with the user input in triggers and transitions.

note

Please see Intent Classification to understand the different ways to create an Intent Classifier in Teneo.

Teneo Learn

When Teneo Learn trains an Intent Classification model, the machine learning pipeline is automatically selected using an algorithm which assigns a pipeline optimized to the number of provided training data examples. Every change to any of the classes in the Class Manager, including addition or removal of one or more classes, activates a new training in Learn as soon as the developer saves the changes in Studio.

Input Processing

Pre-processing

For the Learn Intent Classifier to generalize well on unseen user inputs, the user inputs are pre-processed so that irrelevant features such as punctuation characters or casing do not have an impact on the machine learning prediction.

In general, this means that user inputs are tokenized, lowercased and certain punctuation characters are removed. The following table holds several user inputs in English which are normalized to the same form and as such the Intent Classifier makes the same prediction for all of them.

User inputs
Hello, how are you?
Hello. How are you?
HELLO HOW ARE YOU??
hello how are you

It is vital that the exact same pre-processing that is applied to the user inputs at runtime for the intent prediction is applied to the training data examples that are used to train the model. This means that pre-processing happens in two places: once for the training data examples in Teneo Studio and once at prediction time when the model is called in the Teneo Predict Input Processor.

Input Processors

To perform pre-processing, the language-specific Teneo Input Processors (IPs) are used since they are tailored for the needs of each language.

The following table shows the Input Processors which are used by default to normalize the data; again, note that by default the Simplifiers are also applied.

Language	Input Processors
Standard (all languages except below)	Standard Splitting, Standard Auto Correction
Chinese	Chinese Tokenizer IP
Finnish	Standard Splitting, Finnish Splitting, Standard Auto Correction
Japanese	Japanese Tokenizer, Japanese Concatenator
Turkish	Turkish Analyzer

Please see language specific information in the Input Processors section.

Training Data

To train an Intent Classification model, more than one Class must exist to distinguish between different intents. Each Class has training data that has been added manually to then train the Class.

To successfully add training examples, the following requirements should be taken into consideration:

• Minimum one training example per Class must be provided.
• For optimal performance, a minimum of ten training examples should be provided.
• Identical training data examples are not allowed. The Class Manager prevents developers from saving if two identical examples are detected in the same Class.
• A training data example cannot be empty. The Class Manager also prevents from saving if an empty example is detected.
• To perform a Cross Validation evaluation in the Class Performance area of Teneo Studio Desktop, each Class must contain a minimum of 5 training examples.

warning

In Teneo Studio Desktop, when adding training examples manually, the Class Manager also prevents developers from adding a new example to the list if the exact same example already exists.

tip

Keep an eye on the Suggestions in the backstage of Teneo Studio Desktop, as duplicated examples detected in the same class or different classes are raised here.

Test Data

Test data is used to evaluate the Intent Classification model when running Test Data Evaluation in the Class Performance area of Teneo Studio Desktop. Unlike adding Training Examples, adding Test Examples is not compulsory for the Class.

The test data is made up of two components:

• Test Examples, which are added in the Class Manager
• Linked Test Examples, which are User Intent examples defined in triggers and transitions which use the class in a Class Match

During a Test Data Evaluation, the model is evaluated with all the test data. It is worth noting that the Linked Test Examples cannot be edited in the Class Manager, but direct navigation is possible by double-clicking them.

note

Note that User Intent examples from triggers and transitions are ignored and not displayed as Linked Test Examples if the Flow is disabled.

Similarly to the training examples, the Test Examples must be unique. To avoid duplicated Test Examples, the Class Manager prevents developers from saving if two identical examples are detected. Furthermore, in Teneo Studio Desktop, when adding test examples manually, the Class Manager prevents developers from adding a new example to the list if the exact same example already exists. Duplicated test examples are also removed from the test data when running Test Data Evaluations.

note

Linked Test Examples are only available in the Class Manager in Teneo Studio Desktop.

Class Name

At run time, Teneo Predict Input Processor uses the Class names as annotations for the most probable intent class(es) and denotes the annotation with a confidence score. Therefore, the Class name is visible in any place where annotations are usually seen, for example, in Tryout or scripting, and because of this users are encouraged to give each class a unique and easily recognizable Class name.

Additionally, certain requirements exist for Class Names:

• The Class name must be unique.
• The Class name must be upper case.
• The Class name cannot contain reserved characters nor spaces.
• It is obligatory to add a Class Name. If the Class Name field is left empty, the user will not be able to save modifications until a name is provided.

Teneo Studio automatically suggests the name CLASS when creating a new Class. This should be customized by the user. If the user adds more classes (without customizing the names), Teneo Studio adds an incremental number to the suggested names of the next classes, for example, CLASS, CLASS_1, CLASS_2, CLASS_3, etc. Please see the edit section for more information on editing the Class name.

As the Class names are applied within the Teneo Platform as annotations, these can also be used within TLML syntaxes using the Teneo Linguistic Modeling Language.

How To

Add

Teneo Studio Desktop

info

You can use Copilot to create new training and test examples with the help of Generative AI. Please make sure the LLM Settings are configured in your Account for this option.

To add a new Class to the solution, follow these steps:

1. Open a solution and navigate to the Class Manager section of the top ribbon.

2. In the Class Manager window, click Add Class in the top ribbon; the new class will appear in the Class list.

3. In the Class details view, write the name of the class under Class Name.

4. Now, add examples to the class.

   • Add Training Examples:
     • Write training examples in the Enter Training Example text field.
     • When done, click Add or press Enter to add the example to the list and write a new one.
     • Repeat above steps to introduce the all wanted training data examples.
   • Optionally, select the Test Examples tab to add Test Data Examples.
     • Write test examples in the Enter Test Example test field.
     • When done, click Add or press Enter to add the example to the list and write a new one.
     • Repeat above steps to introduce all the wanted test data examples.

5. Remember to click Save in the Class Manager to preserve the modifications.

Note that it is not possible to save changes if a new class is created but no training data is entered.

tip

Some text editors, such as Windows Notepad or Notepad++, allow to select and drag text which can then be dropped onto the training data field in the Class Manager creating one training or test data example for each line in the original editor.

Teneo Studio Web

To create a new class, follow these steps:

1. First, either:
   • Click the Create button directly in the Classes tile on the Solution dashboard, or
   • In the Class list, click on Create New Class

2. The Create new class opens, write the Name and optionally a description.
3. Then, click Create.
4. The new class opens for edit:
   • Click Add Training Example:
     1. Write the training example in the text field.
     2. Press Enter to write another example, or alternatively Add Training Example.
     3. Repeat above steps to add more training examples.
   • Optionally, click Add Test Example:
     1. Write the test example in the text field.
     2. Press Enter to write another example, or alternatively click Add Test Example.
     3. Repeat above steps to add more test examples.
5. Remember to Save the class to preserve the changes.

Edit

Teneo Studio Desktop

To edit an existing class, follow these steps:

1. Open a solution and navigate to the Class Manager section of the top ribbon.
2. In the Class Manager window, double-click the class name in the Class list, or select the class and click Edit in the top ribbon.
3. Now, perform any needed updates:
   • Add Training Examples:
     1. Edit the name in the Class Name field.
     2. Write training examples in the Enter Training Example text field.
     3. When done, click Add or press Enter to add the example to the list and write a new one.
     4. Repeat above steps to introduce all the wanted training data examples.
   • Select the Test Examples tab to add Test Data Examples:
     1. Write test examples in the Enter Test Example test field.
     2. When done, click Add or press Enter to add the example to the list and write a new one.
     3. Repeat above steps to introduce all the wanted test data examples.
4. Remember to click Save to preserve any modifications.

tip

In edit mode, drag/drop examples between the Training Examples tab and the Test Examples tab to move examples from one tab to the other; multi-select by pressing Ctrl while clicking and drag/drop multiple examples in one.

Teneo Studio Web

To edit a class, follow these steps:

1. Go to the Class list (Solution dashboard > Classes tile > right-pointing arrow)
2. Click the class to edit.
3. Now edit as needed the training data examples by clicking Add Training Example:
   1. Write the training example in the text field.
   2. Press Enter to write another example, or alternatively Add Training Example.
   3. Repeat above steps to add further training examples.
4. Add more test data examples by clicking Add Test Examples:
   1. Write the test example in the text field.
   2. Press Enter to write another example, or alternative click Add Test Example.
   3. Repeat above steps to add more test examples.
5. Remember to Save to preserve any changes performed.

tip

Filter existing examples by writing letter(s) or words in the Search field just below the top ribbon.

tip

Rename the class or add a description by clicking Edit in the top ribbon.

Save

Teneo Studio Desktop

New and modified classes can be saved in two ways:

• Individually by clicking the Save button in the Class details view, or
• all at once, using the Save Classes button in the top ribbon of the Class Manager.

note

Note that the following restrictions are bound to the save action of classes:

• The class name cannot contain spaces or reserved characters. Teneo Studio Web will automatically replace any of these with underscore (_) .
• Every class must contain at least one training data entry.
• Classes cannot contain duplicated training data examples nor duplicated test data examples.

Teneo Studio Web

To save a class, follow these steps:

1. Click Save in the top ribbon.
2. Optionally, add a comment.
3. Click Save to confirm the save action or Cancel to abort the save.

info

The Save button is only visible when changes are performed to an already existing class or if the class has not been saved earlier.

Successful save is confirmed with a "Document saved successfully!" message; by clicking elsewhere in the browser window the message is closed.

note

Note that the following restrictions are bound to the save action of classes:

• A class cannot have an empty training example nor an empty test example.
• A class cannot contain duplicated training data examples nor duplicated test data examples.
• The class name cannot contain spaces nor reserved characters, Teneo Studio Web will automatically replace any of these with underscore (_) .

Close

Teneo Studio Desktop

To close the Class Manager, either:

1. Click the X in the top, right corner of the Class Manager window.
2. Click the Close button available in the top ribbon.
3. In the Class Manager backstage, click Close in the left-side menu.

If unsaved changes are detected, Teneo Studio will prompt the user to Save Changes, Discard Changes or Cancel.

Teneo Studio Web

To close a class in Teneo Studio Web:

1. First, remember to Save if any changes where performed in the class; Teneo Studio will prompt the user to save if unsaved changes are detected
2. Next, simply navigate back by using the back arrow of the browser or by clicking a location in the path available in the top ribbon.

Delete

Teneo Studio Desktop

To delete a class, follow these steps:

1. Select the class to delete in the Class list; multi-select by pressing Ctrl while clicking.
2. In the top ribbon, click Delete.

The selected class(es) are marked for deletion until the next Save action is performed in the Class Manager, or until the user clicks Apply in the Class details section. If a class is marked for deletion by mistake, select the class again and click Cancel in the Class details section.

Remember to click Save in the Class Manager to perform the actual deletion of the selected class(es) and to train a new machine learning model without the deleted class(es).

You can find more information on deleting documents here.

Teneo Studio Web

To delete a class, follow these steps:

1. Navigate to the Class list (Solution dashboard > Classes tile > right-pointing arrow or header).
2. Hovering over the class in the list, an icon of a recycle bin appears in the right side of the window, click on it.
3. The user is prompted to confirm the deletion of the class:
   • Click Delete to delete with usage check.
   • Check Do not check usage and click Delete to delete without usage check.
   • Click Cancel to cancel the action.

You can find more information on deleting documents here.

note

The deletion of a class currently cannot be undone in Teneo Studio Web, for this action please refer to the Recycle Bin available in Teneo Studio Desktop

Filter

Teneo Studio Desktop

In the top ribbon of the Class Manager, two text filters are available:

• one to filter classes, i.e., on the name of the class, and
• one to filter on the examples provided as training data or test data of a previously selected class (or selected classes).

The two filters allows the user to filter separately in the Class list and the training and test examples. Both filters operate directly over the lists and are kept while the user is working on the classes, meaning that the filter isn't cleared if a new class is selected and any current filter is applied when selected another class. The filter(s) can be cleared by manually clicking the Clear Filter icon next to the filter text.

Teneo Studio Web

The Filter documents field in the top ribbon of Teneo Studio Web allows to filter on the displayed list of documents; simply write the wanted letter combination to start filtering over the list. The filter works on the following columns: Name, Description, Author.

Clicking the column headers, furthermore, allows to sort the list of documents a-z or z-a when clicking the Name, Description or Author; clicking either Text Examples or Test Examples allows to sort by the count of examples, while clicking Updated sorts by data/time for creation/last update.

With a class open, it is possible to use the Search field to filter out examples which contain specific letter combinations or words.

Search

note

The options and functionalities described in this section currently only apply to Teneo Studio Desktop.

The information related to a class, such as the name, Id, training data examples and test data examples, is searchable in the Search tab of the main Teneo Studio window.

The Id of a class is available at the bottom of the Class details in the Class Manager when selecting the class (in read-only mode). To copy the Id, simply right-click it and select copy from the context menu.

For more information, please visit Search.

Import Classes

note

The options and functionalities described in this section currently only apply to Teneo Studio Desktop.

The Import Classes button, in the top ribbon of the Class Manager, allows to import classes from an external file (in .tsv format); requirements related to the file are explained below.

To import a .tsv file, follow these steps:

1. In the Class Manager, click Import Classes.
2. Browse to the location of the .tsv file, select it and click Open Alternatively drag/drop the .tsv file onto the Class list.
3. Teneo Studio prompts to confirm the data import, click Yes.
4. Remember to click Save in the Class Manager to preserve the modifications.

Note that the import functionality does not check if the .tsv file contains duplicated data examples; this check is performed by the Class Manager when saving and the user will have to remove any duplicated examples from the Class Manager before being able to save the newly imported classes; also see Save above.

File Requirements

As of Teneo 7.3 it is possible to import classes with both training data examples and test data examples.

The old format, specifying class name and training data example only, is still supported while a new format is available supporting data type, class name as well as test and training data examples.

Below please find further details for the new and old formats.

note

To update an existing class, the class name must be written in upper case in the .tsv file.

New format

The new format allows you to:

• create new classes with training data examples only
• create new classes with training data examples and test data examples
• update existing classes with test data examples and/or training data examples.

The requirements of the file to import are as follows:

• plain text file, saved in the tab-separated values format (.tsv)
• one line per training or test data example
• each line must contain three columns separated by tab with the following information:
  • a valid class name, and
  • depending on the specified data type, either a training or a test data example.
  • the example type: either TRAINING or TEST, note that example type must be written in upper case

Old format

The old format allows you to:

• create new classes with training data examples
• update existing classes with training data examples.

The requirements of the file to import are as follows:

• plain text file, saved in the tab-separated values (.tsv) format
• one line per training data example
• each line must contain two columns separated by tab with:
  • a valid class name
  • one training data example.

UI

Class Manager Window

Teneo Studio Desktop

The Class Manager window can be accessed by clicking the Class Manager button in the top ribbon of Teneo Studio and it consists of two tabs: Home and Class.

The Home tab contains:

• Top ribbon with editing options and buttons; read more.
• Class list displays the list of existing classes in the solution together with the counts of training examples and test examples.
• Class details provides information about the class name and tabs for training examples and test examples.
• Used by panel gives information about where the currently selected class is in use.
• Linked Test Examples displays the list of the positive User Intent examples from the transition/trigger where the class is used as a Match; the Linked Test Examples are by the Class Performance feature.
• Class Id is the unique Id of the selected class.

The Class tab consists of a left-side menu:

• Save Classes allows to save the current, modified version of the classes.
• Close allows to close the Class Manager.

Top ribbon

The Class Manager's top ribbon provide a range of buttons and options for working with classes, the below table lists the different buttons and, when relevant, links to more information are provided.

Section	Button	Comment	More info
Class Manager	Close	Close the Class Manager	Close
	Save Classes	Save modifications performed in one or more classes; clicking the lower part of the button allows to save with or without comment	Save
	Import Classes	Import various classes from a .tsv file	Import classes
Classes	Add Class	Add a new class	Add
	Edit	Edit the selected class	Edit
	Delete	Delete the selected class	Delete
	Filter Classes	Text field; use to filter on the list of classes	Filter
Examples	Filter Examples	Text field; use to filter on the lists of training and test data examples	Filter
History	Class	Open the history of the selected class	History
Stable Version	Set/Unset	Set the current version of the selected class(es) as stable or unset it	Version Flags
Branching	Include/Exclude	Include or Exclude the selected class(es) from branching when working in a Localization setup	Localization setup

Teneo Studio Web

The Class Manager is available on the Solution dashboard, in the Classes tile; clicking the right-pointing arrow (or the header) takes the user to the list of classes.

In this view the user will find:

• Class list contains the list of existing classes (if any) with, among others, the counts of training examples and test examples.
• Order per column header allows you to sort the list by column header (Name, Description, Training Examples, Test Examples, Updated and Author).
• Create button allows you to create a new class.
• Filter documents text filter allows you to filter over the class list.

Practical Examples

Add a Training Example

Teneo Studio Desktop

If you want to add more examples to the Class, you need to do the following:

1. Open the 'Class Manager' and select a Class that needs training examples.
2. Click on 'Edit' in the ribbon bar and add sentences to the training data.
3. Hit 'Save'.

For example, here we have edited a class containing data relating to different coffee types. Here you will see the before and after we added training data for a coffee type 'Cortado'.

info

When you go back to the Tryout window you will see a small text saying, 'Model training in progress', this indicates that the changes you have made are saved and the platform is currently re-training the classes in the machine learning model. This will change to 'Model update completed' when it is done.

Teneo Studio Web

If you want to add more examples to the Class, you will need to do the following:

1. Open the 'Class Manager' and select a Class that needs training examples.
2. Click on 'Add Training Example' and add sentences to the training data.
3. Hit 'Save'.

For example, here we have edited a class containing data relating to different coffee types. Here you will see the before and after we added training data for a coffee type 'Cortado'.

info

When you go back to the Tryout window you will see a small text saying, 'Model training in progress', this indicates that the changes you have made are saved and the platform is currently re-training the classes in the machine learning model. This will change to 'Model update completed' when it is done.

Add a Test Example

Teneo Studio Desktop

It is important to note that training examples can only be added from the Class Manager page once the Class has been created. It is not possible to add training examples from the flow editor page. This will only add 'Test Data' that will be used when testing your bot and not 'Training Data'.

Test data can be viewed and added in the Class Manager. Let's try that now.

1. Open the 'Class Manager' and select a Class.
2. Click on 'Edit' in the ribbon bar and click on the 'Test Examples' tab.
3. Add sentences for the test data and hit Enter.
4. Hit 'Save'.

Teneo Studio Web

It is important to note that training examples can only be added from the Class Manager page once the Class has been created. It is not possible to add training examples from the Flow editor page. This will only add 'Test Data' that will be used when testing your bot and not 'Training Data'.

Test data can be viewed and added in the Class Manager. Let's try that now.

1. Open the 'Class Manager' and select a Class.
2. Click on 'Add Test Example'.
3. Add sentences for the test data and hit Enter.
4. Hit 'Save'.