Skip to main content

Annotations

The sections on this page provide information about how users can add Annotations in their solution:

Concept​

Annotations can be considered 'labels' that are attached to inputs. The power of the annotations come from exactly how they are created as the data they can provide goes beyond what is achievable with standard TLML rule-based engine matching against words, Entities and Language Objects. The annotations can be generated based on context, sentence structure, user specific configurations or more complex machine learning models, etc., and can then be used in the matching process. As a result, annotations contain brand-new data which makes it possible to create syntaxes which more precisely identify sentence elements that are necessary for interpreting the input correctly.

Some annotations are created automatically by Teneo (e.g. by Named Entity Recognizers (NER), Part-of-Speech annotations), but you can also add customized annotations using scripts to match on user input dynamically.

Read more about predefined annotations in the Annotations Reference page.

You can use annotations the same way as Language Objects or Entities in TLML syntax. The prefix looks slightly different; instead of %, you add %$ before the name, as shown below.

explain ner in condition

The main components of an annotation are:

  • its name
  • its (optional) variables

The name should reflect what is annotated. The annotation variables can contain extra information related to the annotation. For example, the annotation USER_ASKS_ABOUT_OPENING_HOURS.TOP_INTENT has a variable containing the confidence score for that specific user intent. Annotation variables can be retrieved with propagation scripts, the same way as NLU variables for Entities or Language objects.

A word, word sequence, sentence, or full input can have one or more annotations. Similarly, the same annotation can be attached to various words or sentences, as seen in the below picture.

multiple annotations per word

Creating Annotations​

You can create your own custom annotations, from the following places in a Solution:

When in doubt, Deferred Annotations is the recommended place to create your annotations, as they allow for the creation of on-demand annotations and provide a way to control potentially expensive actions, such as calls to Gen AI Models or databases, within a solution. Deferred Annotations ensure that these expensive actions are executed only when they are needed and only the minimum number of times.

Additionally:

  • When you add Classes to the Class Manager, corresponding annotations are created by the Input Processor Predict. background.

  • When you add a Gen AI annotator, a Deferred Annotation is created "behind the scene" by the system.

Customizing Annotations​

The Engine Scripting API Javadoc documentation describes in detail the method for creating customized annotations, but the most important parts are explained below:

_.inputAnnotations.add(_.createInputAnnotation(
    				 String annotation_name, 
    				 int sentence_index, 
    				 Set<Integer> words_indices, 
    				 Map<String, Object> annotation_variables))

The following table explains arguments of the createInputAnnotation method.

ArgumentDescription
annotation_nameThe name of the annotation you will use in TLML syntax later. The word, words or sentence will be annotated with this name and it should be written in uppercase letters.
sentence_indexThe index of the sentence that you want to annotate (the first sentence has index 0).
words_indicesThe indices of the sentence words this annotation is assigned to (the first word has index 0, the set may be empty).
annotation_variablesOptional variables you may want to add to the annotation. The annotation variables are provided as a map: the key is a string with the name of the annotation variable, and the value is an object. If you don't want to provide annotation variables, use 'null' instead.

Use of Annotations in TLML syntax​

Conditions can be created so that a user input only is annotated when it matches a certain annotation variable or position in the text.

Predicate Script​

Control matching based on the annotation variable by using a predicate script.

For example, if you wish to match only when the the city annotation has a > 90% confidence score, you can use: %$CITY_MATCH:{lob.confidence > 90}.

Propagation Script - Extraction​

Use a propagation script to pass data along with the annotation and to use within the Flow.

For example, to extract the country for the matched city, you can use: %$CITY_MATCH^{matchedCountry = lob.city.country}. This assumes a Flow or Global Variable called matchedCountry has been defined.

Positional​

Use >> in the syntax to restrict the match to a particular context.

For example, to extract the city a user wants to go to: to >> %$CITY_MATCH.

Used words​

The "used words" of an annotation are the words that the annotation is connected to. They are controlled by the WordIndices parameter of the _.createInputAnnotation method.

multiple annotations per word

Annotation NameTypeUsed Words
EN.LANGSentence(none - sentence annotations do not have used words)
NN.POSSingle word"annotations"
PL.POSSingle word"annotations"
EXAMPLE.WORDSMulti word"annotations", "can", "be", ... (all words annotated)

These used words then define the matching behavior when using:

  • Positional operators, e.g.: >> (directly followed by)

  • Extended And operators, e.g.: &= (same match).

The used words also define the value returned by _.getUsedWords() (and similar) from within a predicate script attached to an annotation: %$EXAMPLE.WORDS^{myVar = _.getUsedWords()}.

Applying Annotations​

Annotations can be applied to 0-n words within a single sentence.

Sentence Annotations​

Setting an annotation to 0 words in a sentence annotates the entire sentence - but none of the words within it (see Used Words).

Use an empty set ([] as HashSet) to annotate an entire sentence: _.createInputAnnotation("EXAMPLE.SENTENCE", 0, [] as HashSet, [:])

For example:

sentence annotation

Code:

_.getSentences().eachWithIndex{s, sentenceIndex ->
    def some_variable_value = ["annotationType": "This is a sentence annotation"]
    _.getInputAnnotations().add(
        _.createInputAnnotation("EXAMPLE.SENTENCE", sentenceIndex, [] as HashSet, ["some_variable_name": some_variable_value, "sentenceIndex": sentenceIndex])
    )
}

Single-word Annotations​

Using a set with a single word index entry (_.createInputAnnotation("EXAMPLE.WORD", 0, [0] as HashSet, [:]) annotates a single word:

single word annotation

Code:

_.getSentences().eachWithIndex{sentence, sentenceIndex -> 
    def some_variable_value = ["annotationType": "This is a word annotation"]

    sentence.getWords().eachWithIndex{word, wordIndex -> 
        _.getInputAnnotations().add(
            _.createInputAnnotation("EXAMPLE.WORD", sentenceIndex, [wordIndex] as HashSet, 
                        ["some_variable_name": some_variable_value, "sentenceIndex": sentenceIndex, "wordIndex": wordIndex])
        )
    }
}

Multi-word Annotations​

Use a set with multiple word index entries (e.g. [0, 1, 2, 7] as HashSet) to annotate multiple words: _.createInputAnnotation("EXAMPLE.ALL_WORDS", 0, [0, 1, 2, 7] as HashSet, [:])

For example, here we annotate all words in the sentence:

multi-word annotation

Code:

_.getSentences().eachWithIndex{sentence, sentenceIndex -> 
    def some_variable_value = ["annotationType": "This is an all-words annotation"]

    def allWordIndices = sentence.getWords().withIndex().collect { word, wordIndex -> wordIndex }
    _.getInputAnnotations().add(
        _.createInputAnnotation("EXAMPLE.ALL_WORDS", 
                       sentenceIndex, 
                      allWordIndices as HashSet, 
                      ["some_variable_name": some_variable_value, "sentenceIndex": sentenceIndex, "allWordIndices": allWordIndices ])
    )
}

Inspecting Annotations​

To see how an input has been annotated, follow the below steps:

  1. Open your Solution and navigate to Tryout in the top ribbon.

  2. Enter some input in Tryout and open the 'Advanced' window. You will find all annotations in the Input section under 'Annotations'.

    tip

    Hovering over an annotation will display additional information that is stored in annotation variables.

inspect ner

Tips and Tricks​

Annotating Multiple Possibilities​

In cases where different annotations could apply to different sets of words in the same input, the different combinations can each be annotated.

For example, consider a solution containing a city matching annotator, with an input containing many possible city matches. Both "new york" and "york" are possible values for mentioned cities, hence both have been annotated as such:

annotating multiple possibilities

With a variable 'city' containing the full detail of the match:

city match example

Similarly, where a word (or combination of words) has a number of possible matches, these matches can be added to the annotation variable value to be used in the matching later for example to achieve automatic - or user driven - disambiguation:

city match example 3