Bulk Import Formats
Currently the Bulk Import functionality is only available in Teneo Studio Desktop
The Bulk import feature of Teneo Studio allows users to create Language Objects, Entities or Question and Answers (Q&A) pairs outside Teneo Studio in csv file format and later import these into a solution.
This page describes the file format as well as the rules for Bulk Import; to learn how to perform a bulk import in Teneo Studio, please see here
Format
The Bulk Import works with the csv (comma delimited) file format; these files can be created with Excel or any other compatible editor.
The supported encoding is UTF-8 and UTF-8-BOM.
Rules
For each document type (i.e. Language Objects, Entities, Question and Answer pairs) there are different rules or specifications which applies to the csv file however, some general rules are shared among the documents:
- Fields are separated by commas (,) or semicolon (;)
- Start tags are prefixed with the number sign (#)
- New line / carriage returns / extra delimiters are ignored
- New lines starting with #ignore allows to omits the entire line during bulk import in Studio providing the option to write headers / comments in the csv file
In the below tabs, please find the rules per document type; the Details section further below provides detailed information of each tag and field.
- Language Objects
- Entities
- Q&A pairs (Flows)
In the csv file for Language Object creation, each line in the file represents a new Language Object and should be preceded with the #language_object tag (indicating the beginning of the Language Object). The tag can be followed by the fields shown in the below tables.
| Tag | Field 1 | Field 2 | Field 3 | Field 4 | Field 5 | Field 6 |
|---|---|---|---|---|---|---|
| #language_object * | name * | alias ** | description ** | syntax * | positive examples | negative examples |
* Mandatory fields with mandatory values These fields must exist in the file and contain values
** Mandatory fields with optional values These fields must exist in the file but the values can be left empty
| Optional tags * | ||
|---|---|---|
| #folder | folder name | |
| #nlu_variables | NLU variables name | NLU variables value |
| #variables ** | variables name | variables value |
* Optional tags should be added to the same line as the #language_object tag they belong to
** Language Object Variables are deprecated as of Teneo Platform 7.0 and users are encouraged to use NLU Variables instead
To create a new Language Object, the user needs to - at least - give the new Language Object a name and write its TLML syntax and at the same time the fields for alias and description must exist in the file (although the user can choose to leave them empty). This means that the import of a Language Object in its simplest form, only providing a name and a TLML syntax, will look similar to the below:
#ignore,Language Object name,Description,Positive examples,TLML Syntax
#language_object,HAPPY.ADJ.LEX, , ,happy/happier/happiest
Or, the same, but in a table:
| #ignore | Language Object name | Description | Positive examples | TLML Syntax |
|---|---|---|---|---|
| #language_object | HAPPY.ADJ.LEX | happy/happier/happiest |
For more examples, please see the Examples sections further below.
The csv file to bulk import Entities can contain the tags and fields visualized in the below table, where the #entity tag indicates the beginning of a new Entity.
| Tag | Field 1 | Field 2 | Field 3 | Field 4 |
|---|---|---|---|---|
| #entity * | name * | description | positive examples | negative examples |
| #string_variable | string variable name | |||
| #script_variable | script variable name | |||
| #entry * | entry value * | value of string variable | value of script variable | |
| #folder | folder name |
* Mandatory fields with mandatory values These fields must exist in the file and contain values
To create a new Entity, the user needs to - at least - give the new Entity a name and provide one entry with a value. This means that the import of an Entity in its simplest form, only providing a name and the entry, will look similar to the below:
#entity,AIRPORTS.ENTITY
#entry,Arlanda
Or, the same, but in a table:
| #entity | AIRPORTS.ENTITY |
|---|---|
| #entry | Arlanda |
For more examples, please see the Examples sections further below.
To import Flows of the type Question and Answer pairs (Q&As), the csv file can contain the fields as visualized in the below table in the specified order. The #question_answer tag indicates the beginning of a new Q&A pair.
| Tag | Field 1 | Field 2 | Field 3 | Field 4 | Field 5 | Field 6 |
|---|---|---|---|---|---|---|
| #question_answer * | flow name * | description | output url | random answer order [true, false] | ||
| #question * | trigger name ** | positive examples | negative examples | order group | syntax match | class match |
| #answer * | answer text * | emotion | ||||
| #folder | folder name |
* Mandatory fields with mandatory values These fields must exist in the file and contain values
** Mandatory field with optional value This field must exist in the file but the value can be left empty
To create a new Question and Answer pair, the csv file has to contain - at least - the #question_answer tag with a flow name, the #question tag and the trigger name field (although it can be left empty), as well as the #answer tag and the answer text; this means that importing a Q&A pair in its simplest format will look similar to the below example:
#question_answer,User says Hello
#question, ,
#answer,Hello to you too
Note that the field for trigger name is left empty (between the two commas next to the #question tag)
Below is the same example, but in a table:
| #question_answer | User says Hello |
|---|---|
| #question | |
| #answer | Hello to you too |
For more examples, please see the Examples sections further below.
TODO triggers
When bulk importing Q&A pairs, if neither a TLML syntax Match nor a Class Match is specified in the file, Teneo Studio will mark the trigger(s) with the TODO label.
Quotes
For Language Objects and Q&A pairs, when creating the csv file in a text editor or similar, the usage of quotes needs a special treatment, where double quotes ("") must be escaped with an extra set of double quotes and, in addition, the entire field must be double quoted.
Example TLML Syntax
"Anderson>>""&"">>(co/company)"
String NLU Variable values
"""New York"""
For string NLU Variable values, Groovy also permits single quotes and they can be added to the csv file without having to be escaped.
'New York'
Both of the above examples of quotes are displayed in Teneo Studio as visualized in the below image.
Details
This section describes in more details the tags and fields for each of the document types.
- Language Objects
- Entities
- Q&A pairs (Flows)
| Tags and fields | Necessity | Description |
|---|---|---|
| #language_object | Mandatory | Indicates the beginning of a new Language Object |
| name | Mandatory | Name of the Language Object * Must be written in capital letters * Must be unique (both within the file and within the solution to which it is imported) * Must not contain spaces, reserved characters, etc. |
| alias | Mandatory field, optional value | Alias of the Language Object * Must be written in capital letters * Must be unique (both within the file and within the solution to which it is imported) |
| description | Mandatory field, optional value | Description of the Language Object |
| syntax | Mandatory | TLML syntax of the Language Object |
| positive examples | Optional field/value, fixed placement | Positive examples of the Language Object * Positive examples must always be written in field 5 (see above table) * Separate examples with a slash (/) |
| negative examples | Optional field/value, fixed placement | Negative examples of the Language Object * Negative examples must always be written in field 6 (see above table) * Separate examples with a slash (/) |
| #nlu_variables | Optional | Indicates that at least the following two fields contain a name and a value of an NLU variable |
| NLU variables name | Mandatory when the #nlu_variables tag is added | Name of the NLU variable |
| NLU variables value | Mandatory when the #nlu_variables tag is added | Value of the NLU variable |
| #variables * | Optional | Indicates that at least the following two fields contain a name and a value of a Language Object variable |
| variables name | Mandatory when the #variables tag is added | Name of the Language Object variable |
| variables value | Mandatory when the #variables tag is added | Value of the Language Object variable |
| #folder | Optional | The folder name is relative to the selected folder in Teneo Studio upon Bulk Import |
| folder name | Optional | Specify a folder path by separating each folder with slash (/) |
* Language Object Variables are deprecated as of Teneo Platform 7.0 and users are encouraged to use NLU Variables instead
| Tags and fields | Necessity | Description |
|---|---|---|
| #entity | Mandatory | Indicates the beginning of a new Entity |
| name | Mandatory | Name of the Entity * Must be written in capital letters * Must be unique (both within the file and within the solution to which it is imported) |
| #entry | Mandatory | Indicates the beginning of a new entry It is mandatory to add at least one #entry tag and value Add several entries by adding the #entry tag (and its entry value) in a new line |
| entry value | Mandatory | Entry of the Entity |
| description | Optional value, fixed placement | Description of the Entity * The value is optional, but if a description is added the placement is fixed |
| positive examples | Optional value, fixed placement | Positive examples of the Entity * The value is optional, but if positive examples are added the placement is fixed * Separate examples with slash (/) |
| negative examples | Optional value, fixed placement | Negative examples of the Entity * The value is optional, but if negative examples are added the placement is fixed * Separate examples with slash (/) |
| #string_variable | Optional | Indicates a new string variable |
| string variable name | Mandatory when the #string_variable tag is added | Name of the string variable * Several #string_variable tags and string variable names can be added to the file |
| value of string variable | Optional | Value of the string variable * Is added to the line of the entry it belongs to * String values do not need to be quoted, quotes are automatically added by Teneo Studio |
| #script_variable | Optional | Indicates a new script variable |
| script variable name | Mandatory when the #script_variable tag is added | Name of the string variable * Several #string_variable tags and string variable names can be added to the file |
| value of script variable | Optional | Value of the script variable * Is added to the line of the entry it belongs to |
| #folder | Optional | The folder name is relative to the selected folder in Teneo Studio upon Bulk Import |
| folder name | Optional | Specify a folder path by separating each folder with slash (/) |
| Tags and fields | Necessity | Description |
|---|---|---|
| #question_answer | Mandatory | Indicates the beginning of a new Q&A pair |
| flow name | Mandatory | Name of the Flow |
| description | Optional value, fixed placement | Description of the Flow * The value is optional, but if a description is added the placement is fixed |
| output url | Optional value, fixed placement | URL of the Output node * The value is optional, but if an output URL is added the placement is fixed |
| random answer order | Optional value, fixed placement | Set the answer order to random by adding the Boolean value "true" * The value is optional, but if a value is added the placement is fixed |
| #question | Mandatory | Indicates a new Intent trigger * Add multiple Intent triggers by adding additional #question tags |
| trigger name | Mandatory field, optional value | Name of the Intent trigger |
| positive examples | Optional value, fixed placement | Positive examples of User Intent * The value is optional, but if positive examples are added the placement is fixed * Separate examples with slash (/) |
| negative examples | Optional value, fixed placement | Negative examples of User Intent * The value is optional, but if negative examples are added the placement is fixed * Separate examples with slash (/) |
| order group | Optional value, fixed placement | Name of the Order group to which the User Intent trigger should be added * The value is optional, but if the name of an Order Group is added the placement is fixed *If no Order Group is specified, a new "Q & A's" group is created upon bulk import to Teneo Studio |
| syntax match | Optional value, fixed placement | Specify the TLML syntax of a TLML Syntax Match * The value is optional, but if TLML Syntax Match is added the placement is fixed |
| class match | Optional value, fixed placement | Specify the Class name of a Class Match * The value is optional, but if the name of a Class is added the placement is fixed * New Classes are added to the Class Manager upon bulk import; the positive examples of User Intent are added as training data examples (at least one positive example must be provided) * In case of various references to the same non-existing Class, the first Intent Trigger in the file referencing the Class will be used to create the Class and training data in Studio |
| #answer | Mandatory | Indicates a new answer * Add multiple answers by adding additional #answer tags * Multiple answers are imported to the same Output node |
| answer text | Mandatory | The answer text to be added to the Output node |
| emotion | Optional value, fixed placement | Name of the Emotion * The value is optional, but if an emotion is added the placement is fixed * The emotion (with the exact same name) needs to exist in the solution when performing the bulk import |
| #folder | Optional | The folder name is relative to the selected folder in Teneo Studio upon Bulk Import |
| folder name | Optional | Specify a folder path by separating each folder with slash (/) |
Fixed placement
The fixed placement for the above labeled fields and tags entails that the preceding fields must exist in the file when wanting to define a latter value although the values can be left empty as long as the field exists. For example, to specify a Class Match of a trigger, the line should start with the #question tag and be followed by the fields for trigger name, positive/negative examples, order group and syntax match (with or without values) and then end with the name of the Class to be used in the Class Match as visualized below:
#ignore,trigger name,positive examples,negative examples,order group,syntax match,class match
#question, , , , , ,CONTACT_CLASS
or visualized in a table:
| #ignore | trigger name | positive examples | negative examples | order group | syntax match | class match |
|---|---|---|---|---|---|---|
| #question | CONTACT_CLASS |
Examples
In the below tabs, please find examples for each of the documents types on how the files for bulk import can be set up. The information in the code blocks can be copied/pasted into an editor which can then be saved in .csv format and bulk imported into Studio. Please, the below examples are only created as examples to help understand how to set up the csv files for bulk import.
If copying/pasting into Excel, select the data and then go to Data and click Text to Columns, in the Convert Text to Columns Wizard leave the setting at Delimited click Next, choose Comma and click Next again and last click "Finish", the data is now available in the appropriate fields and can be edited/updated as wanted. Remember to save the file again in the csv (comma delimited) format before performing the bulk import.
- Language Objects
- Entities
- Q&A pairs (Flows)
Below please find in the code block an example of a bulk import of four Language Objects.
The first line of the file contains the #ignore tag allowing to specify headers in the following fields, followed by the tags and values in the next lines/fields. There are four Language Objects to import: AIRPORTS.DENMARK.LIST, COPENHAGEN_AIRPORT.NN.SYN, ODENSE_AIRPORT.NN.SYN and BILLUND_AIRPORT.NN.SYN, each contain a description as well as a TLML syntax, and the three latter contains examples of NLU Variables.
#ignore,Language Object name,Alias,Description,TLML syntax,Positive examples,Negative examples,NLU variables,variable name,variable value,,,
#language_object,AIRPORTS_DENMARK.LIST,,List of main airports in Denmark,%COPENHAGEN_AIRPORT.NN.SYN / %ODENSE_AIRPORT.NN.SYN / %BILLUND_AIRPORT.NN.SYN,Kastrup/Aalborg airport/ODE/Copenhagen airport,,,,,,,
#ignore,,,,,,,,,,,,
#language_object,COPENHAGEN_AIRPORT.NN.SYN,,Copenhagen/Kastrup airport,CPH/(copenhagen>>airport)/(kastrup>>airport)/(Copenhagen>>airport>>kastrup),,,#nlu_variables,sCity,"""Copenhagen""",#nlu_variables,airportCode,"""CPH"""
#language_object,ODENSE_AIRPORT.NN.SYN,,Odense/Hans Christian Andersen airport,ODE/(odense>>airport)/(hans>>christian>>andersen>>airport),,,#nlu_variables,sCity,"""Odense""",#nlu_variables,airportCode,"""ODE"""
#language_object,BILLUND_AIRPORT.NN.SYN,,Billund airport,BLL/(billund>>airport),,,#nlu_variables,sCity,"""Billund""",#nlu_variables,airportCode,"""BLL"""
The example to import contains the Entity CAPITALS_EUROPE.ENTITY; the first line in the file includes the #ignore tag allowing to add headers and is followed by the information related to the Entity to create. In this case, the Entity contains the string variable "sCountry" and underneath are specified each of the entries marked by the #entry tag and followed by the value of the string variable. At the end, the folder to which the Entity should be imported is specified.
#ignore,Entity name,Description,Positive examples
#entity,CAPITALS_EUROPE.ENTITY,Entity listing the capitals of Europe,Paris/Copenhagen/London/Stockholm/Berlin/Madrid/Rome
#string_variable,sCountry,,
#entry,Amsterdam,Netherlands,
#entry,Andorra la Vella,Andorra,
#entry,Athens,Greece,
#entry,Belgrade,Serbia,
#entry,Berlin,Germany,
#entry,Bern,Switzerland,
#entry,Bratislava,Slovakia,
#entry,Bucharest,Romania,
#entry,Budapest,Hungary,
#entry,Chișinău,Moldova,
#entry,City of Brussels,Brussels,
#entry,City of San Marino,San Marino,
#entry,Dublin,Ireland,
#entry,Helsinki,Finland,
#entry,Lisbon,Portugal,
#entry,Ljubljana,Slovenia,
#entry,London,United Kingdom,
#entry,Luxembourg city,Luxembourg,
#entry,Madrid,Spain,
#entry,Minsk,Belarus,
#entry,Monaco,Monaco,
#entry,Moscow,Russia,
#entry,Nicosia,Cyprus,
#entry,Nuuk,Greenland,
#entry,Oslo,Norway,
#entry,Paris,France,
#entry,Podgorica,Montenegro,
#entry,Prague,Czech Republic,
#entry,Pristina,Kosovo,
#entry,Reykjavik,Iceland,
#entry,Riga,Latvia,
#entry,Rome,Italia,
#entry,Sarajevo,Bosnia and Herzegovina,
#entry,Skopje,North Macedonia,
#entry,Sofia,Bulgaria,
#entry,Stockholm,Sweden,
#entry,Tallinn,Estonia,
#entry,Tbillsi,Georgia,
#entry,Tirana,Albania,
#entry,Vaduz,Liechtenstein,
#entry,Valletta,Malta,
#entry,Vienna,Austria,
#entry,Vilnius,Lithuania,
#entry,Warsaw,Poland,
#entry,Zagreb,Croatia,
#folder,Entities,,
The below example of a bulk import of a Q&A pair contains the Flow "Contact Teneo AI" which has two #question tags (triggers) with each a name and positive examples where one of the triggers have a TLML Syntax Match and the other a Class Match. Three answers are defined with the #answer tags and the folder is specified to be "Teneo AI".
#question_answer,Contact Teneo AI,"Flow answering inputs related to contact, phone, email, etc.",https://www.teneo.ai/contact,TRUE,,
#question,Contact: Main,could you give me the company contact email?/i need to send you a mail/i need your e-mail address/i want to mail you/I need an e-mail address to contact the company/can I get in touch with the company?/could you call me back?/could you give me your contact information?/how can I contact you/i need to make a phone call to the company/i need your phone number/i'd like to contact the company/please give me your phone number!/tell me how can I contact you/we need to contact with the company/what are your contact information,,Contact triggers,,CONTACT_TENEO_AI
#question,Contact: Keywords,phone/email/e-mail/mobile/contact/address,,Contact triggers,(%MOBILE_PHONE.NN.SYN / %TELEPHONE.NN.SYN / %TELEPHONE.VB.SYN / %TELEPHONE_NUMBER.NN.SYN / %ADDRESS.NN.LEX),
#answer,Fill out the contact form on the webpage to receive a call from us!,,,,,
#answer,You can find the contact details for Teneo AI on the webpage I have just opened.,,,,,
#answer,Please find the contact form on the webpage behind me.,,,,,
#folder, Teneo AI,,,,,