Dynamic Carousels from your DialogFlow webhook

Dynamic Carousels from your DialogFlow webhook

After recently starting to work with DialogFlow and Actions on Google, the one thing that I learned is that the documentation is really patchy and often provides snippets of JSON without the context of how everything should fit together. I’ll probably do a series of posts in this vein, but for this inaugural DialogFlow entry I’m going to focus on carousels!

What is a DialogFlow carousel?

Carousels in DialogFlow are used to present the user with multiple options such as a list of products from a shop or results from a search.

Google Assistant Carousel

It’s fairly straight forward to set up static carousel responses in the DialogFlow console. In the “Google Assistant” responses tab, hit “Add Responses” and choose “Carousel Card”. Repeat for the number of cards needed. The blank cards have fields for an image URL, alt text for that image, a title, a description, a key and some synonyms (more on these last two later).

Adding a static carousel card

If you fill in some information, you get the idea. The below is an example using characters from the game Dota 2.

Static carousel items

But what if you want this content to be based on results of a search or some other action? We could always do the search on the web hook side, put all of the data into a context and reference it in the carousel cards using the hash notation (e.g. #results.title1 where results is the name of a context and title1 is the title of the first result). But what if how many cards that we want to show is variable? We may have 5 results in some cases and only 3 in others!

Dynamic Carousels

For users of one of the SDKs, you’re in luck because creating carousels with dynamic number of items and such is baked into the SDK. No JSON wrangling for you!

For the rest of us, ultimately, we need to return JSON in the format the DialogFlow is expecting back for a carousel to show up. Carousel responses have partial documentation. Unfortunately, this documentation does not give you the full JSON payload required. Through some trial and error, using bits from the “custom_payload” response and scouring the internet for information from others; I’ve been able to work out the structure of JSON that is expected!

Here’s the JSON that I have been using:

And from the documentation, we can see that a carousel response looks like this:

You’re not limited to the one message type in a response either. I regularly combine a simple_response that introduces the results, a carousel_card to display results and then a suggestion_chip response to suggest follow on actions to the user.

Handling carousel item selection

Obviously, there’s not much point in giving users multiple options if you don’t know which one that the user has chosen! To receive the user’s choice, we need an intent that can be triggered with the event actions_intent_OPTION. Events are essentially ways of tagging an intent to be able to trigger it programatically (more information on events in DialogFlow can be found here). In this case, when someone makes a choice from a carousel, the event actions_intent_OPTION is fired. There are two different ways of handling this event:

  1. If you only have one carousel, you just need an intent that knows what to do with the carousel choice.
  2. If you have multiple carousels, you need an intent specifically for handling carousel choices. This will capture the chosen option, put it into a context and trigger the correct event.

Note on option 2: You could also do this by having your carousel handling action call the right code internally but to me it seems more correct to not give that responsibility to code that should just be dealing with carousels.

With both approaches, the option that the user selected is put into a context named actions_intent_OPTION. That context looks a little like this:

The value of OPTION will be the key of the selected carousel item. As this context only has a lifespan of 0, it will expire after this call. The easiest thing to do is to take this value and put it into a context that has a longer lifespan. Looking back at the first JSON snippet, there’s a context named carousel with a parameter named followup-event. We can add the option value to this context and then tell DialogFlow to follow the current action up with the action associated with event in followup-event.

To summarise that, here’s the sequence:

Carousel flow

Carousel item keys and synonyms

As mentioned above, whatever you set as the key for your carousel item is what is sent back as the option. This means that you don’t have to try and interoperate what the user has said or typed, DialogFlow will do that for you and let you know what option was chosen.

How will dialog flow know which option the user has selected? If the user taps or clicks a carousel option then it’s easy for DialogFlow to know which option to return to the webhook. When it comes to speech or typing, you can use the “synonyms” feature to make it easier for DialogFlow to make a choice on what was chosen. For example, the screenshot with the Dota heroes has short and alternative names for that hero but we can also add in numbers and ordinals for that choice (first, second, etc). This way when a user says “choose the second one”, DialogFlow will match that to our synonym and pass the associated key to the webhook.

Another option for selecting from carousels is to use the select.number followup intent. This adds in a bunch of common “number selection” phrases and you can use that in your webhook but then you need to know what number corresponds to what option value which may involve storing some more stuff in the context for later recall.

Summary

Here’s a quick summary of what this post was about:

  • Dynamic carousels are not very well documented in the DialogFlow or Actions on Google docs
  • Example JSON to return a response with carousel items from your webhook
  • How to handle the event trigged by a carousel item choice
  • How to have a single intent for handling all carousel item choices
  • An explanation of how carousel item keys and synonyms work and how they can make your assistant app more user friendly

3 thoughts on “Dynamic Carousels from your DialogFlow webhook

  1. Hello Kristian,

    This comment is not directly related to the article above but to the NDC Conference you attended in London last year (see youtube video: https://www.youtube.com/watch?v=esOmLqp6YuE). I found the video extremely interesting since for an academic related project, I have to use visual studio, LUIS and Azure platform. This technology is totally new to me and I feel kind of lost, especially for the programing part since I have never coded in C#. Would you be so kind to share the code you wrote as an example for that conference (the video definition is too low to be able to read the code)? It would help me to use it as a skeleton script for my own project. Also, any other reference to online resources would be greatly appreciated.

    Kind regards,

    Adam

  2. Just a small comment, I believe the matching of “user says” to a carousel tile occurs in Actions on Google (AoG) and not Dialogflow. If the user clicks on a tile then you get the actions_intent_OPTION event, but if the “user says” has to be matched, then your synonyms become important to populate. In the case that AoG cannot match, then AoG it will send Dialogflow a query set to the “user says” and the intent will be a actions_intent_TEXT.

Leave a Reply

%d bloggers like this: