How to build a simple AI assistant based IoT
After getting an Android Things kit at Google Developer Day India, I have been very excited at getting into IoT and building with Android Things. Here I am finally messing around with it.
Hardware
Some of the supported kits are shown below. I am using the NXP Pico i.MX7D for this article. Raspberry Pi 3 and the Intel Edison are also some of the other supported boards with more manufacturers working to add support.
Setup
This consists of basically two steps:
Connect the Hardware
Having max to max fixed a few broken sockets and fixed a few broken stuff at home, building this was very exciting for me.
I followed the following instructions but I only attached the display to the board as I only wanted to mess around with some simple stuff before going into more detailed setups.
I sadly didn’t have a WiFi antenna provided with my kit and had to connect to the internet through an ethernet cable. Also a Rainbow Hat wasn’t provided and hence I decided to make just simple experiments with the kit. Once all the hardware was connected and I powered the kit, it looked like this.
My final setup
Installing Android Things
After the hardware is all connected you have to install Android Things onto the board. I suggest the Android Things Setup Utility.
The following guide will help setup the kit. I will also provide a short brief into installing Android Things.
Once you run the executable file, the below are the setups that you encounter.
Once I was done installing Android Things on the board, I didn’t setup the WiFi as I didn’t have an antenna. I hence used an ethernet cable to connect to the internet. Once it was set up my kit looked like this.
Building and Deploying
Create a new project
Creating a new Android Things project is very straightforward.
Start a new project
Select Android Things as the desired form factor
In my case, I wanted a UI so I selected “Android Things Empty Activity”
Click “Next” and my project was ready
Project Configuration
Since the project creation tools set everything up for you, you don’t have to worry about what went on under the hood to work with Android Things. However, I was curious anyways and noticed a few things.
The Android Things dependency was automatically added to the build.gradle file.
compileOnly 'com.google.android.things:androidthings:+'
The manifest had 2 items of interest
The full explanation for these can be found here. In short, the IOT_LAUNCHERcategory on the intent filter indicates which activity should be launched when run on an IOT device, and the uses-library declaration makes the Android Things library available at runtime.
Building Hello World
From here, building a Hello World UI application for Android Things was no different than for a phone/tablet once the initial project setup was finished.
I kept this very simple and added a single button that, when tapped, showed a “Hello World” toast.
Once the code was ready, I deployed the project like I would to a phone/tablet and my new app was running on my personally constructed Android Things hardware.
The code can be found on GitHub here:
Google Assistant Project
I further decided to make a Google Assistant application which is one of the sample projects. For this project you will require:
Google does provide a DIY kit which can be used with the raspberry pi to do the same. You should be able to hook up a button, an external microphone and speaker to the kit and use that with the below project to build a smart assistant. I made the following project with the above simple setup and a standard mobile earphone.
Setup Project to work with Google Assistant API
First clone the following repository onto your device. It has been modified to run for a standard line input/output using a microphone.
Configure Credentials
Enable the following activity controls in the Google Account you plan to use with the Assistant:
Make sure you check the box that says “Include Chrome browsing history and activity from websites and apps that use Google services”
Device Information
Voice & Audio Activity
Once done with that:
In the Cloud Platform Console, go to the Projects page. Select an existing project or create a new project.
Enable the Google Assistant API on the project you selected.
Click Enable.
Create an OAuth Client ID
Click Other (select Other UI and User data if prompted) and give the client ID a name.
On the OAuth consent screen tab, give the product a name (don’t use “Google” in the name).
Click Create. A dialog box appears that shows you a client ID and secret. (No need to remember or save this, just close the dialog.)
Click ⬇ (at the far right of screen) for the client ID to download the client secret JSON file (client_secret_NNNN.json or client_id.json).
Open a terminal with python support and install the google-auth-oauthlib. The command is as follows:
In your virtual environment, run the command:
pip install google-auth-oauthlib[tool]
Navigate to your top-level project directory.
Copy the client_secret_NNNN.json file over to your projects root directory, then run Google’s oauthlib tool to generate the credentials.
google-oauthlib-tool --client-secrets client_secret_NNNN.json \ --credentials app/src/main/res/raw/credentials.json \ --scope https://www.googleapis.com/auth/assistant-sdk-prototype \ --save
Now Open the code in android studios. Sync the gradle and then deploy like usual. On the first install permissions have been granted to the application. Restart the application
Now the application will run properly. Press the button on the screen and your device will start listening to your voice.
Your voice query gets sent over via the Google Assistant API, and you get a response almost instantaneously. Neat! Since the service is linked to your Google account, you can even tell it to add events to your Calendar and access other information. Ask for a joke, find out tomorrow’s weather, and when you run out of ideas, check out this comprehensive list.
Dave Smith has a great, high-level overview of the Android Things platform. This is a wonderful place to start if you have no idea what Android Things is or want to explain it to someone else.
Rebecca Franks and Wayne Piekarski gave some really nice talks at GDD events last year
Conclusion
I hope this helps people with Android Things supported kits to build a simple Google Assistant application. Again, this has been a lot of fun and it’s great to see how easy it is these days to sprinkle some AI into any project. I still haven’t fully explored the capabilities of my DIY Google Assistant yet, but I think there’s a lot of potential for interesting projects here.
I would also like to extend my sincere gratitude to Nate Ebel and w4ilun for their previous works on this topic which helped me get this far.
I really hope that you find this article useful. I invite you to participate in the discussion in the comments below, I’m always interested to know your thoughts and happy to answer any questions you might have in your mind.
Thanks for reading! :)
P.S Do feel free to connect with me on LinkedIn
