Add a voice user interface
You can extend your application with a do-it-yourself voice user interface. Imagine that your home robot assistant can take voice commands and give you advice like other voice assistants currently on the market.
By using your iOS application, your robot assistant could take requests to provide the current temperature at, say, the golf course.
- To configure the voice interface, you’ll add the following services from IBM Bluemix:
- Watson Speech to Text
- Watson Text to Speech
You’ll also add the Weather Company Data service to your application to get a weather forecast.
- In the existing application, set up the Carthage repository with the required SDKs by creating a cartfile and including the following SDKs:
- Watson-Developer-Cloud SDK for iOS: Simplifies using Watson services in Swift.
- Bluemix Services SDK for iOS: Helps with connectivity to Bluemix services.
- SwiftyJSON SDK: Simplifies JSON operations in Swift.
- In a terminal, issue the following command from the root directory of the application:
cat > cartfile
- Copy and paste the following Carthage GitHub commands:
github "ibm-bluemix-mobile-services/bms-clientsdk-swift-core"
github "SwiftyJSON/SwiftyJSON"
github "watson-developer-cloud/swift-sdk" - Fetch the libraries from Git by running the command
carthage update --platform iOS
. Add in the Xcode references to the following libraries from theCarthage/iOS
directory:
BMSAnalytics.framework
BMSCore.framework
RestKit.framework
SpeechToTextV1.framework
TextToSpeechV1.framework
SwiftyJSON.framework - Create two new .swift files:
String.swift
: Adds the encoding for the network.WeatherData.swift
: Manages the connectivity to the Weather Company Data service.
- Add the following code to these files:
String.swift
import UIKit// Extending String to support Base64 encoding for network requests
extension String {
func fromBase64() -> String? {
guard let data = Data(base64Encoded: self) else {
return nil
}return String(data: data, encoding: .utf8)
}func toBase64() -> String {
return Data(self.utf8).base64EncodedString()
}
}WeatherData.swift
//
// WeatherData.swift
// WatsonIoTCreate2
//
// Created by Marek Sadowski on 2/25/17.
// Copyright © 2017 Marek Sadowski. All rights reserved.
//import Foundation
import BMSCore //querrying Bluemix for Weather Company Data
import SwiftyJSON //working with JSON with Weather forecastclass WeatherData {
let location = "94596%3A4%3AUS" //find the weather forecast for a zip
let username: String
let password: String
let host: Stringvar temp0 = 0
var golf0 = ""init() {
username = "Weather Company Data user name from bluemix"
password = "Weather Company Data password from bluemix"
host = "twcservice.mybluemix.net"
}func getGolf() -> String{
return "there is a forecast for the temperature \(self.temp0) and the \(self.golf0) golf conditions"
}func getCurrentWeather() {
let url = "https://" + host + "/api/weather/v1/location/\(location)/forecast/hourly/48hour.json"
let basicAuth = "Basic " + "\(username):\(password)".toBase64()
let headers = ["Authorization" : basicAuth]let request = Request(url: url,
method: .GET,
headers: headers)
NSLog("sending request")
request.send { (response, error) in
if let error = error {
NSLog("error sending request")
print(error)
return
}
guard let response = response else {
return
}
if response.isSuccessful {
//print(response.responseText!) //what we got from Weather Company Data
let data = response.responseText!.data(using: String.Encoding.utf8)
let json = JSON(data: data!)
//print(json["forecasts"]) //JSON
let forecasts = json["forecasts"].arrayValue
self.temp0 = forecasts[0]["temp"].intValue
self.golf0 = forecasts[0]["golf_category"].stringValue
NSLog("end of weather company data call : golf0 - temp0")
} else {
if response.statusCode! == 401 {
print("Failed to connect to the Weather Company Data service due to invalid credentials. Please verify your credentials in the WeatherCredentials.plist file and rebuild the application. See the README for further assistance.")
} else {
print("Failed Request to Weather Service :: Status: \(response.statusCode), Message: \(response.responseText!)")
}
}
}}
}
- Add two new elements to the UI (
Main.Storyboard
).- Button: place it at the bottom center with the text
MIC
. - Label: place it above the button.
- Button: place it at the bottom center with the text
- Reference these two elements in the code in
ViewController.swift
:
//STT + Weather + TTS
@IBOutlet weak var speechToTextLabel: UILabel!@IBAction func micButtonTouchedDown(_ sender: Any) {
NSLog("mic button pressed down - starting to listen")
} - Add imports:
//STT
import SpeechToTextV1
//TTS
import TextToSpeechV1
//audio!
import AVFoundation - Add the code to the
micButtonTouchedDown
function:
@IBAction func micButtonTouchedDown(_ sender: Any) {
NSLog("mic button pressed down - starting to listen")//STT - listen for the *forecast* command
let usernameSTT = "STT user name from bluemix"
let passwordSTT = "STT password from bluemix"
let speechToText = SpeechToText(username: usernameSTT, password: passwordSTT)var settings = RecognitionSettings(contentType: .opus)
settings.continuous = false
settings.interimResults = false
let failureSTT = {(error: Error) in print(error)}
speechToText.recognizeMicrophone(settings: settings, failure: failureSTT) { results in
print(results.bestTranscript)
self.speechToTextLabel.text! = results.bestTranscript
NSLog("stopping Mic")
speechToText.stopRecognizeMicrophone()if (self.speechToTextLabel.text!.contains("forecast")){
NSLog("found forecast")//fetch weather data from Weather Company Data
let weather = WeatherData()
weather.getCurrentWeather()
//print the weather forecast
print(weather.getGolf())
sleep(1)// some time to get the result from Weather Company Data
print(weather.getGolf())
let textToSay = weather.getGolf()//TTS - say the data
let username = "TTS user name from bluemix"
let password = "TTS password from bluemix"
let textToSpeech = TextToSpeech(username: username, password: password)
let failureTTS = { (error: Error) in print(error) }
textToSpeech.synthesize(textToSay, voice: SynthesisVoice.us_Michael.rawValue, failure: failureTTS) { data in
var audioPlayer: AVAudioPlayer // see note below
audioPlayer = try! AVAudioPlayer(data: data)
audioPlayer.prepareToPlay()
audioPlayer.play()
sleep(4)
NSLog("end of tts")
}}
NSLog("end STT")}
}
Now, when you press the MIC button and ask for a forecast, you can get the weather forecast with the temperature and a recommendation about whether it’s good weather for playing golf. The Watson Speech to Text service listens for the word forecast.