Create Python apps with IBM Watson and IBM Bluemix

Create Python apps with IBM Watson and IBM Bluemix

Course Features

Course Details

If you’re a Python developer who wants to learn how to add IBM® Watson™ cognitive services to your applications, you’ve come to the right place. IBM® Bluemix®, which is IBM’s platform-as-a-service cloud offering, provides starter applications, runtimes, and buildpacks to help you develop cloud-ready applications in Python. This course lays the foundation for building cognitive cloud applications by showing you how to work efficiently in the Bluemix environment, how to handle versioning, and how to add cognitive services to a simple text identification and translation application.
Get started

Bluemix and Python applications

FREE
Decide on your Python and Cloud Foundry Stack versions
6 of 13
First, let’s explore the Python runtimes that are available on Bluemix. Open a web browser and open the Bluemix console on https://console.ng.bluemix.net/. In the Bluemix catalog, note the two starting points for Python. The first is the Python Flask boilerplate. The second is the Python runtime. The second is the Python Click either to see the landing page for Python. Click VIEW DOCS. The documentation describes how to get started and download starter code. It also shows you how to specify a Python version and which runtime versions are available. Take note of these versions.  Over time, newer versions are added to the list with newer versions sliding in from the right and older versions sliding off on the left. When you develop a new application, you will probably select the version to the right or one that matches the version of Python that you installed. When your application is operational, you will want to future-proof it so that on a restart (or code update), it continues to work. Click the Python buildpack link. You should always control which buildpack is being used. The Python buildpack link takes you to GitHub. In GitHub, click releases to see a list of the stable builds. Select an appropriate release and also note the supported cf_stacks. The cf_stack defines the machine or server that the runtime will be deployed to. Like the supported versions of Python, the stack that is supported by Bluemix changes over time. In particular, you need to be aware of the default Bluemix stack.
FREE
Create your application project
7 of 13
You will initially create your Django projects and application outside of Bluemix. That way you will have more control over the project structure. You can start from either the flask boilerplate or the Python runtime. Create the project named projwatson:> django-admin startproject projwatson Your project skeleton should now have been created, which you can run and see the “Hello Django” page and see the administration features. Change the directory to projwatson and start the application:projwatson> python manage.py runserver You should see that the server is running at http://127.0.0.1:8000/. On the https://localhost:8000/admin page, you’ll see a login prompt for to the administration page, but you will not be able to log in because you did not yet created an administrator ID. On the console, notice a message that says that you have unapplied migrations. The migrations refer to your database and models. At this point, the application is using the default Django embedded sqllite3 database. If you want to remove this message, you must run the migrate command as specified in the message. You will need to repeat this every time you make a database or model change. When you do make model changes, the migration is a two-step process, and you should always perform both steps together. On the initial run, the first step will show “No changes detected,” but it’s a good habit to get into every time you deploy your application changes. These are the actions that your application deployment must take every time there is a database schema change, including the initial deployment. Use Control <C> to interrupt the application and then generate the migration SQL:projwatson> python manage.py makemigrations Start the migrations:projwatson> python manage.py migrate Create your administrator user ID:projwatson > python manage.py createsuperuser createsuperuser command requires user interaction, it will not work when you deploy to Bluemix, so...

Continue Reading
FREE
Customize the application
8 of 13
Create a manifest.yml file. Put this file in the project directory, which is the directory where your manage.py file is stored. Add the following information to your manifest.yml file. In this example, the application is called sc-proj-watson, but you must select a unique name. This information also specifies a buildpack and a Cloud Foundry stack. (A stack is a prebuilt root file system (roots) that works in tandem with a buildpack and is used to support running applications.) Check that the combination of the Python runtime and stack is supported by your buildpack. applications: - path: . memory: 128M instances: 1 domain: eu-gb.mybluemix.net name: sc-proj-watson host: sc-proj-watson disk_quota: 1024M buildpack: https://github.com/cloudfoundry/python-buildpack#v1.5.1 cf_stacks: - cflinuxfs2 command: bash ./run.sh In the code above, notice the line with the command: bash ./run.sh. This line launches the application. This mechanism will give you greater flexibility in running other commands, in particular, the Python shell. In this lab, you will use the command-line option. You can start the application on Bluemix in this way if you want greater flexibility in running other commands such as the Python shell. If you base your application code on the boilerplate sample from Bluemix, notice that the manifest.yml file doesn’t have a corresponding command. Instead, it uses a Procfile to specify how to launch your application. You can use either method. Some developers prefer the bash command line. If you use a bash shell file, be sure to save it in a UNIX format: #!/bin/bash if [ -z "$VCAP_APP_PORT" ]; then SERVER_PORT=80; else SERVER_PORT="$VCAP_APP_PORT"; fi echo port is $SERVER_PORT python manage.py makemigrations python manage.py migrate python manage.py shell < initdbadmin.py echo [$0] Starting Django Server... python manage.py runserver 0.0.0.0:$SERVER_PORT --noreload The following line runs the initdbadmin.py Python procedure that initializes the database: python manage.py shell < initdbadmin.py Create this file...

Continue Reading
FREE
Install prerequisites for Watson services
9 of 13
You won’t need the Watson services until you start using them, but you should install them now so that you’re ready to use them later. Create the package that allows your application to make REST calls: projwatson > pip install requests Install the Watson developer cloud SDK: projwatson > pip install watson-developer-cloud Create a runtime.txt file. Its contents must specify the Python version that you want on Bluemix. The sample file for this lab contains this version: python-3.4.3 Create a requirements.txt file to tell Bluemix which package prerequisites to install for your application:projwatson> pip freeze > requirements.txt The sample file for this lab contains these requirements:Django==1.8.5 requests==2.8.1 watson-developer-cloud==0.1.9 Your project directory should look something like this:   Set up application logging. You might prefer to use application logging rather than print commands. By using Python logging, you can keep all logging statements in the code and control how much information is logged. a. Edit the settings.py file, which is in the projwatson\projwatson folder b. Add the code from one of the following two options to the settings.py file. Both options do the same thing: they set the application level logging. You can set log levels to DEBUG, INFO, WARNING, ERROR, or CRITICAL. Option 1: Works with Python 2 and Python 3 import logging logging.basicConfig( format='%(asctime)s:%(name)s:%(levelname)s:%(message)s', datefmt='%m/%d/%Y %I:%M:%S', level=logging.DEBUG) Option 2: Works with Python 3, is the more flexible, and the recommended option when you use Python 3LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'formatters': { 'ourFormatter': { 'format': '%(asctime)s:%(name)s:%(levelname)s:%(message)s', 'datefmt': '%m/%d/%Y %I:%M:%S', }, }, 'handlers': { 'theConsole': { 'class': 'logging.StreamHandler', 'formatter': 'ourFormatter', }, }, 'root': { 'level': 'DEBUG', 'handlers': ['theConsole'], }, }
FREE
Create the application shell
11 of 13
In this section, you’ll use the Watson Language Translation service to detect the language. This simple use case will show you how to get started with the watson-developer-cloud Python SDK. In your Django project, create a new application named watsonlanguage: projwatson > python manage.py startapp watsonlanguage Register the application in the settings.py file:# Application definition INSTALLED_APPS = ( 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'watsonlanguage', )   Edit the project level urls.py file. A best practice is not to overload the project-level urls.py file routing table. Instead, application patterns should point to an application-level urls.py file, which allows applications to be reused across projects. Add in a URL pattern for your new watsonlanguage application:urlpatterns = [ url(r'^wl/', include('watsonlanguage.urls', namespace="watsonlang")), url(r'^admin/', include(admin.site.urls)), ] The r'^wl' specifies the URL pattern, that is, URLs with \wl\ relate to the application for which the routing table can be found in include('watsonlanguage.urls'). < Because the project urls.py file is now pointing to an application-level urls.py file, you need to create the application-level file.   Inside the watsonlanguage application directory, create a new urls.py file.   Edit that file to add patterns for your first application page:from django.conf.urls import include, url from .views import wl urlpatterns = [ url(r'^lang$', wl.index, name='langhome'), ] When you open the web page /wl/lang, Django will look for an index function in the file views/wl.py file.   Create a views subdirectory. In the views directory, create an empty __init__.py file and a wl.py file.   Add the following code to the wl.py file: # -*- coding: utf-8 -*- from django.shortcuts import render from django import forms import logging logger = logging.getLogger(__name__)   class Form_language(forms.Form):   txtdata = forms.CharField(required=True, label="Text to Process", widget=forms.Textarea)   def index(request):   allinfo = {}   form = None    if request.POST:     form = Form_language(request.POST)     if form.is_valid():       data = form.cleaned_data['txtdata']...

Continue Reading
FREE
Add the Watson service
12 of 13
You now have your shell application and can add the Watson service. In Bluemix, add or bind the Watson Language Translation service. You don’t need to restart the application for now. Now, you see the service credentials.   In the wl.py file, import the language translation component from the watson_developer_cloud SDK:# -*- coding: utf-8 -*- from django.shortcuts import render from django import forms from watson_developer_cloud import LanguageTranslationV2 as LanguageTranslation from watson_developer_cloud import WatsonException import logging import json The final change is in the placeholder that you created in the wl.py file. Remember to use the service credentials that you got for Bluemix as the language translation user name and password.   Open the wl.ph file and find this code:if form.is_valid():    data = form.cleaned_data['txtdata']    logger.info("Text to be processed is %s " % data)     lang = "TBD"       allinfo['lang'] = lang   Add code that invokes the language detection service after the line lang = “TBD” and before the line allinfo[‘lang’] = lang: lang = "TBD" [Add code here] allinfo['lang'] = lang       if form.is_valid():       data = form.cleaned_data['txtdata']       logger.info("Text to be processed is %s " % data)        lang = "TBD"       try:         language_translation = LanguageTranslation(username='<user>',                                                    password='<pwd>')           langsdetected = language_translation.identify(data)                                                                   logger.info(json.dumps(langsdetected, indent=2))          logger.info(langsdetected["languages"][0]['language'])            logger.info(langsdetected["languages"][0]['confidence'])                 primarylang = langsdetected["languages"][0]['language']         confidence = langsdetected["languages"][0]['confidence']                   lang = "I am %s confident that it is %s" % (confidence, primarylang)       except WatsonException as err:        allinfo['error'] = err;              allinfo['lang'] = lang     else:       allinfo['error'] = "Form is invalid"

Language identification and translation

FREE
Add translation capability to your application
5 of 7
Make all the changes for this step in the view/wl.py file. Start by updating the code. All of this code is available in GitHub.   Create a function that will encapsulate the service credentials: def getTranslationService():   return LanguageTranslation(username='<uid>',                             password='<pwd>')   Create a new function named identifyLanguage: def identifyLanguage(data):   txt = data.encode("utf-8", "replace")   language_translation = getTranslationService()     langsdetected = language_translation.identify(txt)   logger.info(json.dumps(langsdetected, indent=2))    logger.info(langsdetected["languages"][0]['language'])       logger.info(langsdetected["languages"][0]['confidence'])       primarylang = langsdetected["languages"][0]['language']    confidence = langsdetected["languages"][0]['confidence']     retData = {"language" : primarylang,              "confidence" : confidence}             return retData This function calls the identify function that you coded before. It will allow you to refine the main logic, which you’ll do shortly. Before you do so, you need to add several new functions.   Add a function to check whether a translation for a specific language is supported: def identifyLanguage(data):   txt = data.encode("utf-8", "replace")   language_translation = getTranslationService()     langsdetected = language_translation.identify(txt)   logger.info(json.dumps(langsdetected, indent=2))    logger.info(langsdetected["languages"][0]['language'])       logger.info(langsdetected["languages"][0]['confidence'])       primarylang = langsdetected["languages"][0]['language']    confidence = langsdetected["languages"][0]['confidence']     retData = {"language" : primarylang,              "confidence" : confidence}             return retData This function iterates through all the supported translation models and checks which ones support the required translation.   Add a function that performs the translation: def performTranslation(txt, primarylang, targetlang):   lt = getTranslationService()   translation = lt.translate(txt, source=primarylang, target=targetlang)   return translation   Modify the index method for the page, the core logic of which is shown here:    try:         lang = identifyLanguage(data)         primarylang = lang["language"]         confidence = lang["confidence"]         outputTxt = "I am %s confident that it is %s" % (confidence, primarylang)                if targetlang != primarylang:           logger.info("Language %s is not %s" % (primarylang, targetlang))           supportedModels = checkForTranslation(primarylang,...

Continue Reading

Classification, keywords, and entities

FREE
Add the Natural Language Classifier to your application
5 of 7
Find the credentials for your Natural Language Classifier service on Bluemix.   Import the Natural language classifier service from the Watson Developer Cloud SDK: from watson_developer_cloud import LanguageTranslationV2 as LanguageTranslation from watson_developer_cloud import NaturalLanguageClassifierV1 as NaturalLanguageClassifier   Change the index function to initialize a classification variable: def index(request):   allinfo = {}   outputTxt = "TBD"   targetlang = 'en'   classification = None   form = None    Modify the code to run a classifier on the translated text or run it on the original text if that text is in English: if targetlang != primarylang:           logger.info("Language %s is not %s" % (primarylang, targetlang))           supportedModels = checkForTranslation(primarylang, targetlang)           if supportedModels:             logger.info("We have some supported translation models")             englishTxt = performTranslation(data, primarylang, targetlang)             outputTxt += ", which in english is %s. " % englishTxt             classification = classifyTheText(englishTxt)                       else:             outputTxt += " ,which unfortunately we can't translate into English"         else:           classification = classifyTheText(data)         if classification:           outputTxt += "(and %s confident that it is %s classification)" % (classification['confidence'],                                                                   classification['className'])            Add the module to perform the classification: def classifyTheText(txt):   logger.info("About to run the classification")   classification = {}   nlc = getNLCService()                                                                           classificationList = nlc.list()   logger.info(json.dumps(classificationList, indent=2))   if "classifiers" in classificationList:      logger.info(classificationList["classifiers"][0]['classifier_id'])     if "classifier_id" in classificationList["classifiers"][0]:        classID =  classificationList["classifiers"][0]['classifier_id']        status = nlc.status(classID)        logger.info(json.dumps(status, indent=2))                      if "status" in status and "Available" == status["status"]:          logger.info("Classifier is Ready to use")                           classes = nlc.classify(classID, txt)          logger.info(json.dumps(classes, indent=2))          if "classes" in classes:            className = classes["classes"][0]["class_name"]            confidence = classes["classes"][0]["confidence"]                classification = {"confidence": confidence,                              "className" : className}            logger.info(classification)               return classification The following function first finds the Natural Language Classifier...

Continue Reading
FREE
Keywords and entities
7 of 7
AlchemyAPI offers two categories of services that enable businesses and developers to build cognitive applications that understand the content and context within text: AlchemyLanguage and AlchemyData News. AlchemyLanguage (Rebranded Natural Language Understanding in September 2016) is a collection of APIs that provides text analysis through natural language processing. The AlchemyLanguage APIs can process text and help you to understand sentiment, keywords, entities, high-level concepts and more. In this section, you learn how to use AlchemyLanguage to extract and analyze entities and keywords from text. Entities Returns items such as persons, places, and organizations that are present in the input text. Entity extraction adds semantic knowledge to content to help you understand the subject and context of the text that is being analyzed. The entity extraction techniques used by the AlchemyLanguage service are based on sophisticated statistical algorithms and natural language processing technology, and are unique in the industry with their support for multilingual analysis, context-sensitive disambiguation, and quotations extraction. Keywords Returns important topics in your content that are typically used when indexing data, generating tag clouds, or when searching. The AlchemyLanguage service automatically identifies supported languages in your input content and then identifies and ranks keywords in that content. Sentiment can also be associated with each keyword by using the AlchemyLanguage sentiment analysis capabilities. Related links Getting started with AlchemyAPI AlchemyLanguage AlchemyLanguage demo

Consumability and security

FREE
Make your new functionality available as a REST service
5 of 6
Now that you created a function that both translates and classifies, you will make that function available as a REST API. Edit the application level urls.py file.   Add the route to the new url – process:# -*- coding: utf-8 -*- from django.conf.urls import include, url from .views import wl urlpatterns = [     url(r'^lang$', wl.index, name='langhome'),     url(r'^process$', wl.process, name='apiprocess'),     url(r'^converse$', wl.converse, name='apiconversation'),    ]   Edit the wl.py view file.   Add an import for an HttpResponse:from django.http import HttpResponse   Write the API function:@csrf_exempt   def process(request):    targetlang = 'en'   classification = {"className":"unknown"}   results = {}   theData = {"error":"If you see this message then something has gone badly wrong"}     validRequest = False    logger.info("Checking request method")     if request.method == "GET":      logger.info("Request is a GET")     data = "Hard coded string to test that API is returning something"     validRequest = True     if request.method == "POST":      logger.info("Request is a POST")             form = Form_language(request.POST)     if form.is_valid():       data = form.cleaned_data['txtdata']       logger.info("Text to be processed is %s " % data)        validRequest = True     else:       logger.info("The form is not valid")       if validRequest:     del theData["error"]     try:        primarylang = theData['language'] = identifyLanguage(data)["language"]       logger.info('checking languages')       if targetlang != primarylang:           logger.info('checking translation support')            supportedModels = checkForTranslation(primarylang, targetlang)         if supportedModels:           englishTxt = performTranslation(data, primarylang, targetlang)           classification = classifyTheText(englishTxt)                     else:         classification = classifyTheText(data)        theData['classification'] = classification['className']              except WatsonException as err:        theData['error'] = err;   else:        theData['error'] = "The form data is invalid";        results["results"] = theData     return HttpResponse(json.dumps(results), content_type="application/json") This function checks for GET methods. If a GET is sent to the API, then a dummy...

Continue Reading

Summary

More Courses by this Instructor