MARGO

News

Tutorial – Artificial Intelligence: from prototype to deployment

Develop a smart mobile app with Microsoft Azure Machine Learning Studio and Xamarin

By Mehdi Rahim Data Scientist

02/05/2019

The contribution of AI to our daily life is continuing to grow across various applications it has put within reach, such as voice recognition, face recognition in images, chatbots etc. These technologies use Machine Learning algorithms to establish predictive models tailored to the requirements.

The deployment of AI in the form of a web or mobile app is a key step towards getting more out of it. This involves an API (web service) that will be queried with each new input (new voice command, new image to identify).

In this article we present a practical case study for building an AI model and deploying it in a mobile application, all in less than an hour.

 

Azure Machine Learning Studio (AzureML) for the model, and Xamarin for the deployment

There are a number of Machine Learning libraries, either open-source or commercial, in the form of APIs or software packages. In the present scenario, we will use Azure Machine Learning Studio: it’s a tool for creating a Machine Learning model from a web browser via a graphic interface, so there’s no need to write even a line of code!

AzureML studio contains the essential steps of Machine Learning such as data selection, cleaning, learning and validation.

Xamarin is a suite of development tools in C# for native mobile applications (Android, iOS, Windows). It allows you to develop graphical interfaces both common and specific to each operating system.

The link between AzureML and Xamarin is through the rich AzureML functionality, which makes it possible to deploy a Machine Learning model in the form of a web service. AzureML thus provides templates in C# classes ready for use in Xamarin.

 

Practical example: identifying trees

Here is a fairly playful Machine Learning example: we want to predict the planting year of a tree in Paris from its characteristics (height, circumference and diameter). For this we’ll use data published by the city of Paris open-data portal (https://opendata.paris.fr/), which lists all the trees in Paris. In what follows, we describe the steps needed to recreate this example.

Portail open-data de la ville de Paris

 

Building the model with Azure ML

To use AzureML go to https://studio.azureml.net. Once connected, the tool lets you create projects containing the machine learning experiment. In our scenario, we want to predict the planting year of a tree.

The experiment is made up of a combination of processes represented by bricks. Each brick applies a transformation to the data, the connections representing the sequence of the bricks and hence of the transformations.

We add the different bricks to prepare the data and build the predictive model. The experiment thus contains the following bricks in order:

  • The tree dataset uploaded in csv format (1). Note that AzureML can read several data file formats including flows and databases.
  • Select columns useful to our model (2). We have reserved 4 columns describing the tree species (e.g: oak, plane etc.), its circumference (in centimetres), height (in metres) and the planting year, which is the variable the model has to predict.

jeu de données des arbres azure machine learning

  • Clean the data (3)through transformations like deleting tree records with missing data or extracting the year from the planting date. The clean output is then converted to a dataset format (4) in order to set up the predictive model.
  • Divide the whole set of data into two subsets: (5)one learning subset for building the model and another test subset for evaluating the model. This allows you to measure how well the model generalises data different to that of the learning data, and to avoid the phenomenon known as over-fitting.
  • Train the model on the learning subset (6). We have used the Boosted Decision Tree Regressor algorithm (7), which has the advantages of combining heterogeneous information corresponding to the nature of our variables (dimensions, species etc.) and of requiring minimal programming. The Boosted Decision Tree Regressor algorithm produces a set of rules arranged in the form of a decision tree. In our scenario, the decision involves estimating (or regressing) the planting year of the tree from a set of terms based on its characteristics. We use the term regressing because the algorithm calculates an approximate year (as a quantitative value).
  • Brick scoring (8) lets you test the model on the test subset to evaluate the quality of our model and estimate its precision (9). Thus the model is used to calculate planting years. These estimates are compared to real planting years in order to measure the model’s error rate. From this we see an error margin of +/- six years, which is not bad.
  • In order to use the model on new data, the score model prediction module (11) predicts the planting year of trees based on their description. The model is deployed as a web service through the web service input brick (12)for the data set, and the web service output brick (13) for returning the calculated planting year. The web service consists of a URL that can be called from a client such as a browser or a mobile app.

Construction du modèle avec Azure Machine Learning

 

Creating the mobile application with Xamarin

The objective now is to deploy our model on a native mobile application. We’ll use Xamarin to develop an interface to query the AzureML Studio web service. The application collects the information entered about the tree’s species, circumference and height. It responds with a prediction of the planting date. Here too the coding effort is minimal since AzureML Studio generates C# code for sending the data to the service and getting back the calculated result. Which means we can concentrate on creating the Xamarin application itself.

We need to start by creating a new “(Xamarine) Cross-Platform App” project in Visual Studio.

projet Cross Plateform App Xamarin

Once this is done, you need to pick the following tree view in the solution explorer:

Arborescence dans l’explorateur de solution Xamarin

The first project is for shared code, the others are specific to each platform.

Xamarin Forms is a powerful way to write one set of code which works on all mobile platforms. However you can write specific code in dedicated projects if need be. For example, camera management in Android and iOS is completely different. The specific code then integrates easily into the shared code by dependency injection.

We organise our code with the design pattern recommended for Xamarin projects: Model-View-ViewModel. We’ll use MainPage.xaml as our view, add the class Tree as the Model and the class ArbreViewModel as the ViewModel.

classe arbre xamarin

As with WPF projects, the View is represented by a XAML file. At the time of writing this article, Xamarin does not yet have a graphical interface editor. You therefore have to edit the XAML manually. But Visual Studio makes it easier to do this by providing autofill suggestions.

NB: it is perfectly possible to write the View as a C# class. However doing it in XAML is recommended, for better legibility and better separation between the HMI and the code.

The MainPage.xaml file is modified as follows:

 

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
xmlns:local="clr-namespace:Tree"
x:Class="Tree.MainPage">

<StackLayout>
<Entry Text="{Binding Circonference}" Placeholder ="Circonference (cm)"/>
<Entry Text="{Binding Hauteur}" Placeholder ="Hauteur (m) :"/>
<Entry Text="{Binding Espece}" Placeholder ="Espèce"/>
<Button Text="Envoyer" Clicked="OnSendClicked"/>
<ListView x:Name="ArbreList" SeparatorVisibility="Default" SeparatorColor="Black" HasUnevenRows="True">
<ListView.ItemTemplate>
<DataTemplate>
<ViewCell>
<StackLayout>
<Label Text="{Binding Circonference, Mode=OneWay, StringFormat='Circonference : {0}'}"/>
<Label Text="{Binding Hauteur, Mode=OneWay, StringFormat='Hauteur : {0}'}"/>
<Label Text="{Binding Espece, Mode=OneWay, StringFormat='Espèce : {0}'}"/>
<Label Text="{Binding Annee, Mode=OneWay, StringFormat='Année de Plantation : {0}'}" TextColor="Blue" FontAttributes="Bold"/>
</StackLayout>
</ViewCell>
</DataTemplate>
</ListView.ItemTemplate>
</ListView>
</StackLayout>

</ContentPage>

In this example, the view elements are stacked horizontally on top of each other via the StackLayout

The view consists of three input fields for the tree properties, then a button to send to the AzureML web API, and a list to display the sent requests with their results (prediction of planting year).

The Binding keyword creates the link between the properties of the View elements and those of the ViewModel.

In the following example, the Text property of the input element is linked to the Circonference property of the ArbreViewModel.

<Entry Text="{Binding Circonference}"

Objects are linked in C# in the MainPage class associated with XAML, that defines mainpage.xaml.cs for which the code is:

 

using System;
using System.Collections.ObjectModel;
using Xamarin.Forms;
namespace Tree
{
    public partial class MainPage : ContentPage
    {
        private ArbreViewModel ArbreViewModel { get; set; } = new ArbreViewModel();
        private ObservableCollection<ArbreViewModel> ArbresList { get; set; } = new ObservableCollection<ArbreViewModel>();
        public MainPage()
            {
            BindingContext = ArbreViewModel;
            InitializeComponent();
            ArbreList.ItemsSource = ArbresList;

        }

        private async void OnSendClicked(object sender, EventArgs e)
        {
            var arbreVM = new ArbreViewModel(ArbreViewModel);
            ArbresList.Add(arbreVM);
            await arbreVM.ComputeAnnee();
        }
    }
}

The ArbreViewModel object is defined as the BindingContext of the view. This allows these properties to be linked in the XAML.

An ArbreViewModel List is then defined as the data source to the XAML ListView.

NB: using ObservableCollection is very important. This collection provides notification of changes produced, which allows the view to be updated.

This class also defines the method called when the OnSendClicked button is pressed. It adds a tree to the list and starts the year calculation.

NB: the Async keyword allows asynchronous operation. After adding the tree to the list and using the Await keyword, the method passes control to the year calculation. This long process (sending the http request and waiting for a response from the server) will be done in the background. This is very important as it prevents the graphical interface from freezing. It also allows you to stack queries without waiting for the results of earlier ones.

Now let’s go back to our model. The Tree class alone contains the properties of the tree:

classe arbre xamarin

The ViewModel acts as the intermediary between the view and the model. It sends the properties of the latter to the former.

Le ViewModel sert d'intermédiaire entre la vue et le modèle

To do this it implements the property INotifyPropertyChanged. Each time a property is changed, it triggers the PropertyChanged event to update the view. Here is part of the code for the class in order to illustrate the function:

public class ArbreViewModel : INotifyPropertyChanged
    {
        public event PropertyChangedEventHandler PropertyChanged;
        private void OnPropertyChanged(string name)
        {
            PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(name));
        }
        public string Circonference
        {
            get
            {
                return arbre.Circonference == 0 ? null : arbre.Circonference.ToString();
            }
            set
            {
                int intVal;
                int.TryParse(value, out intVal);
                if (arbre.Circonference != intVal)
                {
                    arbre.Circonference = intVal;
                    OnPropertyChanged(nameof(Circonference));
                }
            }

 

The code for calculating the year is derived from the example given in the Web Service documentation produced by AzureML. Note that this method must be written asynchronously for the reasons given earlier:

public async Task ComputeAnnee()
{
  .
  .
  .

  response  = await client.PostAsync("", new StringContent(jsonReq, Encoding.UTF8, "application/json"));
  .
  .
  .
}

 

NB: use of communication code with the AzureML service has necessitated the installation of a NuGet as defined in the documentation generated. NuGet can’t be Installed on a shared project. It must be installed on all of the target device projects in order to be accessible from the shared project.

To run the application in Windows, the UWP project must be defined as a start-up project before launching. Here is what happens at the start:

définir le projet UWP comme projet de démarrage puis lancer l’exécution

And after sending a request:

Projet UWP Xamarin après le lancement de la requête

To run on Android, the Android project must be defined as a default project. Connect an Android phone to the PC and then run. Here is what happens at the start and after sending a request:

capture android projet xamarin

 

Conclusion

With a few clicks and some lines of code we have just created – for free – the contents and interface of an AI mobile application. It’s possible to extended the use to more advanced data sources such as using a camera or microphone for recognition. The example we have given demonstrates the ease and growing accessibility of AI tools for all, without requiring expertise in data science.


By Mehdi Rahim Data Scientist
Artificial Intelligence (AI)
Machine Learning
Microsoft Azure
Microsoft Xamarin
Mobile
News

Data Science applied to the retail industry: 10 essential use cases

Data Science is having an increasing impact on business models in all industries, including retail. According to IBM, 62% of retailers say the use of Big Data techniques gives them a serious competitive advantage. Knowing what your customer wants and when, is today at your fingertips thanks to data science. You just need the right tools and the right processes. We present in this article 10 essential applications of data science in the field of retail.

31/05/2018 Discover 
News

Modern Android Development: the new contributions of the Java language

Since the release of Android 1.0 in 2008, the Android ecosystem has evolved enormously, but especially in recent years, with the new contributions of the Java language, changes related to terminals and the appearance of new tools. Recommendations and good practices have evolved accordingly.

15/05/2018 Discover 
News

Establishment of a centralised log management platform with the Elastic suite

The volume of data generated by our systems and applications continues to grow, resulting in the proliferation of data centers and data storage systems.  In the face of this data explosion and the investment in skills and resources, decision-makers need sophisticated analysis and sophisticated dashboards to help them manage their systems and customers.

14/05/2018 Discover 
News

A brief introduction to Chatbots with Dialogflow

Nowadays, I'm working on a Chatbot project with Google Dialogflow. In this article, I will present some notions about Dialogflow and Chatbots. Then, I will talk about how to create a simple Chatbot with Dialogflow Platform.

07/05/2018 Discover 
News

Tunisia Digital Summit 2018: key points

Last week, Margo was invited to participate in the Tunisia Digital Summit 2018. This is the second year of this major North African event, held in the centre of Tunis, at the recently re-furbished Laico Hotel. Two days of talks were held on the topics of digital transformation, artificial intelligence, social media and blockchain. Over 40 Tunisians and international speakers attended, as did around twenty local start-ups to show their solutions and provide a specific focus on the major disruptions that digital is causing for companies. Here are some notes on the main topics to draw our attention at this two-day event.

03/04/2018 Discover 
News

A Gentle Introduction to Reinforcement Learning

Reinforcement learning is considered as the science of decision-making: trying to understand the optimal way to make decisions. Nowadays, it is applied in many different fields and has achieved many distinguished accomplishments.

20/03/2018 Discover