How to make your iOS app smarter with sentiment analysis

For quite some time now, I’ve been developing an interest to data analysis to find new ways to improve mobile app. I’ve recently found some time to experiment neural language processing for a very specific usecase related to my daily work, sentiment analysis of customer reviews on fashion items.

Context

I’ve been working in fashion e-commerce for almost four years now, and something really important for any e-commerce is customer reviews. So I was wondering, “what if we could anticipate customer’s review, so we could help them before they even ask”. If you look at Amazon, or App Store reviews, imagine if you could reduce all the negative feedbacks. So I looked into it a bit more closer, and it took me only couple hours to get something working.

So let’s say we have a product page where customer can leave a review and, as a business, I want to limit the negative one but also help my customers. From that point of view, my idea is simple, if the customer types something negative, we would show a button for customer support. But how to know if it’s negative?

Well, that’s the part it’s getting interesting. I’m going to use some basic neural language processing (or NLP) algorithm to do a sentiment analysis of it. This part mix more knowledge of data analysis and a bit of machine learning than iOS development, but bear with me, Apple made it very straightforward for us.

Dataset

First, I needed some data to train my model on. Lucky enough, I’ve found a dataset of women’s clothing reviews on Kaggle with more than 20 000 records. That’s enough to start. It includes rating and review but a recommendation field.

From that data, I need to define what is a “positive” feedback and “negative” one. I’m not a data scientist, I don’t have much background on this, so I only applied a simple logic: if the customer rated the item 4 or 5 (out of 5) OR if the customer recommended the item, then I would consider it positive. Otherwise it would be negative. I believe, I genuinely believe I do the same: if I recommend a service to a friend, or if I leave a good review, it would be at least a 4.

Ideally, you want to keep a neutral review as well, to balance out, but I’ll keep it simple for time being.

From there, I applied that rule to the whole dataset. Now we can train our machine learning model in Swift.

Training model

With CoreML and CreateML, Apple made it very easy for developers to create new machine learning models base on predefined computed one. Since I’m interested into sentiment analysis in text, I’ll be using MLTextClassifier for this. I was surprise how short the code was to build it.

import CreateML
import AppKit

// define the input of dataset as json format
let data = try MLDataTable(contentsOf: URL(fileURLWithPath: "/Users/benoit/Desktop/clothing-review.json"))

// split data, 80% to train, 20% to test
let (trainingData, testingData) = data.randomSplit(by: 0.8, seed: 5)
let sentimentClassifier = try MLTextClassifier(trainingData: trainingData, textColumn: "Review Text", labelColumn: "Sentiment")

// define training accuracy in %
let trainingAccuracy = (1.0 - sentimentClassifier.trainingMetrics.classificationError) * 100

// define validation accuracy in %
let validationAccuracy = (1.0 - sentimentClassifier.validationMetrics.classificationError) * 100
let evaluationMetrics = sentimentClassifier.evaluation(on: testingData, textColumn: "Review Text", labelColumn: "Sentiment")

// define evaluation accuracy in %
let evaluationAccuracy = (1.0 - evaluationMetrics.classificationError) * 100

let metadata = MLModelMetadata(author: "Benoit Pasquier",
                               shortDescription: "A model trained to classify women's fashion review",
                               version: "1.0")

try sentimentClassifier.write(to: URL(fileURLWithPath: "/Users/benoit/Desktop/FashionSentimentClassifier.mlmodel"), 
                              metadata: metadata)

When running, this will take couple minutes. Compare to my previous try in fashion image classifier, it feels way faster.

Parsing JSON records from /Users/benoit/Desktop/clothing-review.json
Successfully parsed 23486 elements from the JSON file /Users/benoit/Desktop/clothing-review.json
Tokenizing data and extracting features
10% complete
20% complete
30% complete
[...]
Iteration 19 training accuracy 0.993544
Iteration 20 training accuracy 0.993597
Iteration 21 training accuracy 0.993703
Iteration 22 training accuracy 0.993756
Finished MaxEnt training in 2.00 seconds
Trained model successfully saved at /Users/benoit/Desktop/FashionSentimentClassifier.mlmodel.

Now that the first version of my fashion sentiment analysis model is ready, let’s test it. Good thing, I can still do that in Xcode Playground.

import NaturalLanguage
import Foundation
import CoreML

let modelPath = "/Users/benoit/Desktop/ClothingSentimentClassifier.mlmodel"

let url = URL(fileURLWithPath: modelPath)
let urlModel = try MLModel.compileModel(at: url)

let model = try NLModel(contentsOf: urlModel)

model.predictedLabel(for: "Dress looks cute") // positive
model.predictedLabel(for: "Dress looks cheap") // negative

model.predictedLabel(for: "Dress looks very cute, but the fabric is bad quality and the zipper is cheap") // negative
model.predictedLabel(for: "Dress looks nice and the fabric is good quality") // positive

Based on the result, it seems working decently, and that blows my mind.

Of course, I can manage to have false positive using more complex and nuanced sentence, with double negation for instance, but for the time I spent on this, I’m very happy with this first result.

What’s the next step?

From here, I can drop this model into an iOS app and from there, when a user pause typing during a review, I can let the device run a quite sentiment analysis on it and show a “call customer support” to prevent him to leave a bad review but also help him improve his user experience overall.

I can also work further on the model itself, the more data you have and the more time you spend on tweaking the model, the better accuracy you’ll get. Each project is different, and I feel that each model should be, to adapt to your own problem.

In conclusion, starting from an user experience problem or business one, we found a way to improve the user experience by using machine learning and simple data analysis concepts. If this seems far from iOS development of day to day, it amazes me how fast it was to prototype a first version. Obviously, the more complex is your app (and product), the more data you would need to refine and keep the accuracy hight.

For now, this one is limited to women clothing items, but it makes me wonder how far we could go to keep the UX of our iOS app really sharp.