Using AI to calculate a fair review score for a venue
In this post, we explain how we use some AI and old-fashioned content scraping to generate a fair review score for a venue. We call this "FairScore" and it makes up part of our product, Barkeeper.
In this post, we explain how we use some AI and old-fashioned content scraping to generate a fair review score for a venue.
We call this "FairScore" and it makes up part of our product, Barkeeper
Content sources
Content is very important for building the FairScore. It relies heavily on reviews posted about a venue on the following services:
- Google Places / Google Maps
- Yelp
- TripAdvisor
Where an API is available to get access to review content, it uses that. Otherwise, the content is scraped and processed. The data from all the sources is cleaned up and stored in a standard schema to make comparisons easier.
Reviews which are 1 star but have no content are ignored. Reviews which are overly negative but have a reply from the owner refuting the claim (or helping the customer) are also ignored.
Additionally, FairScore also checks social media posts for entries about the particular venue that express an opinion about it.
FairScores can change between calculations, since they are partly based on data sources that are updating in realtime.
FairScore also doesn't mind what the language a particular piece of content is in. It's designed to handle multiple languages, like all of the other services we've built for Barkeeper.
Processing and getting a little artificially intelligent
Once the content is cleaned up and on-hand, it gets processed in several ways to help the score be calculated.
First, the content is run through our Sentiment Analysis models. These are RoBERTa based models, specifically building on SiEBERT, Twitter-roBERTa-base, and twitter-XLM-roBERTa-base.
This helps FairScore to weigh the review against the star rating given of the review, and allow for some flexibility between the star rating and the content of the review to influence each other.
For content from social media, FairScore simply checks if the post is positive, negative, or neutral.
The results are then processed through a Clustering system and after a little math, the FairScore is generated.
How long does it take?
From having zero data about a venue, to generating a FairScore usually takes a few seconds.
In Barkeeper, it is much faster as the service is always processing in the background for the venues of customers. Barkeeper also has an extra dimension to its scoring - in-person reviews from members of staff.
Sounds great, can I try it?
You can try FairScore for yourself at https://fairscore.nekotachi.co.uk , use of the service is limited to prevent abuse, and you will see some examples of results there.
As for Barkeeper, it is currently in closed beta. If you are interested in joining the testing group, please send an email to [email protected] to get more details.
