Industry
Real Estate & PropTech
Our contribution
Conversation design, AI training & Ongoing support
Technology
Rasa & LLMs
Union is a US based company that offers software to multi-family property management companies.
In traditional multi-family property management companies, you'll have a dedicated customer support team per property. With Union, such companies now have one support team that are managing all the properties.
Union products enable companies manage leads and keep track of information about property allowing companies to streamline their processes, improve customer service and reduce costs.
How it started
Union has been on the market since 2017 and they always strive to offer better product to their customers. They already helped their clients streamline their customer support operations and manage information across different properties. The next step was to automate some of the repetitive tasks coming to customer support, such as showing available apartments, booking tours, filling in applications and answering other frequently asked questions.
That's where we came in play. We started working together with Union in 2021 to assist them in improving their AI assistant. Union had a list of innovative ideas about how to use AI to optimise their processes and we possessed the complimentary skills to make it reality. We brought expertise in conversation design, AI and chatbot development, working closely with their software engineers to integrate chatbots into their workflows.
In the case-study below, we go into details of our collaborative solutions and the results achieved together with Union.
Challenge
Property management agents have to deal with a lot of repetitive tasks and questions: properties should be shown, differences explained, viewings scheduled. Performing those mundane tasks eats up time from answering more complex questions. Moreover, the time of live agents is costly and they operate within limited business hours. This time and resource constraints lead to missed lead opportunities and potential revenue losses for the property management companies.
Union team realised that they need to automate communication with prospects to help their clients stay on top of the game and increase their agent productivity.
They needed a solution to allow their client scale their operations.
Meet Hobbes
Meet Hobbes - a property management AI assistant that helps answer questions from potential tenants. Hobbes is the very first touch point for the prospects, that welcomes users on the website and tells them more about the property. Hobbes keeps the user engaged, allowing them to ask very specific questions and answering in a personalised way. Hobbes can display available apartments, schedule tours and provide information on rent, fees, policies and more.
Vision
Lead generation
Convert website visitors into pre-qualified leads
Self-service
Allow prospects to discuss the property specifics with AI and get encouraged to schedule a tour
24/7
Engage with prospects outside of office hours
Features
Easily scalable: With every property having different data, we needed to create an AI assistant that can scale and dynamically surface relevant data depending on the property website and current information in the database. Our solution allows to easily scale and add support for extra properties while keeping up to date with database changes.
Knowledgable: Hobbes is trained on 150 curated FAQs and can talk about 12 different topics, including information about units, amenities, community, policies, tours and more.
NLU-powered: People can interact with Hobbes in a free text format. Hobbes has knowledge on a variety of topics, recognises what the user wants and responds to questions in a personalised way, giving the information that was asked, without any irrelevant details.
Proactive: Hobbes asks follow-up questions to engage users and encourage them to see available apartments & schedule a tour.
Smart actions: Hobbes is connected to various smart actions allowing prospects, for example, to manage tours and see units that satisfy their requirements. Hobbes uses AI to know when to trigger each of these actions smoothly.
Human fallback: Hobbes understands when is the right time to connect the user with the agent.
Contextual button suggestions: Using AI, Hobbes displays button suggestions relevant to user's needs.
Featured scenarios
Filtering out floor plans based on specific user request
Explaining pet policies in a personalised and empathetic way
Proactively offering to schedule tours/see apartments
In-depth research and analysis
Examples of FAQs collected by Union
When we started working with Union in 2021, they already had an initial implementation in Rasa and wanted it to be improved.
We started by conducting an in-depth audit of our client's existing conversational AI implementation.
This included:
diving deep into their data: manually reviewing existing FAQs and past conversations
performing an audit of existing Rasa implementation: checking the technical setup and the training data
performing an audit of conversation design: identifying missing knowledge and areas for improvement
This resulted is comprehensive understanding of the work that has already been done paving a path to improve it.
Identifying optimization opportunities
Examples of a new topic discovered using HumanFirst platform
We advocate for data-driven approach to chatbot development. Using HumanFirst platform, we applied advanced Natural Language Processing techniques to analyse patterns in the historical conversation logs. This helped us highlight areas for improvement.
We identified gaps in our client's knowledge base. From analysing real conversations we learnt, for example, that many people wanted to check the status of their application, so we added a flow to support this request.
In total, we added support for 26 new topics.
We also used HumanFirst to restructure the training data to improve precision and recall of intent detection models. HumanFirst platform helped source real user examples to strengthen the training data. Fast forward to the next stages of this project, the updated models achieved ~90% F1 score across 114 defined intents, making sure users are understood even if they speak in different ways.
Expanding knowledge & improving conversation design
We used insights from previous steps to gather missing knowledge, write additional content and reorganise our knowledge in a more-user-friendly way. We redesigned our conversation flows in draw.io, ending up having support for 148 conversation flows.
Snippet of conversation design 1: conversation intro
One of our goals was to make Hobbes more intuitive. We added more personality to Hobbes, rewriting the responses making them sound more friendly and enthusiastic. We rewrote the intro message making it more clear what the user can expect. We also added button suggestions, trying to anticipate the next most probably user need and helping the user learn more about our community. 30% of the messages coming from users come from button suggestions, which we assume means our users find them helpful.
Snippet of conversation design 2: follow-up questions
In a new version, we added follow-up questions to engage more with the user instead of being purely reactive. We made the follow-up questions more relevant by varying the logic for first time users and returning users. Also, there was always an option to connect with an agent, making sure we follow the principle of being helpful.
Snippet of conversation design 3: dynamic responses based on connected property
Our solution is dynamic, meaning that we have one master chatbot that has unified NLU model but can surface different responses per property depending on the data from the database. This means we needed to create different responses for the same question depending on the database values.
We also added more empathy to our conversations, e.g. acknowledging that the user might be upset if our community doesn't allow pets.
Snippet of conversation design 4: multiple levels of fallback
When making a new conversation design, we made sure to always be helpful. We worked on proper error handling, making sure that if we don't understand the user we guide them towards a helpful answer:
If we were not quite sure what the user meant, we tried to make our best guess and offer user possibly related button suggestions.
If we still didn't know what the user meant, they we always offered them to connect to human immediately. This way we designed for 2 levels of fallback, giving different answers the first time the chatbot didn't understand and the second time, forwarding the user to the agent as soon as we understand we can't help.
If we knew what the user was talking about but the data was missing in the database, then we redirected them to customer support agent.
Retrain intent detection model in Rasa
Example of training data in Rasa
Now it was time to put our learnings into practice and implement everything in Rasa.
One of most important parts when working with Rasa is organising training data in a way that makes sense for machine learning models. People can ask the same thing in multiple ways, and we needed to identify the differences. Is "Do you have 2 bedroom available?" same as "Show me two bedroom units?". Is it one intent or two different ones? You can see one more example on the screenshot above, how we put different ways to ask the same question into one intent.
We originally started with 38 intents. We worked on completely restructuring training data using HumanFirst insights. The updated models were trained on real-life examples and achieved ~90% F1 score and accuracy across 114 intents in total.
Migration from Rasa 2 to Rasa 3
Migrating Hobbes from Rasa 2 to Rasa 3 was another crucial milestone in ensuring compatibility and taking advantage of new features and improvements. Some features were removed and some new added and we had to find workarounds to support all our existing functionality.
Then ChatGPT came out 🚀
Migrate from Rasa to GPT
While Rasa 3 was excellent in understanding user queries, its responses inherently remained static and generic. In an effort to add a more personal touch, we made our responses dynamic by creating templates and incorporating entities from user input. We even went a step further by writing variations for each response and presenting them randomly to users, to create the illusion of engaging in a real conversation rather than having an interaction with a robot.
We wanted to provide even a more personalised experience to our users and react to their questions by omitting all the unnecessary information and focusing on the specific thing the user was asking. So we migrated our project into a new architecture that was using GPT models.
New challenges appeared, as we now needed to fight hallucinations and make sure that the generative models are robust. But as we've already have collected data and done a lot of research during all our previous steps, migration went easier.
We continue collaborating with Union to continuously update and improve Hobbes, making it more proactive, helping drive sales and anticipate user needs.
Results
106 properties
Scaled across 106 properties
57k
The chatbot handled more than 57k messages since launch
2.5k
Every month Hobbes help automate 2.5k messages
Testimonial
Ethan L,
CEO of Union
Do you find this case interesting? Let's build something extraordinary together. Tell us a bit about your idea and let's discuss how we can help you with our services.