Saving lives with AI health coaching

Saving lives with AI health coaching | OpenAI

Healthify

Healthify collaborates with OpenAI to improve millions of lives with sustainable weight loss.

Loading…

Share

With over 40 million users, Healthify⁠ is India’s largest health platform, providing health tracking and AI-enhanced health coaching to help users become fit and reverse metabolic disease. Since introducing AI in its coaching and tracking components, Healthify has already helped users lose an aggregate of 25 million+ pounds and with OpenAI’s API, Healthify is greatly simplifying tracking and scaling up coaching.

Healthify’s AI journey

Healthify has been pioneering using AI for driving behavior change in health & fitness for over a decade. By 2018, Healthify already had more than 5 million users and hundreds of nutritionists and trainers exchanging millions of messages with their clients each month, along with tens of thousands of hours of calls and meal & fitness plans each month. Healthify’s data pipeline inherently also came with feedback loops—with knowledge of which messages and plans created higher engagement and impact.

Using this wealth of real-world, contextual information, Healthify made significant strides in AI, notably with the launch of Ria, the world's first AI-powered virtual nutritionist, and Coach Co-pilot—its coach-facing assistant.

Ria utilized hierarchical LSTMs (Long Short-Term Memory) and custom NLU (Natural Language Understanding) systems to accurately recognize user intents and provide relevant answers. By 2020, Ria was handling the majority of user messages directly. Combined with the Coach Co-pilot, this breakthrough allowed coaches to significantly scale their services, managing up to 300 clients simultaneously, while hitting record high NPS through improvements to personalized health coaching.

In 2021, Healthify also introduced Snap, a revolutionary feature designed to simplify calorie counting through photo recognition of food. Snap utilized Convolutional Neural Networks (CNNs) and proprietary models to accurately recognize single food items, particularly focusing on Indian cuisine. This technology not only respected user privacy but also tailored food recommendations by incorporating user-specific contexts. Over time, Snap achieved around 80% accuracy for single Indian foods.

Despite these successes, Healthify encountered challenges:

• Performance. It took multiple iterations to get Snap to recognize food accurately, and it struggled when photos contained multiple foods. As a result, “Snap was only used 10–20% of the time,” said Healthify CEO Tushar Vashisht⁠. Similarly, Ria was rules-based, so it couldn’t truly answer the long tail of valuable yet complex queries about nutrition (e.g., “how did my food yesterday affect my sleep last night?”).
• Scale. For each new country they wanted to add, Healthify needed to spend lots of effort on localizing models for the language, common foods, and exercise routines. “It took us two years to enter Southeast Asia,” Vashisht explained.

The collaboration with OpenAI emerged as a pivotal solution to overcome these limitations, significantly advancing Healthify’s offerings and setting a new standard for innovation in the health technology sector.

> “We integrated with everybody who’s out there. OpenAI was the best.”

Tushar Vashisht, Healthify CEO

Using GPT-Vision and embeddings to instantly track food and scale coaching

The Healthify team believed that AI could increase their products’ accuracy, usefulness, and scalability overnight. With OpenAI’s first-to-market and simple API, the team quickly built prototypes to test model performance. Before they officially chose a provider, the team evaluated numerous model options, including open-source models.

After thorough evaluation, Healthify partnered with OpenAI for multiple reasons:

• Best-in-class accuracy. GPT‑4 Vision instantly surpassed Snap pipeline’s performance; the model by itself recognized foods from across the world and detected multiple food items in photos. GPT‑4 (for Ria) and Whisper (for Coach Copilot) similarly outperformed both their existing pipelines and alternative Generative AI models.
• Easy integration. Once they validated prototypes and it came time to integrate into their production systems, the Healthify team could simply add OpenAI’s API into their existing pipelines. “It was very easy to build proof-of-concepts with OpenAI and use those to build production systems,” said Vashisht.
• Simple fine-tuning. Healthify chose OpenAI in large part because fine-tuning was available out-of-the-box. It was simple to model their data for fine-tuning and upload it to the service. The team was able to successfully ground OpenAI’s models to their use case, without much configuration.
• Embeddings. Healthify used the Embeddings model to solve one of its key problems: how do you match two foods beyond just text matching? They needed to match food names returned by GPT‑4 with food names already in their own system. “GPT is a model of its own. It has its own dictionary of food names. Healthify has its own food names and how to do that matching is something we were trying to figure out,” commented Abhijit Khasnis⁠, VP of Technology. “And when we tested the OpenAI Embeddings model, we realized that cosine similarity matching between the GPT identified food name and our food embeddings gives us high accuracy!”

Millions of lives improved with increased engagement

Snap now employs an ensemble of proprietary models along with GPT‑4 Vision to understand user-specific contexts while ensuring data privacy. Post-image analysis, custom heuristic models are applied to deliver precise food recommendations. This upgrade has elevated Snap’s accuracy to match human recognition, streamlining the process of dietary tracking.

Ria’s latest update represents a significant leap in complexity and capability, incorporating an ensemble of fine-tuned models, including GPT 3.5 and the cutting-edge GPT‑4 Turbo. This advanced setup enables Ria to access and interpret Healthify’s extensive literature and each user’s unique history with the platform. Such integration has transformed Ria into the most comprehensive and intelligent version yet, offering personalized health and nutrition advice at an unprecedented scale.

Across the board, the Healthify team has seen engagement and retention shoot up, leading to better health outcomes for their users:

• Users track food 50% more often with Snap.…