Case Study
Designing AI-Driven solutions for sales efficiency
💡 What to expect in this case study
Context
The Product
CoPilot AI is a SaaS lead generation and engagement tool that leverages AI-driven solutions to create LinkedIn outreach campaigns. It enables businesses to connect with other businesses effectively. However, one of the key product metrics—retention rate—was not performing well, with a high early churn rate.
The Business
Unlike competitors that focus solely on automation, CoPilot AI differentiates itself by prioritizing lead quality over quantity. At the time of this project, the company was shifting from pure revenue growth to sustainable, profitable growth while ensuring alignment with its value proposition.
The Customer
CoPilot AI’s customers use multiple channels to generate leads, with LinkedIn serving as a complementary outreach strategy. The primary customer segments include:
Small business owners (e.g., financial advisors) selling their services.
Sales teams seeking an additional sales channel to expand their pipeline.
Agencies offering prospecting services to their clients.
Their main expectation is a low-effort tool that delivers high-quality leads and, ultimately, results in booked meetings—our primary success metric.
Problem
Are we delivering quality vs. quantity?
Through customer surveys, our design team discovered that most users were unaware of key AI features and how they contributed to lead quality over quantity.
Additionally, our newly formed ML team assessed the accuracy of AI models previously developed by third-party providers. One AI feature, which measured the number of interested leads in a campaign to indicate performance, was found to be inaccurate. This led to a decline in customer trust.
Business
Many customers struggled to see our core market differentiator in action, creating a disconnect between marketing promises and product experience. This contributed to churn and risked eroding trust in our AI suite.
Product
Before I joined this project, the ML team operated in a silo, with little collaboration with PMs and designers, limiting the effectiveness of AI-driven improvements.
Customers
Customer interviews from previous projects indicated a strong interest in using this feature for campaign performance reports. However, its inaccuracy reduced its perceived value.
Solution
Regaining Customer Trust and Unifying the Team
Measuring interested leads (i.e., leads more likely to book a meeting) per campaign is a key way to demonstrate our value proposition (quality over quantity). Ensuring the model effectively supported both business objectives and customer expectations was critical in strengthening trust and improving usability. To tackle this, I championed a cross-functional approach, ensuring ML engineers, PMs, and designers worked as a unified team.
Taking Initiative: A Single Team from Insights to Solution
While not every team member needed to be involved in all steps, sharing customer insights was crucial for crafting an effective solution. As a leader, I encouraged the team to think beyond improving model accuracy and consider how successful sales teams qualify leads. What defines a qualified lead?
Our design and PM teams had already established a customer research hub documenting discovery findings and evaluative insights for all projects we worked on. Leveraging this, I conducted quick interviews with our sales team rather than a lengthy research study. Why? Because:
The sales team actively used our tool and successfully hit revenue goals by refining their lead qualification strategy.
ML engineers were already working on a model that could classify conversations with more detailed labels (e.g., "not interested"), but they lacked clarity on how customers would utilize these insights.
Connecting the Dots: Customer Insights to Solution
Sales teams use frameworks like BANT (Budget, Authority, Need, and Timing) to qualify leads. Based on this system, we identified numerous opportunities to redesign one of our inbox tabs to better support lead qualification.
Our sales team prioritizes responding to leads based on intent. Leads showing immediate interest are highly qualified, so sorting them in the inbox was as important as reporting their numbers.
Analytics revealed that faster responses led to more booked meetings. While this seems intuitive, data confirmed that successful users responded within 1.5 days, compared to an average of 3 days.
Other lead attributes, like “authority,” indicated potential targeting mismatches. Our PM Lead and I mapped this insight into a long-term roadmap, exploring enhancements like automated targeting suggestions.
Sales teams prioritize responding to leads based on intent. Since highly qualified leads show immediate interest, we introduced a new tab to highlight them in the inbox, ensuring visibility was as important as reporting their numbers.
Analytics showed that faster responses led to more booked meetings. Successful users replied within 1.5 days, compared to the 3-day average. To encourage quicker engagement, we prioritized high-intent leads at the top, creating a sense of urgency for customers to respond faster.
We introduced a dual-label system for conversation intent analysis that incorporates a feedback mechanism to improve model training and meet customer needs for accurate reporting. When labels are manually selected, AI suggestions remain available for review and correction, ensuring precision every time.
Measuring the Impact
For Customers
Before this initiative, there was a gap between customer expectations and our product’s performance. Simply showing interested leads did not increase the number of qualified leads, but it did help customers identify them more easily.
After launching the feature:
Response time improved from 3 days to 1 day for customers receiving high-intent leads.
This cohort booked more meetings, validating the effectiveness of surfacing qualified leads.
For the Product
One key initiative I led was ensuring model accuracy (measured by F1 score) became a success metric. I helped ML engineers collect ground truth data, resulting in this being our most accurate model to date.
Tracking how many customers receive high-intent leads and convert them into meetings now allows us to validate if we are delivering on our value proposition. This data-driven approach provides confidence in prioritizing future roadmap decisions, a process I actively contribute to as a leader within our EPD team (Engineering, Product & Design) .
For the Business
As for retention rate, this is a lagging metric that can be influenced by multiple factors. However, we have not yet collected sufficient data to determine its impact on the cohort of customers using this feature, as it has only recently been launched. That said, having data visibility into how well we are delivering on our value proposition is a promising first step.
What's next?
Future Roadmap
Looking ahead, we plan to enhance the model’s capabilities by automating workflows, including:
Automated Follow-ups: Setting reminders for leads with future engagement potential, enabling Sales Development Representatives (SDRs) and Account Executives (AEs) to maintain warm connections.
Timeline: Enables automated follow-ups when the lead’s timing isn't right, streamlining customer outreach.
Authority or Not Interested: Helps identify potential targeting issues, allowing adjustments to improve customer audience selection.
Urgent Lead Notifications: Sending alerts for high-priority leads requiring immediate attention to maximize engagement.
These initiatives build upon our current success as we iterate and refine the solution.
Key Takeaways: The Power of Cross-Disciplinary Collaboration
None of this would have been possible without fostering a unified, cross-functional team. Today, all areas of our EPD team—engineers, ML specialists, PMs, and designers—see the tangible impact of UX research on AI-driven solutions.
Adopting a win-together mindset pays off. By breaking silos and aligning teams under shared goals, we not only improved product performance but also reinforced the role of strategic, user-centered design in AI-driven decision-making.
Ready to create something remarkable?
I’d love to explore how empathy, strategy, and a dash of creativity can elevate your product vision. Drop me a line at laura.rittmeister@gmail.com, and and let’s make it happen—together.