Adding ~80,000 engaged sessions for 40 million users.
Building a High-Impact, Low-Lift feature with AI
I designed a high-impact, low-effort feature, ensuring it was easy for the dev team to implement. I led the ideation, prototyping, and testing process using Google Analytics , VWO, HTML/CSS, JavaScript, and AI tools like ChatGPT and Google Gemini. Working in an Agile two-week sprint alongside the product manager, dev team, and AI, I helped drive measurable results. Because of the tight timeline, I used lean UX processes.
Early data showed a 0.2% increase in time on page, CTR, and session duration—adding approximately 80,000 engaged sessions for 40 million users, with potential conversion rate gains.
Due to confidentiality agreements, some details and visuals have been omitted or altered, so this case study focuses on the design process and outcomes.
Lean UX Process
It Started with Failing Forward
I saw that our key competitor—a big name in gaming—had YouTube videos embedded on their highest-traffic pages. Seemed like a solid move, right? Users interacting with videos = more time on page, which is a key KPI for the company.
Well… not quite. A/B testing results showed the opposite—time on page actually dropped, and clickthrough rates took a hit. So yeah, back to the drawing board.
For this idea, I relied on competitor analysis and quantitative data- without taking the users into account. So, going forward, I incorporated findings from user testing into my ideas. And when I incorporated user testing into my ideas, I came up with a win!
Research & Ideation
Our top competitor had a 'back to top' button on their pages with the most traffic. This sparked the idea.
I combined quantitative and qualitative data to prove the value of this idea. I used google analytics for quantitative data, as the company primarily relied on quantitative data. I used contextual inquiry as qualitative data.
Contextual inquiry and analytics supported the idea of the anchor link.
I collaborated with the project manager to ensure the idea aligned with business goals.
Contextual Inquiry
I introduced qualitative insights to the company. To support the need for a 'back to top' button, I pulled key findings from an ongoing remote contextual inquiry I was conducting. In the contextual inquiry participants used the site naturally while thinking out loud, allowing me to observe and manually record their actions and reactions. This was done over Google Meet, and participants were recruited over instagram stories. They completed a Google Form for eligibility.
I believe that combining both quantitative and qualitative insights makes for the most impactful results.
New Idea
Low Lift
This project requires setting up an anchor link using JavaScript and designing the button using HTML/CSS.
High Impact
Implementing a ‘back to top’ button has the potential to increase time on page by at least 30 seconds. Time on page is a KPI.
Building the A/B Test
Four button variations were tested:
Version A (90% scroll depth)
Version B (post-iframe load, centrally displayed after mouse interaction)
Version C (right corner)
and Version D (90% scroll depth).
Key metrics were time on page and overall session duration.
The site has over 40 million users per month, so we segmented the test to 10% of traffic . The percentages of the users exposed to the variants was equally split between each variant.
For the statistical analyses, the type of test used was a Z-Test, where p < 0.05
A statistically significant result was a ≥0.1% improvement in key metrics. With 40 million users per month, even small gains had a major real-world impact.
Coding with AI
I collaborated with ChatGPT to build the anchor link. I edited ChatGPT's HTML/CSS code, and ChatGPT helped debug it. ChatGPT generated and debugged the JavaScript.
Testing
The test was handed to developers for QA. When it was bug-free and live, we discovered that Version A performed the best out of all the variants, with the highest time on page, session duration, and clickthrough rates. Early results show a 0.2% increase in time on page, CTR, and session duration—adding ~80,000 engaged sessions for 40 million users, with potential conversion rate gains.
Challenges
Ideation was the most time-consuming phase of this project. I generated a lot of ideas, but many were ultimately ruled out after vetting them for feasibility and impact. However, this idea stood out, and it made the final cut. Through this process, I learned how to structure my ideation more effectively: first, I let my ideas flow freely and jot down all of them, without filtering myself. Then, I revisit the ideas and evaluate their potential using quantiative and qualitative research. This approach helps me refine my ideas and focus on the ones with the greatest impact.
Lessons Learned
Small changes = big impact! It’s interesting how so much thought went behind a feature as simple as a button. And that simple feature created a tangible, positive difference according to testing results. I also learned how to use AI to streamline my UX process.
Future Plans
Due to the timeline, there wasn’t time to refine the UI of the button. The test was mostly focused on the functionality of the button. If there was more time, I would make the button look more unique while adhering to WCAG accessibility guidelines and incorporating relevant UI trends.