︎

Test Creation Flow



Background
The main user journey in Shoplift is creating A/B tests. At that point in time, our testing capabilities were limited, but we had a rough roadmap to expand on types of tests to support our users' needs. The existing flow was a wizard flow where the user needed to make one decision at a time.

Release date: September 2023

Users
CMOs, E-commerce managers, Store founders

My Role
  1. Planning the design process and writing the PRD - steps, requirements and timelines with the CPO
  2. User research - Talking to CS and conducting user interviews
  3. Hands-on design and prototyping
  4. Internal usability testing sessions
  5. Delivering design to the development team
  6. QA
  7. Post-launch analysis


Before & after - home page

⚡️ Challanges

  1. User needs and gaps of knowledge were not addressed.
  2. There was no room for expanding the feature
  3. Technical: The app’s code base was being redone (moving from React to Vue) and this was the last piece to update
  4. Visual: We were implementing a new design system and needed to solidify the design.

I researched the problem space
  1. Unmoderated observations (Hotjar recordings)
  2. Met with customer success - I gather initial sentiment about the flow
  3. User interviews - in-depth conversations about user goals and needs
  4. UX and copy review - breakdown of problem areas

Key findings

✅ Feature Priorities

  • Reduce cognitive overload when creating a test
  • Reduce test creation time
  • Help newbies get started by incorporating the test library into the flow
  • Allow the expansion of the flow to more testing capabilities







💡 What I learned

  1. Our original scope was way too broad. We were too eager to serve nice-to-have ideas and lost time and resources to try to build them in.
  2. Talking to customer success early in the process changed our core hypothesis about the friction points and that there’s a difference in expectations between novice and experienced users.
  3. Post launch analysis revealed promising numbers in unexpected areas - we reduced the time to launch a test for first time users. 

Later this metric was correlated to our activation research - users who launch tests and get positive results within their trial period are less likely to churn.


Feature evolution

🥂 Since then

As we grew our user base and test creation kept evolving with their needs:

  1. Testing every type of page in the store.
  2. Adding engagement goals and metrics
  3. Testing global elements and entire stores (Theme level testing)
  4. URL split testing (in beta)
  5. Improved variant previews
  6. Advanced audience segmentation
  7. Various UX improvements.

Link to product demo


Thank You! ✨️
Mark