Test Creation Flow

Background
The main user journey in Shoplift is creating A/B tests. At that point in time, our testing capabilities were limited, but we had a rough roadmap to expand on types of tests to support our users' needs. The existing flow was a wizard flow where the user needed to make one decision at a time.
Release date: September 2023
Users
CMOs, E-commerce managers, Store founders
My Role
Release date: September 2023
Users
CMOs, E-commerce managers, Store founders
My Role
- Planning the design process and writing the PRD - steps, requirements and timelines with the CPO
- User research - Talking to CS and conducting user interviews
- Hands-on design and prototyping
- Internal usability testing sessions
- Delivering design to the development team
- QA
- Post-launch analysis

Before & after - home page
⚡️ Challanges
- User needs and gaps of knowledge were not addressed.
- There was no room for expanding the feature
- Technical: The app’s code base was being redone (moving from React to Vue) and this was the last piece to update
- Visual: We were implementing a new design system and needed to solidify the design.
I researched the problem space
- Unmoderated observations (Hotjar recordings)
- Met with customer success - I gather initial sentiment about the flow
- User interviews - in-depth conversations about user goals and needs
- UX and copy review - breakdown of problem areas

✅ Feature Priorities
- Reduce cognitive overload when creating a test
- Reduce test creation time
- Help newbies get started by incorporating the test library into the flow
- Allow the expansion of the flow to more testing capabilities



💡 What I learned
- Our original scope was way too broad. We were too eager to serve nice-to-have ideas and lost time and resources to try to build them in.
- Talking to customer success early in the process changed our core hypothesis about the friction points and that there’s a difference in expectations between novice and experienced users.
- Post launch analysis revealed promising numbers in unexpected areas - we reduced the time to launch a test for first time users.
Later this metric was correlated to our activation research - users who launch tests and get positive results within their trial period are less likely to churn.

🥂 Since then
As we grew our user base and test creation kept evolving with their needs:- Testing every type of page in the store.
- Adding engagement goals and metrics
- Testing global elements and entire stores (Theme level testing)
- URL split testing (in beta)
- Improved variant previews
- Advanced audience segmentation
- Various UX improvements.
Link to product demo