Customer Beta Testing: AppSumo’s Secret to Successful Deal Launches
By Launching a Select deal on AppSumo has always been a labor of love, with all departments at the company teaming up to make sure we find and showcase the best up-and-coming companies on our site.
These exclusive deals on high-quality software are the entrepreneurial building blocks that AppSumo is known for. Our teams work closely with each Partner to make sure their products meet our standards and that the deal terms are exciting to our audience.
As the Senior Product Manager at AppSumo, I oversee the Customer (Demand) side of our two-sided marketplace.
In my day-to-day, I’m constantly collaborating with internal and external stakeholders and engineering teams and learning from our customers about how we can make our e-commerce experience more valuable. This collaboration and customer focus helps me shape our company’s strategic roadmap and ultimately, what we build for our customers.
With the continued growth of our company came the added growth of our customers’ expectations. We started wondering: “How can we continue to do this at scale, and how can we make sure every deal we launch is consistently successful?”
Enter: the AppSumo Beta-ling program.
How customer beta testing started at AppSumo
At the very beginning, our Beta-ling program was made up of a small but mighty team of three Sumo-lings (our term of endearment for AppSumo customers) from our Facebook group.
Our team members reached out to these people because they regularly posted in-depth, constructive product reviews. Moreover, they deeply understood our brand and what other customers would be looking for in the products we sell.
We noticed that these Beta-lings could clearly articulate what changes would need to be made with either the deal itself or the feature set. That way, we could transform a deal from lukewarm reception to grabbing that “monster” deal status when it finally goes live on our site.
After connecting with these folks, it was pretty easy to come up with a win-win-win situation:
- Beta-lings provide candid feedback on upcoming products in exchange for store credits, help influence the Partner’s roadmap, and contribute to AppSumo’s success.
- Partners enjoy free access to an in-depth product review board, improve their deal terms and product based on feedback, and get set up for success at launch.
- AppSumo gets a repeatable process for vetting upcoming deals and making sure we always exceed customer expectations.
How the AppSumo Beta-ling program works
As we continued to add more Sumo-lings to the Beta-ling program over the years, we developed a more formalized review process to make sure no stone was left unturned when we onboarded a Select deal. This is how the program works today:
1. Create test accounts
A few months before every Select product launches on our site, we ask the Beta-lings to create test accounts and kick the tires.
2. The first pass: “does this product belong on AppSumo?”
Beta-lings fill out a high-level review form after testing the product for about an hour.
The goal of this test is to answer the question, “Does this product belong on AppSumo?” Products must receive a passing grade from Beta-lings in this step to continue onto the next step of the review process.
A Beta-ling review of one of our past products:
3. In-depth review with taco ratings
This step is the meat and potatoes of the Beta-ling process. Here, Beta-lings fill out another more detailed, long-form review of the product.
The goal of this test is to answer the question, “What does it take to make this a 5-taco (i.e. 5-star) deal?”
Beta-lings provide a taco rating of the product as-is. This review document is very thorough and encompasses all aspects of the product, including features, integrations, UX, page speed, onboarding flow, competitive analysis, and landing pages.
Another screenshot from a Beta-ling review of one of our past products:
4. Share feedback with partners
Our team shares the Beta-ling feedback with the Partner. The expectation is that any bugs, features, or improvement requests must be fulfilled before the product goes live on the site.
5. Final re-test
Before going live, we conduct a final re-test of the product to make sure that all the bugs, feature requests, and improvements have been completed. Beta-lings then provide a final taco review of the product. This final review gives us a picture of the taco ratings we can expect from our customers when the product finally goes live.
The screenshot below shows a Beta-ling retest review from a past deal. This was before we enforced the rule that a product needs at least a 4-taco review to go live.
However, because we were aware of customer recommendations ahead of time, it helped us manage customer expectations immensely. Therefore, the product ultimately ended up getting a score of 4.36 tacos based on 61 customer reviews.
6. Deals go live!
Finally, if all the requests in step #6 have been met, the product goes live on the site!
Beta testing success examples
We’ve had countless successful product launches on the site since the Beta-ling program began a few years ago, most notably:
- BetterUptime: Average Beta-ling score of 4.5 tacos. Store score of 4.92 from 169 reviews. Juraj at BetterUptime noted that our Beta-ling program helped immensely with preparing the product for launch on AppSumo, especially when it came to bug squashing.
- SerpWatch: Average Beta-ling score of 4.25 tacos. Store score of 4.62 from 48 reviews.
- SwipePages: Average Beta-ling score of 4.16 tacos before any changes. After making improvements and going live: store score of 4.87 from 134 reviews (as of December 9, 2020).
I wanted to share our Beta-ling program with you today in order to showcase how we incorporate customer feedback into everything we do at AppSumo.
After joining the AppSumo Product team last year, I can say from experience that this is the first company I’ve worked for that truly takes customer interests to heart.
Moreover, I love how we’re a bit unconventional with how we gather customer input pre-launch and how hands-on we are when it comes to the success of our Partners.
If you’ve read this far, we want to work with you!
Our team is actively looking for more Beta-lings to join our program. If you’re interested in being a part of AppSumo’s growth and influencing startup product roadmaps, please reach out to our team members at kim@appsumo.com for more information about joining the Beta-ling program.
Shout-out to our amazing team members at AppSumo—especially Kim Tyrone Agapito and Olman Quesada—without whom we would not have been able to build such an incredible Beta-ling program.
I’d also like to acknowledge our fantastic Business Development team and Account Managers, who help make our Select Partnerships so successful. Finally, special thanks to the dedicated Beta-lings, our secret weapon to consistently launching hugely valuable products.