What’s a Beta?

For years people have been arguing about what the real purpose of a “beta” is. I used to argue this point as well, but today the term is so ambiguous that we all just need to accept that the term can be used for many different purposes, and what’s important to you is what you’re trying to accomplish.

Some teams use “beta” as essentially a QA phase. Basically you’re letting your customers help you test the product. Either because you don’t have much QA staffing yourself, or because for your product, there are just too many run-time environments out there for you to replicate and test, so you put the software out in “beta” and then tackle the bugs as they come in. This sounds worse than it is. In truth, there are many bugs that you’ll only find once you launch, especially related to scale and performance. As long as the release has achieved a reasonable level of quality before launching this is fine.

Some teams use “beta” as a gentle deployment mechanism. See https://svpg.com/gentle-deployment/ for the details, but the idea is that you deploy the new release as “beta” while you keep the existing release live, and over time you move users from the old to the new system, and when enough are there, you remove the “beta” tag and you then phase out the old. Think Yahoo Mail’s massive gentle deployment to their new Ajax-based mail client over the past year.

Other teams use “beta” as a way to get user feedback on a product idea. For most types of products there are much less expensive ways to get this feedback than to build and launch the service like this, but for some types of products, it really is the only way to see if the idea is a good one. Especially products introducing something new enough that there isn’t an established or proven market, and you don’t really know if enough people out there are interested enough to make this product idea worthwhile.

All of these uses of “beta” are useful and appropriate in the right situations. The key is that all three of the above cases are actively managed by the product team and site analytics and feedback from users is used to make the product better as fast as possible.

There is, however, one other use of the term “beta” that is not quite as virtuous, but is nevertheless fairly common. Some teams use the term “beta” to represent software that has largely been abandoned by its product team. The software has the “beta” label because nobody wants to stand behind it. It essentially means, “take it or leave it.” This may be because the idea didn’t get traction, or because there aren’t people available to work on it because it’s not a high enough priority, or because business priorities changed, or because the company has a form of corporate ADD and they simply lost interest part way through. The internet is so littered with these “betas” it reminds me of all the orbiting space junk. For much the same reason. It can be easier to launch a new web service than it is to kill it and tell users they have to get off.

I’m not thrilled about this use of “beta” because it can harm the three good uses of “beta” above, as it can make users hesitant to engage with something called “beta.”

If you have a product in this last form of beta, this is where it’s useful to be honest with your users and let them know that you are no longer promising to take this forward. Some teams label these products “sustaining only” meaning you’ll keep it running but that’s all. In any case, engaging with your users is good, and using their feedback to improve the product is even better.