Beta Testing with Google Play

From the perspective of a quality minded application developer, Google’s addition of Alpha / Beta testing facilities to the Play Console is huge.  I recently had the opportunity to test out the new feature and while the experience was mostly pleasant there were a few warts.

The biggest negative I encountered was by far the lack of a means to unpublish a production release without also unpublishing the alpha and beta releases as well.  This is especially dangerous during the initial development of an app; a time where Alpha / Beta testing is most important.  Here’s the scenario:

You’ve recently completed your first working prototype of your app and want to start alpha testing it using Google Play.  You’ve got your application configured in Google Play with a status of “unpublished”, the prototype .APK is uploaded as an alpha and you are ready to go.  This is where you realize that your alpha testers will not actually see your app until you “publish” your app.  Why this is necessary I cannot say but it is.  Once the app has been published, both the Alpha and Beta .APKs should become available to those on your white list via the Play Store and so long as there is no Release .APK defined, access will be limited to those on the white list.

Here’s where it starts to get dangerous.  While not difficult to navigate I found the process of deploying a new .APK poorly documented and extremely error prone. It was only through trial and error that I ultimately discovered what appears to be the intended workflow; not only can an .APK be directly uploaded into the Alpha, Beta and Release tracks but promotion of .APK’s is also possible via multiple methods.  If you do happen to make the mistake of uploading and deploying an Alpha .APK as a Beta, so long as you’ve got a previous release it appears possible to revert.  If however you’ve NEVER released a public .APK and you accidentally release an Alpha or Beta .APK there is no way to unpublish only the Release APK.  At this point you have 3 options:

1. Unpublish the entire app and lose use of Alpha / Beta testing via the Play Console
2. Create a dummy .APK with the most restrictive compatibility requirements possible such as compatibility ONLY with devices running Android 1.6.
3. Move up your release schedule date to today and hope for the best.

Sadly, none of these are very good options.  For whatever it’s worth, it does appear to be possible to enter “Advanced Mode” on the Play Console and “Deactivate” the Production .APK:

Attempting to save such a change fails with the message “The application could not be saved.  Please check the form for errors.” but mysteriously no form errors are displayed. It’s possible that this functionality will be fixed sometime in the future.  But then again also possible that the visibility of the “Deactivate” button is the bug and that is what ends up being fixed.

At the end of it all, the moral of the story is be very careful when deploying new Alpha / Beta .APKs because there is currently no forgiveness from Google for those who make mistakes.