So, about that Beta test

So we FINALLY managed to get the Beta builds out of the door on the 30th of November 2015, a full year after I had originally expected. That's scoping for you, ladies and gentlemen.

The purpose of a Beta test is to expose the game to a select, small audience consisting of volunteers; they will (hopefully) play the game and provide some feedback about the technical performance and gameplay among others.

So as we were approaching the point where the game was kind of feature complete and polished up, we started the process of recruiting volunteers on Facebook and forums like Touch Arcade. We got about 70-odd volunteers, and since I wanted specific and pointed feedback I created a google form that asked specific questions about many aspects of the game.

Many of these questions were based on what we knew were possible issues; for example, we were a little unsure about how effective the tutorials were. This caused us to ask a few detailed questions about how well the player understood the tutorial, and if they were too long. We were also unsure about the difficulty curve so we asked pointed questions about it.

The builds were distributed through TestFlight (for iOS) and Google Play (for Android). Strangely, many of the iOS testers seem not to have received the invite, and had to be added again using an alternate email ID. It's possible to see on iTuned exactly when each email ID was notified, and if/when the game was installed.

Feedback started trickling in within a day or so, and oddly enough it was almost all on Facebook messenger in the form of a conversation. Till date, only five (5) people have actually bothered to fill up the form. In hindsight, it would seem to be a chore to actually go back to your Gmail inbox, find that email and then the link to the form.

We have received approximately 15 or so responses in all forms (Facebook messages and the form) so far. Some of the responses were brief and not very useful, others were quite detailed and a few were very useful.

Primarily, we were worried about our retention as a function of the difficulty curve. I have a difficulty spike in Level 6 that I'm not too sure about, and a few of the users confirmed what I suspected; that spike was too sharp for a primarily casual audience. A few testers confirmed that they had stopped playing after that level.

Another feature that we were planning and were looking for validation from the Beta was the 'Save Me!' feature that allows players to save their level progress in a particular session. If they 'die' after collecting half the eggs and pickups. This was a bit tricky to design due to the non-linear nature of the levels, but we think we may have found a way to make it work.

​We also got a great preview from AsidCast here, so that really buoyed our spirits quite a bit. 

All this notwithstanding, the real test will be when we soft launch Bird of Light. That's planned for January 2016, so fingers crossed. Watch this space! 

---