As an entrepreneur and game developer, I’ve begun to encounter what I like to call, “the prototype paradox”. Let me explain:
When creating a new product for market (in my case a 3 dimensional board game for Massif Games), most experts will tell you not to spend much time making your prototypes.
Why? Because chances are, you will barely make it through one testing period, before you find 20 things that need to be changed. You shouldn’t waste your time and resources making something that will just get remade anyhow.
This advice is sound. In making “The Mountaineer“, I’m on my 10th or 11th iteration, and still have things to adjust! And I’m sure I’ll keep finding things right up until the day I sign a manufacturing contract. And after all 1000 copies are printed.
As a perfectionist who must get everything right the first try, this experience is equivalent to being strapped to a chair in a Russian prison and being forced to listed to Rebecca Black sing “Friday” over and over again while a KGB agent electrocuted your testicles.
However, many prototypes rely on potential customers using your prototype and giving feedback (especially in the game industry). Enter Mr. Prototype Paradox:
A prototype is supposed to require little time and energy to make; yet, the less time and energy you spend making your prototypes, the worse your feedback will be from the average tester. And the whole point of a prototype is to test your product idea on its value proposition in the market!
A consumer’s feedback for prototypes that are not visually attractive and/or intuitive to use, has inherent bias and inaccuracy that makes it harder to find the real problems. When making “The Mountaineer”, I used basic images and graphics for my game board, and designed the playing cards to really only have basic text on them. In terms of mechanics and testing the game, this was all that was needed, and everything would be easy to modify in the future.
However, when I had people test this prototype, the lack of visually intuitive playing cards and graphics significantly increased playtime and resulted in negative player feedback on the appearance of the game, not the mechanics. This resulted in me getting feedback that was essentially useless, as I already planned to hire a professional artist, and was more concerned with how the game actually worked.
In my more recent prototypes, I have erred on the side of too much time and work, but the results have been interesting. I’m now getting feedback on the actual mechanics of the game, how long it takes, the emotions and competition it evokes, and what would make it “more fun”.
This makes sense. Imagine if you were testing an engine for a new sports car, but only had the chassis of a rusted truck for testing. Say you asked everyday consumers to test the vehicle and give you feedback on the engine. I’m guessing that the top comments you would hear would be the need to put it in a cooler looking car, and not necessarily whether the torque to rpm ratio felt right.
So. Does this mean that the advice of experts is wrong, and that we should spend more time and energy on making our prototypes look amazing? Not necessarily. We just need to add one more requirement to this concept:
Test your crappy prototypes on professional and expert users, not the average customer.
As I tested my game with multiple people, I found that board game hobbyists and strategic thinkers provided much more valuable feedback than those who were “casual players”. And using the sports car example above, I’m sure that Kimi Räikkönen (Professional Racer) would provide you much better feedback on your engine than your average Joe Smith. (Sorry Joe!)
Wait until things are a little more developed, before testing prototypes on your mainstream customer. (This is completely unrelated to finding the value proposition of your product! Always test your product’s value proposition on the main customer segment!)
And, as with many things in life, balance is key. Try to make a prototype that requires low energy and time, but gives the testers a bit of a “wow, this is cool” experience! Truth be told, in spite of these lessons I’ve learned, I still spend too much time on my prototypes! Damn perfectionists! 😉
What are your thoughts? What have you experienced when testing prototypes? I’d love to hear your comments and advice!
Take a quick watch of me and two friends testing a prototype for “The Mountaineer”.