Jul 02 2009

Testing testers

Bottle jacks header image

One of the challenges most independent developers will face is finding good testers for their applications. I’ve been relatively lucky with Hyperspaces — my users tend to take the time to tell me what’s going on when they submit a crash report, or start a new conversation on GetSatisfaction. What I have had troubles with is finding enough people willing to help test upcoming releases — right now, I have a great new release of Hyperspaces ready for a few keen souls to try out, but nobody has downloaded it! If you’re a user — please — give it a go, let me know what you find. If you’re a developer, read on…

So what do other independent developers do in this situation? My tests (unit and functional) tell me that the release is OK, but after some earlier mishaps I’m much more cautious about just throwing out a new update via Sparkle. I don’t have the money to spend on hordes of formal testers, and I’m pretty sure people do actually use and love my app — is it right to harness that goodwill for my testing purposes? Or should I just be saving up to pay some testers? What’s a reasonable amount? Questions, questions, questions…

Comments

Gravatar for Dean.

I’ve got around 100 users test­ing Hahlo4 at the moment, but I’ve prob­a­bly only had about a dozen pro­vide feed­back. Luck­ily that dozen have actu­ally pro­vided good, help­ful feed­back, they even sent screen­shots of the crazy errors they were see­ing allow­ing me to almost instantly know what was going wrong and where. I’ve tried using Get­Sat­is­fac­tion, but with­out mas­sive suc­cess, it mainly turned into a forum for ask­ing why hahlo isn’t native, or why they have to login constantly.

Posted by Dean on

Gravatar for Tony.

I’m find­ing that Get­Sat­is­fac­tion is turn­ing into really shitty forum soft­ware for my prod­ucts — it’s not help­ing stem the tide of sup­port requests, and I believe users end up being con­fused by the absolute mess of the main list­ings. I’m gen­uinely think­ing about drop­ping it in favour of more tra­di­tional sup­port channels.

Do you use the same testers each time? (like on call beta bashers?)

Posted by Tony on

Gravatar for Neil.

I would be happy to take your money to run tests.

Test 1: Appli­ca­tion has name …check!

Test 2: Appli­ca­tion has icon …check!

Test 3: Appli­ca­tion is appli­ca­tion …check!

BAM! I accept paypal ;)

Posted by Neil on

Gravatar for Tony Arnold.

Oh, Neil — how would I ever test my apps with­out you around? :P

Posted by Tony Arnold on

Gravatar for Dorian John­son.

You might try MacDe​vel​oper​.net. They cater to this exact situation.

(full dis­clo­sure: I was pre­vi­ously a back-​end devel­oper for MD, but I no longer have ties to it, so I’m not mak­ing any money on this suggestion.)

Posted by Dorian John­son on

Gravatar for Tony Arnold.

Thanks Dorian — I’ll give them a look. I think my prod­ucts could really ben­e­fit from some pro­fes­sional testing.

Posted by Tony Arnold on

Sorry, this conversation has finished.

This post is a bit old now, so I've closed the conversation. If you're keen to keep talking about it, please email me directly.