unit testing question

[Eric]

Hi everyone,

I would think this has been discussed before, but I haven't seen it
anywhere.

On the one hand I've seen people advocate putting all tests in a separate
text package. This makes production deployment easy, but coerces you into
expose methods of some sort with public access, solely to validate the test.
And I guess the opposite would be true for white box testing; you can leave
your accessors less than public, but you'll wind up deploying those tests
into production.

Any thoughts on this? FAQs I should know about?

Thanks in advance,

- Eric

[Darrell Grainger]

On Mon, 19 Apr 2004, Eric wrote:
Hi everyone,

Hi Eric,
I would think this has been discussed before, but I haven't seen it anywhere.

I've been lurking for a few years and I haven't seen this discussed here.
On the one hand I've seen people advocate putting all tests in a separate text package. This makes production deployment easy, but coerces you into expose methods of some sort with public access, solely to validate the test. And I guess the opposite would be true for white box testing; you can leave your accessors less than public, but you'll wind up deploying those tests into production. Any thoughts on this? FAQs I should know about?

For the projects I have worked on packaging and memory footprint have been
issues. Therefore, including the test code in the shipped product was not
acceptable. If the customer is downloading the information or you require a
larger amount of memory to run the application then I'd advise you create a
test interface and put the test code in seperate packages that do not ship
with the product.

One thing that seems standard from all the texts I've read and people I've
talked to (plus it just makes sense) is that you cannot test the code then
take the test code or interfaces out before shipping it. Any alteration of
the code would require you to retest. Taking the test code or interface out
would require you to retest... but you removed all the ability to test it.

This is why many products have backdoors. Microsoft product installation
had a special key that lets you install it. It was there so the testers did
not have to generate keys to test installation. It remained because
removing it would require retesting. Many games have secret codes so the
tester can get to parts of the game that normally would take the best
players weeks to reach.

If you do make a test interface, don't make it too obvious. I have seen
situations where customers have been able to easily figure out the
interface and started using it for their own purpose. On the next release
you change the test interface and all the customers hacks fail. If the
customer became dependent on the interface they might complain. If the
interface is cryptic enough that they are never confident of how it works
they will not grow to trust and use it. Therefore they will not be
disappointed if you change it (they will probably assume they never
understood it in the first place).

--
Send e-mail to: darrell at cs dot toronto dot edu
Don't send e-mail to vice.president@whitehouse.gov