When the test tooling finally runs the first time, it won’t find anything. And runs after that, that might find something. Those runs aren’t free either, as the system will keep changing, the tooling will need maintenance.
Done well, this will allow you to release software more frequently with less risk.
Done poorly, this costs you time, money, and energy.
Richard Bradshaw, a trainer/organizer the Ministry of Test and no slouch himself, recently presented at the Selenium Conference on how he fell into this trap as well.
Before Richard’s talk, the folks at SearchSoftwareQuality asked me to put together a small piece on it, which, with a little editing on their side, was published two days ago as How to perform test automation maintenance.