For a domain that has never been verified, the domain will be suspended if not verified within the specified timeframe (see above). This means that the website and email service associated with the domain will stop working. You will see a page with a notice about ICANN's RAA and instructions to verify the contact.
For a domain that has been verified previously, if you don't verify, the website/email will continue to work. There won't any negative impact. The pending change to registrant information, however, won't go through.
Once a domain is suspended, only a successful contact change can un-suspend the domain.