There is a way to turn it off:
I tried to provide a link to an article on PC Magazine's web site explaining how to turn it off, but apparently there are no links allowed here???
OK, just open System Preferences and select the General settings. Un-check the box next to "Restore windows when quitting and reopening apps". That's it. It might auto-resume the first time you quit and re-start an app, but the next time it will not.
I can now open Yummy without connecting to the last server. Safari now opens to my homepage, too.
Thanks again for pointing me in the right direction.