Yup, that's how I have always started my games; small town, make up the rest as we go.
For my upcoming game I am waiting to create the pantheon until after my players come up with their characters. Except for the god of Death, I know what she is going to be like. :-)