This game is a lot of fun for young babies and toddlers. If you have a young baby put them in the center of the circle. Once they get a little older they can have fun putting on and taking off the hats as well.
Collect all the hats in your house (or at least a good number of them if you happen to collect hats)
Have everyone sit in a circle and put on a hat. Yell "Switch!", you can either pass the hats around the circle or all throw your hats in to the pile and grab another one.
That's it! I know it sounds simple, but young toddlers and babies love seeing their family members in different hats, especially silly ones!