The easiest place to start is www.geocaching.com and read the stuff there. Put in your address and it will show you all the hidden treasures in your area. It's up to you to read the description and hints and figure it out.
We've been doing this since 2004 but really slowed down after the first year. Our GPS device was clunky and the kids got very involved in other things. Then I got a smart phone and we're back on the trail. In the last few weeks, I've used my bike to get to the locations and try to find the stuff!
It's a game of physical and intellectual curiosity, and you can do as much or as little as you want. There's no cost for the basic entry to the website, and gets you outside! You see places and things you might never have known were there -- like the park the other day where I found the bunny and the snake! And yesterday morning my friend and I went all over the small town next to ours, following a history clue/cache. We even had to do some math to get where we were going!
Yesterday afternoon after caching in the morning, I went to the Farmer's Market, so here are three photos to make your mouth water!