Apple has posted a Q&A style page detailing info on the tracking issue:

3. Why is my iPhone logging my location?
The iPhone is not logging your location. Rather, it’s maintaining a database of Wi-Fi hotspots and cell towers around your current location, some of which may be located more than one hundred miles away from your iPhone, to help your iPhone rapidly and accurately calculate its location when requested. Calculating a phone’s location using just GPS satellite data can take up to several minutes. iPhone can reduce this time to just a few seconds by using Wi-Fi hotspot and cell tower data to quickly find GPS satellites, and even triangulate its location using just Wi-Fi hotspot and cell tower data when GPS is not available (such as indoors or in basements). These calculations are performed live on the iPhone using a crowd-sourced database of Wi-Fi hotspot and cell tower data that is generated by tens of millions of iPhones sending the geo-tagged locations of nearby Wi-Fi hotspots and cell towers in an anonymous and encrypted form to Apple.

That answers a question I had with my first generation iPod touch. I remember driving around and it would somehow still know roughly where I was when I pulled up the Map app, even though the first gen touch didn't have GPs/cellular/etc. to be able to pull that information down.

and

Software Update
Sometime in the next few weeks Apple will release a free iOS software update that:

reduces the size of the crowd-sourced Wi-Fi hotspot and cell tower database cached on the iPhone,
ceases backing up this cache, and
deletes this cache entirely when Location Services is turned off.

So conspiracy over. Go back to worrying whether Obama is a US citizen or not.