Right now there is a big disparity between the two. The iMacs use the latest Intel architecture (Sandy Bridge) where the
Mac Pros not only use the older Nehalem/Westmere architecture but also 18 month old specifications as Apple don't update during a system's life-cycle, unlike Intel who do offer better clock-rates over time.
In terms of value for performance the iMacs are much better. This is in part because of the newer components and part because it is priced for consumers and competes against competitor's consumer systems where the Mac Pro is priced for business/professional use. As far as raw performance, unless you are getting a 6 or 12 core Mac Pro the iMac has better CPU performance.
If you can wait then new Mac Pros should be available in November or January and will be better performers than the current iMacs. If you really need something now I'd look at an official Apple refurbished model or a used model (you should still be able to get Applecare on a used model as they only came out a year ago). If you are comfortable swapping processors then you can get a 6-core 3.33GHz CPU now for $600 if you want a lot of processing power.
Bottom line: if you aren't looking at spending $4,000+ on a Mac Pro go with an iMac.