Keeping things simple. Pure VPN performance is determined by the encryption algorithm used. If we're talking about IPSEC, then it's a function of the IKE Phase 1 hand shake which can be either SHA or MD5 along with the selection of Diffie-Hellman Group (1, 2, 5, or 7.) After this, there is the Phase 2 part which is the actual encryption of the payload. Again the encryption algorithm here will determine the performance of the connection. For IPSEC, your choices range from DES, 3DES, or AES (128, 192, or 256 bit.) When running a remote client to VPN end point server, your performance will be a function of the aforementioned parameters (for IPSEC) and the hardware of your two end points. With modern day computers, the performance hit is reduced when running a VPN client. So the endpoint server will typically dictate your overall performance. For VPN hardware appliances like concentrators, routers, and firewalls, it all depends on the processor in the unit and the amount of memory available.
As an example, I used to run a VPN connection to a Netgear FVS318 v1 VPN endpoint router. The transfer rate I would get with this router was about .8 Mbps over my local LAN connection. Next I bumped up my hardware to a FVS338 which was able to provide me performance that was close to maxing out my wireless G setup. Now, my VPN connections to a Cisco ASA 5505 which more than exceeds the capacity of my wireless G setup to keep up. I haven't had a chance to do actual wire speed tests but will do it some day when I'm bored.
The above discussions are based on on a single client connection to the VPN end point server. But in many VPN deployments, you're talking about more than one connection. Obviously, when you start increasing the number of clients or site to site tunnels, you're going to reach a point where the VPN end point will start to degrade in performance. So this also has an impact on the throughput each connection experiences.
So with the above discussion, yes, VPNs inherently have overhead which will degrade performance versus a comparable non encrypted connection. But you can buy appropriately sized VPN appliances which can provide performance at the client's wire speeds. These VPN devices are not something home users would purchase nor many businesses as the cost is extremely prohibitive. We're talking about buying a very nice luxury car to being a very decently sized house type prices.
The discussion I have put out there also doesn't take into account any actual network infrastructure considerations like VPNs over WAN links and such. Also, in some situations, VPN performance can be impacted by MTU frame sizing.