Could go the other way then. Not have a swap at all. That forces pages to be freed, rather than paged out, but that means "bloat" from applications stays in RAM, if you ever got the situation where the system would rather cache say disk blocks, than keep "dirty" pages in memory. If you do reduce it, I'm sure it'll work, unless you do start using memory intensive stuff, perhaps a Virtual Box or VMware for example. Last time they tried to change the VM, so it *had* to be as big as RAM, it caused too many problems, as lots of ppl had expanded RAM without swap. The distro can't take the risk though and set it up "wrong".
If you're short of space, perhaps you could not have swap, format and use it for something, then use a swap file if you ever need 3GB of VM?
PS. I installed Win98SE today, and guess what it crashed! And needed a few reboots and though it's "working" it's still not right and gonna need a re-do.