Even using the Task Manager may not work completely. AVG loads drivers that remain loaded even when the program is shut down. The best way to stop it is to disable its services, temporarily turn off its startup program, and reboot.
Remove the check mark from the AVG startup program(s) with:
Startup Control Panel Startup Control Panel (Exe version for Vista)
and then use the services console.