You will need to get your Linksys manual out and check what the admin login to it is probably 192.168.1.1, log onto the router using with an ethernet cable plugged into one of its four ethernet ports directly from one of your PC's.
From Internet Explorer type
http://192.168.1.1 and the router administration page will pop up. If the password has not been changed the user name and password will both be admin, this should log you into the router where you can check to see if DHCP has been enabled. I believe on the Linksys router this is enabled by default.
Then check on both of your PC's through the Network Connection box in control panel that they are both set to be assigned an IP address automatically. To do this right click on your connection then from the new window that pops up click on Internet Protocol(TCP/IP) and then the properties button the radio button should be checked against Obtain an IP Address automatically.
If this is already done you should be able to connect off either PC onto the internet, setting up wirelessly is slightly different, post back if you need more help.