With the "how far away is the port?" question, I thought you really might want to know "how far away is the AIRPORT?"
The "port" is right in the City...Getting to it from ANY hotel in the Financial District, Fisherman's Wharf, Union Square or Nob Hill area is relatively simple: Take a SHORT taxi ride...
The key for those flying in from out of town is how far is the AIRPORT from the port...and the answer is...about 12 to 15 miles depending on which route you take and how many wrong turns you take...With typical traffic, it will take you a good half an hour...
Do NOT stay down by the Airport...nothing really down there except convenience to the Airport...although there is an abundance of hotels down there at much lower prices than in the City...
If you can find a reasonable rate at one of the major Fisherman's Wharf area hotels, that's a great location...For value, I like the Holiday Inn/Financial District...it's one of the nicest Holiday Inns in that chain and usually you can find some great rates...Make sure you ask about deals and special rates...The Holiday Inn is very convenient to Chinatown and North Beach (the areas you want to go to to EAT) and to the Cable Cars, etc.
Nob Hill (Mark Hopkins, Fairmont) is very pricey...but definitely first class...
Union Square has a lot of smaller, boutique-y hotels...some are very nice, but it's hit-or-miss...
Good luck...