You don't say when you are going to be there but here is info direct from the Gardens in reply to a question I asked a while ago. You should do a ships tour to hit those early dates and times I think depending on when your ship actually gets there...... for the rest of the summer things are more flexable. Tours are available from downtown Victoria near the center of town and the Harbour. It's a fair piece from the dock as I recall....but its been a while since i was there. So unless your into a brisk walk the cab fare would likely defeat the price difference. Lots of tourist assist places throughout downtown/harbour area/
Garden Hours:::::
"Our summer season begins on June 15th with gates closing at 10:30pm. We are
however, staying open on a few dates prior to June 15th to accommodate some of
the early cruise ships. These dates are May 21, 28, 29, June 4, 5, 11, 12, when our
gates will close at 8pm, grounds at 10pm. "