Swiftsure Bank can be pretty amazing fishing on its day, definitely worth experiencing, and not really that far of an adventure trip for Victoria/Sooke folks.
I don't know you or your seamanship, but it's out there ... got to have your ducks in a row, all safety systems in place, redundancies, etc. You have plenty of boat for offshore, but have to watch the weather and if things aren't looking good, be willing and able to call the day quits ... sometimes you can't tell how snotty it is until you get out there, wind and swell are predictable, but the current can make it awful ... hard to know.
Lots of tips in the archives of this forum for fishing out there. Search for the coordinates of the Chicken Ranch ... the area around there and towards J Buoy would be good enough for halibut and salmon fishing on a given day. Drift for halibut, and deep trolling for salmon with spoons or plugs. (150-200') Best of all, IMHO, when the coho are thick, non stop surface action ... an annoyance when targeting springs, but I love it....turn off the motor, put on a jig, loosen the drag, and have some fun!
You need to program the coordinates of the closure into your chart plotter so that you can be sure you're not drifting into the closure. You'll see the pattern when you get out there, motor to edge of chicken ranch, drift through, pull gear and do it again....can pretty much follow the crowd.
And if all that's too much, The Beach can be super good on its day too. Worth experiencing even if just for something different. You tend to fish shallower than Sooke, 25-40 feet, so when a big one hits, it is right on the surface already....awesome fights.
Crabbing in San Juan Harbour can be very good but tends to get fished out in Aug. For other stuff, can check out sea lion rock at carmanah and nearly always see humpbacks somewhere offshore.
Good luck out there!