Well I'd use a 3" Forstner bit to cut the hole, not a router. Here's a link so you know what I'm talking about: http://www.amazon.com/Steelex-D1024-3-Forstner-Bit/dp/B0000DD0KZ
A forstner bit is more precise but usually not used in a handheld drill situation. A keyhole saw may be too imprecise for the rudder although a dedicated (not interchangeable) keyhole saw is one option. The depth of the cut would be the other problem here.
A VERY slow drill setting is required. A high torque, low speed drill suitable for use with a diamond cutter hole saw would be best. i.e. max 450rpm.
In the boat building industry you would normally use a guide that you bolt to the rudder or mast to drill such a hole. I don't have a ready reference for such a guide but you may have luck simply asking a local boat-builder (very very nicely) to borrow one! It is easier to drill three mounting bolts into the rudder to hold the guide and later backfill with a quality (read "marine") epoxy.
A boatbuilder I worked with had one such mount specially made. It is simply a flange turned from steel - the harder (the steel) the better but also more expensive. Three bolt holes in the collar allow for mounting. The centre hole is the exact right dimension for either a drill bit or forstner bit.
The problem on the rudder is the non-flat surface. A piece of packing timber, crafted to match the curved surface to a flat plate is the best idea.
As an alternative a special drill-bit like a fish-tail can be used. These bits can be used at high speed to cut an accurate hole at the size you want. It all depends how exact you need to be.
Whatever you choose, trial the hole and hub cuts in a similar grade timber first. Whichever way you go the standard solution is a custom solution!
I would suggest a plug cutter for the hub and FAMAG is the best, made in Germany. A matched pair for 3" i/d and o/d is the best option for an exact match. This is the normal procedure for such a boatbuilding operation.
Good Luck!