Boring To Seat A Ball Bearing

I have a task where I have to cut a few holes or bore to seat a ball bearings. The part is aluminum bar, nearly 36 inches long and therefore this work can not be done on my 9x19 Grizzly G4000 lathe. The other machine I have my mini-mill. I have tried test cuts using my milling cutters. I ran into two problems.
1. I don't have milling cutter of the right diameter to suit the bearing diameter.
2. The surface finish I got when I plunged the end mill into aluminum was not smooth like I would...

Boring To Seat A Ball Bearing