I don't think you have to remove the bull gear to get the brake housings and bull pinion out of the 1650. There is an o-ring in there to seal it, that could be making it pull out  hard, and it could be rusty for some reason.

That thrust bearing is the most likely source of the needle bearings you found, but just to be sure, can you wiggle the brake shafts on each side? Rotation is ok, but if they can move in and out or wiggle up and down, forward and back, then that thrust bearing is probably the problem.