My Oliver 1650 did the same thing.  The steering column just wobbled all over the place.

On mine, one of the screw pins that are on each side of the hand pump where it hinges just loosened up.  All it needed was some tightening.  There's a lock nut on the hinged bolt that has to be tightened when the screw is tighened.