You could hava a bent lever or the slots on the rails are worn badly or the end of the shift lever is worn; or it could be a loose or worn fork. There is a lot to check. They do come real close to the deash when everything is tight. I put a toggle switch in for my ignition for that reason. I was pulling a load of corn up a hill and had to shift down to 5th gear;  and as I did I hit the ignition switch and shut it off. Luckly my reflexes were better then and I got it turned back on befor the moter stopped.
Gary