Starter/ignition issue

[pontiacvince]

I am having an issue that has just started. I got in my '71 and started it and the starter stayed engaged. I turned it off and tried to restart, and there was nothing. Went out and tapped on the starter housing and it eventually started and I drove home. Tried to restart, and the starter engaged with the key in the "On" position (not the start position). Did that repeatedly. Just replaced the starter with a new one with new solenoid. No change. Still engages the starter with the key in the "On" position. Could this be the ignition switch or something else I'm missing???