OK, bounding box/sphere work well but they don't give you any more info on what type of collision of where it happened. For my ball I came up with the idea to do some path and find every point on the outside of the ball, currently 80 points. Now, when the collision is detected I run through the points and check to see if any are inside(or on) the plane and if so then I now have an exact point of collision. It works very well for the ball hitting dead on the top but now I have issues when it hits the side. There's the case which it would hit both(corner) and sometimes just one or the other but I can't seem to get it right. Also, any ideas on how to do a response to this?

I have a 2d Vector for speed and position of both the ball and box.

A simple method would be determine the normal of the collision (if it's a box or plane, then it's the normal of the surface) and then do a normal.Reflect(incomingVelocity) to get the rebounded velocity vector.