You will not have to start over for MonoGame. Virtually all of your code should be able to remain completely intact without change. The only real trick will be sorting through the content pipeline, which is an area that MonoGame has always been behind the curve on. I try to stay up-to-date on MonoGame, but I'll admit I don't know it as well as I wish I did. The last I heard, you should be able to use XNA's content pipeline to build XNB files, which you then dump into your MonoGame project as raw assets. (MonoGame doesn't have a content pipeline, except perhaps now as a beta. At least, that's the latest I've heard.) So you use XNA's content pipeline and everything should work from there.
Do you have any custom shaders? They might require a bit of tweaking. I haven't gone through that process myself, so I'm not sure. In most places, MonoGame uses OpenGL instead of DirectX as the underlying rendering framework. OpenGL doesn't understand HLSL. It has its own version. I know the MonoGame team was working on a cross-platform variation that could be converted to HLSL or GLSL at run time or compile time, but I don't know what the state of that is at this particular moment. Worst case scenario, you may need to rewrite your shaders. (I don't think it will be that bad, but that's the worst case scenario. Maybe somebody else can chime in with respect to this as well.)
At any rate, no, you aren't talking about starting over. You'll have to work through some content pipeline issues, and perhaps some shader issues, but I've seen many people report that it took on the order of hours (not weeks or months) to get things converted over.
