Hi Rune!
This is certainly the million dollar question nowadays . The conditions for these kind of systems have changed a lot just during the last year mainly due to the rapid development of high resolution megapixel cameras. For about a year or two ago it was the cameras that limited performance. CPU was not a problem. Today it is the other way around. MP cameras demand so much CPU power which then makes the CPU the limiting factor when using MP-cameras. A new interesting scenario. So giving you an exact answer is impossible. So you need to test and see how your over all system reacts.
As you say the system you use today works fine. Main reason for that I would say is the Xeon X5650 which is a 6 core 12 threads processor. Video processing love to use as many threads as possible. When you run this what is the CPU load?
Going from 720p to 1080p is a factor of 2. Calculations are made at least 2 or 3 times in NCS. Also going from 25fps to 30fps also demands more CPU. And finally how many FPS do you need for the recording? And this is for five cameras. It will certainly be much more calculations and the need for more CPU power, but how much more compared to what you have today? Here is another post that just now deals with this and I hope he will report som results 100% CPU with 6 x Samsung SND-6013 cams
You have a very good start with that Xeon processor and I would start with buying the cameras, connect them one-by-one and see what happens with the system. Depending on the hardware configuration adding a second Xeon processor might be a solution.
If you like Vivotek go with Vivotek. I am using Vivotek in some systems and I am happy, but the market is just flooded with new MP-cameras. A camera that support ONVIF is good since it is easy to connect to NCS. If not you need to find the correct URL, but I think this is familiar to you.
Sorry, no exact answer, but I hope that the discussion can guide you to make it work. You know where to find me for more questions and I would certainly be interested in any reports how it works for you.
Thanks and good luck,
Henrik