So this has been an adventure and one which doesn’t really shed NCS in a very positive light for me.
I’m going to start with the questions I need answered, which were not fully formed until the end of my discovery in writing this post, but I will post them once up top as a basis for where we are going:
1) How to accurately judge FPS in NCS?
2) Why have I been able to increase my FPS when it comes to recording, but my live view is so achingly slow?
3) Why is NCS apparently so inefficient compared to the alternatives in both properly utilizing the available CPUs resources and using so much of them?
Context: I’ve been using iNCS intermittently for a couple of years and most of the issues I always wrote off due to a an old junky FOSCAM model which was not getting very good FPS. I would say it averaged around 5-7, sometimes I’d see it as low as 3-4, and occasionally it got upwards of 8-10.
I had always discounted all my motion sensing problems (needing to set sensitivity too high to reliable get all person movement, but at the same time this triggered too many light related events) due to the fact this was just an older camera, with poor FPS, on a 2.4 WiFi connection that maybe wasn’t the most reliable.
So, I finally decided to improve my situation by purchasing a Reolink E1 Pro (ONVIF compatible). 4K camera @ 25 FPS with both wired and 2.4/5 Ghz WiFi.
Unfortunately, it seemed that my NCS problems weren’t any better and, in fact, seems the built in camera functions outperform NCS in almost every way (though there are still some reasons I would like to use NCS, and thus me bringing this issue up).
The issue was very low FPS compared to any of the alternatives
So it is a bit hard to measure true FPS in all cases. Not every client reports it directly and those that do, don’t always do it clearly or reliably (looking at you NCS). To understand my issue, it is best to first view this video.
On the left we have NCS showing a live stream of the Reolink. On the top right we have VLC showing a live stream. On the bottom right we have the Reolink client showing the same stream.
You can see by tracking the ms that on the two right images it is possible to make out pretty much one number in every ms, but the NCS video looks choppy, skipping numbers. This is a rough example of how NCS was simply suffering from a much lower frame rate.
As some more specific proof, here is a shot of the recorded framerate of each of these sources:
You can see that NCS is only recording about 9-10 FPS, whereas the other tools are recording in the full 25 fps at 4K.
To add insult to injury I want to note here that in this example the NCS server is actually hard wired to the camera (via switch) whereas the computer which was running the VLC and Reolink streaming/record was on WiFi. IOW, this clearly doesn’t seem to be a network bandwidth issue.
The questions are what exactly is the FPS that NCS is reporting and since it is clearly low in the recordings, why?
So, I’d like an explanation as to how NCS reports FPS(?), because it is a bit confusing. See this image:
In the top left NCS is reporting 24 FPS. Where is it getting this number from?
At 4K it is almost impossible to actually make out what I believe the actual FPS is (another example of how NCS hasn’t kept up with modern display technology). If we try to zoom into the lower-right corner it appears to be showing 6/25. I can only assume this means I’m getting about 6 FPS out of … a maximum of 25? Well where is it determining the maximum of 25 is and why is that different than the 24 reported above?
So what have we established?
a) NCS FPS reporting is confusing and inconsistent (and poorly documented)
b) Regardless of A, it is clear that the actual FPS being shown and recorded is much lower than the other programs.
I did multiple tests with NCS using different quality video streams (set via the ONVIF “Output Resolution” setting in NCS) and measuring FPS and CPU usage of the NCS server. This is what I found:
- At 2560x1440 I get recordings of 9.5 FPS. The NCS CPU gets as high as 50% (but like in all cases mentioned this is a high, and probably averages about 5% lower).
- At 1920x1280 I get recordings of 13.2 FPS. CPU max 40%
- At 1024x768 I get recordings of 22.2 FPS. CPU max at 25%
- At 640x480 I get recordings of 25 FPS. CPU max at 20%
I did these tests last night. Upon revisiting them this morning after a restart of NCS the CPU numbers are even lower. For instance, when viewing the 2560 feed my CPU never goes above about 15%. When recording the stream it maxes at 27% and yet still only gets < 10 FPS.
Also, to be clear the CPU numbers I have given above (in all cases) are the CPU usage of the NCS process. The overall CPU numbers are about 5% higher as nothing else is going on with the system during this time.
So it is clear that the higher the resolution, the more taxed the CPU is. However you don’t buy a CPU to have it sit 50% idle. Why at 1920x1080 (or higher) am I only using 30-50% of my CPU but still having such low frame rates?
Testing methodology:
- Tested both Wireless and Wired, didn’t seem to make a difference
- Tested in NCS using both rtsp and ONVIF feeds (which use rtsp, so shouldn’t matter). Results were the same.
I tried adjusting almost every setting available in NCS and in the end, it seems that the one thing that fixed my issue was the Multi-threading option in the “Encoding/Streaming” options. I’m not sure what the default is/was, but on my system it was set to “Single”. Changing this to Auto or 8 massively increased my FPS on recording!!!
But this is only part of the solution (problem?)!!!
EDIT: So this is the point I need to interject/change tact a bit. I have been discovering some things while I investigate this issue. While all the above information is accurate, I need to make some comments on CPU. These additional findings may make some of the numbers above a bit inaccurate, in total, but still seemingly accurate in comparison with each other (i.e. scale).
I was previously using Task Manager which I’ve recalled is a very poor way on modern systems to monitor actual CPU usage. I’ve switched to using Process Monitor. In PM, when I was set to only a single CPU I was getting CPU utilizations around 60%, when I bumped this up to more CPUs I was able to increase about 20 FPS while utilizing about 80%.
But still, something wasn’t right. I’m still only at 80% CPU. Why wasn’t NCS pushing even harder to maximize my CPU and get a few extra FPS? After all, CPU is meant to be used. It doesn’t help anyone to have 20-30% idle CPU during the most intensive tasks.
This is when I noticed something strange in Task Manager on the NCS server. It only showed my CPU running at 1.6 Ghz, well under half its rated speed. I went ahead and ran a few stress tests. Including Prime95 torture test which maxed out my CPU around 99% and showed speeds closer to 3.5 Ghz.
I also streamed a video on Plex which required the server to encode it and my processor shot up to 80% with similar speeds.
So why, when it needed CPU, did NCS not actually use not only more utilization, but didn’t ramp up speed as well? Well, I think the answer is because NCS is not coded properly. I don’t know enough about designing software in this regards, but I can tell you I don’t have the issue on other programs which require heavy CPU utilization.
I was able to solve this (at least in testing), but changing my Windows Power Plan from “Balanced” where the Minimum power state of the CPU rate can be as low as 5%, to “High Performance” where the rate is locked in at 100% all the time. You shouldn’t need to configure your system in this way for well designed applications. The idea is that the CPU only uses what it needs and doesn’t keep its clock rate up when there is nothing to do. As soon as there is something to do however it should increase to the maximum it is designed for. This is how every other program I tested behaves, but not NCS.
Here you can see exactly what I mean. First, here is P95 running a stress test, properly behaving when in Balanced mode:
Now here is NCS, configured to use all CPUs running in balanced mode:
Only 1.58 GHz and 80% CPU!
Now when we switch Power Plan to High Performance:
Processor doubles in speed and CPU utilization drops to 50%.
And guess what? Full 25 FPS recording!
As another contrast, and to be a bit gracious to NCS. The Reolink client running on the same system has a bit of the same issue. Namely it seems to not increase the CPU clock when encoding in Balanced mode and only takes full advantage if the power plan is set to High Performance.
However, even in this case when recording a 2560x1440 stream at 25FPS it only uses 30% of the CPU in balanced and only 15% in High Performance mode when it is actually utilizing the processor.
NCS still seems very inefficient in this regard.
Additionally, even after all of this, while I can seemingly get recording to give m my full 25 FPS at 4K, I cannot seemingly get anything better than about 7-9 FPS when viewing the live feed. I seem to have no issues with a live feed over VLC or Reolink’s software (or ONVIF DM).
So to reiterate what I need help with resolving:
- How to accurately judge FPS in NCS?
- Why have I been able to increase my FPS when it comes to recording, but my live view is so achingly slow?
- Why is NCS apparently so inefficient compared to the alternatives in both properly utilizing the available CPUs resources and using so much of them?
I want to close this with an aside on the state of NCS:
Specifically, I want to briefly state how disappointing the lack of development on NCS is. As has been said in multiple threads it seems that Henrik is the only employee doing anything by minding these forums. And while his level of knowledge of the product is high, he is mostly now relegated to apologist for the poor outdated features of NCS. For some reason (paid marketing) NCS still comes up as a recommended solution on various “best of” type blog posts for a PC based DVR solution, but outside of those rankings, no one talks about this software. Just Google “netcam studio site:reddit.com” and you will see there has been virtually no talk since 2018 of this product and even what’s out there pales in comparison to almost any other solution.
Sure, the 2 camera free limit, Window-based, and non-subscription model has some benefits, but only goes so far when the technology just doesn’t seem to work as well as what else is out there. I truly hope NCS gets some much needed updating for performance, camera compatibility (ease of configuration), and motion tracking algorithms.
I’ve been working on this issue for hours now. Hopefully my formatting above didn’t get to straggling and the overall narrative is still cohesive. Hopefully this information helps someone else and hopefully someone (i.e. Henrik) can provide at least some insight.
thanks.
