50% cpu for Intel i5-7400, CRF 30, Ultrafast, at 10FPS. What can I do about it?


#1

Hi

I have 9 cameras on an Intel i5-7400, at 10fps and my system is almost at 50% cpu. I am using an Intel 640 graphics card (integrated). Current recording settings: CRF 30, Ultrafast, at 10FPS. Decoding encoding is set to Hardware_DXVA2.

How can I reduce the cpu requirement and add more cameras? I can buy a GPU card. I would also prefer to add more compression, but right now that is very problematic.

I will probably need to add at least 5 more cameras, if not 8.
Please advise,


#2

Hi,
This depends very much on the resolution of the cameras. A higher resolution demands a lot from the CPU. Our experience is that using Intel’s integrated graphics with DXVA2 really do not reduce the cpu load that can be measured.
What is the resolution of the camera? Do you use mjpeg or h.264 from the camera? In general mjpeg use more bandwidth on the LAN, but less cpu power compared to h.264. Is the fps limited from the camera?
If you need a total of 16 cams I would recommend an i7 processor with the actual settings of NCS and of the cams. The i5-7400 have 4 cores, but no hyper-threading so it have only 4 threads. The encoding/decoding process works best with many threads.

Read also this post Compression Causes recording to fail for very high resolution cameras - tests and results

-Henrik


#3

Thank you Henrik,

My camera resolutions are 720p. So it should not be a big issue. I will check for the setting between h.264 and mjpeg. It seems odd to need an I7 processor for 16 cams, though my knowledge of the subject is limited. Not sure what the NCS is, and I cant seem to find a good reference that explains that.

If intel’s integrated graphics dose not reduce the cpu usage. Dose another card do? (GForce for example?).

Thanks for the response!, would love to hear back from you.


#4

Hi again,
Well 720p is 1280x720 pixels and it uses a lot of cpu load. The processor have 4 cores, but no hyper-threading which works in the wrong direction. So far I have not seen anyone that have reduced the cpu load dramatically using DXVA2. In one system I have an NVIDIA card, but nothing special. Best is to search the internet to read about people’s experience. So far video coding is not that big hit with DXVA2.
I don’t know what FPS you get from your cams, but you can test to reduce the FPS in the configuration of each camera. You can also test to reduce the Frame Interval in the setting of Motion detection for each camera.

In a way you already have the result for your system. Add 5 more cameras and see how it reacts and experiment with the different parameters. If you want more compression I promise you that the CPU load will increase very fast. Most likely the compressed videos will have problems as in the link I referred to above including a solution. Increase compression and see what happens with the cpu load.
The best or only way is to start experimenting and connect more cameras and see what is necessary for your application.

-Henrik


#5

Thanks a lot Henric,

I will try that as well. FPS is 10 on all cameras. This would mean that:

  1. I need a specialized computer (I7) for my security system.
  2. No point in adding a specialized card.
  3. For the compression I need to use a larger HDD, and remove that option. (Sadly). This is the cheapest way.

FYI: I tried the VLC decoding/encoding, but no major changes in the performance. after some further checks I have found that it uses the same hardware acceleration. I’ll try to see what people use for that, or if there is any specific VLC plugin for decoding encoding video.

Is there a way to use FFMpeg with NetcamStudio?

Best
Zav


#6

Netcam Studio use the ffmpeg codec. The ffmpeg codec have been around for many years and more or less all softwares working with video coding use that piece of software. It´s very efficient and together with NCS that we have really put a lot of effort into for optimizing and reducing the cpu load.
-Henrik


#7

Hi Henrik,

I now have 16 cameras (Foscam, 720p, 10FPS, 40 CRF) on my system. The network is holding easily (on Lan) but the cpu load is ~100%. Jumps 50-80 when at 13, which is very close, but still not there.

Thank you for your previous answers. I have some more though.

Is there any way to connect two servers running on two different computers. Make them respond to the same netcam studio viewer (8100)? (Or add another storage location to the viewer)? I can network the drives to allow them available to one of the servers…

If I cannot connect the servers, is there a way to get the Netcam studio UI separately? Run the web api and just point the service

I do understand the recommendation is to use an I-7 (4/8 or 6/12) in this case, but wouldnt multiple computers be better?
Does it matter what generation the CPU is (for the I-7)?

Best, and thanks again,
Zav


#8

Hi Zav,
Thanks a lot for your feedback. Very valuable and interesting!
With 16 720p cameras I understand that it is 100% cpu load. If I understand correct with only 13 cams the CPU load is 50-60%? Interesting that adding 3 cams send cpu load to 100%. If that is correct, for me that indicates a bottleneck in the system, decoding or encoding. If you have all 16 cams running in the GUI of NCS X with NO motion detection or recording is the cpu load still at 100%? Or is it when all cams are running with motion detection or recording enabled?

To answer your questions it is no and no. But, here are some suggestions what you can do:
I give you an example what I am doing.
I have a simple computer running NCS. To that computer I have 2 USB cameras connected. On that computer the cameras are recorded continuously since I am mainly interested in the audio. I call this NCS-slave.
I have a server running NCS with 18 cameras in motion detection with VGA to 920p. CPU load is 95%-100%. I call this system NCS-master.
When I connect to NCS-master with windows or web client I want to have all 18+2 cameras. Therefore, i have connected NCS-slave to NCS-master like this How to connect one NCS server with another NCS server

Positive is that now I have all cameras live in only one GUI. Negative is that searching the Library I only have the 18 cameras connected to NCS-master. For Library from the USB cameras I must connect to NCS-slave.

Depending on your application you can also do this. In NCS there is an HTML generator that generates html code that can be embedded in a web page. If you have access to a web server you can make a web page for all 16 cameras. This will of cause only be for live viewing.

I have been discussing to do something that I call NCS Central for larger surveillance application. Here we can connect many NCS-computers to a central that collects all information that can be managed in one software. If that is possible I am not sure, but it is probably a lot of development and NCS will be in another market segment with another price structure. So far we have focused on NCS to be more efficient and with a better GUI to that price.

For computers or mainly processors. the i5-7400 has 4 cores/4 threads which is the reason for the high cpu load. In a system I have a Xeon e2-1220v3 which corresponds to an i5-7400 and I experience the same behaviour.

One high-end processor compared to two slower processors. That is of cause possible, but then we are back to the discussion above on how to merge the two NCS systems.

Which processor generation? High cpu speed, many cores, many threads, …
Lately I have been looking into another solution. This application involves a lot of decoding and encoding of video signals compressed in h.264 or now also in h.265. This type of video processing use mainly the ffmpeg codec and utilize massive parallel processing and therefore works best with processors with many cores and many threads. What I can see from all information is that many cores/threads are more important than a high clock frequency. If there is a processor with 4 cores it is important to know that virtualization do not end up in 8 real cores. It is 4 cores and 4 virtual processors. However, for this application that seems not being a problem.

Today it is possible to buy used computers to very reasonable prices. I am looking at high-end server systems that have xeon processors with many cores and many threads or even multiple processors. These processors are not running on the highest cpu frequency, but they have many cores/threads and also ECC which I like for stability. I have not done this yet since the actual system is in another part of the world where I am for the moment, but within a couple of months I will be on the same place :slight_smile:

To come back to your last question about the significance of the generation i7 processor. From my discussion above it is cores and threads that are more important than cpu speed so an older generation would be no problems as I see it.

-Henrik


#9

Hi Henric,

There is no motion detection or any other processes running. I do not require these. I am currently using the inner Decoder since as you said there was no real change in performance going to hardware. I did make sure the the ffmpeg configuration is using the intel integrated graphics card -
as defined in your NetCam directory (HWA). In terms of the cpu usage, the limit for the I5 is ~13 cameras at 720p 10fps. This leads the computer to have peaks of ~80-90% usage with ~50-60 regular. I am using a 2TB HDD, so writing is a bit slow somethings. Writing is set to every 5 mins, due to local security regulation.

In my field there are regulation challenges that might not allow us to use such a master slave configuration - we are required to present the data at any time through one interface, or at least if we don’t provide it we might not pass inspections - a thing I will not risk.

Given your well constructed JSON API, I was wondering if I can get some of the javascript/html source for your web interface. When I inspected your website (:8100) it seems that the only intersection between the website code and the camera system session state is the security token. Everything else is javascript and html? Is there any server code running? I would understand if this is proprietary, though if you provide it I might be able to tweak it for my use, I could create a pseudo server system, that acts as a security token gateway, using NodeJS (ExpressJS), that will connect and present any X number of computers, as long as I have access to the internal network (that is, as long as each server has a local netowork IP).

I was considering these computers as my i7 NCS server:

  1. https://www.amazon.com/HP-Elite-8300-Professional-Refurbished/dp/B076D46673/ref=sr_1_1?s=instant-video&ie=UTF8&qid=1513161809&sr=8-1&keywords=refurbished+i7,
  2. https://www.amazon.com/HP-270-Quad-Core-Processor-Bluetooth/dp/B076DJVTFM/ref=sr_1_5?s=pc&ie=UTF8&qid=1513161911&sr=1-5&keywords=pc&refinements=p_n_feature_fourteen_browse-bin%3A2057441011

what do you think?
For now, as my current solution I would use the i7 computer, together with my current i5 as the gateway web site.


#10

To add to the above,

I should note that if there was any way to integrate web ui to show all the attached servers, then it would be easy to use either:

  1. https://www.amazon.com/Upgraded-Z83-W-PC-x5-Z8350-Graphics/dp/B01N3RO6X5/ref=sr_1_2?s=pc&ie=UTF8&qid=1513162037&sr=1-2&keywords=pc&refinements=p_n_feature_fourteen_browse-bin%3A2057441011
    or 2. https://www.amazon.com/HP-Quad-Core-Windows-10-Refurbished/dp/B01CV9G1BO/ref=sr_1_1?s=pc&ie=UTF8&qid=1513161911&sr=1-1&keywords=pc&refinements=p_n_feature_fourteen_browse-bin%3A2057441011.
    Together with a network/usb/internal drives to create a system for any X number of cameras, where each computer holds up to 10. I think this solution would enable NCS to go much further.

10 Camera System Lag
#11

Oh and, one more thing :slight_smile: Excuse my reposting.

I think you will find these useful:



#12

It´s getting even more interesting! I have to check with our programmer about this what we can do. Interesting solution for viewing. Any of the i7 computers looks good. The 7200 rpm HD is of cause interesting, but a 5400 rpm HD has never been limiting for me. Since you are continuously recording the faster one is probably the better choice or an SSD for a couple of more bucks :slight_smile:
-Henrik


#13

Thanks!

Keep me updated.

Best
Zav


#14

Hi,

Any update on my request?

My i7 is now a 77% when not writing to disk, with the above 16 cams. When someone is viewing though it can jump to 100%. Separating the view server to another computer would be very helpful.

Thanks
Zav


#15

Hi,
Sorry, but no update on this.
-Henrik