High CPU Usage
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
High CPU Usage
Hi there, CPU usage is too high for my servers
why is that happening?
how can I fix this?
50 users at the same time use more than 5Ghz CPU in 4 or 6 cores
is it a bug or something?
is there any workaround for this issue?
why is that happening?
how can I fix this?
50 users at the same time use more than 5Ghz CPU in 4 or 6 cores
is it a bug or something?
is there any workaround for this issue?
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
SecureNAT?
Precautions relating to Performance
By possessing an internal virtual TCP/IP stack, SecureNAT performs the highly advanced process of reassembling the TCP/IP stream packetized once by the TCP/IP stack and further TCP/IP packetizing via the operating system. The overhead resulting from these processes is large, such that throughput via the virtual NAT is considerably decreased when compared to physical maximum throughput, even when using a computer with sufficiently high speed. That is why virtual NAT should not be used for performance-centric applications. As previously stated, virtual NAT is a function which can be used as an alternative when the local bridge function cannot be used for security or technical reasons. Where high-speed methods such as local bridging are available, those methods should be used.
-
- Posts: 12
- Joined: Mon Jan 02, 2023 2:11 pm
- Contact:
Re: High CPU Usage
Unfortunately high CPU is an issue we also encountered doing tests, especially for some VPN protocols such as what you're using. Was CPU usage half when the number of active users was half from what it is now, or did the CPU usage increase unrelated to the increase in users activity?
Free VPN that works. For V2Ray, WireGuard and OpenVPN - https://vpn.fail/
Real-time free proxy list. SOCKS, HTTP & V2RAY - https://vpn.fail/free-proxy
Real-time free proxy list. SOCKS, HTTP & V2RAY - https://vpn.fail/free-proxy
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
Dear @solo, yes I'm using SecureNAt, can you explain me or do you have any documentation for running a local bridge using physical nat?
I can add ethernet to my VM, I tested but when I try to use the local bridge it's getting an error, Virtual Nat is working fine but still has too high usage for the CPU
so tell me how I can use a local bridge between two ethernets instead virtual one, I want to have maximum usage here
thanks a lot
I can add ethernet to my VM, I tested but when I try to use the local bridge it's getting an error, Virtual Nat is working fine but still has too high usage for the CPU
so tell me how I can use a local bridge between two ethernets instead virtual one, I want to have maximum usage here
thanks a lot
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
Hello JellyVPN, from our past conversations I assume that you still use Linux for SE server. If so, a conversion from SecureNAT to local bridge with dnsmasq' DHCP and iptables' NAT is rather simple - we have discussed it recently here and here.
Is the VM on a VPS or LAN PC? What error did you get after adding ethernet?
Please add an ethernet and post from the VM, as code, the output of:
Is the VM on a VPS or LAN PC? What error did you get after adding ethernet?
Please add an ethernet and post from the VM, as code, the output of:
Code: Select all
ifconfig
vpncmd localhost:port /server /password:*** /cmd BridgeDeviceList
//replace: *** with SE admin password
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
I fixed the issue for the local bridge, but still, CPU usage is too high
my servers are VM on ESXi 8
I have Centos 7, Centos 8, Windows Server 2019, 2022, and Ubuntu 22.10
Issues are
1. automatically disconnect users after 1-2 minutes.
2. High CPU usage even small users are connected
3. after some users are connected DHCP won't give IP to new users
thanks for your help Dear @solo
my servers are VM on ESXi 8
I have Centos 7, Centos 8, Windows Server 2019, 2022, and Ubuntu 22.10
Issues are
1. automatically disconnect users after 1-2 minutes.
2. High CPU usage even small users are connected
3. after some users are connected DHCP won't give IP to new users
thanks for your help Dear @solo
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
Would you be able to compare Windows vs Linux servers?
On a Windows Server edition you can replace SecureNAT with native DHCP server and RRAS' NAT.
On a Windows non-server edition you could try "Open DHCP Server" and something like https://www.nat32.com/ for NAT.
On a Windows Server edition you can replace SecureNAT with native DHCP server and RRAS' NAT.
On a Windows non-server edition you could try "Open DHCP Server" and something like https://www.nat32.com/ for NAT.
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
I checked on Windows and Linux with built-in Securenat, and both of them have High Cpu usage and High Memory usage
100% CPU and 100% RAM
I searched a lot and find it
https://github.com/SoftEtherVPN/SoftEth ... ssues/1616
it seems this huge issue is still not solved after years
for 3rd party DHCP it seems not working as expected and it's a very good idea unless Softether becomes more flexible with 3rd party apps
The best solution is to fix Softether Securenat usage
Best Regards
100% CPU and 100% RAM
I searched a lot and find it
https://github.com/SoftEtherVPN/SoftEth ... ssues/1616
it seems this huge issue is still not solved after years
for 3rd party DHCP it seems not working as expected and it's a very good idea unless Softether becomes more flexible with 3rd party apps
The best solution is to fix Softether Securenat usage
Best Regards
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
Thank you for Windows vs Linux server tests!
While we're waiting for SecureNAT fix, can you re-configure the setup as follows?
- disable SecureNAT
- enable local bridge
- offload DHCP+NAT to another PC or a router
While we're waiting for SecureNAT fix, can you re-configure the setup as follows?
- disable SecureNAT
- enable local bridge
- offload DHCP+NAT to another PC or a router
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
Can you explain how is possible to offload secure nat?
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
Thank you Dear @solo
I didn't check at windows due high usage CPU for windows itself
I'm trying to use Linux but not a successful scenario
1. I did a local bridge with a Virtual Tap adaptor (Softether VPN Server)
2. I installed dnsmasq and iptables in Ubuntu 22.10 (config as well with ipv4 forward active and tested)
but not working, I'm sure something is missing here
can you tell me steps until I can figure it how can I solve it
P.S: I installed ocserv on the same server and working very well without any issues by dnsmasq
I didn't check at windows due high usage CPU for windows itself
I'm trying to use Linux but not a successful scenario
1. I did a local bridge with a Virtual Tap adaptor (Softether VPN Server)
2. I installed dnsmasq and iptables in Ubuntu 22.10 (config as well with ipv4 forward active and tested)
but not working, I'm sure something is missing here
can you tell me steps until I can figure it how can I solve it
P.S: I installed ocserv on the same server and working very well without any issues by dnsmasq
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
Hello JellyVPN, this Softether on VPS Using Local Bridge guide is exactly what you ask for.
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
Thanks the guide is very great
But I face a problem and couldn't resolve the issue, even with a lot of searching on Google
==========
Softether start-up script belongs to Centos, I have a script for Ubuntu 22.10 for Softether Startup but I don't know how can I use virtual adaptor for the bridge to this script
==========
I added /etc/init.d/vpnserver based on the guide and only changed the IP Address based on my needs, but still can't use it
also in this folder, there is not file available
LOCK=/var/lock/subsys/vpnserver
==========
tap_soft will not give IPv4 to users, just IPv6
I did all the guide step by step, added to Firewall, dnsmasq, and more
But I face a problem and couldn't resolve the issue, even with a lot of searching on Google
==========
Softether start-up script belongs to Centos, I have a script for Ubuntu 22.10 for Softether Startup but I don't know how can I use virtual adaptor for the bridge to this script
Code: Select all
[Unit]
Description=SoftEther VPN server
After=network-online.target
After=dbus.service
[Service]
Type=forking
ExecStart=/opt/softether/vpnserver start
ExecReload=/bin/kill -HUP $MAINPID
[Install]
WantedBy=multi-user.target
I added /etc/init.d/vpnserver based on the guide and only changed the IP Address based on my needs, but still can't use it
also in this folder, there is not file available
LOCK=/var/lock/subsys/vpnserver
==========
tap_soft will not give IPv4 to users, just IPv6
I did all the guide step by step, added to Firewall, dnsmasq, and more
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
For Ubuntu SoftEther start-up use https://gist.github.com/amanjuman/6a40d ... 14e4a3d3b4
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
my problem isn't startup, Local Bridge not working!!!
https://blog.lincoln.hk/blog/2013/05/17 ... al-bridge/
I did all steps correctly, still when user connect doesn't get IPv4
https://blog.lincoln.hk/blog/2013/05/17 ... al-bridge/
I did all steps correctly, still when user connect doesn't get IPv4
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
But the soft-tap bridge is working?
This is the only bridge you need.
This is the only bridge you need.
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
No the problem is tap_soft installed, script for startup is active, but still users can't get IPv4
Something missed or has issue
Something missed or has issue
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 1433
- Joined: Sun Feb 14, 2021 10:31 am
Re: High CPU Usage
To clarify, we're not creating a "Local Bridge" in your VPS context. You use only a soft tap to SE bridge. Typical gotchas of this Linux setup are: missing IP forwarding and restrictive firewall. Review these topics on a very similar dnsmasq/iptables application:
https://www.vpnusers.com/viewtopic.php? ... 926#p97433
https://www.vpnusers.com/viewtopic.php?f=7&t=67987
https://www.vpnusers.com/viewtopic.php? ... 926#p97433
https://www.vpnusers.com/viewtopic.php?f=7&t=67987
-
- Posts: 289
- Joined: Wed Dec 28, 2022 9:10 pm
Re: High CPU Usage
Using Local Bridge and dnsmasq are not hard. You can follow below steps to check the issue stage
1. save your current iptables rule in order to restore it later
Code: Select all
iptables-save > your-file.v4
Code: Select all
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -t raw -F
iptables -t raw -X
iptables -t security -F
iptables -t security -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
if it passed next
4. Disable SecureNat and crate a soft bridge with SE server (check is has been created)
5. Manually assign IP to local bridge created in step 4
Code: Select all
ip addr add 10.11.12.1/24 brd + dev tap_tap
check the IP has been assigned to tap_tap (e.g ip -br a show tap_tap)
6. configure dnsmasq, then restart it and check the status
Code: Select all
interface=tap_tap
dhcp-range=10.11.12.10,10.11.12.250,12h
dhcp-option=3,10.11.12.1
dhcp-option=6,8.8.8.8
Code: Select all
port = 5353
NOTE
If the client/user Internet is too slow/weak they may face ERR_TIMEOUT because SE server DHCP is disabled and dnsmasq is near 3 or more times slower for IP assignment. I have tested with SSTP:
- SecureNAt IP assignments takes 1 or 2 seconds
- dnsmasq IP assignment takes 3 to 10 seconds or fails
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
Thank you Dear @solo
Thank you Dear @shakibamoshiri
problem is solved
But Dear shakibamoshiri I have a concern about CPU usage, with Softether's DHCP CPU usage is hiking
I didn't check with dnsmasq at least for many users, Softether's DHCP after almost 120-200 users won't give IP and server CPU and RAM usages hiking to 100% without dropping even 1 second
after a lot of checks, I got this issue DHCP using a lot of CPU
now I'm concerned about dnsmasq is better or Softether's own NAT
===========================
#Issue 2:
The new Issue I'm facing is with SSTP clients after connect automatically disconnects after a few seconds, I have no clue why it's happening
===================================
#Issue 3:
some users when trying to connect to servers after disconnecting get the below error while trying to reconnect (SSTP Android and iOS)
SSL Connect Error: BROKEN_PIPE
===================================
#Issue 3:
as I understand Dear shakibamoshiri you are providing VPN in Iran, while we have several countries and several servers some users can't connect to some of them
sometimes Irancell, sometimes MCI, and so more
for example, someone easily connects to the USA and the same person can't connect to France, on the other hand, another one can connect to France and can't to the USA
I'm confused, all of them use the same config but issue still exists
Thank you Dear @shakibamoshiri
problem is solved
But Dear shakibamoshiri I have a concern about CPU usage, with Softether's DHCP CPU usage is hiking
I didn't check with dnsmasq at least for many users, Softether's DHCP after almost 120-200 users won't give IP and server CPU and RAM usages hiking to 100% without dropping even 1 second
after a lot of checks, I got this issue DHCP using a lot of CPU
now I'm concerned about dnsmasq is better or Softether's own NAT
===========================
#Issue 2:
The new Issue I'm facing is with SSTP clients after connect automatically disconnects after a few seconds, I have no clue why it's happening
===================================
#Issue 3:
some users when trying to connect to servers after disconnecting get the below error while trying to reconnect (SSTP Android and iOS)
SSL Connect Error: BROKEN_PIPE
===================================
#Issue 3:
as I understand Dear shakibamoshiri you are providing VPN in Iran, while we have several countries and several servers some users can't connect to some of them
sometimes Irancell, sometimes MCI, and so more
for example, someone easily connects to the USA and the same person can't connect to France, on the other hand, another one can connect to France and can't to the USA
I'm confused, all of them use the same config but issue still exists
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 289
- Joined: Wed Dec 28, 2022 9:10 pm
Re: High CPU Usage
Personally I prefer using SE Secure NAT it is match faster in terms of DHCP IP allocation and assignment BUT we know NAT is a kind of high CPU consumption process and it is better to delegate this to Linux Kernel which is highly optimized. Since we give this process to OS, IP allocation and assignment will be slower BUT less pressure will be on SE server and respectively on CPU.JellyVPN wrote: ↑Wed Jan 25, 2023 4:03 amThank you Dear @solo
Thank you Dear @shakibamoshiri
problem is solved
But Dear shakibamoshiri I have a concern about CPU usage, with Softether's DHCP CPU usage is hiking
I didn't check with dnsmasq at least for many users, Softether's DHCP after almost 120-200 users won't give IP and server CPU and RAM usages hiking to 100% without dropping even 1 second
after a lot of checks, I got this issue DHCP using a lot of CPU
now I'm concerned about dnsmasq is better or Softether's own NAT
===========================
#Issue 2:
The new Issue I'm facing is with SSTP clients after connect automatically disconnects after a few seconds, I have no clue why it's happening
===================================
#Issue 3:
some users when trying to connect to servers after disconnecting get the below error while trying to reconnect (SSTP Android and iOS)
SSL Connect Error: BROKEN_PIPE
===================================
#Issue 3:
as I understand Dear shakibamoshiri you are providing VPN in Iran, while we have several countries and several servers some users can't connect to some of them
sometimes Irancell, sometimes MCI, and so more
for example, someone easily connects to the USA and the same person can't connect to France, on the other hand, another one can connect to France and can't to the USA
I'm confused, all of them use the same config but issue still exists
dnsmasq
I did not tested it with high number of users but according to others , it seems to be a better choice for large scale use cases
#Issue 2:
if they can connect successfully and disconnected after a while mostly could be their ISP issue or like Iran deliberately done by ISPs. In this regard we cannot expect a long stable connectivity.
#Issue 3:
I never git this (BROKEN_PIPE) with SSTP, which client you use?
This error is common with SSH ing to a server and again mostly cased by ISP
#Issue 4:
disclaimeras I understand Dear shakibamoshiri you are providing VPN in Iran,
I setup VPN for companies and mostly I used OpenCoonect but got interested in SE as well recently
WE DO NOT SELL VPNS
All of them of terrible. none of them are good but we have use them. they are pretty unstable and expensivesometimes Irancell, sometimes MCI, and so more
Yes this is truefor example, someone easily connects to the USA and the same person can't connect to France, on the other hand, another one can connect to France and can't to the USA
I explained it here
https://www.vpnusers.com/viewtopic.php? ... 011#p97757
Stable VPN connections needI'm confused, all of them use the same config but issue still exists
1. stable server
2. stable network
3. working protocols
Number 2 and 3 are hard to find in Iran :)
-
- Posts: 47
- Joined: Sun May 25, 2014 3:37 pm
- Contact:
Re: High CPU Usage
Thank you Dear Shakiba for your information
Issue #2 still exists:
customer use VPN Client Pro on Android and sometimes get error (SSL Connect Error: BROKEN_PIPE)
I don't know the reason and can't find a solution yet
any help or clue will be great
Issue #2 still exists:
customer use VPN Client Pro on Android and sometimes get error (SSL Connect Error: BROKEN_PIPE)
I don't know the reason and can't find a solution yet
any help or clue will be great
#1 Security, Speed, and customer service
JellyVPN - https://jellyvpn.com
JellyVPN - https://jellyvpn.com
-
- Posts: 289
- Joined: Wed Dec 28, 2022 9:10 pm
Re: High CPU Usage
To me the issue is the network.
Practically you have these solutions and two are based on double-vpn.
First (double-vpn)
If you can have server in Iran, buy and use it as hop-1 and CC it to your end-hop
pors
- less disconnection
- much more stable
- almost all protocols work
cons
- hiding your identity
- rarely ISPs in Iran give semiofficial bandwidth (1 to 1) and you have keep buying more traffic
Second (double-vpn)
If the "First" one was not possible for you, but a server in Turkey which has the closed route to Iran and make that Turkey's server as hop-1
pros
- less disconnection
- much more stable
- no need to hide
- you may can buy semiofficial bandwidth
cons
- hard to find working protocols
Third (Normal vpn)
just a server in Turkey. As I said Turkey has the closest route to Iran. ping could be near 70ms. which to Germany is near 120 to 150 , to USA more than 200 ms.
Lastly at the moment I am wring this reply, no ISP in Iran has stable network. Even domestic servers somethings cannot ping each other.