Windows Server 2012 L2TP Issue

Post your questions about SoftEther VPN software here. Please answer questions if you can afford.
Post Reply
nepal0621
Posts: 4
Joined: Sun Oct 05, 2014 5:08 pm

Windows Server 2012 L2TP Issue

Post by nepal0621 » Sun Oct 05, 2014 5:31 pm

Hi.
I'm trying to setup L2TP over IPsec connection. My configuration is:
Two subnets (one provider), say "Network 1" and "Network 2". Both subnets behind routers. Both routers have static public IPs. On both routers opened 500 & 4500 UPD ports.
Host A - PC with Windows Server 2008 R2 installed (Network 1)
Host B - PC with Windows Server 2012 R2 installed (Network 2)
On both SoftEther VPN Server Installed.
I CAN connect to Host B from Host A with native windows client.
I CAN connect to Host B from Windows 8.1 at network 1.
I CAN connect to Host B from Macbook with native client from Network 1 connected over Wi-Fi (!!)
I CAN connect to Host A from Windows Server 2003 installed as Hyper-V guest on Host B (!!!)
I can do all this things "out-of-box" without any additional settings. Only username, password, pre-shared key. All firewalls are enabled with default settings.
But I CAN NOT do one thing that I really need - connect from Host B to Host A >:( I got error 789. Has anyone faced a similar problem?

redbean
Posts: 8
Joined: Tue Sep 30, 2014 12:33 pm

Re: Windows Server 2012 L2TP Issue

Post by redbean » Sun Oct 05, 2014 5:57 pm

> Error 789: The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer.

It sounds like you are trying to connect the two host devices over L2TP. Is it very important you do it like this? An alternative could be to set up a cascading connection between the two host devices. Functionally, this could probably do what you want.

VPN error 789 is a security settings or third party software problem. Perhaps some security settings on Host A are misconfigured. I saw this suggestion on another forum:
> To resolve this problem, you are able to make use of a preshared key on both ends of the VPN connection. The role of implementations supports L2TP/IPSec gateway to gateway VPN with a preshared key used for Internet Key Exchange (IKE) authentication.

nepal0621
Posts: 4
Joined: Sun Oct 05, 2014 5:08 pm

Re: Windows Server 2012 L2TP Issue

Post by nepal0621 » Sun Oct 05, 2014 6:05 pm

redbean wrote:
> It sounds like you are trying to connect the two host devices over L2TP. Is it very
> important you do it like this? An alternative could be to set up a cascading
> connection between the two host devices. Functionally, this could probably do what
> you want.
Have not quite understand what you mean (my English is poor). I'm need to connect from Host B to Host A to have access to Network 1 (over local bridge). All other described things done just for test.

redbean
Posts: 8
Joined: Tue Sep 30, 2014 12:33 pm

Re: Windows Server 2012 L2TP Issue

Post by redbean » Sun Oct 05, 2014 8:29 pm

nepal0621 wrote:
> Have not quite understand what you mean (my English is poor). I'm need to connect
> from Host B to Host A to have access to Network 1 (over local bridge). All other
> described things done just for test.

It sounds like a cascading connection is what you need. I'll try to explain how to set it up. Since you are using windows, I assume you are also setting up your host servers with the SoftEther Server Manager. I also assume you have set up a virtual hub on each of the hosts.

Connect in your server manager to host B. Select your virtual hub > manage virtual hub > Manage cascade connections (bottom right in the newly opened window). Create a new connection to host A - fill in host name, the name of the host A hub, and enter a user name and password (which you created on host A beforehand).
This is the easiest way to access the network.

If you need it, here is the latest version for Server Manager: http://www.softether-download.com/files ... -intel.exe

nepal0621
Posts: 4
Joined: Sun Oct 05, 2014 5:08 pm

Re: Windows Server 2012 L2TP Issue

Post by nepal0621 » Sun Oct 05, 2014 8:48 pm

redbean wrote:
> It sounds like a cascading connection is what you need. I'll try to explain how to
> set it up. Since you are using windows, I assume you are also setting up your host
> servers with the SoftEther Server Manager. I also assume you have set up a virtual
> hub on each of the hosts.
Not exactly. SoftEther VPN Server on Host B installed just for test. In production mode I hope to have ability to connect to Host A (with SoftEther VPN Server installed and virtual hub configured) from Host B with native Windows Server 2012 vpn client!

Because I can connect to the Host A from the "Network 2" from the VM with the Windows Server 2003 (hosted on Host B), I am convinced that all the other components (network, routers, SoftEther VPN Server etc.) works right and configured properly, and the problem with some limitations of the Windows Server 2012.

BTW, after that changes http://support.microsoft.com/kb/240262 i'm getting 809 error :) But stuff like this http://justworks.ca/blog/what-happened- ... ws-72008r2 does not help :(

redbean
Posts: 8
Joined: Tue Sep 30, 2014 12:33 pm

Re: Windows Server 2012 L2TP Issue

Post by redbean » Sun Oct 05, 2014 9:25 pm

nepal0621 wrote:
> Because I can connect to the Host A from the "Network 2" from the VM with
> the Windows Server 2003 (hosted on Host B), I am convinced that all the other
> components (network, routers, SoftEther VPN Server etc.) works right and configured
> properly, and the problem with some limitations of the Windows Server 2012.

Might be true - I don't know how to fix the errors you got, unfortunately. But should you consider my suggestion, it will likely do what you want it to do.

nepal0621
Posts: 4
Joined: Sun Oct 05, 2014 5:08 pm

Re: Windows Server 2012 L2TP Issue

Post by nepal0621 » Thu Oct 09, 2014 1:32 pm

I think I found the cause of the problem but can't find solutions :(
If the information here https://kb.meraki.com/knowledge_base/tr ... n#error789 true, the IPsec connection can not be established because the client does not have "IKE and AuthIP IPsec Keying Modules"-service running.
So it is! But it does not start (error 13876: load failed). There is written that it may be due to the 3rd party VPN Software (SoftEther VPN i guess).
Is there any workaround to this issue?

Update: A service "IKE and AuthIP IPsec Keying Modules" on a Host B does not start until the option "L2TP over IPSec" in the SoftEther VPN Server (also installed on the Host B) is enabled. When I disable this option, L2TP/IPsec connection to Host A from Host B with native windows client establishing correctly. So it's not a problem of the OS, but simply a software conflict.
There are any suggestions on how it can be avoided?

thisjun
Posts: 2458
Joined: Mon Feb 24, 2014 11:03 am

Re: Windows Server 2012 L2TP Issue

Post by thisjun » Thu Oct 23, 2014 7:07 am

You can't use IPsec server and client at same time.
Because server and client use same port.

gctss
Posts: 3
Joined: Thu Mar 19, 2015 7:57 pm

Re: Windows Server 2012 L2TP Issue

Post by gctss » Tue Jul 14, 2015 2:28 pm

This is more of an FYI for anyone looking this up....

If you have Softether Server Manager installed, and have "enable L2TP Server Function" Checked

The services "IKE and Auth IP IPsec Keying Modules" on Server 2012 will not start

Post Reply