WFP redirection and Metro Style apps

Question

Hopefully this is the correct forum for my questions...  I am maintaining an application that does email encryption/decryption via a localhost proxy.  The current implementation uses an LSP to redirect email traffic to a user process on localhost which proxies the traffic.  This architecture has worked for many years on many versions of Windows.  It still works for Windows 8 desktop email applications.  However, it does not work for the Metro email app in the Windows 8 previews.  I assume it does not work because of the limitation imposed on Metro apps about connecting to localhost (although I guess it could also be an incompatibility with the LSP as I see that those are being deprecated in Windows 8).  

Questions:

  1) will LSPs not work at all in Windows 8 RC?

  2) if I write a WFP driver to replace the LSP I assume it will work to redirect desktop email app traffic, but will it redirect Metro app email traffic?

  3) does the prohibition on connecting to localhost apply to the WFP driver filtering Metro app traffic?  That is, if my (yet to be written) WFP driver tries to redirect email traffic to localhost, will it work for Metro apps?

  4) if the answer to #3 is "no", is there any way to locally proxy tcp/ip traffic from Metro apps?

Thanks!

Answer 1

1) LSPs will continue to load on Win8.  That they function the way you intend them in some scenarios may not be the case.  If you are using the Hardware Certification Kit to get your driver signed, then you will fail the tests for that.  LSP support will be limited by Microsoft.  The point is, to stop using them.

2) There is a bug right now with redirection and metro apps.  The fix should be in an drop relatively soon.  Once this is fixed, then the redirection will work for both modern and native apps.

3) yes this will work

A question from me now :)  why must you use redirection to a local service?  Couldn't you achieve the same using a STREAM callout (i.e. take the buffer you see @ STREAM, clone the data, block it, send the cloned data to a kernel workitem for processing, then inject it back when deemed OK.

Hope this helps,

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 2

Thanks so much for the reply and for the answers.

A follow-up question:

  5) just to be clear: are you saying that when the known bug is fixed, my existing LSP will be able to redirect Metro Style app traffic to localhost?  Or does this comment only apply to WFP drivers?

As to why we continue to think in terms of redirection, the answer is simple: inertia.  The current product has worked well for many years now and we have limited resources for re-designing the code.  However, we highly appreciate your opinion on the proper way to accomplish our goal.  We will certainly evaluate this approach when writing our new WFP driver, especially if that is the only way to do it  :-)

Thanks again!

Answer 3

I can only speak for WFP.  A side effect of the fix may be allowing the LSP to work, however that is not part of our design goals, as LSPs are on the deprecation path.

Hope this helps,

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 4

Hi there -- I'm the "program manager" (PM) for Winsock.  Your LSP solution isn't going to work for Metro style apps, the "bug fix" isn't going to make it any better (it's fixing a related issue but not this issue).  We don't allow LSPs in Metro style apps for two reasons:

1. It's a security hole.  Your LSPs would run in the context of the running Metro style app, but without going through the App store checks

2. Too often, it's a bad experience for our customers. LSPs cause enough problems that there's even a special Microsoft Fix-it tool that removes all LSPs from a system.  (What kind of problems?  For example, one LSP that's well-rememembered here was returning incorrect information on the number of bytes it was sending -- so programs that correctly looped until all their data was written out were going into infinite loops)

In addition, there are other paths through the stack that won't trigger your LSP at all.

The same statements apply to TDI filters, except that in addition to the above they also cause the network stack to use a slower code path, reducing the network speed for everyone. 

Lastly, the latest Hardware Certification Kit includes tests for both LSPs and TDI filters; having any will cause the certification to fail.  The tests will be applied to everything that gets certified -- including the OEM machine tests.  Which means that if you want to ship your software pre-installed on a machine, the OEM that you're shipping on will fail the certification tests. 

And now the good news: I recently talked to a room full of people using the Windows Filtering Platform, and not a single one had complaints about it -- no about the documentation, not about the performance, and not about the samples. 

---

Network Developer Experience Team (Microsoft)

Answer 5

Thanks for all the information.  Don't get me wrong, I completely understand about the reasons for deprecating LSPs.  I'm only trying to figure out what work we must do eventually and what work we must do for Windows 8.

Thanks again!

Answer 6

Hello,

I am happy that I have found this thread. It has made things more clear about LSP and Metro apps.
Basically, we just need to migrate from LSP to WFP.

But I still have a question about NSP (Winsock Namespace Service Providers). Will they work in Metro?
If no, what is the migration path for them?

(Please let me know if I need to open a new thread).

Thanks,
Sergey

Answer 7

Yes NSPs are still supported and should work with Metro apps.

Hope this helps,

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 8

According to our tests Namespace Providers (NSP) did not work properly for Metro apps in the recent RTM build (9200.16384 x64).
Strange, with the same tests everything was all right back in March (Consumer Preview build).

I have created test programs in VS 2012 and run them under debugger to check whether MyNSP.dll loads or not.
(You can simply make a copy of mswsock.dll and rename it to MyNSP.dll).
 
If I install NSP for name space NS_DNS (12) with WSCInstallNameSpace() it loads properly into a usual desktop application.
Here is what happens when I call gethostbyname("microsoft.com"):
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\NapiNSP.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\msvcrt.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\sechost.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\pnrpnsp.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\nlaapi.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\mswsock.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\dnsapi.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\winrnr.dll'.
'DesktopApp.exe' (Win32): Loaded 'C:\Windows\System32\MyNSP.dll'.

If I call something like this in Metro app:
StreamSocket ^clientSocket = ref new StreamSocket(); 
clientSocket->ConnectAsync(ref new HostName(L"microsoft.com"), L"http");

I see the following:
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\Windows.Networking.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\Windows.Networking.Connectivity.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\FirewallAPI.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\ws2_32.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\nsi.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\mswsock.dll'.
...
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\NapiNSP.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\pnrpnsp.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\nlaapi.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\winrnr.dll'.

MyNSP.dll is not loaded but other NSPs from Microsoft like NapiNSP.dll, winrnr.dll, etc have been successfully loaded.
If I change namespace for MyNSP from NS_DNS (12) to anything else e.g. NS_TCPIP_LOCAL (10), Metro apps start to load it properly:
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\MyNSP.dll'.

Finally if I use Metro version of Internet Explorer or a build-in web Metro API like MyWebView->Source = ref new Uri("http://microsoft.com");
I see this:
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\ws2_32.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\nsi.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\winhttp.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\IPHLPAPI.DLL'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\winnsi.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\mswsock.dll'.
'MetroApp.exe' (Win32): Loaded 'C:\Windows\System32\dnsapi.dll'.

Neither MyNSP.dll nor standard Microsoft NSPs are loaded.

Is it a bug? If it is the expected behavior, what is the migration path for NSP?

Thanks!

Answer 9

I have forwarded this on to the appropriate Microsoft contact.  You should be hearing back soon.

Thanks,

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 10

Post Consumer Preview, support for NSP's with modern apps was disabled.  This is by design.

Thanks,

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 11

Thanks for the fast response.

If NSPs do not work anymore what should we use instead? Maybe you can suggest something?
Basically we need to capture DNS requests and tunnel them through a secure tunnel to the remote server.
NSP worked just fine for this scenario.

Answer 12

I have asked the feature team owning NSPs for their suggestions.  If they don't have any, then I'll offer you the WFP solution.

Thanks,

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 13

So any news from the team owning NSP?
Unfortunately, WFP doesn't offer any options to work with the name system. Interception of raw DNS protocol is a big pain. So the lack of NSP support in Win8 apps looks like a real drawback.

Even if you do not support NSP in metro applications, I really hope that you fix a problem in the current implementation. If I disable system NS_DNS provider with WSCEnableNSProvider call. All metro applications stop working. This doesn't make sense because metro applications do not depend on NSP anyway.

This behavior causes a problem when a software installs its own NSP provider and disables the system one. Metro applications cannot load the new provider but they also do not use the system one.
Obviously metro applications should just ignore the settings done by WSCEnableNSProvider.

I really hope that you address this because the problem is quite severe and is very hard to diagnose.

Answer 14

The NSP owners had the following response:

   "If this is on x86 and they can change system level components, they could look at the NRPT"

NRPT is the Name Resolution Policy Table.

Here is a link to a starting point:
     http://technet.microsoft.com/en-us/library/ee649207(v=WS.10).aspx

This is now beyond the scope of WFP, so please start a new thread in the networking forums for any additional information on the proposed solution.
   http://social.msdn.microsoft.com/Forums/is/peertopeer/threads and possibly
   http://social.msdn.microsoft.com/Forums/en-US/winappswithnativecode/threads

Thanks

---

Dusty Harper [MSFT]
Microsoft Corporation
------------------------------------------------------------
This posting is provided "AS IS", with NO warranties and confers NO rights
------------------------------------------------------------

Answer 15

Are you kidding?

LSP is wonderful tech and you closed it? Dont be Apple, stay Microsoft.

You limit developers now!

WFP does not allow that can LSP!!!