Among the many new features delivered in Exchange 2013 SP1 is a new method of connectivity to Outlook we refer to as MAPI over HTTP (or MAPI/HTTP for short). We’ve seen a lot of interest about this new connection method and today we’ll give you a full explanation of what it is, what it provides, where it will take us in the future, and finally some tips of how and where to get started enabling this for your users.
What is MAPI over HTTP?
MAPI over HTTP is a new transport used to connect Outlook and Exchange. MAPI/HTTP was first delivered with Exchange 2013 SP1 and Outlook 2013 SP1 and begins gradually rolling out in Office 365 in May. It is the long term replacement for RPC over HTTP connectivity (commonly referred to as Outlook Anywhere). MAPI/HTTP removes the complexity of Outlook Anywhere’s dependency on the legacy RPC technology. Let’s compare the architectures.
MAPI/HTTP moves connectivity to a true HTTP request/response pattern and no longer requires two long-lived TCP connections to be open for each session between Outlook and Exchange. Gone are the twin RPC_DATA_IN and RPC_DATA_OUT connections required in the past for each RPC/HTTP session. This change will reduce the number of concurrent TCP connections established between the client and server. MAPI/HTTP will generate a maximum of 2 current connections generating one long lived connection and an additional on-demand short-lived connection.
Outlook Anywhere also essentially double wrapped all of the communications with Exchange adding to the complexity. MAPI/HTTP removes the RPC encapsulation within HTTP packets sent across the network making MAPI/HTTP a more well understood and predictable HTTP payload.
An additional network level change is that MAPI/HTTP decouples the client/server session from the underlying network connection. With Outlook Anywhere connectivity, if a network connection was lost between client and server, the session was invalidated and had to be reestablished all over again, which is a time-consuming and expensive operation. In MAPI/HTTP when a network connection is lost the session itself is not reset for 15 minutes and the client can simply reconnect and continue where it left off before the network level interruption took place. This is extremely helpful for users who might be connecting from low quality networks. Additionally in the past, an unexpected server-side network blip would result in all client sessions being invalidated and a surge of reconnections being made to a mailbox server. Depending on the number of Outlook clients reconnecting, the re-establishing of so many RPC/HTTP connections might strain the resources of the mailbox server, and possibly extend the outage in scope (to Outlook clients connected to multiple servers) and time, caused by a single server-side network blip.
Why MAPI over HTTP?
You are probably asking yourself why the Exchange team would create a complete replacement for something so well-known and used. Let us explain.
The original Outlook Anywhere architecture wasn’t designed for today’s reality of clients connecting from a wide variety of network types – many of these are not as fast or reliable as what was originally expected when Outlook Anywhere was designed. Consider connections from cellular networks, home networks, or in-flight wireless networks as a few examples. The team determined the best way to meet current connection needs and also put Exchange in the position to innovate more quickly was to start with a new simplified architecture.
The primary goal of MAPI/HTTP is provide a better user experience across all types of connections by providing faster connection times to Exchange – yes, getting email to users faster. Additionally MAPI/HTTP will improve the connection resiliency when the network drops packets in transit. Let’s quantify a few of these improvements your users can expect. These results represent what we have seen in our own internal Microsoft user testing.
When starting Outlook users often see the message “Connecting to Outlook” in the Outlook Status bar. MAPI/HTTP can reduce the amount of time a user waits for this connection. In the scenario when a user first launches Outlook the time to start synchronization improved to 30 seconds vs. 90 seconds for Outlook Anywhere for 70% of the monitored clients.
Improvements are also delivered when clients are resuming from hibernation or simply re-connecting to a new network. Testing showed that 80% of the clients using MAPI/HTTP started syncing in less than 30 seconds vs. over 40 seconds for Outlook Anywhere clients when resuming from hibernation. This improvement was made possible as MAPI/HTTP implements a pause/resume feature enabling clients to resume using an existing connection rather than negotiating a new connection each time. Current sessions for MAPI/HTTP are valid for 15 minutes, but as we fine tune and expand this duration, these improvements will be even more noticeable.
Improvements aren’t limited to end users. IT administrators will gain greater protocol visibility allowing them to identify and remediate situations faster and with more confidence. Due to MAPI/HTTP moving to a more traditional HTTP protocol payload, the ability to utilize already known tools common to HTTP debugging is a reality. IIS and HttpProxy logs will now contain information similar to other HTTP based protocols like Outlook Web App and be able to pass information via headers. At times in the past, certain debug procedures for RPC/HTTP were only available via proprietary internal Microsoft tools. This move should put all customers on the same level playing field as far as what tools are available for debug purposes.
Exchange administrators also will find response returned by Autodiscover for MAPI/HTTP to Outlook is greatly simplified. The settings returned are just protocol version and endpoint URLs for Outlook to connect to Exchange mailbox and directory from inside or outside the customer’s corporate network. Outlook treats the URLs returned as opaque and uses as-is, minimizing the risk of connectivity breaking with future endpoint changes. Since MAPI/HTTP, like any other web protocol, simply sends an anonymous HTTP request to Exchange and gets back the authentication settings, there is no need for Autodiscover to advertise the authentication settings. This makes it easier to roll out changes in authentication settings for Outlook.
MAPI/HTTP puts the Exchange team in position to innovate more quickly. It simplifies the architecture removing dependency on the RPC technologies which are no longer evolving as quickly as the customers demand. It provides the path for extensibility of the connection capabilities. A new capability that is on the roadmap for Outlook is to enable multi-factor authentication for users in Office 365. This capability is made possible with the use of MAPI/HTTP and is targeted to be delivered later this year. For a deeper look at this upcoming feature you can review the recent Multi-Factor Authentication for Office 365 blog post. This won’t stop with Office 365 MFA, but provides the extensibility foundation for 3rd party identity providers.
How does MAPI/HTTP work?
Let’s walk through the scenario of an Outlook 2013 SP1 client connecting to Exchange Server 2013 SP1 after MAPI/HTTP has been enabled.
- The Outlook client begins with an Autodiscover POST request. In this request Outlook includes a new attribute that advertises the client is MAPI/HTTP capable with the attribute X-MapiHTTPCapability = 1.
- The Exchange server sees the request is coming from a MAPI/HTTP capable client and responds with the MAPI/HTTP information including the settings on how to connect to the mailbox using MAPI/HTTP. This assumes the MAPI/HTTP has been configured and enabled on the server.
- The Outlook client detects the new connection path and prompts the user to restart Outlook to switch to use the new connection. While the restart is pending Outlook will continue using Outlook Anywhere. We recommend you deploy the latest Office client updates to provide the best user experience. The updates remove the prompt and clients are allowed to make the transition at the next unprompted restart of Outlook.
- After the restart, Outlook now uses MAPI/HTTP to communicate with Exchange.
So now we have a clear set of advantages you can offer users, let’s review the requirements to enable MAPI/HTTP.
Server Requirements: Use of MAPI/HTTP requires all Exchange 2013 Client Access Servers to be updated to Exchange Server 2013 SP1 (or later). The feature is disabled by default in SP1 so you can get the servers updated without anyone noticing any changes. If you are an Office 365 Exchange Online customer you won’t have anything to worry about on the service side of deployment.
Client Requirements: Outlook clients must be updated to use MAPI/HTTP. Office 2013 SP1 or Office 365 ProPlus February update (SP1 equivalent for ProPlus) are required for MAPI/HTTP. It is recommend you deploy the May Office 2013 public update or the April update for Office 365 ProPlus to eliminate the restart prompt when MAPI/HTTP is enabled for users.
Prior version clients will continue to work as-is using Outlook Anywhere. Outlook Anywhere is the supported connection method for those clients. We do plan to add MAPI/HTTP support to Outlook 2010 in a future update. We will announce timing when we are closer to its availability.
How to get ready
Part one of getting ready is to get the required updates to your servers and clients as described in the prior section. Part two of getting ready is evaluating potential impacts MAPI/HTTP might have on your on-premises servers. Again if you an Office 365 customer you can ignore this bit.
When you implement MAPI/HTTP in your organization, it will have an impact on your Exchange server resources. Before you go any further you need to review the impacts to your server resources. The Exchange 2013 Server Role Requirements Calculator has been updated to factor in use of MAPI/HTTP. You need to use the most recent version of the calculator (v6.3 or later) before you proceed. MAPI/HTTP increases the CPU load in the Exchange Client Access Servers. This is a 50% increase over Exchange 2013 RTM, however it is still lower than Exchange 2010 requirements. As you plan be mindful that deploying in a multi-role configuration will minimize the impact to your sizing. Again use the calculator to review potential impacts this may have in your environment. This higher CPU use is due to the higher request rate with several short-lived connections, with each request taking care of authentication and proxying.
To provide the best MAPI/HTTP performance you need to install .NET 4.5.1 on your Exchange 2013 servers. Installing .NET 4.5.1 will avoid long wait times for users thanks to a fix to ensure the notification channel remains asynchronous to avoid queued requests.
The change in communication between Exchange and Outlook has a small impact on the bytes sent over the wire. The header content in MAPI/HTTP is responsible for an increase in bytes transferred. In a typical message communications we have observed an average packet size increase of 1.2% versus Outlook Anywhere for a 50 KB average packet. In scenarios of data transfers over 10 MB the increase in bytes over the wire is 5-10%. These increases assumes an ideal network where connections are not dropped or resumed. If you consider real world conditions you may actually find MAPI/HTTP data on the wire may be lower than Outlook Anywhere. Outlook Anywhere lacks the ability to resume connections and the cost of re-syncing items can quickly outweigh the increase from the MAPI/HTTP header information.
Now deploy MAPI/HTTP
Now that you have prepared your servers with SP1, updated your clients, and reviewed potential sizing impacts you are ready to get on with implementing MAPI/HTTP. It is disabled by default in SP1 and you must take explicit actions to configure and enable it. These steps are well covered in the MAPI over HTTP TechNet article.
A few important things to remember in your deployment.
- Namespace: MAPI/HTTP is a new endpoint on CAS and can utilize both an internal namespace and an external namespace. For more information on how to properly plan your namespace design, see Namespace Planning in Exchange 2013 and Load Balancing in Exchange 2013.
- Certificates: The certificate used in Exchange will need to include both the internal and external MAPI/HTTP virtual directories to avoid any user certificate prompts, thus consider if the names exist on your certificates. Refer to the certificate planning post for additional help planning.
- MAPI/HTTP Configuration: Enabling MAPI/HTTP is an organizational configuration in Exchange, you won’t have the ability configure this for a subset of servers. If you require more specific control you can control the client behavior with a registry key.
NOTE: If you require more specific control you can control the client behavior with a registry key on each client machine. This is not recommended or required but included if your situation demands this level of control. This registry entry prevents Outlook from sending the MAPI/HTTP capable flag to Exchange in the Autodiscover request. When you change the registry key on a client it will not take effect until the next time the client performs an Autodiscover query against Exchange. To disallow MAPI/HTTP and force RPC/HTTP to be used. HKEY_CURRENT_USERSoftwareMicrosoftExchange]
“MapiHttpDisabled”=dword:00000001 To allow MAPI/HTTP simply delete the MapiHttpDisabled DWORD, or set it to a value of 0 as below. HKEY_CURRENT_USERSoftwareMicrosoftExchange]
- Connectivity: An important final consideration is to verify load balancers, reverse proxies, and firewalls are configured to allow access to the MAPI/HTTP virtual directories. At this time UAG SP3 is not compatible with MAPI/HTTP. The UAG team has committed to deliver support for MAPI/HTTP in a future update.
How do I know it is working?
There are a few quick ways to verify your configuration is working as expected.
1. Test with the Test-OutlookConnectivity cmdlet
Use this command to test MAPI/HTTP connectivity:
Test-OutlookConnectivity -RunFromServerId Contoso -ProbeIdentity OutlookMapiHttpSelfTestProbe
This test is detailed in the MAPI over HTTP TechNet Article.
2. Inspect MAPI/HTTP server logs
Administrators can review the following MAPI/HTTP log files to validate how the configuration is operating:
%ExchangeInstallPath%LoggingMAPI Client Access
%ExchangeInstallPath%LoggingMAPI Address Book Service
3. Check Outlook connection status on clients
You can also quickly verify that the client is connected using MAPI/HTTP. The Outlook Connection status dialog can be launch by CTRL-right clicking the Outlook icon in the notification area and selecting Connection Status. Here are the few key fields to quickly confirm the connection is using MAPI/HTTP.
HTTP (v/s RPC/HTTP for Outlook Anywhere)
Actual server name (v/s GUID for Outlook Anywhere connections)
MAPI/HTTP provides a simplified transport and resulting architecture for Outlook to connect with Exchange. It enables improved user experiences to allow them faster access to mail and improves the resilience of their Outlook connections. These investments are the foundation for future capabilities such as multi-factor authentication in Outlook. It also helps IT support and troubleshoot client connection issues using standard HTTP protocol tools.
As with all things new you must properly plan your implementation. Use the deployment guidance available on TechNet and the updated sizing recommendations in the calculator before you start your deployment. With proper use it will guide you to a smooth deployment of MAPI/HTTP.
Special thanks to Brian Day and Abdel Bahgat for extensive contributions to this blog post.
| Technical Product Manager
We collected a number of questions which frequently came up during the development, internal dogfooding, and customer TAP testing of MAPI/HTTP. We hope these answer most of the questions you may have about MAPI/HTTP.
Can MAPI/HTTP be enabled/disabled on a per-server basis?
No, it is an organization-wide Exchange setting. The user experience mentioned during database failovers when one server is not yet MAPI/HTTP capable made the functionality to turn MAPI/HTTP on and off per server not a viable solution.
Can MAPI/HTTP be enabled/disabled on a per-mailbox basis?
No, there is not currently a Set-CasMailbox parameter to enable/disable MAPI/HTTP for a single mailbox.
I updated the registry key to disable MAPI/HTTP on a client but the connection didn’t change?
The registry entry simply controls what Outlook sends to Exchange about its MAPI/HTTP capability during an Autodiscover request. It does not immediately change the connection method Outlook is using nor will it change it with a simple restart of Outlook. Remember, the Autodiscover response Outlook gets only has MAPI/HTTP or RPC/HTTP settings in it so it has no way to immediately switch types. You must allow Outlook to perform its next Autodiscover request and get a response from Exchange after setting this registry entry before the change will take place. If you must attempt to speed along this process, there are two options.
- Set the registry entry as you wish, close Outlook, delete the Outlook profile, and then restart Outlook and go through the profile wizard. This should result in an immediate switch, but any settings stored in the Outlook profile are lost.
- Set the registry entry as you wish, close Outlook, delete the hidden Autodiscover response XML files in %LOCALAPPDATA%MicrosoftOutlook, and restart Outlook.
- Restart Outlook once more to complete the switch.
What happens if a user’s mailbox database is mounted on a MAPI/HTTP capable server and then a database failover happens to a non-MAPI/HTTP capable server?
E.g. When not all mailbox servers in a DAG are MAPI/HTTP capable and MAPI/HTTP has already been enabled in the organization, and then a mailbox failover takes place between the SP1 (or later) and pre-SP1 servers. Additionally this could happen if you move a mailbox from a MAPI/HTTP capable mailbox server to a server that is not MAPI/HTTP capable.
In the above example Outlook would fail to connect and when Autodiscover next ran the user would get an Outlook restart notification warning because MAPI/HTTP is no longer a viable connection method due to the mailbox being mounted on a pre-SP1 server. After the client restart the client profile would be back to utilizing RPC/HTTP.
Note: While a mix of MAPI/HTTP capable and non-capable mailbox Exchange servers in the same DAG are supported in an environment with MAPI/HTTP enabled, it is very strongly not recommended due to the possible user experience outlined above. It is suggested the entire organization be upgraded to SP1 or later before enabling MAPI/HTTP in the organization.
What if a user accesses additional mailboxes where MAPI/HTTP is not yet available?
Outlook profiles can continue access additional resources using non-MAPI/HTTP connectivity methods even if the user’s primary mailbox utilizes MAPI/HTTP. For example a user can continue to access Legacy Public Folders or Shared Mailboxes on other Exchange servers not utilizing MAPI/HTTP. During the Autodiscover process Exchange will determine and hand back to Outlook the proper connectivity method necessary for each resource being accessed.
If MAPI/HTTP becomes unreachable will a client fallback to RPC/HTTP?
No, a user’s profile will never attempt to use RPC/HTTP if MAPI/HTTP becomes unavailable because the original Autodiscover response only contained one connection method to use. There is no fallback from MAPI/HTTP to RPC/HTTP or vice versa. Normal high availability design considerations should ensure the MAPI/HTTP endpoint remain accessible in the event of server or service failures.
Is MAPI/HTTP replacing Exchange Web Services (EWS)?
No, MAPI/HTTP is not a replacement for EWS and there are no plans to move current EWS clients to MAPI/HTTP.
Is RPC/HTTP being deprecated as an available connection method?
Over time this may take place as non-MAPI/HTTP capable Outlook versions age out of their product support lifecycle, but there are no immediate plans to remove RPC/HTTP as a valid connection method.
What authentication methods does MAPI/HTTP support?
A huge architectural improvement by moving to MAPI/HTTP is that MAPI/HTTP is abstracted from authentication. In short authentication is done at the HTTP layer, so whatever HTTP can do, MAPI/HTTP can use.
Does moving the MAPI/HTTP negatively affect the Lync client at all?
No, the Lync client uses the same profile as configured by Outlook and will connect via whatever connectivity method is in use by Outlook.
Is there any kind of SDK/API for third party application usage of MAPI/HTTP?
The MAPI/HTTP protocol is publically documented (PDF download) and has the same level of documentation support as RPC/HTTP. There are no plans to update the MAPI CDO library for MAPI/HTTP and third party companies are still encouraged to utilize Exchange Web Services as the long-term protocol for interaction with Exchange as discussed in Exchange 2013 Client Access Server Role article.
What are the client requirements for MAPI/HTTP?
Use of MAPI/HTTP requires Outlook 2013 clients to obtain Office 2013 Service Pack 1 or the February update for Office 365 ProPlus clients. MAPI/HTTP is planned to be ported to Outlook 2010 at a future date. At this time no other version of Windows Outlook supports MAPI/HTTP.
How does this affect publishing Exchange 2013 via ARR, WAP, TMG, UAG, B2M, ABC, BBD …
Publishing Exchange 2013 with MAPI/HTTP in use does not change very much. You will need to ensure devices in front of Exchange handling user access to CAS are allowing access to the Default Web Site’s /mapi/ virtual directory.
At this time UAG SP3 is not compatible with MAPI/HTTP even with all filtering options disabled. UAG plans to add support for MAPI/HTTP in a future update.
Still want more information? Review the following sessions on this topic from Microsoft Exchange Conference 2014
Outlook Connectivity: Current and FutureOutlook Connectivity: Current and FutureOutlook Connectivity: Current and Future
What’s New in Outlook 2013 and Beyond
What’s New in Authentication for Outlook 2013