Exchange 2010 Performance Counters

When you have performance related issues and concerns that are the basic things to look for in exchange 2010?

Possible causes of poor performance ,

Performance issues may raise when mail server is sized wrong and unable to satisfy the current load, the basic counters, Memory, CPU, Disk averaged read write. The typical indication of poor performance , outlook popup indicates retrieval data from problem.ExcahngeServer.local etc..

What to look for?

Isolate if the issues is specific to one client or effecting more then one user.

Understand and investigate your CAS Servers , “CAS ARRAY”, remember when MAPI ( Outlook) client opens up MAPI request to connect to it’s own mailbox the, MAPI request will not directly go to Mail Box servers, it will go to your CAS server and CAS will talk to MBX server ( MAPI) behalf of requester.

Use your tool box and set of tools in there in E210. the most common one is Performance Troubleshooter.

I want to talk about MSExchangeIS\RPC Averaged Latency and the changes in the counter recommended thresholds.

in Exchange 2003 and 2007

MSExchangeIS\RPC Averaged Latency < 10ms

The value has changed as the business logic

MSExchangeIS\RPC Averaged Latency < 250ms

The new model in E210 Client MAPI request shown as below. The MAPI client ( Outlook) who want to connect to it’s mailbox database has to fallow below logic.

Outlook Client ( MAPI Users)---------(MAPI)------------> CAS ----------(MAPI)-----------MBX Server

MBX--------------(MAPI)------------->CAS----------(MAPI)---------->Outlook Client ( MAPI Users)

• Averaged Latency should be less than 250ms at all times……
• The generally-accepted warning threshold for MSExchangeIS\RPC Requests is less than 70 at all times…..

What would be considered as bottleneck ?

• Although store.exe has 500 threads in Exchange Server 2007 / 2010, consuming more than 70 at one time is evidence of a bottleneck.

 

Performance Monitoring Getting Started Guide

Respectfully,
Oz Casey, Dedeal ( MVP north America)
MCITP (EMA), MCITP (SA)
MCSE 2003, M+, S+, MCDST
Security+, Project +, Server +
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)

Outlook Connection & CAS ARRAY

If you are in the process of moving into Exchange 2010 deployment or designing it most likely you have heard CAS ARRAY. I am going to list basic information you may want to pay attention to make sure you are covered up.

1. MAPI Connections in Exchange 2010 is handled by CAS servers , this is what is called “Middle Tier”

2. Outlook clients wants to connect to its mailbox talks to CAS server, CAS server talks to Mailbox Server behalf of Outlook client.

3. Outlook clients talks to mailbox server directly only for public folder data…

4. RPC Client Access service (MsExchangeRpc) answers the RPC endpoint.

5. CAS ARRAY is single contact point for all client connections with in the AD Site.

6. Create CAS ARRAY even though you have single CAS Server.

 

 

The way outlook clients connects to their respected mailboxes have changed and come along way in Exchange 2010 and will continue to evolve on next versions.

In Exchange 2010 , when outlook opens up it checks its configured profile and tries to locate Home server property.(msExchHomeServerName) The home Server property returns the Exchange Server name and DNS lookup provides the IP address of the Home server to outlook client, then Outlook established TCP connection on port 135 to

RPC Endpoint mapper the home server.

CLIENT ACCESS CAS ARRAY

• One ARRAY PER AD SITE
• RPC CAS Array Name , resolvable internally ONLY !!!
• Array Does not provide any load balancing

If the CAS ARRAY NAME is resolvable from outside , delete the A record if not here is the behavior of outlook client from outside.

Outlook fires up

First thing it will do, to locate the Home server property specified within the outlook profile and the name resolves in external DNS and Outlook client is handed out External IP, Outlook tries to establish MAPI session on port 135 TCP, since this will fail, outlook will stop working and fall into retry stage, eventually client who is trying to connect will get exhausted and will try HTTPS connection which will let it go trough the firewall and work. This creates significant delay.

So the rule  is simple

Outlook.MyCompany.internal= 10.10.10.100 ( VIP IP ) Internal DNS ONLY !!!!!

Reference:

http://technet.microsoft.com/en-us/library/ee332317.aspx

Respectfully,
Oz Casey, Dedeal ( MVP north America)
MCITP (EMA), MCITP (SA)
MCSE 2003, M+, S+, MCDST
Security+, Project +, Server +
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)

Move Active Database… DAG Exchange 2010

When you do maintenance or need to bounce one of the  DAG members, you may want to move Active databases to another server. If you have been using EMC ( GUI) you have noticed it is not efficient and doing repetitive work. Sooner or later you need to switch doing Exchange related task from power shell otherwise , trust me you will run into many problems.

To see your databases

Get-MailboxDatabase

Move-ActiveMailboxDatabase DB1 -ActivateOnServer R1E1 -MountDialOverride:None -confirm:$false

• DB1= Replace this with your DB Name
• R1E1 is the server I will be activating DB1 on
• confirm:$false  is going to take care of annoying confirm option

Move-ActiveMailboxDatabase -Server R1E2 -confirm:$false

This will move all databases on R1E2 Server

Source

http://technet.microsoft.com/en-us/library/ee364750.aspx

Respectfully,
Oz Casey, Dedeal ( MVP north America)
MCITP (EMA), MCITP (SA)
MCSE 2003, M+, S+, MCDST
Security+, Project +, Server +
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)

How to set Client Access server array

If you find out you have not set your Client access array and would like to set it up here are quick steps to get the work done. If you are not certain what the CAS array is here is Exchange team Demystifying the CAS Array Object  series part1 and part2

Also very nice detailed explanation about the RPC Client access server is here

Exploring Exchange 2010 RPC Client Access service

Exchange 2010 Middle Tier

In order to facilitate this move to the middle tier, two new services were created that run on the Client Access Server role: the RPC Client Access service and the Address Book service.

The RPC Client Access service handles all data connections for mailboxes and the Address Book Service handles all data connections for access to Active Directory. One notable exception is public folders. For public folder connections, clients connect to the RPC Client Access service on the Mailbox role, and not the Information Store.

How Does it work ?

• Outlook client checks its configured settings and finds out  where to connect to get the user mail from ( This case it is  CAS ARRAY = outlook.smtp35.org=10.10.10.150
• The DNS Server the client is talking too,  has A record matches the RPC Client Access Server outlook.smtp25.org=10.10.10.150
• Each Database has attribute called RPCclientAccessServer which is equal to CAS ARRAY name. ( this is where the magic happens……)

so in this case as fallows.

Outlook clients start talking to CAS servers and CAS Servers will talk to MBX Servers behalf of outlook ( MAPI ) clients. Now if you pay attention you can see the beauty of this design and its benefits.

Example one: You do not have any HLB in your environment what are your supported options after configuring CAS ARRAY ?

Option 1, you have created CAR ARRAY called outlook.yourCompany.local and assigned this to unique IP Address some thing like this

Outlook.YourCompany.Local = 10.10.10.100

Now you have two CAS Servers

CAS1=10.10.10.100 CAS2=10.10.10.101

Now here is how I can configure CAS ARRAY and have all RPC Connections to go to CAS1 Server.

Outlook.YourCompany.Local = 10.10.10.100 CAS1=10.10.10.100

Option 2,

Outlook.YourCompany.Local = 10.10.10.100  ( A Record in DNS) Outlook.YourCompany.Local = 10.10.10.101  ( A Record in DNS)

As you can see half of the RPC connections go to CAS1 other Half will go to CAS2.

Of course there are problems with these configurations such as if CAS1 goes down, what happens to outlook client connected to CAS1 ? until these clients go back to their configured DNS and query Outlook.YourCompany.Local they wont get connected.

Option 3,  would be getting HLB ( Hardware load Balancer) and make more intelligent application base load balancing on the CAS Servers.

To create the Client Access server array……

New-ClientAccessArray -fqdn <FQDN_of_array> -site <SiteName>

To tag the existing databases

Set-MailboxDatabase <MailboxDatabaseName> –RpcClientAccessServer <Casarray>

To set for multiple databases:

Get-MailboxDatabase | Set-MailboxDatabase –RpcClientAccessServer CAS-Array.YourConpany.com

 

Respectfully,
Oz Casey, Dedeal ( MVP north America)
MCITP (EMA), MCITP (SA)
MCSE 2003, M+, S+, MCDST
Security+, Project +, Server +
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)