Exchange 2016 CU 7 Issues / bugs and frustrations 

If you are upgrading to CU7 you might run into issues listed below. Interruption on the mail flow. The issue seems to be caused by permission changes on the registry key called “IDStore”

This is another undocumented surprise with CU7.  Also keep an eye on your existing IIS settings, SSL offloading existing setting seems to be wiped out with the CU7 install.

OU pickler ( config file ) is not working after config file is getting wiped out also known CU upgraded gift , Microsoft seems to be neglecting to get this fixed.

IF more found I will update the article in the feature.

The transport process failed during message processing with the following call stack: System.UnauthorizedAccessException: Access to the registry key 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\WorkerTaskFramework\IdStore\ProbeDefinitionIDConflicts' is denied.    at Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)    at Microsoft.Win32.RegistryKey.CreateSubKeyInternal(String subkey, RegistryKeyPermissionCheck permissionCheck, Object registrySecurityObj, RegistryOptions registryOptions)    at Microsoft.Win32.RegistryKey.CreateSubKey(String subkey)    at Microsoft.Office.Datacenter.WorkerTaskFramework.WindowsRegistryKey.CreateSubKey(String path)    at Microsoft.Office.Datacenter.ActiveMonitoring.DefinitionIdGenerator`1.RecordConflictInRegistry(String keyName, String value)    at Microsoft.Office.Datacenter.ActiveMonitoring.DefinitionIdGenerator`1.GenerateDefinitionId(String definitionKey, Boolean isNotification)    at Microsoft.Office.Datacenter.ActiveMonitoring.NotificationItem.Publish(Boolean throwOnError)    at Microsoft.Exchange.Protocols.Smtp.Legacy.MailSmtpCommand.LogMailFromSmtpResponseEventIntoCrimsonChannelPeriodically(String eventIdentifier)    at Microsoft.Exchange.Protocols.Smtp.Legacy.MailSmtpCommand.OutboundProcessResponse()    at Microsoft.Exchange.Protocols.Smtp.Send.SmtpOutSession.HandlePostParseResponse(SmtpCommand command)    at Microsoft.Exchange.Protocols.Smtp.Send.SmtpOutSession.InvokeResponseHandler(SmtpCommand command)    at Microsoft.Exchange.Protocols.Smtp.Send.SmtpOutSession.StartProcessingResponse(Byte[] buffer, Int32 offset, Int32 size, Boolean overflow)    at Microsoft.Exchange.Protocols.Smtp.Send.SmtpOutSession.ReadLineComplete(IAsyncResult asyncResult)    at Microsoft.Exchange.Net.LazyAsyncResult.WorkerThreadComplete(Object state)    at Microsoft.Exchange.Net.LazyAsyncResult.Complete(IntPtr userToken)    at Microsoft.Exchange.Net.LazyAsyncResult.ProtectedInvokeCallback(Object value, IntPtr userToken)    at Microsoft.Exchange.Net.NetworkConnection.InvokeRecvCallback(LazyAsyncResultWithTimeout`1 asyncResult, NetworkResult result)    at Microsoft.Exchange.Net.NetworkConnection.ReadLineDataAvailable(IAsyncResult asyncResult)    at System.Net.LazyAsyncResult.Complete(IntPtr userToken)    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)    at System.Net.ContextAwareResult.Complete(IntPtr userToken)    at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)    at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)    at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

Issue:  Inbound e-mail stops with following error (user not getting inbound e-mail)

Symptoms:  All Exchange 2016 services seems to be running, transport service looks fine.

User experience:  No incoming e-mail, outgoing e-mail continues to work

Solution:

To remediate the issue listed above, Assign network service ( This computer)  Full Access rights as shown below.

• ·        HKEY_LOCAL_MACHINE
• ·        SOFTWARE
• ·        Microsoft
• ·        ExchangeServer
• ·        v15
• ·        WorkerTaskFramework
• ·        IdStore

'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ExchangeServer\v15\WorkerTaskFramework\IdStore

Navigate the register key listed above on your Exchange 2016 CU7 server. Make a right click, permissions and select ( Local computer) Network service is local computer account. Assign Full permissions and close the register.

Verify mail flow id restored.

Oz Casey, Dedeal

Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

Check to see if Active Directory Account is Enabled/Disabled

Simple Tip:
Using the simple PS code below we can check to see if AD account for given user is enabled or disabled. Exploring other  properties with following one liner PS  Get-ADUser $user  -Properties * | gm
will also provide more options. You can check any of the available properties with same way and accomplish the task.

# Check to see if AD User Exist #(1).Collect user Info $user               = "Casey.dedeal" $status           = Get-ADUser $user  -Properties * | select LockedOut $LockedOut  = $(try {$status} catch {$True}) #(2).Write Output write-host "()_Checking $user AD Account Status" -f yellow #(3).Check the account status If ($LockedOut -eq $True) { write-host "()_.$user is locked out" } Else { write-host "()_.$user is NOT locked out"}

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

Skype4B crashes , when attemping to reply to notifications.

Recently, we found out Skype for business start crashing with the symptom explained below. After working around the issue, we figured it out the issue is caused by recently installed security update

KB4011040” 

Issue :  When user clicks on the notification message at the bottom right screen , Skype4B closes entirely. When user opens Skype4B and initiates IM to another user it works.

Cause : Recently installed “Security update for Skype for business KB4011040”

Solution: Install Office update KB4011099 to fix the issue.

September 5, 2017, update for Office 2016 (KB4011099)

https://support.microsoft.com/en-us/help/4011040/descriptionofthesecurityupdateforskypeforbusiness2016september12-2017

https://www.microsoft.com/en-us/download/details.aspx?id=55944

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

How to use Power-Shell to Perform SMTP Relay Test

In most cases, when application owner claiming, e-mails aren’t relayed through the existing internal application relay front end, you will need to perform few basic troubleshooting steps. Bear in mind asking basic questions up front will make your job easier.

What is the IP address of your application?

What platform your application is running from? (In this scenario we will assume the application is installed

on Windows Serer.

What is the host name ( FQDN) or IP address your application is configured to relay through

Do you have any logs on your application showing SMTP handshake (you are asking TCP/IP handshake).

Most   cases you won’t get this detailed information

If telnet service is installed you can sent e-mail from command line and observe the response. If SMTP relay

Front end accepting e-mails, you will need to figure it out what is happening on the transport and delivery

layers.

Here is simple PS code can be used to simplify the test from Application Server. You will need to change few variables to make this cod work in your environment

$smtpServer = "relay.smtp25.org"

$smtpFrom   = "smtp_relay@.smtp25.org "

$smtpTo     = "casey.dedeal@.smtp25.org "

clear-host #()Variables $subject    = "Testing SMTP Relay" $body       = "smtp relay testing" $smtpServer = "relay.smtp25.org" $smtpFrom   = "smtp_relay@.smtp25.org " $smtpTo     = "casey.dedeal@.smtp25.org " #()Message $Computer = $env:computername $message  = " sent from " #()Subject body $messageSubject = $subject $messageBody = $body + $message + $Computer #()Sending $smtp = New-Object Net.Mail.SmtpClient($smtpServer) $smtp.Send($smtpFrom,$smtpTo,$messagesubject,$messagebody)

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

PowerShell Check if Server is alive

Use the script below to check if a server is alive # replace the server name to make it fit into your needs.

$server = “dc1.smtp25.org” If (Test-Connection $server -count 1 -quiet) {   Write-host  “The $server responded” -f yellow } else{ Write-host  “No connection to $server” -f red   Write-host “Script will stop” -f red    Break }

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

Using PowerShell Test-Connection

#(1)_.Check if  Machine is responding Clear-host Write-host $null #(a)_Capture Server name $server = Read-host “_Provide Server Name to Test” #(b)_. Test Connection If (Test-Connection $server -count 1 -quiet) {   Write-host "The $server responded" -f yellow } else{ Write-host  "No connection to $server" -f red   Write-host "Script will stop" -f red    Break; }

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

Connect Remote Exchange Server Function

You can use below function to make connection to remote PS in Exchange 2016.

You will need to change following string $server ="EXC01.smtp25.org" to make it work for you environment.

# Connect Remote PS Exchange 2016 Server, Specify Server to connect.

# Change the server FQDN with server name you desire. Function Connect_E16 {     param(         [Parameter( Mandatory=$false)]         [string]$server ="EXC01.smtp25.org" )         $session = New-PSSession -ConfigurationName Microsoft.Exchange `         -ConnectionUri http://$srv/PowerShell/ `         -Authentication Kerberos          Import-PSSession $session -AllowClobber } Clear-host write-host $null Write-host "_Connecting Exchange 2016 Servers" $Connect_E16

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

Exchange 2016 OU Picker is not Showing Organizational Units.

Issue: Exchange 2016 CU5 , OU Picker is not showing Users within EAC ( Exchange Admin Center)

Cause: Web.config file needs to be modified “GetListDefaultResultSize” needs to be added with > number of existing OU ( Organizational Units) within the environment.

Notes: Web.config file is located on default exchange install directory under ECp folder “Program Files\Microsoft\Exchange Server\V15\ClientAccess\ecp”

It needs to be added with notepad ( opened with administrator privileges)

Each CU install will wipe these settings, therefore web.config file needs to be saved each time prior installing CU on Exchange 2016 Servers.

I have added few simple scripts to help you to develop baselines to implement in your environment

Web.config file is not server dependent, single file can be modified/saved and moved around Exchange 2016 servers, if desired.

• Log onto Exchange 2016 Server, via RDP (Remote Desktop Protocol)
• Provide your credentials at the logon.
• Click windows tile and select PS (PowerShell)
• Select to open PowerShell with administrator privileges
• Type following press enter (note the installed directory)

$exinstall\ClientAccess\ecp

• Type notepad on the PowerShell (Open notepad with administrator privileges)
• Click file and select Open on the administrative notepad
• Drill down to ECP directory under Installed directory ;
• Program Files\Microsoft\Exchange Server\V15\ClientAccess\ecp
• Locate web.config ( make sure on the bottom change to all files to locate the web.config)
• Open web.config file

Press CTRL + F to bring up find menu and type “</appSettings>”

• Locate the section
• Add this line ( it does not exist in Exchange 2016 )

# below line must be edited above “</appSettings>” Section within the Web.Config File

<add key=”GetListDefaultResultSize” value="1500" />

• Comments Added ( same line with comments)

<!-- Each CU Install will Wipe This File. --> <!-- Save a backup copy of web.config file --> <!-- After CU install you can copy and paste backup web.config file --> <add key="GetListDefaultResultSize" value="1500" />

• Open another PS window and copy and paste one line to get the number of OU’s in your environment and note the settings.

# You can easily get your current OU count with simple one liner

# Getting OU count (Get-OrganizationalUnit).count

• Close web config
• From PowerShell Type

Restart-WebAppPool MSExchangeECPAppPool

• Perform same task on Each Exchange 2016 Server listed below

Verification

• Open web browser and type following URL press enter, replace the server name with proper server name that is applicable for your environment

https://EX101/ecp/default.aspx

• Provide your credentials
• Click Recipients
• Click New, User Mailbox
• Select new user
• Click Browse
• Make sure Under “Select an Organizational Unit” Section populates the OU Structure

• You will be able to see the OU’s
• PS scripts to

# Connect Remote Exchange 2016 Server, Specify Server to connect.

# Change the server FQDN with server name you have. Function Connect_E16 { param( [Parameter( Mandatory=$false)] [string]$srv ="EXC01.smtp25.org" ) $session = New-PSSession -ConfigurationName Microsoft.Exchange ` -ConnectionUri http://$srv/PowerShell/ ` -Authentication Kerberos Import-PSSession $session -AllowClobber } Clear-host write-host $null Write-host "_Connecting Exchange 2016 Servers" $Connect_E16

# Re-start MSExcangeECPAppPool Service on each Exchange Server

# (1)_.Exchange 2016 Array foreach ($Server in $Servers) { $Servers = (Get-MailboxServer | ? {$_.AdminDisplayVersion -Match "^Version 15" }).name Write-host "_Processing $Server" Invoke-Command -ComputerName $server{ Write-host "()_Re-Starting MSExchangeECPAppPool Service " Restart-WebAppPool MSExchangeECPAppPool Write-host "_()Done" -f yellow Write-host $Null }}

# Check Server Health

<# .NOTES #============================================= # Script : Health_Check_Required_Services.ps1 # Created : ISE 3.0 # Author(s) : casey.dedeal # Date : 08/09/2017 09:23:23 # Org : ETC Solutions # File Name : # Comments : # Assumptions : # #============================================== SYNOPSIS : DESCRIPTION : Acknowledgements : Limitations : Known issues : .EXAMPLE .\Health_Check_Required_Services.ps1 .ps1 Description ----------- Runs the script to perform health check on Exchange 2016 Servers #> clear-host write-host $null #(a)_.Re-start MSExchangeECPAppPool Service On all Exchange Servers $Servers = (Get-MailboxServer | ? {$_.AdminDisplayVersion -Match "^Version 15" }).name #(b)_.Foreloop $Foreloop = foreach ($Server in $Servers) { Write-host "_Checking $Server Health" -f yellow Test-ServiceHealth | select Role,RequiredServicesRunning,ServicesRunning,ServicesNotRunning } #Out-Grid Results $Foreloop | Out-GridView

Make backup for Web.Config File

<# .NOTES #============================================= # Script : Copy_Web_Config_V1.ps1 # Created : ISE 3.0 # Author(s) : casey.dedeal # Date : 08/09/2017 09:31:30 # Org : ETC Solutions # File Name : # Comments : # Assumptions : # #============================================== SYNOPSIS : DESCRIPTION : Acknowledgements : Limitations : Known issues : .EXAMPLE .\Get-Sample.ps1 Description ----------- Runs the script to make copy of Web Config file Specify Exchange 2016 Server Name After each RU install you need to put the file back #> #(a)_.Variables $user = $env:UserName $dom = $env:UserDomain $comp = $env:ComputerName $date = get-date -Format yyyy-mm-dd #(b)_.Source & Dest $Source = "\\EXC01\d$\Program Files\Microsoft\Exchange Server\V15\ClientAccess\ecp\web.config" $Dest = "C:\users\$user\Desktop\" $file = "Web_Config_backup" $Folder = $dest + $file + "_" + $date $collec = $Source write-host $null write-host "--------$dom----------" write-host "$user" write-host "$comp" Write-host "Copying Exchange 2016 Short-cuts" -f yellow write-host "-------------------------" #(c)_. Running foreach foreach ($item in $collec) { If (Test-Path $source){ Write-host -fore yellow "Located $file " New-Item -ItemType directory -Path $Folder Copy-Item -Recurse $item -destination $Folder -Force }Else{ write-host -fore red "Source File cannot be located" write-host -fore yellow " Script Will Stop in 5 seconds" Start-Sleep -Seconds 5 break }}

Oz Casey, Dedeal
Systems Engineer
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

The setup wizard has determined that Certificate Revocation List(CRL) checking is enabled on this computer.

On the Server ( windows 2012 R2)  you are having the issue, make a right click on Windows tile, and select run , type | control | and press enter. From control panel click view and change it to small icons, locate “Internet Options” double click on it to go to properties.Go to advance and drill down to “Security” section uncheck | Check for publisher’s certificate revocation”” click apply and close the window. Installing RU 16 on Exchange 2010 SP3.

Run the setup after the changes. When finished you can revert the changes back.

Oz Casey, Dedeal
MCITP (EMA), MCITP (SA)
Security+, Project +, Server +
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)

Setting Throttling Policies For Cisco Unity

Recently I ran into issues with Cisco/Unity  within the Exchange 2010 environment. I wrote simple script to create Exchange Server Throttling policy based on CISCO’s white paper with their recommended values on the policy. Script can be found on TechNet Scripting Library

https://gallery.technet.microsoft.com/Set-Exchange-Server-60ff7055
Few reminders with Cisco Unity: It is configured to work with EWS.Voice message put into user mailboxes via API , so you wont see Exchange server information within the message headers.  if you are seeing random users not getting their VM, check throttling policy to make sure the recommended values set accordingly.

EWSPercentTimeInCAS               : 300
EWSPercentTimeInMailboxRPC  : 200
EWSFindCountLimit                     : 10000
EWSPercentTimeinAD                  : 100



Oz Casey, Dedeal (MVP North America)
MCITP (EMA), MCITP (SA)
Security+, Project +, Server +
http://smtp25.blogspot.com/ (Blog)
http://telnet25.wordpress.com/ (Blog)
https://twitter.com/Message_Talk (Twitter)