Category Archives: Exchange Server

Need Password prompt in Outlook after clearing everyone’s sessions

Exchange Server, , ,

Long story short one of our vendors got hacked and sent out a phishing scam email using a SharePoint link and session hijacking that was able to bypass MFA. One of my users, who was waiting on a quote from this vendor, clicked it and entered their credentials. A week later their email was used to send out the same phishing email.

We followed the recommended Microsoft steps to clean things up. I did a content search on his account to see what he sent out and then emailed everyone letting them know it was a phishing attempt and if they fell for it they should reset their own passwords and contact their IT people. I then sent out a internal email saying if anyone clicked on the link they also need to change their password and to remind everyone about phishing emails. Then to be extra careful we forced a logout of all 365 sessions using PowerShell:

Connect-AzureAD
Get-AzureADUser | Revoke-Azureaduserallrefreshtoken

Which should have been the equivalent of going into the admin center on each user and clicking “Sign out of all sessions”. We told everyone that they might have to re-login to things and to reboot. Then followed up with some exports of everyones inbox rules, forwarding rules, etc, to make sure no one else was affected. It was a super fun Friday afternoon into evening.

The following Monday 95% of our users are completely fine, no issues, everything works. 5% of our users are getting the “Need Password” prompt at the bottom right in Outlook right after it finishes syncing all folders. If they try to get to public folders, other mailboxes they have access to, etc Outlook hard locks for a couple minutes and then connects and looks fine. Opening up the “Outlook Connection Status” during this time and you can see under status it saying “Connecting….” for each one then disconnected, over and over until it figures it out and connects. But then will go back to Need Password after a couple minutes. But if you simply click on where it says “Need Password” as soon as it pops up it connects the user with no other issues and works fine. So they were operational but with a annoying work around.

So I search google and the majority of things tell people to set a couple of registry keys which Microsoft has a entire article about not doing. Instead I followed the steps in that article which didn’t help at all. Tried the Microsoft support assistant for this issue, also didn’t work. Then moved on to having users change passwords and reboot, disconnect their accounts under “Access work or school” and reconnect, delete the Outlook profile and recreate, etc. All the standard Outlook is broken things to try but nothing seemed to work.

Well it turns out Microsoft put out a advisory that users with a Online Archive in a Exchange Hybrid setup were having trouble if the Autodiscover URL was pointing to the on prem server. So it ended up being a really bad coincidence. But this got me thinking why do I need to point Autodiscover to the on prem server at all if 100% of our mailboxes are migrated. It’s just a extra unneeded step which at best adds a little time to autodiscover compared to going directly to Microsoft and at worst if our internet or Exchange is down could prevent users from connecting.

Time to change that. I deleted my internal DNS autodiscover HOST record and put in a autodiscover CNAME record pointing to Microsofts autodiscover.outlook.com. Did the same for my GoDaddy DNS record. Once that was done I removed the SCP value from the internal server:

Get-ClientAccessService | Set-ClientAccessService -AutoDiscoverServiceInternalUri $Null

Flushed DNS on my own computer, rebooted, and verified Outlook connected without issues. I then updated the SRV records to also point directly up just in case in DNS under Domain -> _tcp -> _autodiscover and pointed that to autodiscover.outlook.com.

It’s now been like this for a couple months with no issues and since we have 0 mailboxes hosted locally there is no reason not to go directly to Microsoft for autodiscover.

A special Rpc error occurs on server xxxxx These certificates are tagged with following Send Connectors

Exchange Server, Security,

So our SAN SSL certificate was coming up for renewal and I really wanted to expand what was covered by it to include more devices. In general anything internal was using a self signed certificate and anything external used the SAN SSL. That worked fine but when it came time for renewal I figured why not just get a wildcard SSL and assign it to anything I could. Also would keep me from having to update subject names when there was a change.

So I bought a Wildcard SSL from GoDaddy and started assigning it to everything. Switched out the certificate on our firewall and VPN clients, mail server, web server, etc. Everything seemed to work fine. Then it came time to remove the old one once things were tested. Removing from IIS was fine, removing from the firewalls likewise fine, but removing from Exchange Control Panel gave the error in the title.

Now we do have a Exchange Online Hybrid deployment setup with centralized mail transport. We use a cloud based Barracuda spam/malware filter so all email in and out of the company goes through them then to our internal mail server. Any mailbox on Exchange Online then goes from our mail server to it and back. Well apparently when I set this up it made a send connector to route mail to Exchange Online and since it uses TLS attached the certificate that was currently being used which I was now trying to delete.

Unfortunately you can’t just go into the Send Connector in the ECP and reassign the certificate but you can do it by following some steps based on the Microsoft Set-SendConnector page. First get the list of your send connectors and the list of your certificates:

Get-SendConnector
Get-ExchangeCertificate

Copy the send connector that was in the error message and also the thumbprint for your new certificate. Next we will use that certificate to pull out the information needed to assign to the send connector and assign it:

$cert = Get-ExchangeCertificate -Thumbprint (your thumbprint)
$tlscertificatename = "<i>$($cert.Issuer)<s>$($cert.Subject)"
Set-SendConnector "Outbound to Office 365" -TlsCertificateName $tlscertificatename

Once your send connectors are updated you should be able to remove the old certificate. Also if you are using TLS on your receive connectors you will want to do the exact same thing but using the Set-ReceiveConnector command.

Note: If your new certificate has the exact same subject name then it might not update. You’ll have to remove it first using:

Set-SendConnector -Identity <connector name> -TlsCertificateName $Null

See https://learn.microsoft.com/en-us/exchange/troubleshoot/mailflow/cannot-remove-installed-certificate for more information.

Exchange Console Commands

Exchange Server, , ,

I have been managing a Exchange server since Exchange 2003. Current on 2010 and planning on upgrading to 2013 in the next month or so. Over the years I’ve been collecting snippets of useful commands which have served me well so here is my go to list. In almost all examples anything being exported/imported is using a directory called c:\ExFiles and in my examples the server is EXServer. Replace (alias) in all examples with the actual mailbox alias:

Export out Mailboxes as PST

Export list of all mailboxes by alias
Get-Mailbox| Select Alias | Export-CSV C:\ExFiles\Alias.csv

Export mailboxes based on a csv list:
foreach ($i in (Import-Csv C:\ExFiles\Alias.csv)) { New-MailboxExportRequest -Mailbox $i.Alias -FilePath "\\EXServer\c$\ExFiles\$($i.Alias).pst" }

Export individual mailboxes:
New-MailboxExportRequest -Mailbox (alias) -FilePath "\\EXServer\c$\ExFiles\(alias).pst"

Export out only Calendar items from a mailbox then import them into another mailbox

Export calendar items from a mailbox:
New-MailboxExportRequest -Mailbox (alias) -FilePath \\EXServer\c$\ExFiles\TempPSTFile.pst -IncludeFolders "#Calendar#"

Import calendar items to a mailbox:
New-MailboxImportRequest -Mailbox (alias) -FilePath \\EXServer\c$\ExFiles\TempPSTFile.pst -IncludeFolders "#Calendar#"

Check Import/Export commands and clear then

Check import status:
Get-MailboxImportRequest

Check export status:
Get-MailboxExportRequest

Clear completed import requests:
Get-MailboxImportRequest -Status Completed | Remove-MailboxImportRequest

Clear completed export requests:
Get-MailboxExportRequest -Status Completed | Remove-MailboxExportRequest

Retention Policy Commands

Get all user mailboxes without a retention policy
Get-Mailbox -ResultSize unlimited -RecipientTypeDetails "UserMailbox" | Where-Object {$_.RetentionPolicy -eq $null}

Assign the default retention policy to all mailboxes without one
Get-Mailbox -ResultSize unlimited -RecipientTypeDetails "UserMailbox" | Where-Object {$_.RetentionPolicy -eq $null} | Set-Mailbox -RetentionPolicy "Default Archive and Retention Policy"

User Permissions

Export out a list of users who have access to other mailboxes other then their own
Get-Mailbox | Get-MailboxPermission | where {$_.user.tostring() -ne "NT AUTHORITY\SELF" -and $_.IsInherited -eq $false} | Select Identity,User,@{Name='Access Rights';Expression={[string]::join(', ', $_.AccessRights)}} | Export-Csv -NoTypeInformation c:\ExFiles\mailboxpermissions.csv

Remove a users permission from all mailboxes
Get-Mailbox | Remove-MailboxPermission -User (UserID) -AccessRights FullAccess -InheritanceType All

Remove a users permission to a single mailbox
Remove-MailboxPermission -Identity (alias) -User (UserID) -AccessRights ReadPermissions -InheritanceType All

Mailbox and Database Information

Get the overall status of each database:
Get-MailboxDatabase -Status | ft name,databasesize,availablenewmailboxspace -auto

Export out the size of each mailbox in a database:
Get-MailboxStatistics -Database "Mailbox Database 1" | Select DisplayName, ItemCount, TotalItemSize | Sort-Object TotalItemSize -Descending | Export-CSV C:\ExFiles\MBSizes1.csv

Get size of a single mailbox
Get-MailboxStatistics -identity alias | Select DisplayName, ItemCount, TotalItemSize

Deleting Items

Delete all disabled mailboxes:
Get-MailboxStatistics -Database "Mailbox Database 1" | where {$_.DisconnectReason -eq "Disabled"} | foreach {Remove-StoreMailbox -Database $_.database -Identity $_.mailboxguid -MailboxState Disabled}

Delete a individual disabled mailbox:
Remove-StoreMailbox -Database "Mailbox Database 1" -Identity (alias) -MailboxState SoftDeleted

Delete recoverable deleted items based on a alias list
foreach ($i in (Import-Csv C:\ExFiles\alias.csv)) { Search-Mailbox -Identity $i.alias -SearchDumpsterOnly -DeleteContent }

Delete all recoverable deleted items
Search-Mailbox –identity (alias) –SearchDumpsterOnly –DeleteContent

Hopefully these help someone out as much as they have me.