AADSync - Increase error limit 5000 - stopped-error-limit

Sometimes large orgs have large errors that they need ignored.  By default, Azure AD Sync (the new DirSync) has a limit of 5000 errors before it will cease synchronizing.  Of course, you could also filter the OUs with known errors to not be synchronized, I can still see a use case where you would want known errors to continue reporting without preventing sychronization.

If you are hitting these error limits, and you are seeing the errors as permission denied, then you have not completed the custom permissions from the AADsync instructions here.  Those instructions are clear, but not given in an ideal manner, so check here and here for scripts for deploying the current permissions to your AAD service account.

Using the miisclient.exe in AADSync, I was seeing a stopped-error-limit at 5000 errors, and was only getting about two-thirds of the users synchronized.

After IM'ing with a colleague from Microsoft, I had the answer.  Here's a five year old KB article.

I've pasted and updated the content here:
Summary
Cause: Certain synchronization activities result in the creation of temporary error conditions that are eventually resolved once the synchronization has had the opportunity to finish running. In environments where extremely large numbers of objects are being processed the number of these errors may exceed the default error limit of 5000 causing the synchronization process to terminate before it has finished processing all objects.

More Information
You can increase the error limit to a sufficient value so that synchronization can finish processing all objects before the error limit is reached. Given the opportunity to finish, the synchronization process will eventually resolve the temporary error conditions. The error limit is configured by adding the ErrorLimit (REG_DWORD) entry to the following registry subkey:

If you are using MIIS / ILM / IIFP
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\miisserver\Parameters

If you are using AADSync
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\ADSync\Parameters

If you are using FIM 2010 or DirSync
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\FIMSynchronizationService\Parameters

The value is an integer in the range of 0-100000.
• Value set to 0 = Error limit set to 100000
• Value in the range of 1-99999 = Error limit set to value
• Value set to 100000 = Error limit set to 100000
• Value set greater than 100000 = Error limit set to 100000
• No key present = Default error limit set to 5000
Note The miiserver service must be restarted after modifying this registry key.

PowerShell and testing connectivity with Test-NetConnection

I have been working in IT long enough to see people stick to their old habits begrudgingly, and I had the realization today that one of the ones I was still holding on to was an inherent need for telnet.exe as a troubleshooting tool.

Let's face it, it is compact, a low security risk to have installed, but it having not been included by default on modern Operating Systems has forced me to memorize import-servermanager and add-windowsfeature telnet-client (and before that servermanagercmd -i telnet-client)

The fact that I can iterate those without looking them up is practically a sign that the tool might need a refresh.

Enter Test-NetConnection.   Now, I won't claim to have discovered or wrote it, but I use PowerShell daily and just learned of it today.

It's new since Windows 8.1 and Windows 2012 R2, which came with PowerShell 4.0.

Like I said - nothing new - it's already been called the SuperPing once.

So what can we do with it?

First off, run it alone is a basic Internet connectivity test using just test-netconnection.  This reaches out to a server at Microsoft to see if you are online.  Great to use programatically like this:

if (test-netconnection) { 
    # Download stuff
    } else { 
    # Error about your Internet
    }

You can also use it to check specific connectivity using something like:
Test-NetConnection -ComputerName edge.extrateam.com -Port 5061 -InformationLevel Detailed -Verbose

Some additional tips:

• -ComputerName can be shortened to -CN
• -commonport can be used for things like RDP, HTTP, SMB and PING
• -traceroute can also be used to display hops to a destination, and -hops can be used to specify the maximum hops to be displayed

 As you can see, this gives us a BUNCH of great information:

1. Multiple DNS records returned (only shown with the Detailed InformationLevel)
2. Which IP we connected to
3. And a successful TCP test!

Have to admit, I might be skipping nslookup and telnet in future deployments, and I hope you do too.   If only it could do UDP tests!

Install-PowerShellOptions.ps1 - Script to deploy and connect to on premises and cloud Microsoft services

I developed the Install-PowerShellOptions.ps1 script for one reason.  I am constantly looking for the right shell download, or connection string, and after years of having my handy EXOConnect.ps1, I wanted one that also pre-loaded the modules for me.

The menu is now interactive, and build the Install or Connect options based on what software is already installed on your PC.  When you choose an Install Step the script will download and launch the installers required for your OS.  Some require Elevated access (they should prompt you)  The connect steps allow you to connect the module/session for the requested PowerShell service.  Once the module loads (or fails to load, check for errors) you are returned the menu, at which time you can load an additional module, or hit 99 to exit and begin using the modules you have loaded. 

Please report any bugs or feature requests via comments on this page or on the Q&A on the Download link. If you are upgrading from older versions of versions, I recommend manually uninstalling the old bits, then running this script to download and deploy the latest code.

Download:
Install-PowerShellOptions.ps1 - Version 1.6

Alternate Download and older versions

Supported OS:

• Windows 8/8.1
• Windows 2012/2012 R2

Features:

• Installs software required to connect to Cloud PowerShell services
• Connects to On premises Exchange and Lync/S4B services with minimal prompting
• Connects to Office 365's Exchange Online, SharePoint Online, Azure AD and Lync/S4B Online PowerShell services with minimal prompting
• Minimal logging of options run

Version History:

• 1.0 - Original Version  :)
• 1.1 - Fixed Lync Online Connector import-module
• 1.2 - Added Azure IaaS PowerShell support, made menu more interactive, checking installs and recommending install or connect (coded in upgrade, but logic not in place yet), Ability to set cloud and on premises credentials per session 
• 1.3 - Updated SPO URLs,  Set ignoreCA/CRL on Lync/Exchange on premises to skip SSL warnings, added S4B language, fixed AAD prereq menu bug
• 1.4 - fixed the recall credentials for Office 365, updated SPO online version for download
• 1.5 - Updated S4B Powershell and SPO powershell 
• 1.6 - Updated Sign in Assistant, Azure AD version, S4B Online and SPO downloads/versions.  Deprecated Azure PowerShell - it changed too often to keep up with, and I am seeing Azure CLI taking a stronger role there.

Wishlist:

• Show Options in colors based on which modules are loaded versus not
• Ability to save tenantID/Lync/S4B/Exchange Server hostnames

Known Issues:

• If you have Lync 2010/2013/S4B admin tools installed locally, they WILL conflict with the LYO module.  See my blog post here on common issues with LYO.
• The install options do not UPGRADE. If you need an update, uninstall the older versions first.
• On Windows 8.0, after an install step there are several things that require PowerShell to be restarted before you can run the connect steps.
• The RSAT-ADDS tools require you to run the script as admin

Thanks to:

• Pat Richard, Lync MVP for allowing me to borrow some of his most excellent modules from Set-CsFeatures.ps1

Lync 2013 Archiving to Exchange 2013

For compliance reasons, I've always recommended that customers configure Lync archiving.  Unlike email, IM messaging is usually not a very large data set, and while the Lync client does have the option to save the conversation history in Outlook and Exchange, those folders are visible to end users and they can also empty and clean those folders out to cover tracks.

Ever since Exchange 2013 launched, I have been aware of the Lync archiving integration to it, but frankly have not had many customers that both had Lync 2013 and Exchange 2013 deployed until fairly recently. 

While reading about these integrations, I learned that the Exchange based archiving is NOT equal to the SQL based archiving.  You must place the users into an in place or litigation hold within Exchange in order for the data to be discoverable in Exchange's eDiscovery portal.

Let me say that a different way.  Unless you place a user on hold in Exchange, no Lync archives will be written to the user's mailbox.

I had spent a lot of time reading before this became clear to me, so hopefully this saves someone else some time.

Reading material:

• Doug's blog - Archiving Options in Lync Server 2013
• Exchange 2013 - Archive Lync conversations and meeting content to Exchange states "This requires placing the user on In-Place Hold"
• Lync 2013 - Configuring Microsoft  Lync Server 2013 to use Microsoft Exchange Server 2013 archiving, in Step 3 states that the Set-CSUser object has four settings for the ExchangeArchivingPolicy attribute:
• Uninitialized. Indicates that archiving will be based on the In-Place Hold settings configured for the user's Exchange mailbox; if In-Place Hold has not been enabled on the user's mailbox then the user will have his or her messaging and Web conferencing transcripts archived in Lync Server.
• UseLyncArchivingPolicy. Indicates that the user's instant messaging and Web conferencing transcripts should be archived in Lync Server rather than in Exchange.
• NoArchiving. Indicates that the user's instant messaging and Web conferencing transcripts should not be archived at all. Note that this setting overrides any Lync Server archiving policies assigned to the user.
• ArchivingToExchange. Indicates that the user's instant messaging and Web conferencing transcripts should be archived to Exchange regardless of the In-Place Hold settings that have (or have not) been assigned to the user's mailbox.
• S4B - Planning for Archiving 

By default, all users will be Uninitialized.  The UseLyncArchivingPolicy means the user will only use SQL based archiving.  The NoArchiving is pretty self-explanatory.  The last option, ArchivingToExchange sounds like what I had thought Exchange archiving was originally, but the article also states this is an optional step only if your Exchange and Lync servers are in different forests:

"If Lync Server 2013 and Exchange 2013 are located in different forests then it is not enough to simply enable Exchange archiving in the archiving configuration settings; that will not result in instant messaging and Web conferencing transcripts being archived in Exchange. Instead, you must also configure the ExchangeArchivingPolicy property on each of the relevant Lync Server user accounts."

So, if you use Exchange 2013 archiving for Lync, and you have not been setting users to be in an in place hold, you do not have discoverable Lync data and need to add them to a hold, or rely on your SQL based logging instead!