Microsoft Lync 2010 RC preparation, pre-req's and installation

With the release of Microsoft's newest version of OCS, named Lync 2010, I decided it was time to do a basic walkthrough of the install process. This process is detailed very well in this deployment guide. This posting is more of a consolidated version of this document (minus the edge deployment for now) Another really good reference is the Getting Started guide which reviews new terminology, features and concepts.

Top Exciting Changes to the deployment
#1 – Powershell installation. You can now deploy roles from script once you understand the install needs.
#2 – Much improved certificate planning wizards.
#3 – Centralized Configuration that allows for more planning, testing and allows for rolling back changes as well.


Reviewing the DNS, certificate, and Edge configuration in that document after reading this walk through will help with some of those particular steps that I didn't think I could consolidate clearly in this entry.

Install pre-requisite Windows features using Powershell and reboot.
import-module servermanager

add-windowsfeature NET-Framework,web-server,web-http-redirect,web-asp,Web-Scripting-Tools,Web-Net-Ext,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Http-Logging,Web-Http-Tracing,Web-Windows-Auth,Web-Cert-Auth,Web-Filtering,web-asp-net,Web-Log-Libraries,Web-Client-Auth,RSAT-ADDS,telnet-client

Re-run Windows Update for the new roles.

Install the Visual C++ redistributable (Lync 2010 will install for you if do not)

Install Silverlight (yep, now you HAVE to - unless you don't want to admin from the server)

Start the \setup\amd64\setup.exe from the installation media


Choose Prepare AD - before doing ANY of this, I recommend reading my article on how to properly back up your AD schema. This is pre-release code, and this shouldn't be done in a production active directory!

Create DNS entries as needed (check document in the introduction above!)

Prepare Schema (this looks different now!)


Prepare the forest and the domain next. After this, you will see some new users in your Users Canonical folder in AD:


Done with AD steps, time to deploy the Central Management Service


This checks prereq's and installs things like SQL Express, and UC components:


Install Topology Builder and the Administrative Tools


At this point, we have some new tools in our start menu:


We will need to run the Topology Builder here and choose a New Topology.

Define a new topology

• Specify your primary and any additional SIP domains
• Define the first site. (More on sites later, but these are different from AD sites!)
• Enter Location information

New Front End Wizard

• Specify FQDN and Enterprise/Standard PoolSelect features
  

• Choose collocated server roles for AV Conferencing and Mediation servers.
• Associate server roles with the front end pool
  

• Specify a SQL store - if you chose Standard edition, everything here is greyed out.
• Define a file share - this will need to be manually created with Domain Admins Full Control and Everyone Read.
• Specify a web services URL, by default this is internal FQDN of the pool, but if you want it later to be an external FQDN it is changeable.

You can now review your topology and once complete, choose to publish it.

Once you have successfully published your topology, you can begin to install the Lync 2010 Server, choosing to retrieve directly from the Central Management Store (CMS)

This pulls your topology from what you published and beings installing the local configuration store, which in a single server configuration, is likely the same server. In a larger environment, you might want your topology and planning on a VM somewhere, separated from other roles.

Setup Lync Server components - based on machine name, it will pull from your published topology.

Request and assign an SSL certificate for the new server. Those who know OCS 2007 planning will appreciate the UI changes here.

Start services and confirm they are running

Launch the Lync Server Control Panel. If prompted for a URL, it's https://<server.or.pool.FQDN>/cscp and can be viewed from other machines as well.

Part Two will be coming soon, and I will delve into user enabling, and review DNS entries and client configurations more. Part 3 will likely review deploying a Lynx Edge server.

Exchange 2010 - Import-mailbox - Error Approving Object -2147221219

Called PSS today due to an issue with a mailbox import refusing to work.

import-mailbox error:
[PS] C:\temp>Import-Mailbox -identity test-pstfolderpath c:\temp\test123.pst
Error was found for swinc ([email protected]) because: Error occurred in the step: Approving object. An unknown error
has occurred., error code: -2147221219
StatusCode : -2147221219
StatusMessage : Error occurred in the step: Approving object. An unknown error has occurred.

When you run an Exchange Best Practices Analyzer (BPA) and it warns on this, you should fix it. Even though the instructions mention storage groups and are clearly based on Exchange 2007. I asked the technician about this and he stated that the Exchange 2010 specific fix is not yet published, but was in their internal KB still. The same fix describe in this BPA result will also address the import-mailbox error shown above.

BPA Error:


This links you to:
http://technet.microsoft.com/en-us/library/dd535374%28EXCHG.80%29.aspx

The Fix:
Copy the distinguished name attribute of a mailbox database on the server
Paste this value into the homemdb attribute of the System Attendant object in AD (which installs blank by default)
Restart the Exchange Information Store service
Restart the Exchange System Attendant service
Run Import-mailbox again

Success!

Odd domain controller errors in Exchange 2010 EMC after some DC moves.

After demoting some domain controllers, Exchange was functional and healthy, but we were seeing the occasional error when loading recipient lists, or drilling into a specific mailbox's settings such as the one below:

I would try a second time and get to the data, but the issue was annoying. Then I found this here:
http://social.technet.microsoft.com/Forums/en-GB/exchange2010/thread/cb231da0-2b06-41ce-9f4a-f11d408cf07f

The fix for me was not the highlighted one in that thread, but Adam Miceli's comment:

"Delete this file: c:\users\<specific user>\appdata\roaming\microsoft\mmc\Exchange Management Console"

This COMPLETELY worked and we no longer receive the above errors.

Enabling Network Level Authentication in Windows XP

With the advent of Windows Vista, Windows 7, and Windows 2008, the Microsoft RDP client was updated to support NLA, or Network Level Authentication. Seen below, the selected option allows for the most secure RDP experience.



The downside of this is that if you run older clients, specifically, Windows XP - the newest RDP client doesn't support NLA, so you receive this error when attempting to connect.





The common workaround for this is to choose the less secure option of allowing connections from computers running any version of Remote Desktop. While this may be fine for some organizations, it might not be as well. I wanted a better work around since it would mean I needed to ask a customer to change this setting. I searched and found this AWESOME workaround here.

On the Windows XP workstation you use to RDP into the server/workstation:

1. Click Start, click Run, type regedit, and then press ENTER.
   
2. In the navigation pane, locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
   
3. In the details pane, right-click Security Packages, and then click Modify.
   
4. In the Value data box, add tspkg on a new line. Leave any existing data that is specific to other SSPs, and then click OK.
   
5. In the navigation pane, locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders
   
6. In the details pane, right-click SecurityProviders, and then click Modify.
   
7. In the Value data box, add an additional comma, a space, and then credssp.dll - leave any data that is specific to other SSPs, and then click OK.
   
8. Exit Registry Editor.
   
9. Restart the computer
   

After rebooting, you will be able to RDP to newer OS's with the more secure NLA SSP.

From Simon G - here is a VBS script to do this in case you need to do it en masse.


const HKEY_LOCAL_MACHINE = &H80000002
lsaKey = "SYSTEM\CurrentControlSet\Control\Lsa"
lsaValue = "Security Packages"

hostname = "."

Set regObj = GetObject( "winmgmts:{impersonationLevel=impersonate}!\\" & hostname & "\root\default:StdRegProv" )

regObj.GetMultiStringValue HKEY_LOCAL_MACHINE, lsaKey, lsaValue, stringValues

size = Ubound( stringValues ) + 1

ReDim Preserve stringValues( size )
stringValues( size ) = "tspkg"


regObj.SetMultiStringValue HKEY_LOCAL_MACHINE, lsaKey, lsaValue, stringValues

secProvKey = "SYSTEM\CurrentControlSet\Control\SecurityProviders"
secProvValue = "SecurityProviders"

regObj.GetStringValue HKEY_LOCAL_MACHINE, secProvKey, secProvValue, theValue

theValue = theValue & ", credssp.dll"

regObj.SetStringValue HKEY_LOCAL_MACHINE, secProvKey, secProvValue, theValue

Documenting Exchange 2003 Users in preparation for migrations

You learn something new every day. Today, I got schooled in Exchange 2003 data collection. Of course, I am familiar with ExBPA. However, I had never seen this great script that collects SG/DB paths, and database sizes.

Then after being so used to Exchange 2007 and using Powershell cmdlets like get-mailboxstatistics and Glen's infamous mbsizereportv5.ps1 script, I was looking for the same in Exchange 2003 and facepalm'd myself when it was as simple as right click export.

Exchange 2010 DAG backups using Windows Server Backup

I wrote a while back on Exchange backups for 2007 and 2010 using Windows Server backup. Then I wrote here about product support for Exchange backups.

That first link is a little dated as the screens are from Windows 2008 and Exchange 2007 SP2. The Windows 2008 R2 screens are slightly different.

Rehashing the differences only, the VSS Full options are in a slightly different place:


Under Advanced Settings, there is an exclusion tab (I leave mine blank, but I could exclude things like Exchange binaries, etc) but more importantly is the VSS Full backup option:



Beyond that, the next new bit of information is about backing up DAG copies. Microsoft states that the Windows Server Backup (WSB) does not support passive database copies.

"However, the built-in support for Windows Server Backup is for active copies only. You can't use Windows Server Backup to back up passive copies."

This leaves you with minimal options.. If I have in this example, 3 databases, and 3 servers, I need to either move all the databases to one server and back it all up, or I need to back have backup jobs on each server, only backing up the disk with an active database.

The problem with the latter option is that the databases might move as maintenance or issues occur, and then you would have a failed job on those databases.

So I opted to move all the databases to be active on one server, run a complete backup, and then re-disperse the databases.

Here's the batch script I created. A few things to know here.

1) Learn how to use wbadmin.exe
2) You will need to change the volumes to match your server. Running mountvol.exe from an elevated command prompt will list all volumes on your server for you. Do note, the -allCritical does not automatically choose drives without system files on it (aka, Exchange disks)
3) You will need to change the username/password seen in the script to match your environment. User rights should be Backup Operators, and Exchange Org Administrator.

rem - Launch Powershell with Exchange cmdlets and run ps1 script to move all databases to on server

PowerShell.exe -command ". 'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1'; Connect-ExchangeServer -auto; c:\installers\move-databasestoexch1.ps1"

rem - Launch a backup
rem - determined volume mount points using moutvol.exe

wbadmin start backup -backupTarget:\\bdc\Exchangebackups\exch1 -include:\\?\Volume{1be73a5a-052f-11df-82c2-806e6f6e6963}\,\\?\Volume{26934b04-09f5-11df-a2b7-002564f8c664}\,\\?\Volume{26934b0b-09f5-11df-a2b7-002564f8c664}\,\\?\Volume{26934b0e-09f5-11df-a2b7-002564f8c664}\,\\?\Volume{1be73a5b-052f-11df-82c2-806e6f6e6963}\ -user:[email protected] -password:yescleartext -vssFull -quiet

rem - Launch Powershell with Exchange cmdlets and run ps1 script to move all databases back to separate servers.

PowerShell.exe -command ". 'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1'; Connect-ExchangeServer -auto; c:\installers\move-databasestopreferred.ps1"


Here's the move-databasestoexch1.ps1 script:

Move-ActiveMailboxDatabase DB1 -ActivateOnServer EXCH1 -MountDialOverride:None -Confirm:$false
Move-ActiveMailboxDatabase DB2 -ActivateOnServer EXCH1 -MountDialOverride:None -Confirm:$false
Move-ActiveMailboxDatabase DB3 -ActivateOnServer EXCH1 -MountDialOverride:None -Confirm:$false

And the move-databasetopreferred.ps1 script:
Move-ActiveMailboxDatabase DB1 -ActivateOnServer EXCH1 -MountDialOverride:None -Confirm:$false
Move-ActiveMailboxDatabase DB2 -ActivateOnServer EXCH2 -MountDialOverride:None -Confirm:$false
Move-ActiveMailboxDatabase DB3 -ActivateOnServer EXCH3 -MountDialOverride:None -Confirm:$false

I was able to schedule the backup using task scheduler and it works!

It certainly is not elegant. I would much prefer a backup product that looks for active mailbox databases and backs up accordingly. I would also enjoy not having a password in clear text in a batch file as a backup solution. But it works for now. I am planning on adding a "DAG support" column to my backup product matrix soon, and I will say "with scripting" for Windows Server Backup and link to this article.

PowerShell for Backup Planning

To size a backup system, you need to decide on a few key pieces of information: retention range, amount in GB of file data to be backed up & change rate on that data. For applications, you will need to know SQL database sizes and change rates. Same for Exchange.

luckily with powershell, it's quick and easy. I'm a lazy scripter so I'm sure this script could be much improved, but it worked for my purposes.

to send server name, volume name, volume size, and free space to a .csv file:

$computer="server1","server2","server3"
$credential=get-credential
ForEach ($item in $computer)
{
$disk = get-wmiobject -credential $credential -query "select * from win32_logicaldisk where DriveType='3'" -computername $item
foreach($drive in $disk)
{
$item + "," + $drive.Name + "," + [int]($drive.size/1gb) +"," +[int]($drive.freespace/1gb)|out-file test.csv -append
}
}

This script will ask you to login with administrator credentials, then output information for all local hard drives (no removable drives or CD-ROMs), and output it to a .csv file.

For clusters, it will only retrieve the primary member's information.

What about SQL? We can simply search for .mdf files and list their sizes. We could also use this same bit to search for LDF files or any other file type. This will output to a .csv file the Server name, the path of the .mdf file, and the file size.

$computer="Server1","Server2","Server3"
$credential=get-credential
foreach ($svr in $computer)
{$dbFiles = Get-WmiObject -credential $credential -Class CIM_DataFile -Filter "Extension = 'mdf'" -ComputerName $svr;
$dbFiles | ForEach-Object { ($_.csname) + "," + ($_.Name) + "," + ($_.filesize/1gb)}| out-file testsql.csv -append

}

Notice that both scripts loop through an array to find data. You could also use an active directory query to populate the array instead of entering the names manually.

by the way, I love how I can convert bytes to GB just by dividing by "1GB" - powershell has built in constants for translating file sizes. You can also divide by MB or KB.

Errors moving mailboxes to an Exchange 2010 DAG

I ran into this today, doing my first production DAG with a copy on a kind of slow connection.

Error: Move for mailbox '/o=First Organization/ou=First Administrative Group/cn=Recipients/cn=user' is stalled because DataMoveReplicationConstraint is not satisfied for the database 'Database' (agent MailboxDatabaseReplication). Failure Reason: Database a409ab86-ce24-4fcf-bd2a-14fd633090aa does not satisfy constraint SecondCopy. Some database copies are behind.

Sure enough, a quick check of get-MailboxDatabaseCopyStatus showed that my CopyQueueLength was fairly high on the server across the WAN. As a result, my mailbox moves were failing with the above error. However, they don't fail right away, the StatusDetail shows StalledDueToHA. Some stayed there up to two hours waiting for the log shipping to catch up on the remote server before failing.

To show a more detailed output on move progress, I was using:
Get-MoveRequest Get-MoveRequestStatistics | ft displayname,*stat*,perc*,totalmailboxsize

So what Exchange 2010 is doing here is smart. Exchange Active Manager doesn't want that CopyQueueLength to be over 10 files, or the replay queue length over 50. More constraints here.

The workaround is to disable this limit, so your moves can occur and the seeding can occur over time. This is one of three Microsoft recommended fixes. One is fix your database health, one is upgrade your WAN. This third one is a workaround that should be reconfigured after the initial mailbox moves.

Set-MailboxDatabase -DataMoveReplicationConstraint None

The default here is SecondCopy. More information on the other settings at the link above. This change DOES require a restart of the Exchange Replication Manager service. Be forewarned, if you have a queue length already, the replication manager will hang on stopping and attempt to complete the copies before stopping, so it might take some time.

Of course, once your moves are done, and your database's CopyQueueLength is normalized, you should re-enable this constraint using:

Set-MailboxDatabase -DataMoveReplicationConstraint SecondCopy

Normalizing Phone Numbers to E.164 format in Excel

Recently, I had the need to import some users for a large company. In order to populate as much of their Active Directory as possible, they wanted their phone numbers to be in a standardized format. Both Microsoft and Cisco have standardized on E.164 (additional information here) as a numbering standard, which basically starts with + [country code] + phone number.

This particular customer is US based only, so all the numbers in their spreadsheet had a US country code of 1. If I had a multinational organization, some additional coding would need to be done to account for other country codes.

My major need was to simply re-input all the different numbering standards the various internal organizations had inputted their phone numbers as. In other words, normalization. This helps to set up AD for later integration of OCS, or other VoIP systems, as well as Exchange 2007 or Exchange 2010 UM.

Either way, the Excel formula I was using here was the following:
=CONCATENATE("+1",SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE((SUBSTITUTE(A2,"(","")),")","")," ",""),"-",""),".",""),"x",";ext="))

In logical order..

1. Replace ( with null
   
2. Replace ) with null
   
3. Replace space with null
   
4. Replace hyphen with null
   
5. Replace period with null
   
6. Replace x with ";ext=" (which is the E164 standard for non-DID numbers)
   
7. Concatenate the +1 country code
   

Here again without the horrible color:
=CONCATENATE("+1",SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE((SUBSTITUTE(A2,"(","")),")","")," ",""),"-",""),".",""),"x",";ext="))

The end result, computer readable phone numbers!

OCS reporting GUI interface!

Saw this on the OCS team blog.

"The report allows you to enter the SIP URI of any 2 users that you want to view archived messages from. If you enter “Any User” (case sensitive) for either of the user input boxes, you are able to view any message from any user to a specific user as well as any user to any other user. You can use the Start Date and End Date to narrow down the search to a specific date range. Once you have entered all of the inputs, click on View Report."

Exchange 2010 Backup Product Support Matrix

Well, the #1 recent article here is my Exchange 2007 and Exchange 2010 Backup how to. The reason, of course, is that Exchange 2010 has been out since November 2009 and select few have yet announced or released Exchange 2010 support, and many companies are still trying to find how to backup Exchange 2010 in their production environment.

While I have not used many of these yet, I am hoping to try to demonstrate them all and update this post as new information comes. However, if you have made a significant investment in any of the below (or others, email me!) and are waiting for support, the best thing you can do is ask your vendor. The more input received, the more important you are making it for them!

Here is a chart of my findings thus far:

Vendor	Product	Exchange 2010 Support	Exchange 2010 SP1 Support	DAG support
Commvault	Commvault	Non-DAG in 8.0, DAG in 9.0	Unknown	Yes in 9.0
Symantec	NetBackup	Yes in 7.0	Unknown	Yes in 7.0
Symantec	Backup Exec 2010	Yes, in Backup Exec 2010, must run x64 on BE Server	Yes, in Backup Exec 2010 R2	Yes in Backup Exec 2010
CA	Arcserve	Yes	Unknown	Yes
EMC	Networker	Yes	Unknown	Yes
EMC	DataDomain	Unknown	Unknown	Unknown
EMC	Avamar	Yes	Unknown	Yes
i365	eVault	Expected by EOY	Expected by EOY	Unknown
Microsoft	DPM 2010	Yes	Yes	Yes
Microsoft	Windows Server Backup	Yes	Unknown	Yes

By the way, if you know of ANY updates, feel free to comment, or email me directly at [email protected]. If I said Unknown above it's more likely that your web site didn't make this information readily accessible. If I receive emails from the appropriate domain names, I will post it as official, more so if you can provide a link!

Uninstalling Exchange 2003 - 8240 error - 0x80072030

Ran into this one today, and hadn't encountered it in quite some time.



This error is due to the fact that someone had listed their own user as the postmaster account in Exchange 2003, and that person had since left the organization and had their account deleted.

So the error above is shown:
Setup encountered an error while checking prerequisites For the component "Microsoft Exchange":
0X80072030 (8240): There is no such object on the server.

This is easily fixed in ADSIEdt.msc - first, let's find out the user.



I've clipped the name to protect the guilty :)

As you can see, the deleted Objects distinguishedName here shows the user selected as the msExchAdminMailbox has been deleted.

In ADSIEdit, browse to CN=Configuration, DC=DOMAINNAME,DC=Com\CN=Services\CN=Microsoft
Exchange\CN=EXCHANGEORGNAME\CN=Global Settings\CN=Message Delivery. Go to the properties of Message Delivery, and scroll down to the msExchAdminMailbox.

Now, we need to fix the error. Pick who you want to be the new admin (I picked Administrator) and put their DN in this field. In my case, this is CN=Administrator,CN=Users,DC=Domain,DC=com. If you aren't sure of your DN, browse to your user in ADSIEDIT and you can directly copy/paste the DistinguishedName from your user object into this field.

Once done, no service restarts needed, as the uninstaller is just checking these fields.. You can now select Remove for Exchange 2003!

Exchange 2010 Management tool start up problems

Something that has been posted a LOT on the Exchange 2010 Forums on Technet - people with issues starting the EMC or EMS in Exchange 2010. Many of these step from the slightly different management via WinRM. The Microsoft Team blog posted a GREAT write up on how to troubleshoot the different common errors and address them all!

http://msexchangeteam.com/archive/2010/02/04/453946.aspx

Migrating PKI from Windows 2003 to Windows 2008 R2

Many customers are running into the need for a Windows 2008 or newer PKI infrastructure in order to enroll and auto enroll newer client operating systems like Windows 7, Vista, and Windows 2008 Server.

Actually, many business customers found the lack of certificate support in Vista (without upgrading their CA's later) as one of the reasons it wasn't business ready. With Windows 7 being almost 10 years newer than Windows XP, many business customers are ready for a software refresh and Windows 7 has enough other appealing features to help that decision along.

There are basically two routes to go; in place upgrade or migration. The only time I would attempt an in place is on a VM so that a snapshot could easily be taken and rolled back in the case of a failure. A migration gives a fresh start, but requires some additional time to complete because between steps you need to wait for certs to issue to clients.

Because certificates are fairly sensitive information, I won't post screen caps, but rather overview the process.

Research and Design

Research what your existing CA is in use for. Anything it has issued needs to be either determined to be invalid (expired, not in use, not needed) or documented as something to replicate on the new CA. The other decision on design is around what CA architecture and hierarchy you want or need. Depends on the size and complexity of your organization this can differ greatly. For most organizations under 2000 users, I would say a single CA is sufficient, and if an additional are needed, use the PKI planning guides that Microsoft provides, or better yet, read Komar's 2k8 PKI book.

Implement and Re-Issue certs

Depending on your usage, this could take a long time. Audit existing certificates, revoke the ones that are not in use or expired, and start re-issuing them on the new CA architecture. For larger organizations, this may take months to complete. Luckily, you can choose to have both CA's active. I recommend changing the certificate templates on the old CA to read only, and no longer allow enroll and auto enroll as you migrate each template type successfully, this way, the old CA still validates certificates issued that you haven't updated while you can work on updating them, without any noticeable downtime.

Decommission Legacy CA

The "easy" part for sure. Removing a CA (Unlike uninstalling Exchange) there are no checks or audits to make sure you did everything correctly. If you didn't notice that your Cisco ASA or VPN Concentrator had a certificate issued and miss it, it may cause some issues for you. I recommend stopping and disabling your legacy CA for a few days or even weeks (this depends on your comfort level, and organization) before you make the decision to decommission. Even then, before you decommission, I would also really recommend taking a complete backup of the server.