Performing Clean Active Directory Migrations and Consolidations

Active Directory Migration Challenges

Over the past decade, Active Directory (AD) has grown out of control. It may be due to organizational mergers or disparate Active Directory domains that sprouted up over time, but many AD administrators are now looking at dozens of Active Directory forests and even hundreds of AD domains wondering how it happened and wishing it was easier to manage on a daily basis.

One of the top drivers for AD Migrations is enablement of new technologies such as unified communications or identity and access management. Without a shared and clearly articulated security model across Active Directory domains, it’s extremely difficult to leverage AD for authentication to new business applications or to establish the related business rules that may be based on AD attributes or security group memberships.

Domain consolidation is not a simple task. Whether you're moving from one platform to another, doing some AD security remodeling, or just consolidating domains for improved management and reduced cost, there are numerous steps, lots of unknowns and an overwhelming feeling that you might be missing something. Sound familiar?

One of the biggest fears in Active Directory migration projects is that business users will lose access to their critical resources during the migration. To reduce the likelihood of that occurring, many project leaders choose to enable a dirty migration; they enable historical SIDs which carry old credentials and group memberships from the source domain and apply them to the new domain. Unfortunately, enabling historical SIDs proliferates one of the main challenges that initially drove the migration project. The dirty migration approach maintains the various security models that have been implemented over the years making AD difficult to manage and near impossible to understand who has what rights across the environment.

Clean Active Directory Migrations

The alternative to a dirty migration is to disallow historical SIDs and thereby enable a clean migration where rights are applied as-needed in an easy-to-manage and well articulated security model. Security groups are applied on resources according to an intentional model that is defined up-front and permissions are limited to a least-privilege model where only those who require rights actually get them.

All consolidation or migration projects aren't the same. The motivations differ, the technologies differ, and the Active Directory organizational structure and assets differ wildly. Most solutions on the market provide point A to point B migrations of Active Directory assets. This type of migration often contributes to making the problem worse over time. There's nothing wrong with using an Active Directory tool to help you perform an AD forest or domain migration, but knowing which assets to move and how to structure or even restructure them in the target domain is critical.

Enabling a clean migration and transforming the Active Directory security model requires a few steps to be followed. It starts with assessment and cleanup of the source Active Directory environments. You should assess what objects are out there, how they’re being used, and how they’re currently organized. Are there dormant user accounts or unused computer objects? Are there groups with overlapping membership? Are there permissions that are unused or inappropriate? Are there toxic or high-risk conditions in the environment? This type of intelligence enables visibility into which objects you need to move, how they're structured, how the current domain compares to the target domain, and where differences exist in GPO policies, schema, and naming conventions. The dormant and unused objects as well as any toxic or high-risk conditions can be remediated so that those conditions aren’t propagated to the target environment.

Once the initial assessment and cleanup is complete, a gap-analysis should be performed to understand where the current state differs from the intended model. Where possible, the transformation should be automated. Security groups can be created, for example, based on historical user activity so that group membership is determined by actual need. This is a key requirement for numerous legal regulations.

The next step is to perform a deep scan into the Active Directory forests and domains that will be consolidated and look at server-level permissions and infrastructure across Active Directory, File Systems, Security Policies, SharePoint, SQL Server, and more. This enables the creation of business rules that will transform existing effective permissions into the target model while adhering to new naming conventions and group utilization. Much of this transformation should be automated to avoid human error and reduce effort.

Maintaining a Clean Active Directory

Once the migration or consolidation project is complete and adherence to the intended security model has been enforced, it’s vital that a program is in place to maintain Active Directory in its current state. There are a few capabilities that can help achieve this goal.

First, a mandatory periodic audit should be enforced. Security Group owners should confirm that groups are being used as-intended. Resource owners should confirm that the right people have the right level of access to their resources. Business managers should confirm that their people have access to the right resources. These reviews should be automated and tracked to ensure that these reviews are completely thoroughly and on-time.

Second, tools should be implemented that provide visibility into the environment answering questions as they come up. When a security administrator needs to see how a user is being granted rights to something they should perhaps not have, they’ll need tools that provide answers in a timely fashion.

Third, a system-wide scan should be conducted regularly to identify any toxic or high-risk conditions that occur over time. For example, if a user account becomes dormant, notification should be sent out according to business rules. Or if a group is nested within itself perhaps ten layers deep, you want an automated solution to discover that condition and provide related reporting.

Finally, to ensure adherence to Active Directory security policies, a real-time monitoring solution should be put in place to enforce rules, prevent unwanted changes via event blocking, and to maintain an audit trail of critical administrative activity.

Complete visibility across the entire Active Directory infrastructure enables a clean AD domain consolidation while making life easier for administrators, improving security, and enabling adoption of new technologies

About the Author

Matt Flynn has been in the Identity & Access Management space for more than a decade. He’s currently a Product Manager at STEALTHbits Technologies where he focuses on Data & Access Governance solutions for many of the world’s largest, most prestigious organizations. Prior to STEALTHbits, Matt held numerous positions at NetVision, RSA, MaXware, and Unisys where he was involved in virtually every aspect of identity-related projects from hands-on technical to strategic planning. In 2011, SYS-CON Media added Matt to their list of the most powerful voices in Information Security.

Audit for Active Directory

NetVision launched a new web site focused on our solution for reporting and monitoring on Windows and Active Directory. The solution is pretty cool. If you're having trouble understanding how to generate reports or apply real-time monitoring for Active Directory, you should check it out.

This is not a SIEM solution designed to collect logs from as many sources as possible. This is a very focused solution on Identity information in Active Directory. The listener is embedded into Active Directory so that we're not reliant on the security event logs, which provides non-repudiable proof of events that are taking place. And it means that we're not limited to the information provided by the event log (there is a limited set of attributes available for a user object change, for example). We can tell you what changes were made, when the change occurred and who initiated it. All of which is valuable audit and compliance information.

We also have advanced filtering capability on the listeners so that you can filter events by type, object, or actor. This means that you only collect relevant data which reduces storage and makes it easy to get to the data you want on the reporting side. We can tell you things like user attributes and group memberships, changes to user accounts or groups, inactive user accounts, OU changes, file system Access Control List changes, file system access attempts and file adds or changes. And we provide policy and report templates that make it easy for you to get up and running.

So take a look and let us know if you have questions.

More on LDAP and the Novell-Microsoft Bridge

While at Tech-Ed, I picked up some literature on the efforts of Novell and Microsoft to build bridges with each other toward interoperability. I found that they also have a web site on this topic where you can still read the original press releases and FAQs from November 2006. Conspiracy theories aside, interoperability is probably a good thing.

While I was looking for more info on their planned directory & identity interoperability, I came across an article titled Novell eDirectory vs. Microsoft Active Directory. I can assure you that it's not part of the interoperability literature. In fact, I wouldn't be surprised if it's soon taken off line to preserve the recently positive flow of energy between these two companies. While it's clearly and unapologetically written to sway would-be buyers toward Novell's directory product, it's quite an interesting read for those evaluating LDAP options. If you read my introduction to LDAP directories and are ready for a more thorough drill-down that doesn't require you to read the LDAP RFC, it's a good next step. Novell provides a thorough business level view of what to look for in a directory server.

Incidentally, I didn't find much on their plans for directory interoperability. From what I could find, it sounds like any identity interoperability will initially be at the user access layer and not at the data storage layer. If you have any more details, please comment!

What is a Directory? ...LDAP, AD and ADAM

Note: This is not content for LDAP professionals. This is
content for the LDAP-confused. Proceed with caution.

Overview

An LDAP directory is simply a type of database that is hierarchical in structure. LDAP is the protocol used to interact with that database. So, when you hear someone refer to putting data into LDAP, what they really mean is using the LDAP protocol to put data into a directory (a hierarchical database that communicates via LDAP).

Relational Databases

The databases used most frequently across today's IT landscape (Oracle, SQL Server, MySQL, etc.) are relational in structure. That is, data is stored in tables and data is connected to other data by way of relationships between tables. For example:

In this sample database, we see that Joe Johnson is a Clerk that reports to John Smith. Joe's job title is not in the same table as his name, but we know what it is because of the relationship between the tables. This is a relational data structure and it's the structure that's commonly used when we use the term database.

LDAP Directory Structure

An LDAP directory is hierarchical in structure. Joe's data would look like this in a directory:A directory uses the concept of an object to represent Joe's data. The directory schema defines the type of objects that will be allowed in this directory and what attributes are associated to those objects. Above we see that an object exists for Joe Johnson with attributes for employee number, first name, last name, job title and manager.

Directory Servers

There are a number of directory servers available on the market, including but not limited to:

• Sun Directory Server
• Novell eDirectory
• Oracle Internet Directory
• IBM Tivoli Directory Server
• OpenLDAP
• Fedora Directory Server

Note: I purposely left Microsoft off this list because I'll
cover AD and ADAM in the next section.

Each of the directory servers listed above (and most others) support a common protocol for storing, accessing, modifying and managing data. This protocol is LDAP. By standardizing on a common protocol, application developers can write their applications to store data in any LDAP compliant directory server. Organizations can choose to implement whichever directory server they prefer and feel fairly confident that an LDAP-ready application will be able to consume or interact with their data.

Vendor-neutrality is theoretically achievable with relational database servers as well. I say theoretically because the big database vendors provide add-ons to the standard SQL language that are convenient for developers but sometimes challenging for portability. The main reasons the industry moved away from relational and toward directories for some uses are as follows:

• Query Speed: LDAP directories typically provide faster query response times
• Ease of Management: LDAP's hierarchical structure makes it easy to assign rights and privileges to subsets of users as well as to enable delegated administration over portions of the directory tree.
• There are probably others, but I believe these are the two big ones - please let me know if I'm missing anything here.

Microsoft Active Directory

With the release of Windows 2000 and the migration from Windows NT as their business networking platform, Microsoft introduced Active Directory (AD). AD is an LDAP directory at its core but it doesn't necessarily play by all the same rules. Because AD is used to manage a network environment - authentication, access rights, user management, etc., AD needed to be much more than just a database.

Note: AD is a predominant part of most IT organizations today. As I travelled to various organizations as an IdM consultant, I almost never encountered an organization that didn't have a mature AD environment containing some of the most up-to-date employee data in the entire organization. Based on my experience, AD has become the de-facto enterprise directory for most organizations.

As the identity management market grew throughout 2000 and 2001, organizations realized the need to leverage LDAP directories more and more. Because AD was already widely deployed and held good data, many organizations wanted to leverage AD -- or at least AD data -- for their identity infrastructure. AD administrators however recognized the need to protect their AD infrastructure and most decided against using AD for anything outside of its core network management role. This is why Microsoft introduced ADAM.

Microsoft ADAM

ADAM (Active Directory Application Mode) is based on the same core LDAP technology on which AD is based. But ADAM installs without all the overhead of the OS and network management capabilities that AD provides. ADAM is a stand-alone LDAP directory store similar to those provided in the list above. ADAM does not require AD to run -- it's not an extension of a given AD environment. But, it can be used to augment the functionality of AD. For example, some organizations use ADAM to store employee information for white pages or other applications that they prefer not to query AD directly. Other organizations use ADAM to store non-employee data while relying on AD for employee information. One interesting use of ADAM is as an authentication proxy. You can send an authorization request to ADAM and have it pass the request to AD - thereby leveraging AD credentials while disallowing applications to connect directly to AD.

Conclusion

A directory is just a type of database. LDAP is a way to communicate with a directory. ADAM is a stand-alone directory and not a feature of AD. There are lots of LDAP directory varieties to choose from.

Let me know if I should include additional info here. My main point was to tackle the simple points made in the conclusion. I run into confusion on those topics quite often.

re: Synchronization versus Virtualization

In his entry on Novell’s Cool Blogs, Volker Scheuber discusses the use-cases for virtual directories versus metadirectories and asks for input. Considering my number of previous posts on this topic, I felt an obligation to respond.

I've done a significant amount of services work around both metadirectory and virtual directory technologies. My last company had both and my current company offers neither. The bottom line is that if I were designing an IdM solution, I would want both options to be available. As with any enterprise technology, there's usually more than one way to accomplish a given goal. Most IdM data integration requirements can be met with either meta or virtual technologies. However, there are some scenarios that, for all practical purposes, require one or the other. A while back, I posted some common scenarios where a virtual directory can be really useful.

Even for some of these scenarios though, you may be able to achieve a similar result by standing up another data store and setting up data sync rules with a metadirectory. But it often requires a significant amount of programming that would need to be maintained in addition to the new data store (LDAP or other) and related infrastructure.

Here’s an example:

Let’s say you were deploying a COTS application for insurance claim processing and security is important. The app uses an Oracle database for system data and is built to leverage an enterprise LDAP store (eDirectory, Active Directory, etc.) for authentication. Each employee that needs access to the system already has an account and credentials in the enterprise directory (ED). The system leverages the ED’s groups to grant or deny access. And of course the ED user needs to be enabled.

Due to the sensitive nature of the material, a requirement is made that the system should not allow access to the app unless the employee is punched-in to the company time-clock system. So, if an employee were to piggy back off of a manager’s home VPN connection, they could not access the system unless physically punched-in at work where all claim processing personnel are monitored. The time-clock system has a MySQL database with both system and user data. The APIs for the claim system and the time-clock system are both difficult to work with and complex to maintain.

What are the various ways to meet this requirement?

You could potentially try to work with the API’s and write a custom authentication module that would connect to the time-clock system and verify that a user is punched in before granting access. This would require some internal training, development time and will need to be maintained as either vendor releases new versions in the future.

You could implement a metadirectory solution that sets the ED user as disabled each time status changes in the time-clock system. This may require some custom code, but should be fairly easy to implement. However, we don’t want to disable users in the ED each time they’re punched out. They still need access to HR benefit info and other systems. Not to mention that disabling a user in the ED may trigger other unwanted events to occur (de-provisioning).

You could have the metadirectory solution switch to a different attribute instead of disabling the user, but you’re back to writing a custom authentication module. Also, these metadirectory options will likely introduce some latency because metadirectory sync jobs are typically scheduled. So, there may be some window of time between when the user punches out and when the system denies access. This time window can be exploited to circumvent the intended security controls.

Or – you could have the claims app connect to a virtual directory that is configured to join data from the ED and the MySQL database in a way that is transparent to the claims app. The app sees the virtual directory as an enterprise LDAP directory, but when the user is not punched in, the user appears to be disabled. This is generally a very simple thing to configure for a virtual directory. And if the claims app expects a single group to grant or deny access but the ED groups are arranged differently, the virtual directory can display group memberships however is most appropriate for the app.

Let’s further suppose that the business rules are changed to grant restricted access for the user when punched in but at an alternate location. The virtual directory can be easily configured to set a flag indicating restricted access based on the requestor’s IP address. This is not easily accomplished with a metadirectory solution.

And it’s a plus that the same virtual directory can be used to display user data differently for any number of systems that need access to it – and without altering or moving the data. As we move toward increased regulations and encrypting sensitive data where it lives, reducing the number of data stores that need to be managed and encrypted is an advantage in itself.

Having said all that, I can come up with a number of examples where a metadirectory is clearly a better option than a virtual directory, but I think those are generally easier to see.

Hope this is useful.