Performing Clean Active Directory Migrations and Consolidations

Active Directory Migration Challenges

Over the past decade, Active Directory (AD) has grown out of control. It may be due to organizational mergers or disparate Active Directory domains that sprouted up over time, but many AD administrators are now looking at dozens of Active Directory forests and even hundreds of AD domains wondering how it happened and wishing it was easier to manage on a daily basis.

One of the top drivers for AD Migrations is enablement of new technologies such as unified communications or identity and access management. Without a shared and clearly articulated security model across Active Directory domains, it’s extremely difficult to leverage AD for authentication to new business applications or to establish the related business rules that may be based on AD attributes or security group memberships.

Domain consolidation is not a simple task. Whether you're moving from one platform to another, doing some AD security remodeling, or just consolidating domains for improved management and reduced cost, there are numerous steps, lots of unknowns and an overwhelming feeling that you might be missing something. Sound familiar?

One of the biggest fears in Active Directory migration projects is that business users will lose access to their critical resources during the migration. To reduce the likelihood of that occurring, many project leaders choose to enable a dirty migration; they enable historical SIDs which carry old credentials and group memberships from the source domain and apply them to the new domain. Unfortunately, enabling historical SIDs proliferates one of the main challenges that initially drove the migration project. The dirty migration approach maintains the various security models that have been implemented over the years making AD difficult to manage and near impossible to understand who has what rights across the environment.

Clean Active Directory Migrations

The alternative to a dirty migration is to disallow historical SIDs and thereby enable a clean migration where rights are applied as-needed in an easy-to-manage and well articulated security model. Security groups are applied on resources according to an intentional model that is defined up-front and permissions are limited to a least-privilege model where only those who require rights actually get them.

All consolidation or migration projects aren't the same. The motivations differ, the technologies differ, and the Active Directory organizational structure and assets differ wildly. Most solutions on the market provide point A to point B migrations of Active Directory assets. This type of migration often contributes to making the problem worse over time. There's nothing wrong with using an Active Directory tool to help you perform an AD forest or domain migration, but knowing which assets to move and how to structure or even restructure them in the target domain is critical.

Enabling a clean migration and transforming the Active Directory security model requires a few steps to be followed. It starts with assessment and cleanup of the source Active Directory environments. You should assess what objects are out there, how they’re being used, and how they’re currently organized. Are there dormant user accounts or unused computer objects? Are there groups with overlapping membership? Are there permissions that are unused or inappropriate? Are there toxic or high-risk conditions in the environment? This type of intelligence enables visibility into which objects you need to move, how they're structured, how the current domain compares to the target domain, and where differences exist in GPO policies, schema, and naming conventions. The dormant and unused objects as well as any toxic or high-risk conditions can be remediated so that those conditions aren’t propagated to the target environment.

Once the initial assessment and cleanup is complete, a gap-analysis should be performed to understand where the current state differs from the intended model. Where possible, the transformation should be automated. Security groups can be created, for example, based on historical user activity so that group membership is determined by actual need. This is a key requirement for numerous legal regulations.

The next step is to perform a deep scan into the Active Directory forests and domains that will be consolidated and look at server-level permissions and infrastructure across Active Directory, File Systems, Security Policies, SharePoint, SQL Server, and more. This enables the creation of business rules that will transform existing effective permissions into the target model while adhering to new naming conventions and group utilization. Much of this transformation should be automated to avoid human error and reduce effort.

Maintaining a Clean Active Directory

Once the migration or consolidation project is complete and adherence to the intended security model has been enforced, it’s vital that a program is in place to maintain Active Directory in its current state. There are a few capabilities that can help achieve this goal.

First, a mandatory periodic audit should be enforced. Security Group owners should confirm that groups are being used as-intended. Resource owners should confirm that the right people have the right level of access to their resources. Business managers should confirm that their people have access to the right resources. These reviews should be automated and tracked to ensure that these reviews are completely thoroughly and on-time.

Second, tools should be implemented that provide visibility into the environment answering questions as they come up. When a security administrator needs to see how a user is being granted rights to something they should perhaps not have, they’ll need tools that provide answers in a timely fashion.

Third, a system-wide scan should be conducted regularly to identify any toxic or high-risk conditions that occur over time. For example, if a user account becomes dormant, notification should be sent out according to business rules. Or if a group is nested within itself perhaps ten layers deep, you want an automated solution to discover that condition and provide related reporting.

Finally, to ensure adherence to Active Directory security policies, a real-time monitoring solution should be put in place to enforce rules, prevent unwanted changes via event blocking, and to maintain an audit trail of critical administrative activity.

Complete visibility across the entire Active Directory infrastructure enables a clean AD domain consolidation while making life easier for administrators, improving security, and enabling adoption of new technologies

About the Author

Matt Flynn has been in the Identity & Access Management space for more than a decade. He’s currently a Product Manager at STEALTHbits Technologies where he focuses on Data & Access Governance solutions for many of the world’s largest, most prestigious organizations. Prior to STEALTHbits, Matt held numerous positions at NetVision, RSA, MaXware, and Unisys where he was involved in virtually every aspect of identity-related projects from hands-on technical to strategic planning. In 2011, SYS-CON Media added Matt to their list of the most powerful voices in Information Security.

Reduce Risk by Monitoring Active Directory

Active Directory (AD) plays a central role in securing networked resources. It typically serves as the front gate allowing access to the network environment only when presented with valid credentials. But Active Directory credentials also serve to grant access to numerous resources within the environment. For example, AD group memberships are commonly used to manage access to unstructured data resources such as file systems and SharePoint sites. And a growing number of enterprise applications leverage AD credentials to grant access to their resources as well.

Active Directory Event Monitoring Challenges

Monitoring and reporting on Active Directory accounts, security groups, access rights, administrative changes, and user behavior can feel like a monumental task. Event monitoring requires an understanding of which events are critical, where those events occur, what factors might indicate increased risk, and what technologies are available to capture those events.

Understanding which events to ignore is as important and knowing which are critical to capture. You don't need immediate alerts on every AD User or Group change which takes place but you want visibility into critical high-risk changes: Who is adding AD user accounts? ...adding a user to an administrative AD group? ...making Group Policy (GPO) changes?

Active Directory administrators face a complex challenge that requires visibility into events as well as infrastructure to ensure proper system functionality. A complete AD monitoring solution doesn't stop at user and group changes. It also looks at Domain Controller status: which services are running, disk space issues, patch levels, and similar operational and infrastructure needs. There are numerous technical requirements to get that level of detail.

AD administrators require full access in the environment which presents another set of challenges. How do you enable administrators to do their job while controlling certain high-risk activity such as snooping on sensitive data or accidentally making GPO changes to important security policies? Monitoring Active Directory effectively includes either preventing unintended activities through change blocking or deterring activities through visible monitoring and alerting.

Monitoring Active Directory Effectively

Effective audit and monitoring solutions for Active Directory address the numerous challenges discussed above by providing a flexible platform that covers typical scenarios out-of-the-box without customization but also allows extensibility to accommodate the unique requirements of the environment.

Data collection is the cornerstone of any Active Directory monitoring and audit solution. Collection must be automated, reliable, and non-intrusive on the target environment. Data that can be collected remotely without agents should be. But, when requirements call for at-the-source monitoring, for example when you want to see WHO did it, what machine they came from, capture before-and-after values, or block certain activities, a real-time agent should be available to accommodate those needs. The data collection also needs to scale to the environment’s size and performance requirements.

Once data has been collected, both batch and real-time per-event analysis are required to meet common requirements. For example, you may want an alert on changes to administrative groups but you don’t want alerts on all group changes. Or you may want a report that highlights all empty groups or groups with improper nesting conditions. This analysis should provide intelligence out-of-the-box based on industry expertise and commonly requested reporting. But it should also enable unique business questions to be answered. Every organization uses Active Directory in unique ways and custom reporting is an extremely common requirement.

Finally, once data collection and analysis phases have been completed, AD monitoring solutions should provide a flexible reporting interface that provides access to the intelligence that has been cultivated. As with collection and analysis, the reporting functionality should include commonly requested reports with no customization but should also enable report customization and extensibility. Reporting should include web-accessible reports, search and filtering, access to the raw and post-analysis data, and email or other alerting.

An effective Active Directory monitoring solution provides deep insight on all things Active Directory. It should enable user, group and GPO change detection as well as reporting on anomalies and high-risk conditions. It should also provide deep analysis on users, groups, OUs, computer objects, and Active Directory infrastructure. Because the types of reports required by different teams (such as security and operations) may differ, it may be prudent to provide slightly different interfaces or report sets for the various intended audiences.

When real-time monitoring of Active Directory Users, Groups, OUs, and other changes (including activity blocking) are important, the solution should provide advanced filtering and response on nearly all Active Directory events as well as an audit trail of changes and attempts with all relevant information.

Benefits of Active Directory Monitoring

The three most common business drivers for Active Directory monitoring are improved security, improved audit response, and simplified administration. Active Directory audit and monitoring solutions make life easier for administrators while improving security across the network environment. This is especially important as AD becomes increasingly integrated into enterprise applications.
Some common use-cases include:

• Monitor Active Directory user accounts for create, modify and delete events. Capture the user account making the change along with the affected account information, changed attributes, time stamp, and more. This monitoring capability acts independent of the Security Event log and is non-reputable.
• Monitor Active Directory group memberships and provide reports and/or alerts in real time when memberships change on important groups such as the Domain Admins group.
• Report on failed attempts in addition to successful attempts. Filter on specific types of events and ignore others.
• Report on Active Directory dormant accounts, empty groups, unused groups, large groups, and other high-risk conditions to empower administrators with actionable information.
• Automate event response based on policy with email alerts, remediation processes, or record the event to a file or database.

Active Directory Monitoring and Reporting doesn't need to feel complicated or overwhelming. Solutions are available to simplify the process while providing increased security and reduced risk.

About the Author

Matt Flynn has been in the Identity & Access Management space for more than a decade. He’s currently a Product Manager at STEALTHbits Technologies where he focuses on Data & Access Governance solutions for many of the world’s largest, most prestigious organizations. Prior to STEALTHbits, Matt held numerous positions at NetVision, RSA, MaXware, and Unisys where he was involved in virtually every aspect of identity-related projects from hands-on technical to strategic planning. In 2011, SYS-CON Media added Matt to their list of the most powerful voices in Information Security.

Active Directory Unification and Attribute Cleanup

I recently posted about Active Directory Unification. The main points were (1) that there is value in AD consolidation and (2) that there's a right way to do it to meet the intended goals.

Sander Berkouwer posted earlier this month on Active Directory attribute integrity. He makes the point that with all the tools Microsoft provides to enable tighter management of identities and access (FIM, ADFS, ADRMS, DAC), Active Directory Cleanup is more important than ever. Berkouwer writes:

"When these attributes are inconsistent, access to files, apps, partners and cloud functionality becomes inconsistent. If you think it won’t happen to you, think twice. During the first internal Microsoft deployment of Dynamic Access Control, attribute inconsistency was the first encountered problem."

Absolutely.

Most people that I speak with jump into the benefits that cleanup will have on the AD Unification process. The reality is that the real value of cleanup is enabling the right functionality and access controls after the unification process is complete. (Of course, as I wrote, it's never really complete - it's not a onetime event.)

It's worth making the distinction.

Active Directory Unification

[This is a partial re-post of an entry on the STEALTHbits blog. I think it's relevant here and open for discussion on the concepts surrounding clean migrations and AD unification.]

It’s no secret that over the past decade, Active Directory has grown out of control across many organizations. It’s partly due to organizational mergers or disparate Active Directory domains that sprouted up over time, but you may find yourself looking at dozens or even hundreds of Active Directory domains and realize that it's time to consolidate. And it probably feels overpowering. But despite the effort in front of you, there’s an easy way and a right way.

Domain consolidation is not a simple task. Whether you're moving from one platform to another, trying to implement a new security model, or just consolidating domains for improved management and reduced cost, there are numerous steps, lots of unknowns and an overwhelming feeling that you might be missing something. Sound familiar?

According to Gartner analyst Andrew Walls, “The allure of a single AD forest with a simple domain design is not fool’s gold. There are real benefits to be found in a consolidated AD environment. A shared AD infrastructure enables user mobility, common user provisioning processes, consolidated reporting, unified management of machines, etc.”

Walls goes on to discuss the politics, cost justification, and complexity of these projects noting that “An AD consolidation has to unite and rationalize the ID formats, password policy objects, user groups, group policy objects, schema designs and application integration methods that have grown and spread through all of the existing AD environments. At times, this can feel like spring cleaning at the Aegean stables. Of course, if you miss something, users will not be able to log in, or find their file shares, or access applications. No pressure.”

Walls offers advice on how to avoid some of the pain. “You fight proliferation of AD at every turn and realize that consolidation is not a onetime event. The optimal design for AD is a single domain within a single forest. Any deviation from this approach should be justified on the basis of operational requirements that a unified model cannot possibly support.”

What does this mean for you? Well, the most significant take-away from Walls’ advise is that it’s not a onetime event. AD Unification is an ongoing effort. You don’t simply move objects from point-A to point-B and then pack it in for the day. The easy way fails to meet the core objectives of an improved security model, simplified management, reduced cost, and a common provisioning process (think integration with Identity Management solutions).

If you take everything from three source domains and simply move it all to a target domain, you haven’t achieved any of the objectives other than now having a single Active Directory. There’s a good chance that your security model will remain fragmented, management will become more difficult, and your user provisioning processes will require additional logic to accommodate for the new mess. On a positive note, if this model is your intent, there are numerous solutions on the market that will help.

STEALTHbits, of course, embraces the right way. “Control through Visibility” is about improving your security posture and your ability to manage IT by increasing your visibility into the critical infrastructure.


If you'd like to learn more about the solution, you can start by reading the rest of this blog entry or contact STEALTHbits.

Rule-Based Log Correlation: An Alternative Approach

In an article at SYSCON Media, Gorka Sadowski writes about SIEM technologies and specifically about the complexity of event correlation.

Why Rule-Based Log Correlation Is Almost a Good Idea: The Future of SIEM

He points out that there are some challenges with static rule-based correlation. But, he calls it "the engine for the first generation of [SIEM]". That sounds about right. What scares me is that the future solutions to which Sadowski alludes look even more complicated. So, there may be a trade off to get the perceived increase in value.

I have an alternative solution that simplifies things for the SIEM. Over the past few years at NetVision, we've had a number of organizations interested in the NVMonitor solution (now called StealthINTERCEPT) because of its advanced filtering and from-the-source event collection. It doesn't rely on logs and enables a highly advanced ability to filter events as they happen eliminating the need for after-the-fact correlation.

For example, when looking at Active Directory Security Group events, you can return only changes to high-risk groups or changes to business-line groups that are not made by a specified subset of users (even if they may be a domain administrator). These events are pre-filtered and sent to the SIEM only when appropriate. It can also block events, btw, and send the event to the SIEM as an "attempt" rather than an actual event. And of course, it has it's own alerting and response mechanisms built in for real-time, contextual response.

Improved data collection on key source systems may be a better alternative to mathematic modeling from the event archive. Perhaps not in every case, but on core security infrastructure like Active Directory where rules are definable and today's challenge lies in the ability to implement, it's not only better, it's here today and already proven in production environments.

A new tool for Identity Management: AD Event Push SDK

I had a brainstorm last night. Most Identity Management systems poll Active Directory to pull changes on a periodic basis. And admittedly, that approach is often sufficient. But, here's an alternative that gets you real-time push updates from AD with the person who made the change (which is often useful for work flow/approval and audit trail purposes):

A small utility service that sits on the DC waiting for changes
Picks up events in real-time
Sends event info to a DLL that you compile

The DLL determines what to do with the info based on:
- Who did it
- What groups the person is a member of
- What happened (add/mod/del)
- Which object or object-type was affected (admin accounts, service accounts, etc)
- Which attributes changed
- In which OU the event occurred
- etc.

Possible outcomes may be to trigger an Identity Management process or work flow, open a help desk ticket, generate an alert, etc.

The programming would be very similar to building an ILM management agent.

So, IDM friends, would this be useful? If you have a project in mind or just want to play with it, let me know. NetVision has had this agent for years but we add a lot of capability around it. I'm wondering if there's some value for you if we strip it down to a single DLL that you would control. Let me know.

How to clean up years of permission bloat

Since joining NetVision a few years ago, I've spoken to countless organizations who are faced with clean up duty. For years, administrators have granted permissions, added group memberships, created countless new security groups, delegated rights in Active Directory and have been mostly in a reactive mode. That is, they grant permissions in response to some member of the business asking for new rights. Unfortunately, business managers have not had motivation to request that permissions be revoked when appropriate. So, in many cases, there are hundreds or thousands of security groups that nobody seems to know what they're for or how they should be used. And some big percentage of the user population has access to files/folders that they shouldn't.

In an ESJ article titled Coming Clean: Getting a Handle on Permissions and Group Memberships, NetVision CEO David Rowe discusses the challenge and explains how you can regain control over network access rights.

ADSI, ADO, and DNS

I discovered a few interesting technical bits this morning that I haven't seen before.

First, DNS operating order (as you may know) is:
(1) Check local host name
(2) Check hosts file
(3) Check DNS servers

If you query for domain name on an Active Directory domain controller, it doesn't resolve in the first step. So, you'd need proper DNS entries for the domain (or an entry in Hosts). I would've thought a query on a DC for domain name would resolve immediately, but when using a client that relies on DNS (like an ADSI script), it doesn't resolve.

Next, I found that a simple LDAP lookup (using domain name / rootDSE) via ADO resolved fine where a similar script using ADSI did not. So, apparently, ADO does NOT rely on DNS to attach to the rootDSE but ADSI DOES rely on DNS.

When evaluating whether to use ADO or ADSI, I recall that ADSI was generally easier, but there may be improved performance with ADO for larger record sets. I'm not sure if that's true, but that's what I remember reading years ago. I wonder if use of DNS should be an additional consideration in some cases?

TEC 2010: Imanami

While walking the floor at TEC 2010, I spoke to a few vendors who agreed to provide a quick video message. The first is Imanami who was demonstrating a pretty interesting solution for managing Active Directory group memberships.

TEC 2010: Active Directory Family

The day one keynote speech this morning was presented by Microsoft's Conrad Bayer. One of the key take-aways from this morning's keynote for me for a consistent theme throughout the talk that Microsoft's Identity & Access solutions are now all part of the same product group. The Identity & Access group's solutions include Active Directory, ADFS federation, RMS rights management, FIM life cycle identity management, PKI/Certificates, identity synchronization, etc.

Bayer also talked about the future of these solutions and briefly discussed that ADFS could evolve to become an authorization server. Specifically, he talked about attributes and claims being the core components of authorization. The idea would be that ADFS could sit in between local and remote directory environments and provide answers to standards-based requests about claims. Bayer was asked later about the challenges around the idea that, for AD, groups are equivalent to roles, but other systems' roles require more than just group memberships. His answer pointed back to attributes and claims as being the way to meet those business requirements and seemed to say that applications would be where you would manage roles. The application would define and manage roles while leveraging the AD infrastructure to answer access-related questions via claims. He didn't say it (or even suggest it), but I wonder if this is a move toward a completely different paradigm than one based on roles. Perhaps roles will never be the right answer since what we've all seen is that in reality, people don't fit nicely into a pre-defined set of business roles.

Another thing that caught my ear was Bayer's point that Smartcards and Certificates are becoming more important as environments move to distributed and cloud-based solutions. Could it finally be the year of PKI? BTW - I see 'the year of PKI' as a modern-day proverb about something that it perpetually about to happen but never really does. Having said that, I'm a fan of PKI as a technology and can see that his point has some validity. The fact that a particular solution is in the cloud is not necessarily the problem. The bigger problem is that there are a variety of apps moving into the cloud each with different security models and underlying security mechanisms. PKI technology might help us figure out how to provide a manageable solution for that complexity.

At the end of the day, I think Microsoft made the right move by bringing these technologies together, but it sounds like it'll be a while before we see a truly unified, native/out-of-the-box set of identity features such as point and click federation, PKI, or rights management.

Querying AD from SQL Server

This is a great how-to article on querying Active Directory from within SQL Server. I've written in the past about using Virtual Directory technology to query SQL data via LDAP. This is the reverse and I can envision many use-cases where this would be useful. This isn't exactly new technology, but it's a new write-up on how it's done and very easy to follow.

For example, you could use this approach to extend the information available to an application without doing any data synchronization or introducing new data sources. If the application's logon ID is the user's email address, you could query AD based on that email and get info about the user's group memberships, attributes, manager, location, etc. and have that returned to the application as if the data were stored in the local app's database.

...another useful approach to keep in your development toolbox.

Contest for AD Admins

NetVision is running a contest for AD administrators. There's a relatively good chance to win $1000 if you can come up with a good caption for the cartoon provided. The idea here is simple. We're looking for some feedback and perspective from administrators. In exchange, we're giving away some cash. No tricks. Instead of paying for advertising that reaches the wrong audience half the time, we're paying the audience directly. Please pass it along to your AD admin friends.

Quick Reference Guides for Windows and AD admins

Active Directory UserAccountControl – Common values related to access rights.

Windows File System Permissions – As labelled in the Windows Security dialog with descriptions for both folders and files.

ADAM Pass Through Authentication to AD

A few years ago, I wrote a post that referenced ADAM passthrough authentication to Active Directory without providing much info. Since then, people searching on that topic have found themselves on my blog (probably hoping for more information). So, I thought I'd cover it in more detail.

Disclaimer:
I haven't actually done this in a few years, so my information may be out of date, but I'm sure someone will speak up if I'm wrong. ...they always do ;) I will assume you know what ADAM is, why you'd use it, where to get it, and how to install & configure it.

Why:
A few quick scenarios where pass through authentication is useful:

1. You want to put a portion of your Active Directory users into the DMZ for authentication by publicly-facing applications, but you don't want to expose an AD DC in the DMZ. In this scenario, the app can leverage a DMZ'ed ADAM for authentication. ADAM will still need to make a request to a DC, so AD is partially exposed, but in a more controlled way.
2. You want to leverage AD credentials for application authentication, but the app wants to store information about users that is not currently in AD and you don't want to extend the schema. You could stand up an ADAM instance, extend its schema however you want, enable passthrough authentication, and point the app to ADAM instead of AD.
3. You have an app that is used by people that have an AD account AND people that don't. And your app only accepts a single authentication store.

Here's what you need to know about using ADAM for pass through authentication to AD:

• The ADAM installation process gives you the ability to import the schema for UserProxy objects from a file called ms-userproxy.ldf – you'll need to import that object to enable the passthrough functionality. You can do it after the install if you need to.
• For an account to perform a passthrough authentication (aka a bind redirection) from ADAM, the account must be configured as a UserProxy object. A standard user account can not authenticate through to Active Directory.
• The UserProxy object has an attribute called ObjectSid, which is critical to this functionality. For passthrough authentication to work, the account's ObjectSid must be populated with the SID of the associated Active Directory user account (this will actually work with any security principal object).
• When a UserProxy account attempts to bind to the ADAM instance, ADAM recognizes the account as a proxy and forwards the authentication request to Active Directory.
• Passthrough authentication only works for accounts that live in the forest to which the ADAM server is joined (or to a trusted domain or forest). So, that's how ADAM knows where to send the request.
• Passthrough authentication only works with simple binds. So, the password is being passed to ADAM in clear text. You'll want to be aware of that and use SSL as appropriate.

That's pretty much all you need to know to get passthrough authentication working. As I recall, it's that simple.

Comments/Corrections:
PLEASE leave a comment if you're here because you want to do this for a reason I haven't mentioned above or if you have additional information. Am I wrong? Did I leave anything out?

More Information:
Microsoft refers to this as Bind Redirection for ADAM Proxy Objects. So, that's the terminology you'll want to use to find more information.

Managed Service for AD Monitoring

The engineers at NetVision were incredible visionaries back in 1995 when they built some of the first identity management tools (for Novell-Microsoft sync) and identity audit tools. (check out their site from 1999) Their directory knowledge is unparalleled and now, that spirit of leadership is alive again.

NetVision recently brought to market the industry's first managed service offering for monitoring and reporting of identity & access information on core network directory platforms (Microsoft Active Directory and Novell eDirectory) and their related file systems.

We have effectively removed all the typical obstacles - software, hardware, configuration, setup, maintenance, etc.. The solution is delivered via an appliance allowing us to handle the heavy lifting (while the data stays close to home):

• WE do the configuration to match your environment
• WE install, configure and lock down the OS
• WE install and configure the supporting software (database, report engine)
• WE install and configure the solution, including setup of policies, reports, and customizations
• WE handle all of the patch management and upgrades
• WE monitor the system for performance
• WE provide policy and report updates when needed

The only thing left for our customers to do is to enjoy the critical data that they wouldn't get anywhere else (and the extra time they now have on their hands).

Please say hello to SIMON.

Better data from Active Directory for your SIEM

If you Have or are Planning to Have:

• A SIEM solution (ArcSight ESM, RSA enVision, Novell Sentinel, IBM TCIM)
• An enterprise Log Management solution (LogLogic, TriGeo, SenSage)

And your employees log on to:

• Microsoft Active Directory / Windows
• Novell eDirectory / NetWare

And you're unhappy with the solution's ability to:

• Get complete information from the directory or file system
• Filter which information is collected
• Generate highly relevant alerts based on filtered event data and custom policies
• Collect event data directly from the source (independent of system logs)
• Apply decisions or alerts based on WHO is performing the action
• Report on ANY combination of objects and attributes in the directory
• Report on who is opening or modifying files, folders, or file system permissions

THEN ...Please give us a call.

I recently wrote a paper discussing how we (NetVision) extend the ability of SIEM or log management solutions by getting better, more reliable, and more relevant information directly from what is arguably your most critical source (the network directory). The paper isn't publicly available (it's not that kind of paper). So, let us know and we'll pass it along or we can save you the trouble of reading and just explain it.

Situational Awareness in Logs & Events

Anton Chuvakin put together a great list of reading on logs. Are they useful? Are they painful? And more. Included in the list is Michael Baum's brilliantly titled post Life after SIEM. Situational Awareness is next. Baum discusses SIEM technologies and the next evolution. I love his idea of bringing situational awareness into the equation. It's a great way to describe what happens when your monitoring solution does more than compile data. When it is intelligent.

At NetVision, one way that we're working to achieve intelligent monitoring is to limit our scope to the core network platforms. This is where your employees authenticate each morning and their entry point into the network. I see the network authentication as the launchpad into the network. And once you're in, you have potential access to systems and assets.In the real world, this usually means Microsoft Active Directory or Novell eDirectory (and their respective file systems). These core systems are incredibly strategic to overall information systems security. And I posit that they deserve more careful consideration than simple log scraping. Many of our customers agree. They feed our data into their enterprise SIEM or log management solution.

In many smaller organizations (SMEs), full-blown SIEMs and massive log management solutions may not be necessary. For them, full insight into the core network system (often Active Directory and Windows) provides answers to most of their security audit questions. This is especially true if Active Directory is considered strategic and is used by other systems for authentication or authorization (as is often the case with Sharepoint). And also true in environments that rely on Active Directory to feed accounts, attributes, or group memberships into a provisioning system.

To get back to the point, the more an organization leverages Active Directory strategically, the more valuable the concept of situational awareness can be within the monitoring solution. For us, it means having different business rules depending on event variables or policies that update in real time based on environmental changes.

I don't agree that SIEMs are dead. But organizations seem to want more than stockpiles of data. And it's extremely important to use context when processing events from core systems.

Ouch

Dave Kearns calls my argument smoke and mirrors and labels it FUD. His argument is that the Global 2000 have more users and are therefore more important? Should their needs drive solutions for the mid-market?

Dave, I don't think the number of users is even relevant. What is relevant is the experience of those customer organizations and how they can meet their requirements. The number of infrastructures is more relevant than the number of end users in this discussion. I don't think a huge amount of them have a need or desire for multiple user directories. They seem to run off of AD and seem to prefer to have apps leverage AD instead of figuring out how to use a virtual directory (or metadirectory for that matter). Where is the FUD in that? Where is the smoke? What would be my motive to raise smoke and mirrors?

The discussion of how should Oracle build a product is very different than whether customers should consider metadirectory as an alternative. I think they should. I think there are still plenty of environments that could benefit from that approach. But I conceded Clayton's point -- if Oracle wants to build a virtual directory into it's suite to enable flexibility for customers, that's great. I just don't think a virtual directory is the answer to everything (and I spent a lot of time discussing the various use-cases that cry for one).

I would just hate to have people shy away from a good technology because some people say it's no good anymore. That doesn't make sense.

Ultimately, we might agree. Dave's conclusion is one that I've echoed over and over:

The need for, and uses of, virtual directories is growing and is still a few years away from peaking.

Let's just not declare something dead because it no longer seems cool to the in-crowd. It's OK to take a pragmatic approach to whatever challenge your organization is facing. That's my point.

Metadirectories: What's left to say?

If you haven't been following the flurry of conversation since my post last week stating that metadirectories aren't dead, well you're in luck. We couldn't have asked for a better recap of the conversation than the one provided by Ian Yip (although I think he gave Nishant a bum rap on this one).

There were so many different angles explored that I'm not really sure where to start or what's left for me to say.

• I'll restate that I see perfect use-cases for both metadirectory and virtual directory. Now and in the near future. In the far future, there will probably be better ways to achieve the same goals.


• Also, it sounded like Clayton took my comments to mean that "everyone needs to be using Active Directory for everything", which was (I think obviously) not the intent. My point is that although the top 500 or 1000 companies may have a number of directories for various strategic uses, there are probably 20x that number of companies that use only Active Directory as the central and primary user store because of it's network and email integration. And those companies might like for their application vendors to offer direct plug-in to AD as an option.
  
  Plugin to LDAP might be another good option and virtual directory technology would be a great enabler to incorporate various vendors, schemas and even relational databases through that single mechanism. But those mid-market companies probably would prefer not to take on the complexity of virtual directory (even if relatively simpler than writing numerous connections) if they could just use AD natively. And I think some percentage of the Fortune 1000 would see AD as strategic enough to ask the same as well.
  
  That's my guess based on a customer perspective as opposed to a software vendor's ideal state of architecture. And I don't think this is limited to companies who are 100% Microsoft shops. AD just has a very far reach and because it holds email in most of those companies it will already have an account for every employee, be available, etc..
  
  I don't think any of this should be seen as threatening to the role that stand-alone directories or meta- or virtual- directories play. The difference in viewpoint between me and Nishant & Clayton (if I can group them together) might be in the types of customers we've been talking to. There are still a ton of companies out there that aren't super-strategic about their Identity Management architecture. Or that just want a point solution because it fits the current business needs.

I think that's it. For now.

[UPDATE] - forgot one:

• Bavo, I wasn't requiring that the HR database is the primary source for account creation and status. I also wasn't telling you that the HR database should be the primary source for Identity information. (However, I think it's more true than you think.) I was stating a requirement (one that I've seen many times). HR has been deemed THE authoritative source for employee existence in a majority of the companies I've worked with. My experience seems to differ from yours. [That's at least interesting! ...and one of the reasons I blog – to engage with people that have different experiences.]
  
  Yes, companies struggle with getting HR updated for the employee's start date. But, I've actually seen more than one customer implement a complicated AD-to-HR-back-to-AD process to accommodate for the issue. One customer integrated the candidate review system into the provisioning system. I think the reason for HR being authoritative is usually for deprovisioning. They want a disabled HR account to ripple downward.
  
  I think what you call the IDM system assumes a provisioning solution with work flow and its own internal store. These are luxuries that are not always available. In my scenario, the cost and complexity of a provisioning solution is probably overkill based on the requirements. And that's my point. There are scenarios where the simplicity of a metadirectory are not only sufficient to meet the requirement but actually a bit more of an elegant way to meet the requirements.

OK, now I'm really done for the night.

What to Monitor in Active Directory

If you manage an Active Directory infrastructure, you probably know that you should be monitoring activity or data or something. But what exactly needs to be monitored? Well, as I say in my latest paper, there is no one-size-fits-all prescription for Active Directory monitoring. But, there are five items that carry particular interest. In this paper, I go into detail on each of the five – what needs to be monitored and why.

For many mid-market organizations, these five may cover 80% of security monitoring needs – especially for organizations that are strategic about their use of Active Directory. As the title says, it's strictly limited to Active Directory, so don't look for firewall logs or changes to virus protection files. There's a short excerpt here if you'd like to take a peek.

If you're a security or Identity Management consultant, feel free to contact me directly and I'll be happy to send you a copy.

...The 5 Most Critical Points for Active Directory Security Monitoring