Weighing in on Persistent Cache

Ash, my experience says that Mark is correct. I believe the top vendors can all brag about similar throughput. But, my understanding is that's only what the VD puts on top of the process. There's still the back end data lookups, etc. To Mark's later point, that may not be a big deal either if those sources perform well.

Let's use a telecom example:

In a scenario where the VD serves an attribute that is a composite of multiple attributes from various sources (a mixed ODBC and LDAP call) or across numerous sources (customer databases from companies that merged or partner) and the attribute is needed to make a decision (does the subscriber get this feature) in real-time (the time between hitting "send" and hearing a ring) for millions of requests each second, effective use of cache can help – even though throughput is already relatively quick. In many (perhaps most) enterprise identity infrastructure uses, cache may not be of enormous value – or at least it's not the most compelling reason to use a Virtual Directory.

I can tell you that customers ask for it whether they need it or not. Probably because they have performance or availability concerns. But, from what I've seen the performance concerns are usually unfounded (unless the back end systems have serious problems). And VD cache isn't a great way to provide redundancy because it's implemented at an attribute level and cached based on use. If the idea is to put the entire data set somewhere else, you could argue that it'd be better to just have another directory instance there and do real-time synch (replication).

My opinion is that it's a nice feature to have in the tool bag when needed, but it's not always needed.

A few interesting Identity findings

User-Centric vs. Enterprise Identity

Dave Kearns offers a concise explanation of the core difference between user-centric identity and enterprise identity. His summary:

Enterprise-centric identity management is really all about tying together all the activities and attributes of a single entity into a readily accessible (and reportable and auditable) form. User-centric identity is about keeping various parts of your online life totally separated so that they aren’t accessible and no report can be drawn.

I like the simplicity of this explanation. I think it really captures the essence of the difference in an understandable way.


Management Profile

In this article from ComputerWorld, the Director of IS, strategy and architecture at Universal Service Administrative Co. is profiled. He talks about his current project:

An IAM framework will allow for customer information of applicants and contributors to remain consistent across IT platforms while spanning new and legacy systems and applications. My goal is to have one authoritative repository for contributors' and applicants' access information that will be used in managing a secure access control infrastructure. I believe that identity and access management will become an underpinning technology that IT leaders need to address.

He goes on to say that Identity Management is the most critical technology of the year. It's nothing earth shattering, but I always give priority to real customer insights.


Interesting Service Offering

I've discussed the idea of outsourcing identity and managed identity services, but CoreBlox, a company founded by ex-Netegrity folks, have this posted in their service offerings:

Dedicated CA SiteMinder Support Professional

It's an interesting twist on managed identity services and one that I think would resonate with customers. I've known a number of companies who would've liked to just outsource the identity support role to someone who knows what they're doing -- without having to hire and without having to pay for a full-time resource who sits around waiting for something to go wrong. One of the things I like about this is that CoreBlox isn't trying to provide a support professional for any identity system. They're focused on the technologies that they know.

So, if you had a provisioning solution from Courion or SAP and Siteminder for Web Access, you might need to go to two different people or companies to get the right support. BUT - that focus on core expertise is a recipe for success (especially in a support role). And likely still more cost effective than hiring, training, and retaining someone to support these complex systems.

Proliferation of Multiple LDAPs

I was questioned today by someone who read my post on Synchronization versus Virtualization, which is topical given the recent blog debates by many of the big names in identity blogging on whether metadirectories are dead. Back then, I was trying to convince the world that there are cases that require a virtual approach. Now, the tables seem to have turned. I've stayed quiet on that debate, but I believe there's still a place for each method. Honestly, I'm not sure why there's such intensity behind that debate – they're just tools. My idea for Enterprise Identity Services Architecture included a layer for Identity Data Services, which is analogous to the proposed Identity Hub or Bus. It has a box for both synchronization and virtualization because there are use cases for both types of technologies often within the same infrastructure.

End of digression.

The question proposed was about how large enterprises (hundreds of thousands of users) are addressing the proliferation of LDAP directories in their environments. The person asking comes from a well-known and reputable company. The first thing I thought is here's the exact reason why we still need metadirectories. Many companies still haven't dealt with the user-store sprawl. In a general sense, I think metadirectory technologies can help get you to a place where you refine and consolidate your infrastructure and can then leverage virtual directory technologies where appropriate for applications that need access to data in multiple stores and/or multiple formats.

What the questioner was really looking for is research that discusses how other companies have handled the challenge. Here's my answer:

My recommendation is to find an experienced group of consultants who have tried various approaches with different clients. There are a number of them out there and I'm happy to give recommendations privately based on geography, technology, comfort-level with small vs. large companies, etc.. Whether you engage them contractually or enter into discussions via user groups, trade shows, etc., real world experience is priceless. Anybody that pushes one particular approach to this problem is probably biased (either by product or by their own limited experience). My experience suggests that most scenarios require a unique approach based on business goals, actual technologies, future plans, etc.. But you probably need someone to spend some time understanding your own scenario before recommending an approach.

Some questions you'll want to explore:

• Which data stores have overlapping data and which are unique?
• Does it make sense to consolidate?
• Is the data mappable across systems? Do they share unique identifiers?
• Where can multiple applications share a single store?
• Where do given applications require access to data in multiple stores?
• What applications or uses are coming in the future?
• Which stores are used for critical apps? What is the up time demand?
• In what format is the data stored?

The answers to these questions will drive your architectural decisions and help you prioritize next steps.

If you'd like to comment and provide a pointer to field research, whitepapers, or contact info for how you can help with this situation, feel free.

Managed Identity Management Services

Identropy has brought to life a long overdue idea in presenting their managed service for identity management. As Corbin Links pointed out, it takes a significant skill set for companies to plan, deploy, and manage an identity management infrastructure. It's not always easy to find the right talent for any given project. And there are pitfalls in most identity management projects as Mark Dixon advises in his list of project success factors.

Identropy isn't the first to offer a managed service for identity -- I know Oracle/WiPro and probably others have tried it. But, Identropy is the first I've seen from a smaller services organization. In the services world, smaller generally means quick, flexible, and ready to respond to market needs and changes.

Forrester says we're ready for managed Identity services. They're betting on Mycroft-Talisen to successfully offer a managed service for identity. I don't know if they've brought it to market yet, but the combination of Mycroft and Talisen has the talent to make it happen.

Anybody have other experiences with managed/outsourced identity management offerings?

Time to rethink your IdM strategy?

"The unexamined life is not worth living"
- Socrates

"The unexamined Identity Management solution is not worth the investment"
- Me

While not quite as eloquent as the original, my point is that we should apply Socrates' theory to business and IT strategy just as we apply it to the way we live our lives. I gave a nod to a blog post a while back that described Identity Management as a lifestyle. Yesterday, in an article entitled CIO Jury: Businesses face ID management headache, the CIO Jury reminds us that identity management is a lifestyle. You can't set it and forget it like Ron Popeil. You need to make a plan, start to build, re-evaluate, launch some functionality, re-evaluate, build some more, alter the plan, evaluate what's coming next, etc.. There are certainly technology components that will run on their own without constant maintenance, but they live at a very tactical level. At the strategic level, it's important to continually re-think the plan. The business climate as well as the technology landscape are both dynamic in nature. Change is constant. Keep this in mind during planning and be prepared to switch gears if needed.

In a recent MaXware webinar, a customer described the process of backing out of a big investment in IdM and starting again with an entirely new product set. The webinar will be available on the MaXware site and it's definitely worth the viewing time. One important take-away for me was that you can forfeit a major investment that's not working, completely shift gears and come out successful. It takes some guts and some ingenuity, but it can pay off. The webinar presented an excellent example of the value of constant re-evaluation.

Another point I want to make is that while planning an Identity services architecture, it's vital to build an environment that is flexible and adaptable. Take a look at Identicentric's idBUS product. They've got the right idea. Build a flexible service-oriented middleware layer that enables you to quickly adapt the front end apps or the back end infrastructure to the ever-changing business requirements. I've also talked in the past about Virtual Directory as a data abstraction layer providing similar capability but at a data access level rather than an application access level. Think about these approaches and how else you might enable your Identity Services infrastructure to adapt quickly to changing requirements.

And rethink your Identity Management strategy. It's always a good time for that.

Burton Document on Identity Data Services

The Burton Group published a new research document today:

Enabling Identity Data Services: New Developments in Identity Tooling Provide a Good Start
You can find it under: Identity and Privacy Strategies

The article touches on many of the same concepts that I've been grappling with lately related to identity data services. They discuss the value of an identity interface layer. The concepts they present are more advanced than what I've been talking about, but I think they're based on the same underlying business drivers, which is encouraging. It's good validation. And I learned a few new things. It's well worth the read if you have a Burton account. Go check it out.

Starting Point for Identity Services

A while ago, I posted about the Identity Management Continuum. I want to revisit that topic from another angle. As we move toward a service-orientation and establish a more component-ized architecture approach, it becomes easier to insert, remove or swap any one particular piece of the puzzle. The continuum still exists - an Identity Services infrastructure is dynamic, should be implemented in phases and should be cyclically re-examined in accordance with business goals. But the starting point clearly varies based on the needs of the organization. For some companies, a particular set of functionality is important. They want to reduce the number of user passwords, provide self-service password recovery, provide a single sign-on service or user-enable a new application. For other companies, the goal is simply to start building an identity infrastructure. Either way, the solution design goals are the same - to implement an open and flexible identity services infrastructure so that components can be added, improved or removed as business needs change.

One solution that makes this architectural approach extremely accessible is an identity services data abstraction layer. An abstraction layer unties the bind between identity services and identity data. Identity data is typically stored in multiple locations, structures and formats throughout an organization. Mapping each identity service to all of those data repositories is a daunting effort. One way of building an abstract layer is by creating an enterprise directory that holds all of the organization's identity information. This can be effective if an organization knows all of the identity and application requirements for this uber-repository. Unfortunately, application requirements change. And data repositories change. And new services rise and fall. Due to the dynamic nature of the business environment, enterprise directories are difficult to manage and maintain. There's often a trade-off between enabling new services and just keeping up with status quo.

An alternative approach is to leverage virtual directory technology as a data abstract layer. Virtual directory technology not only effectively maps identity services to identity data to meet current requirements, it also enables the organization to rapidly adapt to changing business needs. Migrating data repositories to new formats or structures is a seamless experience for the consuming applications and services. Adding new services is easy because identity data is accessible in a single location via customizable views.

I've often heard people discuss virtual directory technology as cutting edge technology that's only adopted by advanced organizations that have well-developed identity infrastructures. I propose that we (as the IdM community) ought to be encouraging the businesses we serve to look at virtual directory technology in the earliest stages of design. Since our process is cyclical and our requirements are dynamic, it's extremely important to be able to adapt to ever-changing identity needs. Virtual directory is a no-brainer for this scenario. If implemented early enough, organizations can greatly reduce the workload of implementing and managing identity services each step of the way - say by 20% - 25%. Isn't that the portion of the effort associated with integrating identity services and applications with the underlying data?

Identity Services Drill Down

In my last post, I presented an overview illustration of Enterprise Identity Services. Here I'll drill down into each layer to provide some further explanation. For each layer, I'll present an overview, its interactions with other layers, current implementation landscape and emerging technologies.

Identity Services

The Identity Services industry has used many terms to describe the overall umbrella under which all (or most) of its products and services fit. Among them are Directory Services, Identity Management, and Identity & Access Management with Identity Management being the most widely used term. Since Identity Management has come to be used in many cases specifically for user provisioning and delegated admin... And since an identity platform typically provides a service out to the larger organization... And since the larger IT industry is moving toward a software-as-services model... it makes sense to me that we start using the phrase Identity Services to describe the umbrella under which our products and services fit.

The Identity Services layers are:

• Access and Policy Services
• User Services
• Identity Data Services
• Data Storage Services

Access and Policy Services

This is the layer at which policies and access permissions are enforced. As people attempt to access resources throughout an organization, this layer grants or denies access based on user identity information and access policies.

The Access and Policy services layer relies upon the identity data services and data storage layers to provide accurate and up-to-date information about users, rights and policies. The user services layer relies upon this layer to control access to its own services.

Commonly found in this layer are Web SSO and Enterprise SSO applications. But implementations vary widely. In the past, access management was handled individually by each individual application. It was difficult to manage and control. The ideal future-state scenario is to have a single centralized enterprise access control platform that every application would leverage. Most organizations today that have already begun to offer identity services are somewhere in the middle.

Emerging technologies in this layer include federation products and standards, network-based access control, user-centric solutions and standardized policy management infrastructures (XACML).

User Services

User services include software and services that enable business efficiency and functionality. Generally, services at this layer are driven by user interaction. Managers manage their employees, grant and deny access, perform attestation duties and run audit reports. System users manage their own information, reset and recover passwords and request access permissions.

The user services layer relies upon the access and policy layer to ensure that people can only perform operations to which they're entitled. It relies on the identity data services layer to achieve the proper view of the data it needs to act upon. And it obviously relies on the data storage layer to store the results of operations performed at this layer.

Typical applications found in this layer include user provisioning and deprovisioning systems, password reset and recovery applications, access management tools and audit reporting. The majority of traditional Identity Management and Identity Services projects and discussions are based in this layer.

Technology emerging in this layer includes advanced attestation reporting, improvements to provisioning, workflow, self-service and password management services and web service enablement.

Identity Data Services

The identity data services layer provides vital services to the overall identity services infrastructure. At the top two layers (access & policy services and user services), systems interact with data that resides on the data storage layer. Each system and application has its own set of requirements and its own data needs. This presents a great challenge for organizations implementing identity services. Identity data is spread out across organizations in network OS and email systems, HR systems, application databases, identity-specific directories and more. This layer provides identity services and applications access to those heterogeneous and disparate data stores in an organized and controlled manner.

The identity data services layer relies upon the access & policy layer to present valid credentials for the user, service or application requesting data access. It presents data out to the access & policy and user services layers but its primary interaction is with the data storage layer where its job is to act as a librarian of sorts. It catalogs identities across systems, joins accounts in multiple systems and presents the data in the format and structure requested by its consumer.

Metadirectories and data synchronization tools have traditionally ruled this layer. Many companies have made progress in identifying identity data throughout their organization and using synchronization products to create reliable sources for identity. Some companies create an enterprise directory that stores a common attribute set that is available to the identity services infrastructure for consumption. Others use synch tools to maintain the existing data stores. Typically, though, there are challenges associated with allowing applications and services to directly access your organizations data stores. Enter Virtual Directory technology. Virtual directories have been steadily proving their worth and will likely emerge as the dominant technology in the identity data services layer. A virtual directory can present a virtualized view of identity data out to the identity services infrastructure for consumption. The view can be customized based on the user or application making the request and can be transformed into multiple formats as appropriate for each consumption point.

Emerging technologies at the identity data services layer includes additional built-in connectors for meta- and virtual directories as well as adoption of emerging protocols and standards.

Data Storage Services

The data storage services layer is the simplest to understand. Organizations have had this layer in place for years. This layer holds the databases, directories and other data storage systems that hold identity data throughout the organization. Typically, there are many different types, structures and sizes of identity data stores in a single enterprise.

The data storage services layer may interact with each of the other layers to provide identity data as appropriate. In some cases, the data storage layer is masked by the identity data services layer so that the upper two layers do not directly interact with the data storage layer.

This layer is by far the most mature in terms of technology and adoption. LDAP directories and relational databases have proven their usefulness and are well understood amongst IT professionals. The implementations are as plentiful as the organizations that rely upon them.

Emerging technologies in this area include the adoption of new data storage standards, data virtualization (not to be confused with virtual directories), improved security & encryption and general feature & functionality improvements.

Enterprise Identity Services

A few months ago, I generated this view of the enterprise identity services landscape. The main goal of identity and access management solutions within an enterprise is to manage users' and applications' access and interactions with corporate assets.

There are a number of service layers that comprise the overall identity infrastructure that enable, restrict and audit this access. Each layer may be comprised of multiple technologies. As I've written in the past, Identity solutions will be implemented as services as we move into the future. Each layer should be built on open protocols that facilitate easy communication across layers. Companies should store and manage data in whatever format they're most comfortable. And employees should be empowered to perform their job functions without technical restrictions.

The layers of enterprise Identity Services as I see them are:

• Access and Policy Services
• User Services
• Identity Data Services
• Data Storage Services

Note: updated 7/25 with some minor enhancements

Identity Management for SOA

From my soon-to-be-released MaXware whitepaper titled Identity Management in a Service Oriented Architecture:

As companies continue to deploy identity management solutions throughout 2006 and 2007, they should do so with an eye toward their enterprise SOA strategy. Organizations that implement identity management solutions will benefit most over the long term when IdM solutions are offered as a service. Companies are beginning to move toward Service Oriented Architecture models and adoption is already gaining momentum. IdM will be a critical component of SOA infrastructure securing service access to applications as well as access to the services themselves. Applications will be looking to access identity data throughout the organization over standard service protocols like DSML and SPML. MaXware is uniquely positioned to meet the current and future needs of organizations looking to build, manage and secure an SOA infrastructure.

There is certainly a significant amount of hype about SOA, but that doesn't necessarily mean that it's only hype. SOA done right has the potential to greatly reduce application integration costs, which has been an albatross for many large organizations. I expect a gradual adoption over the next few years as software vendors provide SOA hooks into their software. I don't really see companies rolling out enterprise-wide SOA infrastructure, but as more apps become service oriented, communication between platforms will become much easier to facilitate. Organizations will obviously need to consider security for the SOA infrastructure but they should also expect that Identity Management solutions (provisioning systems, identity data stores, etc) should be available as a service over standard protocols. If you're interested in the whitepaper, check our web site -- it should be available within the next few days.