Types of Transformation

There are as many different kinds of transformation projects as there are companies considering them – every project has its own special issues based on the state of the application at the start, the business drivers, and the desired goal. But, we can make some broad categorizations in order to understand something about some broad categories of approach.

Two approaches are characterized by modest, short term investment and focused goals.

Spot Changes: In some cases, the application as a whole may not need transformation, at least not immediately, and one can resort to a sharply focused solution. A spot change will not modernize the application or gain the agility benefits of a modern architecture, but can be a pragmatic, short term solution to an immediate problem. Spot changes are not appropriate for general architectural shifts, but can be used when there is an isolated function needing new capabilities. However, such localized modifications only perpetuate any architectural issues which exist and may even make them worse. One example of a spot change is adding an isolated web services component without moving the application to a message orientation in any other way.

Stepwise Transformation: Alternatively, one can consider one of several types of stepwise transformation in which a general architectural goal is determined and then applied on one problem area at a time, typically beginning with those having the highest ROI benefits, but continuing systematically until much or all of the application has been transformed over a number years. One variation on this approach might involve secondary transformations of selected subsystems when a significant number of the components have received their initial transformation, e.g., stepwise transformation to a Service-Oriented Architecture and then creating a new user interface technology for a subsystem when it is largely converted to services. This approach requires a higher budget commitment than merely continuing with normal maintenance, but as the application is gradually modernized, the maintenance benefits of SOA are gradually realized.

There are two primary strategies for a more complete, short term transformation:

Single Focus Complete Transformation: In this approach, some key element needing transformation is identified and the application undergoes a major rewrite to transform this one area, possibly with secondary attention to other areas. The result can be a substantially altered application, e.g., moving from a character to graphical interface, but the system has not been fully modernized or re-architected since it is basically a component for component transformation. In full modernization, the components of which the application are composed often change significantly from that in legacy applications, not just in internal implementation, but in overall packaging.

Full, Model-based Transformation: Alternatively, there is full model-based transformation in which one extracts business rules from the existing code, supplemented with traditional Object-Oriented Analysis and Design (OOAD) techniques, to create a full OOAD model of the application and then one generates a complete new application from this model using Model-Driven Architecture (MDA) techniques. The result of this transformation is fully modern in all respects and provides a mature, top quality application which can continue to evolve rapidly based on the model driven approach. While it is the most extensive change of any approach, it has the potential for significant efficiencies and leveraging through the use of tools and thus potentially can take less effort and time than a single focus transformation.

Obviously, there are many variations on these basic categories. One might point to a project in which the UI was replaced and separated into its own layer and SOA features were added and ask how can that be considered a Single Focus Transformation. But, in response, one can ask whether the data access layer was also separated, whether the SOA changes were as complete as one would have done with a “from scratch” design, and whether the components are largely equivalent between the two or have been rethought based on modern principles. In most cases, there are still significant compromises in the result of a single focus transformation compared to what one would have produced designing the application today according to modern principles and that is the reason I make the contrast. This compromise is understandable since such transformations can be extremely expensive and each company has to make its choices about what changes are important for its business drivers. But, I also think it is important to recognize that the optimum design has not been achieved.

These four broad categories cover a very wide range of expense and complexity. They also cover a wide range of the degree to which the application becomes transformed from a very local change to near re-invention. Each applies to a particular type of business need and capability and each has a different impact. Thus, one must select a strategy which fits the business drivers, level of need, type of need, budget, available skills, long term goals, and many other factors.

UPCOMING: Next time I will talk about some of the ways in which can choose amongst these options.

For services related to this material, see my website.

More About Business Drivers

Business drivers for Transformation come in many different flavors and the perception of them often changes during the needs assessment process. At the start of the assessment process, a need may be obvious and well-recognized by much of the company or it may be an obscure problem, perhaps recognized by only one person and it may not have occurred to that person better software could fix anything. Drivers may be common to many companies or very idiosyncratic to a specific company and its own business processes. And, while it is perhaps most common to think of business drivers in term saving existing costs or improving efficiency, there are also drivers which relate to increasing revenues or creating new business opportunities. One also often thinks of business drivers in terms of their size, either relative to the impact on the business or the perceived difficulty of a solution.

Among the business drivers which are quite often the trigger for considering transformation are those which are obvious and big. Some common examples are:

• Changing the user interface of some or all of the application to a more modern form or to a web interface for remote access;
• Integrating multiple systems in a point to point fashion, which becomes increasingly burdensome as more systems are added. This is common in companies which are growing by acquisition;
• A demand for substantial supply chain integration, which might come from a shift in business practice in the industry. This category first expressed itself significantly with the move to EDI, but more recently there is a demand for more real-time and complete integration, often through web services.

While this type of big obvious driver is easily recognized, it is also the kind of driver which can cause a company to despair since it is so overwhelming in scope and difficulty. At the same time, they can also be drivers which have substantial financial impact on the company and thus are worth the effort and cost. However, one has to look at each driver and each company individually since mere obviousness does not necessarily imply a real business need. Moving to a more modern user interface is one example of a very obvious possible driver that may or may not have any real business justification for a particular company or where there may be a real justification that only applies to a small part of the application.

Drivers which are specific to the company and application are more difficult to talk about in a general way. Typically, though, they are cases where either

• A business process has been deeply hard-wired into the software and now that business process needs to change; or
• Where the process itself hasn’t necessarily changed, but industry expectations about the speed with which that process must operate have changed dramatically and the mechanisms used to implement the process are no longer adequate. There are many dramatic examples of this kind of shift in the financial sector such as the move from three day settlement of stock transactions to near real time resolution.

This type of driver is often characterized by movement from satisfaction to dissatisfaction gradually over a period of time as that which was once adequate becomes less and less so until it becomes an acute business problem. In the most extreme cases, particularly if there are regulatory issues involved, this type of driver can be the primary one instigating consideration of transformation.

Another large group of drivers might be characterized as pervasive issues which are typically under-recognized. One example is code which has undergone years and years of patches upon patches with no overriding architectural guidance and which has gradually become very brittle and difficult to maintain. This condition of brittleness is often accepted simply as “the way things are” without the recognition that the current state is not only much worse than it used to be, but also that substantial cost savings could be realized if conditions were improved. Similarly, with the change in the pace of business change, the time required to make changes for new business processes or rules, which may have been perfectly adequate some years ago, now is too slow to keep up. Some of this time requirement can come from outdated development processes, but a lot often comes from the application architecture itself. One of the key goals in modern architectures is “nimbleness”, the ability to make changes quickly, confidently, and with little effort.

Another common opportunity which is partly of this last type, but which is more likely to be recognized by the individual business is having an application architecture which is inherently monolithic and centralized, but a business which has evolved to be geographically distributed, perhaps even globally. Especially if there is a need to keep systems running when communication lines are down or if there are individuals who are only irregularly connected, moving from a monolithic structure can imply dramatic changes in application architecture.

A common theme among the major business drivers for transformation is that the scope of the change or the nature of the change required is very different than the routine maintenance operations which have constituted the day to day management of the application in the past. This can easily lead to a sense of panic since not only is there no existing budget for such work, but the apparent scope of the project can greatly exceed any previously undertaken. This issue is underscored when the staff and management have no experience in the type of work involved, e.g., a long history working with character user interfaces and a sudden need to create graphical or web applications, possibly not even using ABL.

UPCOMING: Next time I will talk about different types of transformation as a first step in understanding how to meet this seemingly overwhelming challenge.

For services related to this material, see my website.

Business Drivers for Transformation

Because of the atmosphere of comfortable inertia around legacy ABL applications, they often experience a sort of benign neglect – small, short-term, immediate projects get attention and larger scope issues never quite make it onto the table. A developer might bring up one of these more general issues in response to something in an on-line forum or conference. Or an astute manager might be aware of the issue from prior experience or reading. But, even when such issues surface, the pressure of immediate work or the uncertainly of the correct response or the absence of a budget often causes them to be neglected. While the specific stimulus that triggers a company to consider some form of transformation varies from company to company, in most companies there are other issues lurking below the surface which may or may not be currently visible. Some of these are specific to the business of the company and others are likely to be present in many similar companies.

Before making transformation … or replacement … decisions, it is important to make all issues about the existing application visible because looking at the whole can easily change the decision. For example:

• Looking at all issues might reveal that the triggering stimulus isn’t the most important issue in terms of business need;
• There are often synergies with other needs that will substantially alter the ROI from any given change or lower costs over the long term; or
• One transformation strategy might seem to be indicated based on the main stimulus, but another one might be preferred if a larger set of issues is considered together.

Occasionally, a company will have a well-documented list of these other issues, but all too often the needs and issues are not visible, if only because one has to have some expectation of action to bother putting them on a list. If a need seems vague, or difficult to address, or expensive, it may never make it to such a list.

As a part of this review of business drivers, it is important to examine the hopes and needs and opportunities for the future. Knowing a requirement will exist one or two or even five years into the future allows present decisions to be made with that need in mind. This allows more knowledgeable selection among alternatives, searching for synergies, and planning for incremental change which can be both less disruptive and less expensive. Thinking about the future isn’t just about what one is going to have to do, but also about what might be possible. It is quite common for there to be opportunities to do something new which have far greater impact on the business than the things which are mandatory.

Thus, the first step in any transformation process is really an analysis of how the existing software is and isn’t supporting the business and what opportunities exist for changes in the software that would benefit the business. While this analysis has technical implications, it isn’t really an analysis about technology, but an analysis of business issues and opportunities. The results of such an analysis can be quite surprising, especially if the company is not used to thinking this way.

For example, if the trigger was dissatisfaction with the legacy ChUI interface, one might find that there were actually few good business reasons to pay for a change to GUI or WUI beyond making people “feel good”. That doesn’t mean that one shouldn’t look at ways to transform the interface, but it may mean, especially if budgets are limited, that there is another area which is more important in the short term. It might also mean that, instead of focusing on changing the UI, one might adopt an overall strategy such as a gradual move to a service-oriented architecture (SOA) because there are multiple projects with short term ROI that can benefit from SOA … and … moving to SOA will gradually make it easier to change the UI.

This is an example of how looking at the whole set of business issues together can lead to a strategy that is different and better than one would adopt focusing only on the initial trigger issue.

UPCOMING: Next time I will talk about some of the kinds of issues which may be discovered by a review of this kind.

For services related to this material, see my website.

Why Do Companies Start Transformation Projects?

While development shops in companies with OpenEdge applications are as varied as development shops everywhere, there is a tendency for companies with OpenEdge applications to minimize their expenditures on computing. Ironically, this characteristic derives from the low cost of ownership of OpenEdge. Since it is rare for a company to require a full time DBA, as many databases need, the DBA role is a part time job for one of the developers. Since one can typically “load and go” to upgrade to a new version of Progress, unlike the re-engineering for version incompatibilities typical in many languages, there is no budget item for version changes. Since many applications just run and run without much attention, there is no budget for upkeep. Since it is typically inexpensive to make small new reports or screen changes, companies will often keep minimal staff and will resist the idea of expanding to make changes which don’t have an immediate and obvious benefit. Thus, the combination of long term upward compatibility, low cost of ownership, and ease of modification means that there are many companies who have OpenEdge applications of considerable age which have been very inexpensive to own and which often are doing a good job in supporting the business since years and years of small modifications have tailored the software to the company’s specific business needs.

As long as this situation lasts, companies will rarely instigate any meaningful transformation. Possibly a developer whose schedule isn’t too pressed might undertake a small cleanup project in relationship to a module that needs work or might undertake to write a new module using some new techniques, but the vast bulk of the application is likely to remain unchanged. Moreover, it is an extremely common pattern that new work is done in imitation of the architecture that already exists. This can be as explicit as actually copying old code and editing it for a new purpose or it can be just looking at how tasks are done in existing code as a model of how to accomplish similar tasks in new code. Not infrequently, this practice is even enforced by management under the belief that it is more economical or efficient than trying to implement anything new. This practice does mean that the application retains its consistency, but it also means that flaws and limitations written into the system years and years ago are persisted into new code where there is no need.

Given this comfortable, self-reinforcing inertia and low cost, how is it that companies ever decide to do any kind of transformation? Often, this comes in the form of some kind of crisis … and, even if it doesn’t start out looking like a crisis, it often turns into one. Perhaps there is a need to web-enable parts of the application for access by remote users, customers, or vendors and the current application is not only not web-enabled, it is still character interface. Or, maybe it isn’t web screens that are required, but web services for supply chain integration with customers or vendors. Often such demands come from competitive pressures or from having a supply chain partner that is much larger and thus in a position to dictate terms of business. Perhaps it is just new management who takes one look at the character interface of the existing software and announces that it is time for something modern, even if it takes buying a new package and spending heavily to fit it to the business as well as the current package fits.

While the user interface is a common stimulus of the decision to consider some kind of transformation, there are many other possible sources … anything which clashes with the existing architecture of the application will have a similar effect. For the user interface it is the fact that older OpenEdge applications almost universally have the user interface, business logic, and data access all mushed together so that it is very difficult to change any one piece. Another common stimulus is a company that grows from working from a single location to having many locations, possibly geographically diverse, and wanting to manage them as a whole while providing the ability for each location to continue to operate even if communication lines break down. Or, perhaps there is some logic change in a core business process due to changes in business practice or regulatory changes and one discovers that this logic which needs changing is imperfectly duplicated in multiple parts of the application.

The pace of business is changing rapidly in the modern world and processes which used to be handled by month end reconciliations are now often required in days, hours, or even minutes. Old, paper-oriented batch processes that were adequate for the slower pace are unmanageable in a faster paced world, but changing these processes from batch to real time and imposing automated, intelligent management of those processes is typically not possible by making only simple changes to the old application. The whole structure of the application needs to change, at least in the key areas.

Because of the pattern of comfortable inertia previously discussed, companies often don’t have the internal resources for handling this type of change. They often have only enough staff to barely keep pace with a small stream of enhancement requests. Management often has no background or experience with modern architectures and tools. They may have long experience with the OpenEdge product line, but often the tools and techniques which they know are the ones used to originally construct these applications in the 1980s or early 1990s, not the tools and techniques which are available in OpenEdge today. Not knowing what to do, makes the crisis considerably worse than it might otherwise be, since there is no background for understanding when a small, focused change will be sufficient and when something more drastic is required.

Thus, it is easy for companies facing this sort of crisis to decide that they are going to have to start over with a new application which covers the capabilities that are now perceived as needed. But, this is an extremely expensive decision, not only because new software must be purchased, but because one has to go through the cost and disruption of making the change, and one needs to figure in the expense of modifying the new application so that it becomes as well tuned to the business as the current one already is. But, if one doesn’t understand what kind of transformations are possible and has no experience managing such larger projects, it can seem simpler to start over again than to fix what one has. However, this can be a very expensive choice and can easily have risks and downsides which are not apparent from looking at flashy demos.

Do you have stories about the crisis your company is facing? If so, please contact me as I would like to collect stories about what drives companies to consider transformation and possibly I can help you face the crisis.

UPCOMING: Next time I will talk some about the business analysis which should precede any technology decision.

For services related to this material, see my website.