Instant Enterprise REST Accelerates the Software Driven Business

Software Driven Business is a consensus goal.  But real challenges exist: the time, cost and complexity of building such apps is substantial.  Business Agility – and strategic business advantage – is lost.

We need another revolution – Instant Enterprise REST – that provides Business Agility using business-level specifications rather than low-level code, and delivers Enterprise-class scalability, integration, enforcement and extensibility.  It’s now a reality with Instant Enterprise REST.

Software Driven Business: Consensus Vision

Businesses have seen the value in providing mobile and tablet apps that bring the business into the hands of customers and employees.  They provide information at their finger tips – wherever they are.

Industry Leaders like CA have pioneered the vision of  a Software Driven Business.  They argue persuasively that strategic business advantage lies in Time to Market and Time to Decision:

“reveal the need for speed in the application economy.  As companies transform into software-driven enterprises, bringing high-quality applications to market faster becomes one of the most critical differentiators.”

The Business Agility Gap

While there is consensus around this vision, there is a substantial gap in realizing the Software Driven Business.   It centers around Agility – time to market.  As CA argues, this drives strategic business advantage.

This problem manifests both to Business Users and IT, although differently.  You might have been party to a discussion like this:

Business Users are frustrated about how long it takes to create systems, and revise them.  They see problems that look nearly as simple as a spreadsheet take weeks… to months.

How can it months for IT to build a system that takes days on a spreadsheet?

IT is no less frustrated.  They understand the deep technology it takes to build Enterprise-class systems:

We’re working 90 hours a week.  And falling behind.

Gap Analysis

For apps about critical corporate data, there’s general consensus that the time and cost for such systems are about evenly split between backends and front ends.  And there’s nearly universal consensus that, independent of the UI technology, that RESTful APIs deliver the backend data.

But the backend is far more than basic data access.  A “SQL Pass-through” – simply restifying SQL data – does not meet Enterprise-class requirements to scale, integrate and enforce:

• Scale – APIs require Pagination to address large result sets, Nested Documents to reduce latency, Optimistic Locking to ensure concurrency.  These are not provided in a simple SQL Pass-through – you must program them, by hand.
• Integrate – a wizard can produce an API from schema objects, but it cannot address multiple databases, or integrate non-SQL data sources such as ERP, other RESTful services, or NoSQL.
• Enforce – an API needs to enforce our security (down to the row level), and the integrity of the data.  These are significant tasks, which are sadly often placed in client buttons where they cannot be shared.

Providing these Enterprise class services takes significant time, expertise and expense.  Business Agility is reduced.

IT is essentially being forced to cover inadequate technology infrastructure.  The Business Users are right: if the Business Specification is clear, then that ought to be enough:

A clear business specification should be sufficient.  Everything else is just friction.

The vision of the Software Driven Business requires Business Driven Software that pre-supplies the infrastructure.  We are not seeking 10 or 15%.  We are looking for orders of magnitude.  Our vision must be:

We should be able to create RESTful APIs (mainly) from business specifications, not low level code.

It should be no more difficult to create a system than it is toimagine it.

Business-Driven Software: Instant Enterprise REST

Business Driven Software is more than just a clever play on words.  It’s a real implementation that delivers this vision, and we call it Instant Enterprise REST.

It consists of 3 core technologies:

1. Enterprise Pattern Automation – creates APIs that with Enterprise-class scalability built-in (pagination, nested documents, optimistic locking, etc)
2. Declarative – specify your API, integration and enforcement policies with spreadsheet-like rules in a simple point-and-click UI
3. Extensibility – enables the RESTful APIs to invoke your existing logic, inside or outside the JVM, via standard server-side JavaScript.

The combination of these 3 technologies enables you to create RESTful APIs for database backends – half your system – 10 times faster.  Let’s briefly examine them below.

Technology 1: Enterprise Pattern Automation

There are well known patterns in the data domain, describing data structure and access via SQL.  There are also well-known patterns for managing SQL data in the context of RESTful services.

Well known patterns can be automated.  Let’s imagine a service (say, a server accessed via a browser) that automates these patterns, as described below, just by connecting the service to a database:

• Schema Discovery – tables, views, stored procedures: The system creates a complete (default) API for each schema object.  Note this includes Stored Procedures, which often represent a significant investment.
• Enterprise Pattern Automation: the resultant API provides well-known services for Filter, Sort, Pagination, Optimistic Locking, handling Generated Keys and so forth.

So, the service has provided a default Enterprise-class API, instantly.  So, literally seconds into your project, you can test your running API:

Not enough, not done, but a great start.

Technology 2: Declarative

Declarative is the key (“what, not how”).  It has had striking impacts on domains where there are well-understood underlying patterns.  Max Tardiveau has put it well:

Whatever can be declarative, will be declarative.

For example, spreadsheets are declarative – and they gave birth to the PC industry.  And SQL is declarative – itself an industry.  Two game-changers.

So, the challenge is to apply the spirit of declarative to REST integration and enforcement.  The stakes are high – success can deliver breathtaking agility.

Declarative Integration: Multi-Database Custom API, Point and Click

Enterprise Pattern Automation provides a good start, but the API is not rich.  It is a flat, single-table API, really just “restified” SQL.  What we really need is

• Nested Documents – returning multiple types (e.g., an Order, a list of Items, and a list of contact names) in a single call can reduce latency (vs. a separate call for each type).  REST is perfect for this.
• Multi-database APIs – a RESTful server provides the opportunity to integrate multiple databases in single call, shielding clients from underlying complexity.

Nested Documents are easy: define them by simply selecting tables (via a User Interface or Command Line).  Foreign Keys are used to default the joins.  Add the ability to choose / alias columns, and we’re on the way to a pretty good API.

But what about databases that have no Foreign Keys?  Or multi-database APIs?

Leveraging the schema does not mean we are limited to it.  All we need to do is:

• Provide a means to define “Virtual” Foreign Keys for the service (i.e., stored outside the schema)
• Extend this to Foreign Keys between databases

We now have a rich, multi-database API.  Defined declaratively as shown below, no code required, running in minutes, ready for client development:

Declarative Enforcement: Integrity Logic, with spreadsheet-like rules

So now consider enforcement, specifically database integrity.  A very significant portion of any project is the multi-table validations and computations that define how the data is processed.

“Your code goes here” means, well, a lot of code.  We need a more powerful, more declarative, paradigm.

In a spreadsheet, you assign expressions to cells.  Whenever the referenced data is changed, the cell is updated.  Since the cells references can chain, a series of simple expressions can solve remarkably complex problems.

What if we did the same for database data?  We could assign derivation expressions to columns, and validation expressions to tables.  Then, the API could “watch” for requests that change the referenced column, and recompute (efficiently) the calculated column.  Just as in a spreadsheet, support for chaining and proper ordering is required and implicit.

To address multi-table logic, such expressions would need to address references to related tables.  It’s only at this point that the logic becomes seriously powerful.

Let’s take an example.  To check credit in a Customer  / Purchaseorder / Lineitem application, we could define spreadsheet-like expressions such as:

There is actually a sub-branch of declarative that addresses this: Reactive Programming.  Here it’s declarative,since you don’t need to code a Observer handler.

The result is that the logic above can be fully executable.  No need to code Change Detection / Change Dependency – it’s invoked and enforced automatically by the API in reaction to RESTful updates.  SQL handling is also implicit, including underlying optimizations (caching, pruning etc).

The impact is massive – the 5 expressions above express the same logic as hundreds of lines of code.  That’s a massive 40X more concise.  Game changer.

And quality goes up, since the rules are applied automatically.

Declarative Enforcement: Security, filter expressions for role/table

We can provide an analogous approach to security: define filter expressions for roles (like SalesRep), so that when a table is accessed by the role, the API adds the filter.  That way, a user with that role sees only the rows for which they are authorized.

Technology 3: Standards-based Extensibility

Declarative is great, but you’re probably thinking “ok, but you can’t solve every problem declaratively”.  And you’re dead right.

Business Value requires that we integrate a declarative approach with a procedural one that is familiar, standards-based, and enables us to integrate existing software.

Automatic JavaScript Object Model

The first phase of many projects is to build an ORM for natural programmatic access to data: JPA, Hibernate, Entity Framework.  It’s not a small project, and cumbersome to maintain as changes occur.

In fact, the Object Model can be created directly from the schema.  So, you’d have an object type for Purchaseorder, for Lineitem, and so forth.  The model provides access to attributes and related data, and persistence services.  You could then use it as shown below.

JavaScript seems like the best language choice: reasonable across technology bases (everybody uses JavaScript), and its dynamic nature eliminates code generation hassles.

JavaScript Events

In addition to accessors and persistence, the JavaScript objects are Logic Aware.  That is, the save operation above executes any rules associated with OrderAudit (e.g., updated-by), and JavaScript Events.

Here is a sample event for the PurchaseOrder object, where you access the JavaScript Object Model via the system-supplied row variable:

Extensible Logic

Auditing is a common pattern.  It should be possible to solve this once in a genericmanner, then re-use it (e.g, to audit employees, orders and so forth).

So, Instant Enterprise REST should enable you to provide Extensible Logic – load your own JavaScript code, and invoke it.  So, the code above could become:

where auditFromTo creates an instance of OrderAudit, sets the foreign key, sets like-named attributes, and saves it.

Pluggable Authentication

Most organizations have existing data stores that identify users and their roles, such as Active Directory, LDAP, OAuth, etc.  Security should integrate with such systems as a function of enforcing row/column access.

Standard deployment

Finally, the system should deploy in a familiar manner: available on the cloud, or an on-premise virtual appliance or war file.  Standards also enable integration with related critical infrastructure, such as API Management, ERP Systems, etc.

See a project in 3 minutes

To see how it all fits together, you can view this video to see a full project built: from concept, through initial implementation, and an iteration cycle.  Actual project time was about half an hour.

Instant Enterprise REST: Business Agility

Instant Enterprise REST enables us to close the Agility Gap in realizing the Software Driven Business vision.  We can now create important portions of our software in largely business terms, rather than technical terms.

This offers major advantages:

• Time to Market: spreadsheet-like rules are 40X more concise.  Instant REST eliminates all the SQL / REST / JSON boilerplate.
• Simplicity: team members can learn the basics of Espresso in days, and be as productive as rocket scientists using alternative technologies
• Leverage Expertise and Software: Espresso is built on standards like REST, JavaScript, and Event Oriented Programming.  You can call out to existing software, and extend the rule types by identifying your own patterns and loading their implementations into Espresso.
• Quality: at the defect level, automatic invocation and ordering eliminate large classes of bugs.  At the architectural level, centralized enforcement factors logic out of the client buttons where it can be shared, audited for compliances, etc