Lifecycle Microservices With GenAI Tools

Lifecycle Development With AI

We have seen a huge shift in the way developers and consultants are using Generative AI (GenAI) tools to create working microservices. A new tool named WebGenAI begins the process with a simple prompt to create a complete API microservice with a running React-Admin user interface and has the ability to iterate and add new features or even logic.

WebGenAI is built on top of the existing Python open-source framework ApiLogicServer. The entire project can be downloaded as runnable Python code or a Docker container to use locally. It also pushes each iteration to GitHub and you can run the application using Codespaces. This is usually where the beginning of the full microservice lifecycle starts.

Greenfield Project (Ideation)

When a new project is started, it usually begins with paper documents that "explore and explain" the scope and direction, use cases, and workflows of the project. WebGenAI takes a prompt/model-driven approach to explore and visualize ideas. The ability to "iterate" over prior prompts to include new functionality helps the stakeholder and SME capture basic functionality.  While this generative approach will never be the final release, each iteration will help get the project closer to the vision and capture requirements in real time. WebGenAI, shown below, begins with a simple prompt: "Create a dog-walking business".

The result is a complete running application with React-Admin pages, sample data, and the ability to download the entire source to explore locally or run from GitHub Codespaces.

Connect an Existing SQL Database

The WebGenAI tool also has a feature called ConnectDB to prompt for an SQL database (or Excel Workbook), but this is intended to be used in conjunction with the local Docker version or in-house cloud deployment. Not many enterprises will want to put their corporate database credentials into a public website. However, using the local Docker version, WebGenAI can take advantage of an existing database schema to build a complete running application and create an API (based on JSON API) for each selected table. The application that is created will allow the stakeholder to visualize data and navigate between parent/child relationships. While iteration is possible, this would not be the main use case for GenAI.

Local Installation

ApiLogicServer is an open-source project built on Python 3.12 based on SQLALchemy ORM, Flask, and SAFRS/JSON API.  Once your virtual environment is ready, a simple installation using pip will include SQLAlchemy ORM 2.x, Flask, LogicBank, and other libraries to start and run your downloaded project locally. (Note: WebGenAI also has a running Docker version to skip the local installation). In this example, Python and VSCode have already been installed.

python -m venv venv
source venv/bin/activate
(venv)pip install ApiLogicServer
cd myDownloadedProject
code .
# Press F5 to start and run in VSCode

The Developer Journey

WebGenAI will let you see your prompt and project come to life: the actual open-source code can then be downloaded to a local development platform to use with your own IDE (VSCode, PyCharm, IntelliJ, etc). Once you download the project, you can use ApiLogicServer to organize it into folders (Note: An IDE and Python installation and configuration are required to run locally):

• api - Expose API endpoints and define custom API endpoints.
• config - Defines the project variables
• database - Defines the SQLAlchemy ORM 
• logic - Declarative rules (derivations, events, and constraints)
• security - Declarative role-based access control
• devops - Scripts to build and deploy Docker containers
• test  - Behave testing-driven tools
• ui - Both react-admin and Angular applications (using Ontimze from Imatia)

Natural Language Logic

Business Logic is a critical component of any microservice. Explore tools like OpenAPI, ChatGPT, and Copilot to see if they can take advantage of LogicBank, an open-source rules engine to generate declarative rules (e.g. sum, count, formula, copy, and constraints). This is very similar to working with a spreadsheet in 3 dimensions (e.g., rows/columns and parent/child tables). Code completion based on the model and Copilot integration makes the developer experience very friendly.

It is amazing to see Copilot in the IDE turn business user statements like these: 

Enforce the Check Credit requirement (do not generate check constraints):
1. Customer.balance <= credit_limit
2. Customer.balance = Sum(Order.amount_total where date_shipped is null)
3. Order.amount_total = Sum(Item.amount)
4. Item.amount = quantity * unit_price
5. Store the Item.unit_price as a copy from Product.unit_price

. . . into LogicBank Rules:

  Rule.constraint(validate=models.Customer,       
        as_condition=lambda row: row.Balance <= row.CreditLimit,
        error_msg="balance ({round(row.Balance, 2)}) exceeds credit ({round(row.CreditLimit, 2)})")

    Rule.sum(derive=models.Customer.Balance,      
        as_sum_of=models.Order.AmountTotal,
        where=lambda row: row.ShippedDate is None and row.Ready == True) 

    Rule.sum(derive=models.Order.AmountTotal,      

    Rule.formula(derive=models.OrderDetail.Amount, 
        as_expression=lambda row: row.UnitPrice * row.Quantity)

    Rule.copy(derive=models.OrderDetail.UnitPrice, 

Declarative logic sits between the API and SQLAlchemy ORM/SQL database. This allows all API CRUD use cases to be handled consistently and new rules added or changed without having to worry about the order of operations (this is done using a runtime DAG to manage the order of operation) much like a spreadsheet. 

Integration Events

ApiLogicServer explains that this is a 40x improvement over writing logic code by hand. Logic is applied to Attributes (derivations) or Entities (events and constraints). If the logic is a bit more complex, a Python function can be called to complete the processing. ALS can make calls to a Kafka producer, external API systems, or use any of Python libraries (e.g. math, Stripe, Email, etc.) in the function.

If the microservice needs to interface with external systems like Kafka, email, payment system, or business process models like GBTec, the rules engine has a series of event types (early, row, commit, or flush). For example, a flush event is called after all the logic rules have been fired and the data has been written to the ORM (returning auto-increment keys). This would be the point: to call an external system and pass a pre-configured payload. These events act like webhooks attached to specific API entities to integrate other systems (e.g., notify order shipment, update payment processing, send an email message, start a business process).

    def send_order_to_shipping(row: models.Order, old_row: models.Order, logic_row: LogicRow):
        """ #als: Send Kafka message formatted by OrderShipping RowDictMapper

        Format row per shipping requirements, and send (e.g., a message)

        NB: the after_flush event makes Order.Id available.  Contrast to congratulate_sales_rep().

        Args:
            row (models.Order): inserted Order
            old_row (models.Order): n/a
            logic_row (LogicRow): bundles curr/old row, with ins/upd/dlt logic
        """
        if (logic_row.is_inserted() and row.Ready == True) or \
            (logic_row.is_updated() and row.Ready == True and old_row.Ready == False):
            kafka_producer.send_kafka_message(logic_row=logic_row,
                                              row_dict_mapper=OrderShipping,
                                              kafka_topic="order_shipping",
                                              kafka_key=str(row.Id),
                                              msg="Sending Order to Shipping")
            
    Rule.after_flush_row_event(on_class=models.Order, calling=send_order_to_shipping)

Custom API Endpoints

This is another one of the key features of ApiLogicServer.  Not only will it expose the ORM entities as JSON API endpoints, but the developer can create custom API endpoints to perform specific tasks.  

One example of this is the Ontimize Angular User Interface that is created from the generated API. Ontimize would normally send HTTP requests to their own Java/Swing server.  By exposing a custom endpoint, ApiLogicServer acts like a bridge for all GET, PATCH, POST, and DELETE and returns a formatted JSON response to Ontimize. So all the Ontimize UI features are now supported without having to rewrite the front-end framework.  

Authentication and Declarative Security

Security is easily added using the command line tools. ApiLogicServer offers a local Keycloak Docker container (pre-configured). An alternative is to use the “sql” provider type to run a local SQLite authentication database and model. Other authentication models can be easily added (e.g., LDAP, Active Directory, OKTA, or OAuth).

$als add-auth --provider-type=keycloak --db-url=localhost

The KeyCloak configuration settings are stored in the config/config.py file and can be easily changed for test and production deployment.

RBAC: Role-Based Access Control

Security can be enabled at any point in the development process.  Once enabled, the developer will need to create roles to be assigned to the various users. Roles declare general CRUD access to API endpoints (read, insert, update, and delete). A user can have one or more roles, and specific grants can be added to modify role access to an API which includes row-level security and tenancy filters.

DefaultRolePermission(to_role = Roles.tenant, can_read=True, can_delete=True)
DefaultRolePermission(to_role = Roles.employee, can_read=True, can_delete=False)
DefaultRolePermission(to_role = Roles.customer, can_read=True, can_delete=False)
DefaultRolePermission(to_role = Roles.sales, can_read=True, can_delete=False)
DefaultRolePermission(to_role = Roles.public, can_read=True, can_delete=False)

GlobalFilter(   global_filter_attribute_name = "Client_id", 
                roles_not_filtered = ["sa"],
                filter = '{entity_class}.Client_id == Security.current_user().client_id')
              

GlobalFilter(   global_filter_attribute_name = "SecurityLevel",  
                roles_not_filtered = ["sa", "manager"],
                filter = '{entity_class}.SecurityLevel == 0')

#############################################
# Observe: Filters are AND'd, Grants are OR'd 
#############################################
GlobalFilter(   global_filter_attribute_name = "Region",  # sales see only Customers in British Isles (9 rows)
                roles_not_filtered = ["sa", "manager", "tenant", "renter", "public"],  # ie, just sales
                filter = '{entity_class}.Region == Security.current_user().region')
        
GlobalFilter(   global_filter_attribute_name = "Discontinued",  
                roles_not_filtered = ["sa", "manager"],         
                filter = '{entity_class}.Discontinued == 0')
        
Grant(  on_entity = models.Customer,
        to_role = Roles.customer,
        filter = lambda : models.Customer.Id == Security.current_user().id,
        filter_debug = "Id == Security.current_user().id")     
        
Grant(  on_entity = models.Customer,
        to_role = Roles.sales,
        filter = lambda : models.Customer.CreditLimit > 300,
        filter_debug = "CreditLimit > 300")    

Command Line Tools

The lifecycle of any project involves change. The API design may introduce new tables or columns, the ORM data model can change, the UI components need new lookup relationships or a new special API can be introduced. ApiLogicServer offers several command line tools and options to “rebuild-from-database” or “rebuild-from-model”.  These commands can rebuild the ORM or the UI components. There are also GenAI commands that can be used to create new applications from prompts.

Debugging API and Rules

Using VSCode on a local desktop allows the developer to run the microservice and place breakpoints to explore rules and custom API endpoints. This is a must-have for any developer learning a new system to see and understand what is going on and how to fix issues. A nice feature is the ability to link directly to GitHub for each iteration and run the project using GitHub Codespaces.

Behave API Testing

This style of test-driven development (TDD) begins with a feature and scenarios that need to be tested. The implementation is a simple Python program that breaks the scenario steps down into simple instructions.  While I have not tried to generate these tests with Copilot, this may be a nice future feature request. 

Feature: Salary Change

  Scenario: Audit Salary Change
     Given Employee 5 (Buchanan) - Salary 95k
      When Patch Salary to 200k
      Then Salary_audit row created

  Scenario: Manage ProperSalary
     Given Employee 5 (Buchanan) - Salary 95k
      When Retrieve Employee Row
      Then Verify Contains ProperSalary

  Scenario: Raise Must be Meaningful
     Given Employee 5 (Buchanan) - Salary 95k
      When Patch Salary to 96k
      Then Reject - Raise too small

Lifecycle With GitHub 

All of the components from ApiLogicServer can be checked into GitHub (models, logic, configurations, tests, UI components, etc.) to support multi-developer teams.  

DevOps Deployment

The ApiLogicServer directory, devops, provides a series of directories to build an image, use NGINX, and deploy Docker containers.  This takes DevOps a long way down the road to making the project visible on a cloud or on-premise server.

User Interfaces 

ApiLogicServer provides a react-admin UI (seen in WebGenAI) that allows exploration and navigation of data. There is also an Ontimze Angular application (from Imatia) which provides a more full-featured UX developer experience. Both of these are created from a yaml model file which can be edited and the pages regenerated from the yaml. Ontimize gives the UI/UX team a pre-configured and ready-to-run set of pages for all CRUD operations for every API Endpoint. Ontimize is a mature extensible Angular framework including charts, PDF reports, maps, and editable grids. Ontimize will introduce TypeScript, Node.js, and NPM into the development stack, but the look and feel of an Angular application will move the UX closer to production.

Summary

WebGenAI starts the process of prompting, iteration, and generation of an API microservice. Once the base model is ready, the developer team takes over to modify the model, add logic and security, write tests, create a custom UI, and build and deploy Docker containers to the cloud. While this is not exactly low-code, this is a rich platform of integrated services that make the development of a running microservice a new standard combining AI generation with open-source platform tools. You can try a limited web version by providing your GitHub or Google information. Your entire application can be downloaded as a Docker container or a complete source library running the ApiLogicServer framework.