Ultrafast Application Development With MicroStream and PostgreSQL Integration Using Jakarta EE

Integrating MicroStream and PostgreSQL, leveraging the new Jakarta EE specifications known as Jakarta Data, presents a powerful solution for developing ultrafast applications with SQL databases. MicroStream is a high-performance, in-memory object graph persistence library that enables efficient data storage and retrieval. At the same time, PostgreSQL is a widely used, robust SQL database system known for its reliability and scalability. Developers can achieve remarkable application performance and efficiency by combining these technologies' strengths and harnessing Jakarta Data's capabilities.

This article will explore the integration between MicroStream and PostgreSQL, focusing on leveraging Jakarta Data to enhance the development process and create ultrafast applications. We will delve into the key features and benefits of MicroStream and PostgreSQL, highlighting their respective strengths and use cases. Furthermore, we will dive into the Jakarta Data specifications, which provide a standardized approach to working with data in Jakarta EE applications, and how they enable seamless integration between MicroStream and PostgreSQL.

By the end of this article, you will have a comprehensive understanding of how to leverage MicroStream, PostgreSQL, and Jakarta Data to build high-performance applications that combine the benefits of in-memory storage and SQL databases.

Facing the Java and SQL Integration Challenge

The biggest challenge in integrating SQL databases with Java applications is the impedance mismatch between the object-oriented programming (OOP) paradigm used in Java and the relational database model used by SQL. This impedance mismatch refers to the fundamental differences in how data is structured and manipulated in these two paradigms, leading to the need for conversion and mapping between the object-oriented world and the relational database world.

Java is known for its powerful OOP features, such as encapsulation, polymorphism, and inheritance, which enable developers to create modular, maintainable, and readable code. However, these concepts do not directly translate to the relational database model, where data is stored in tables with rows and columns. As a result, when working with SQL databases, developers often have to perform tedious and error-prone tasks of mapping Java objects to database tables and converting between their respective representations.

This impedance mismatch not only hinders productivity but also consumes significant computational power. According to some estimates, up to 90% of computing power can be consumed by the conversion and mapping processes between Java objects and SQL databases. It impacts performance and increases the cost of cloud resources, making it a concern for organizations following FinOps practices.

MicroStream addresses this challenge with its in-memory object graph persistence approach by eliminating the need for a separate SQL database and the associated mapping process. With MicroStream, Java objects can be stored directly in memory without the overhead of conversions to and from a relational database. It results in significant performance improvements and reduces the power consumption required for data mapping.

By using MicroStream, developers can leverage the natural OOP capabilities of Java, such as encapsulation and polymorphism, without the need for extensive mapping and conversion. It leads to cleaner and more maintainable code and reduces the complexity and cost of managing a separate database system.

In the context of a cloud environment, the reduction in power consumption provided by MicroStream translates to cost savings, aligning with the principles of the FinOps culture. Organizations can optimize their cloud infrastructure usage and reduce operational expenses by minimizing the resources needed for data mapping and conversion.

Overall, MicroStream helps alleviate the impedance mismatch challenge between SQL databases and Java, enabling developers to build high-performance applications that take advantage of OOP's natural design and readability while reducing power consumption and costs associated with data mapping.

While addressing the impedance mismatch between SQL databases and Java applications can bring several advantages, it is vital to consider the trade-offs involved. Here are some trade-offs associated with the impedance mismatch:

• Increased complexity: Working with an impedance mismatch adds complexity to the development process. Developers need to manage and maintain the mapping between the object-oriented model and the relational database model, which can introduce additional layers of code and increase the overall complexity of the application.
• Performance overhead: The conversion and mapping process between Java objects and SQL databases can introduce performance overhead. The need to transform data structures and execute database queries can impact the overall application performance, especially when dealing with large datasets or complex data models.
• Development time and effort: Addressing the impedance mismatch often requires writing additional code for mapping and conversion, which adds to the development time and effort. Developers need to implement and maintain the necessary logic to synchronize data between the object-oriented model and the relational database, which can increase the development effort and introduce potential sources of errors.
• Maintenance challenges: When an impedance mismatch exists, any changes to the object-oriented model or the database schema may require mapping and conversion logic updates. This can create maintenance challenges, as modifications to one side of the system may necessitate adjustments on the other side to ensure consistency and proper data handling.
• Learning curve: Dealing with the impedance mismatch typically requires understanding the intricacies of both the object-oriented paradigm and the relational database model. Developers must have a good grasp of SQL, database design, and mapping techniques. This may introduce a learning curve for those more accustomed to working solely in the object-oriented domain.

It is essential to weigh these trade-offs against the benefits and specific requirements of the application. Different scenarios may prioritize various aspects, such as performance, development speed, or long-term maintenance. Alternative solutions like MicroStream can help mitigate these trade-offs by providing a direct object storage approach and reducing the complexity and performance overhead associated with the impedance mismatch.

Enough for today's theory; let’s see this integration in practice. It will be a simple application using Java, Maven, and Java SE. 

The first step is to have an installed PostgreSQL. To make it easier, let’s use docker and run the following command:

docker run --rm=true --name postgres-instance -e POSTGRES_USER=micronaut \
           -e POSTGRES_PASSWORD=micronaut -e POSTGRES_DB=airplane \
           -p 5432:5432 postgres:14.1

Ultrafast With PostgreSQL and MicroStream

In this example, let’s use an airplane sample where we’ll have several planes and models that we’ll filter by manufacturer. The first step of our project is about the Maven dependencies. Besides the CDI, we need to include the MicroStream integration following the MicroStream relational integration, and furthermore, the PostgreSQL driver.

<dependency>
   <groupId>expert.os.integration</groupId>
   <artifactId>microstream-jakarta-data</artifactId>
   <version>${microstream.data.version}</version>
</dependency>
<dependency>
    <groupId>one.microstream</groupId>
    <artifactId>microstream-afs-sql</artifactId>
    <version>${microstream.version}</version>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.2.14</version>
</dependency>

The second step is to overwrite the configuration for using a relational database. First, create DataSource, and then we’ll inject it and then use it on the StorageManager.

@ApplicationScoped
class DataSourceSupplier implements Supplier<DataSource> {

    private static final String JDBC = "microstream.postgresql.jdbc";
    private static final String USER = "microstream.postgresql.user";
    private static final String PASSWORD = "microstream.postgresql.password";

    @Override
    @Produces
    @ApplicationScoped
    public DataSource get() {
        Config config = ConfigProvider.getConfig();
        PGSimpleDataSource dataSource = new PGSimpleDataSource();
        dataSource.setUrl(config.getValue(JDBC, String.class));
        dataSource.setUser(config.getValue(USER, String.class));
        dataSource.setPassword(config.getValue(PASSWORD, String.class));
        return dataSource;
    }
}


@Alternative
@Priority(Interceptor.Priority.APPLICATION)
@ApplicationScoped
class SQLSupplier implements Supplier<StorageManager> {

    @Inject
    private DataSource dataSource;

    @Override
    @Produces
    @ApplicationScoped
    public StorageManager get() {
        SqlFileSystem fileSystem = SqlFileSystem.New(
                SqlConnector.Caching(
                        SqlProviderPostgres.New(dataSource)
                )
        );
        return EmbeddedStorage.start(fileSystem.ensureDirectoryPath("microstream_storage"));
    }

    public void close(@Disposes StorageManager manager) {
        manager.close();
    }
}

With the configuration ready, the next step is to create the entity and its repository. In our sample, we’ll make Airplane and Airport as entity and repository, respectively.

@Repository
public interface Airport extends CrudRepository<Airplane, String> {
    List<Airplane> findByModel(String model);
}

@Entity
public class Airplane {
    @Id
    private String id;
    @Column("title")
    private String model;
    @Column("year")
    private Year year;

    @Column
    private String manufacturer;
}

The last step is executing the application, creating airplanes, and filtering by the manufacturer. Thanks to the Jakarta EE and MicroProfile specifications, the integration works with microservices and monolith.

public static void main(String[] args) {

try (SeContainer container = SeContainerInitializer.newInstance().initialize()) {
    Airplane airplane = Airplane.id("1").model("777").year(1994).manufacturer("Boing");
    Airplane airplane2 = Airplane.id("2").model("767").year(1982).manufacturer("Boing");
    Airplane airplane3 = Airplane.id("3").model("747-8").year(2010).manufacturer("Boing");
    Airplane airplane4 = Airplane.id("4").model("E-175").year(2023).manufacturer("Embraer");
    Airplane airplane5 = Airplane.id("5").model("A319").year(1995).manufacturer("Airbus");
    Airport airport = container.select(Airport.class).get();
    airport.saveAll(List.of(airplane, airplane2, airplane3, airplane4, airplane5));
    var boings = airport.findByModel(airplane.getModel());
    var all = airport.findAll().toList();
    System.out.println("The boings: " + boings);
    System.out.println("The boing models avialables: " + boings.size());
    System.out.println("The airport total: " + all.size());
}
System.exit(0);
}

Conclusion

In conclusion, the impedance mismatch between SQL databases and Java applications presents significant challenges in terms of complexity, performance, development effort, maintenance, and the learning curve. However, by understanding these trade-offs and exploring alternative solutions, such as MicroStream, developers can mitigate these challenges and achieve better outcomes.

MicroStream offers a powerful approach to address the impedance mismatch by eliminating the need for a separate SQL database and reducing the complexity of mapping and conversion processes. With MicroStream, developers can leverage the natural benefits of object-oriented programming in Java without sacrificing performance or increasing computational overhead.

By storing Java objects directly in memory, MicroStream enables efficient data storage and retrieval, resulting in improved application performance. It eliminates the need for complex mapping logic and reduces the development effort required to synchronize data between the object-oriented model and the relational database.

Moreover, MicroStream aligns with the principles of FinOps culture by reducing power consumption, which translates into cost savings in cloud environments. By optimizing resource usage and minimizing the need for data mapping and conversion, MicroStream contributes to a more cost-effective and efficient application architecture.

While trade-offs are associated with impedance mismatch, such as increased complexity and maintenance challenges, MicroStream offers a viable solution that balances these trade-offs and enables developers to build ultrafast applications with SQL databases. By leveraging the power of Jakarta Data specifications and MicroStream's in-memory object graph persistence, developers can achieve a harmonious integration between Java and SQL databases, enhancing application performance and reducing development complexities.

In the rapidly evolving application development landscape, understanding the challenges and available solutions for impedance mismatch is crucial. With MicroStream, developers can embrace the advantages of object-oriented programming while seamlessly integrating with SQL databases, paving the way for efficient, scalable, and high-performance applications.

• Source: MicroStream Integration on GitHub