DZone Spotlight

Horizontally scalable data stores like Elasticsearch, Cassandra, and CockroachDB distribute their data across multiple nodes using techniques like consistent hashing. As nodes are added or removed, the data is reshuffled to ensure that the load is spread evenly across the new set of nodes. When deployed on bare-metal clusters or cloud VMs, database administrators are responsible for adding and removing nodes in a clustered system, planning the changes at times of low load to minimize disruption to production workloads. In the world of applications running on container-orchestration platforms like Kubernetes, containers can be terminated or restarted due to frequent deployments, scheduling issues, node failures, network issues, and other unexpected changes. In this article, we’ll dive into how applications maintaining a distributed state can keep track of nodes entering and leaving the cluster. Workload Management Applications on Kubernetes run as containers inside pods. Since managing individual pods for distributed applications is cumbersome, Kubernetes provides higher-level abstractions that manage the pod for you. Deployments are suitable for running stateless applications like web servers, where all pods are equal and indistinguishable. No pod has a particular identity and can be freely replaced. StatefulSets are better suited for stateful applications, where pods are expected to have a distinct identity and are not freely interchangeable. StatefulSets are the appropriate workload management choice for our particular example of a data store application. Additionally, creating a Kubernetes Service will expose the distributed application to clients as a single, named network application. Tracking Pod Lifecycles Within a Service You have a StatefulSet up and running with a Service that clients can talk to, which finally brings us to the main problem of tracking pod creation: deletion and modification in real-time. Kubernetes assigns a stable hostname to each pod in a StatefulSet. A major drawback to using the hostname to address pods directly is that DNS results are cached, including results of failed requests made against unavailable pods. Enter the EndpointSlices API, which provides a scalable tracking of network endpoints within a Kubernetes cluster. With the appropriate selectors, this can be used to track the lifecycle of pods within a specific Service, as long as the Service has a selector specified. Specifically, the application can invoke the List and Watch API endpoints. First, the application creates a Kubernetes API client. Go import ( ... discovery "k8s.io/api/discovery/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/watch" "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" ... ) func main() { ctx, cancel := base.InitContext() defer cancel() // Create a kubernetes API clientset k8sRestConfig, err := rest.InClusterConfig() if err != nil { log.Fatalf("error getting in-cluster config: %v", err) } k8sClient, err := kubernetes.NewForConfig(k8sRestConfig) if err != nil { log.Fatalf("error creating k8s client: %v", err) } Next, the application invokes the List endpoint to retrieve the currently available backends. Note that the app label is the value of the Service selector label. This ensures that the List and Watch APIs target the pods in our specific Service. Go appLabel := "your-service-label" podNamespace := "your-service-namespace" // List the current service endpoints. endpointSlice, err := k8sClient.DiscoveryV1().EndpointSlices(podNamespace).List(ctx, metav1.ListOptions{ LabelSelector: appLabel, Watch: true, }) if err != nil { log.Fatalf("error listing endpoint slices: %v", err) } Now that the application has the current set of addresses, it can monitor for changes from this state by invoking the Watch API. This is done in a background loop to allow the application to proceed with the rest of its work. The List API provides a checkpoint called ResourceVersion that we can instruct the Watch to start observing events from. The Watch API pushes the results into the ResultChan channel. Go // ResourceVersion is like a checkpoint that we're instructing the Watch to start from. resourceVersion := endpointSlice.ResourceVersion go func() { // Start a watch on the service endpoints watch, err := k8sClient.DiscoveryV1().EndpointSlices(podNamespace).Watch(ctx, metav1.ListOptions{ LabelSelector: appLabel, Watch: true, // Start watching from the appropriate resource version ResourceVersion: resourceVersion, }) if err != nil { log.Fatalf("error watching endpoint slices: %v", err) } // Loop until the context is done for { select { case <-ctx.Done(): break case event := <-watch.ResultChan(): handleWatchEvent(watch); } } }() // Start the server / do other work } The application can then handle the Watch events appropriately. Go func handleWatchEvent(event watch.Event) { // Cast the event into an EndpointSlice event endpointSlice, ok := event.Object.(*discovery.EndpointSlice) if !ok { log.Fatalf("unexpected event object") } // From https://pkg.go.dev/k8s.io/apimachinery/pkg/watch#Event // Object is: // * If Type is Added or Modified: the new state of the object. // * If Type is Deleted: the state of the object immediately before deletion. // * If Type is Bookmark: the object (instance of a type being watched) where // only ResourceVersion field is set. On successful restart of watch from a // bookmark resourceVersion, client is guaranteed to not get repeat event // nor miss any events. // * If Type is Error: *api.Status is recommended; other types may make sense // depending on context. switch event.Type { case watch.Added: // Handle Added event case watch.Modified: // Handle Modified event case watch.Deleted: // Handle Deleted event case watch.Error: // Handle Error event } } It's important to note from the docs that the Added and Modified events send the full new state of the object. Therefore if the application is maintaining the endpoint addresses internally, they don't need to be incrementally updated. The final step is to ensure that your application has the permissions to call the EndpointSlices API resources YAML apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: server-endpoints-role rules: - apiGroups: ["discovery.k8s.io"] resources: ["endpointslices"] verbs: ["list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: server-endpoints-rolebinding subjects: - kind: ServiceAccount name: server roleRef: kind: Role name: server-endpoints-role apiGroup: rbac.authorization.k8s.io Conclusion The EndpointSlices Kubernetes API provides a scalable, extensible, and convenient way to monitor pod lifecycle events in a Kubernetes Service in real-time, enabling seamless load balancing in stateful distributed applications.

Have you ever wondered what happens when you send a message to friends or family over the Internet? It’s not just magic — there’s a fascinating technology at work behind the scenes called WebSocket. This powerful protocol enables real-time communication, allowing messages to flow seamlessly between users. Join us as we dive deeper into the world of WebSocket! We’ll explore how this technology operates and even create a simple application together to see it in action. Get ready to unlock the potential of real-time communication! What Is a WebSocket? WebSocket is a communication protocol that provides full-duplex communication channels over a single, long-lived connection, which means we can transfer data in both directions simultaneously. Unlike traditional HTTP requests, where a client sends a request to a server and waits for a response, WebSocket allows both the client and server to send and receive messages independently and concurrently. This is achieved through a persistent connection that remains open for real-time data exchange. For this blog, we are going to use Jakarta, the enterprise edition of Java, to implement WebSocket. Before we dive deeper into WebSocket, let's take a look at Jakarta EE. What Is Jakarta EE? Jakarta EE (formerly Java EE) is a set of specifications that extend the Java SE (Standard Edition) with a collection of APIs for building enterprise-grade applications. It provides a robust framework for developing scalable, reliable, and secure applications that can run on various servers and cloud platforms. WebSockets in Jakarta WebSockets in Jakarta EE offer a powerful and efficient way to enable real-time communication between clients and servers. Jakarta EE's WebSocket API simplifies the process of building real-time applications by providing a robust framework for managing WebSocket connections. With its event-driven model and seamless integration with other Jakarta EE technologies, developers can create interactive, responsive applications that enhance user engagement and experience. WebSocket Protocol Let's jump back to WebSocket and learn about the WebSocket protocol. The WebSocket protocol is designed to provide full-duplex communication channels over a single TCP connection, making it ideal for real-time applications. Here are the key aspects and fundamentals of how the WebSocket protocol works: 1. Establishing a Connection Handshake Process The connection starts with a handshake initiated by the client through an HTTP request. This request includes specific headers indicating that the client wishes to establish a WebSocket connection. Server Response If the server supports WebSocket, it responds with a status code 101 (Switching Protocols) and confirms the upgrade. 2. Data Framing Messages After the connection is established, data is transmitted in the form of messages. Each message can be text (UTF-8) or binary data. Frames WebSocket messages are divided into frames. Each frame contains a header that includes information like whether the frame is a final frame or if it’s part of a larger message, as well as the payload length and masking information. 3. Message Types Text frames: These frames contain UTF-8 encoded text messages. Binary frames: Used for binary data, such as images or files Control frames: These include frames for closing the connection and ping/pong frames for keep-alive functionality. 4. Closing the Connection Close Frame Either the client or server can initiate the closing of the WebSocket connection by sending a close frame, which includes a status code and an optional reason for the closure. Acknowledgment Upon receiving a close frame, the other party must respond with its own close frame to confirm the closure. 5. Keep-Alive Mechanism Ping/pong frames: To maintain an active connection and check if the other party is still connected, WebSockets can use ping/pong frames. The server sends a ping frame, and the client responds with a pong frame, helping to keep the connection alive. 6. Security Considerations Secure WebSockets (WSS): Similar to HTTPS for HTTP, WebSocket connections can be secured using the WSS protocol, which encrypts the data transmitted over the connection using TLS. Setting Up Jakarta WebSocket To better understand WebSocket, let's build a small app that implements WebSocket. In this project, we are going to use Open Liberty, which is an open-source Java application server designed to support enterprise Java applications developed by IBM. Setting up Jakarta WebSocket on Open Liberty is straightforward, as Open Liberty supports Jakarta EE APIs, including Jakarta WebSocket. 1. Install JDK 11 or above. 2. Set the JAVA_HOME environment variable. For example: Linux/macOS: export JAVA_HOME=/path/to/jdk Windows: set JAVA_HOME=C:\path\to\jdk 3. Download Open Liberty: Go to the Open Liberty starter page. Choose the appropriate release and download the .zip file. 4. Extract the files: Extract the downloaded .zip file to your preferred directory. 5. Update server.xml file in liberty/config with the WebSocket configuration: XML <featureManager> <feature>jakartaee-10.0</feature> <feature>microProfile-6.1</feature> <feature>webProfile-10.0</feature> <feature>websocket-2.1</feature> </featureManager> 6. Create the WebSocket Java Class: In src/main/java/com/openLibertyWebsocket/rest, create a class named ChatEndpoint.java: Java package com.openLibertyWebsocket.rest; import jakarta.websocket.OnClose; import jakarta.websocket.OnMessage; import jakarta.websocket.OnOpen; import jakarta.websocket.Session; import jakarta.websocket.server.ServerEndpoint; import java.io.IOException; import java.util.Collections; import java.util.HashSet; import java.util.Set; @ServerEndpoint("/chat") public class ChatEndpoint { // Maintain a set of all active WebSocket sessions private static final Set<Session> sessions = Collections.synchronizedSet(new HashSet<>()); @OnOpen public void onOpen(Session session) { sessions.add(session); System.out.println("Connected: " + session.getId()); } @OnMessage public void onMessage(String message, Session senderSession) { System.out.println("Received: " + message + " from " + senderSession.getId()); // Broadcast the message to all connected clients synchronized (sessions) { for (Session session : sessions) { if (session.isOpen()) { try { session.getBasicRemote().sendText("User " + senderSession.getId() + ": " + message); } catch (IOException e) { e.printStackTrace(); } } } } } @OnClose public void onClose(Session session) { sessions.remove(session); System.out.println("Disconnected: " + session.getId()); } } This Java class, ChatEndpoint, defines a WebSocket server endpoint for a simple chat application using the Jakarta WebSocket API. This allows clients to connect to the /chat endpoint, send messages, and broadcast messages to all other connected clients. Class Annotation @ServerEndpoint: @ServerEndpoint("/chat") indicates that this class is a WebSocket endpoint at the URL /chat. When a WebSocket client connects to ws://<server-address>/chat, it interacts with this endpoint. Session management with Set<Session>: sessions: A static, synchronized Set of Session objects, representing all active WebSocket connections. The Collections.synchronizedSet(new HashSet<>()) ensures thread-safe access, allowing concurrent modification when clients connect or disconnect. Methods for WebSocket Events: Each WebSocket lifecycle event has a corresponding method in this class, annotated appropriately to handle client connections, messages, and disconnections. @OnOpen: The onOpenmethod is called when a client establishes a connection. Adds the client’s Session to the sessions set, allowing the server to track active connections. Prints a message to the console confirming the client’s connection with its unique session ID. @OnMessage: The onMessagemethod is called whenever a connected client sends a message to the server. Receives the message as a String and the Session of the sender (senderSession). Logs the received message and the sender’s session ID for debugging. Broadcasting: Uses a synchronized loop to send the message to all active clients in the sessions set: If the session is open, the sendText method sends the message to that client. Adds a prefix (e.g., User <session-id>) to identify the sender in the message broadcast to all clients. @OnClose: The onClosemethod is called when a client disconnects. Removes the client’s Session from the sessions set. Logs a disconnect message with the session ID, helpful for tracking connections and debugging. The entire backend code is available in this repository. 7. Run the project using Maven: mvn liberty:dev 8. Test the chat application. Open two browser windows. Use the browser console as the WebSocket client (in Chrome, press F12 > Console tab). Connect each tab to the WebSocket server by running the following JavaScript. JavaScript const ws = new WebSocket("ws://localhost:9080/ws-blog/chat"); ws.onopen = () => console.log("Connected to chat"); ws.onmessage = (msg) => console.log("Message from server: " + msg.data); ws.onclose = () => console.log("Disconnected from chat"); // Send a message to the chat function sendMessage(text) { ws.send(text); } // Example: To send a message, type sendMessage("Hello from user!"); 9. Start chatting: In each console, use sendMessage("Your message") to send messages. You should see messages from both users appearing in both browser consoles, creating a live chat experience. 10. After testing the WebSocket application, we can proceed to create a chat user interface using React.js. Here is the sample code that we have used to create one. import React, { useEffect, useState, useRef } from 'react'; const Chat = () => { const ws = useRef(null); // Use useRef to persist WebSocket instance const [messages, setMessages] = useState([]); const [input, setInput] = useState(''); const [username, setUsername] = useState(''); const [isUsernameSet, setIsUsernameSet] = useState(false); useEffect(() => { // Initialize WebSocket connection only once ws.current = new WebSocket('ws://localhost:9080/ws-blog/chat'); // Handle incoming messages ws.current.onmessage = (event) => { const newMessage = event.data; // Remove random prefix if present const displayMessage = newMessage.replace(/User\s[\w-]+:\s/, ''); setMessages((prevMessages) => [...prevMessages, displayMessage]); }; // Clean up WebSocket on component unmount return () => { if (ws.current) { ws.current.close(); } }; }, []); // Function to send a message with the specified username const sendMessage = () => { if (ws.current && ws.current.readyState === WebSocket.OPEN && input) { const messageToSend = `${username}: ${input}`; // Use specified username ws.current.send(messageToSend); setInput(''); // Clear the input field } }; // Set the specified username const handleSetUsername = () => { if (username.trim()) { setIsUsernameSet(true); } }; return ( <div> <h1>WebSocket Chat</h1> {!isUsernameSet ? ( // Username input screen <div> <input type="text" value={username} onChange={(e) => setUsername(e.target.value)} placeholder="Enter your username" /> <button onClick={handleSetUsername}>Start Chat</button> </div> ) : ( // Chat interface <div> <div style={{ border: '1px solid #ccc', height: '300px', overflowY: 'scroll', marginBottom: '10px' }> {messages.map((msg, index) => ( <div key={index}>{msg}</div> ))} </div> <input type="text" value={input} onChange={(e) => setInput(e.target.value)} placeholder="Type a message..." /> <button onClick={sendMessage}>Send</button> </div> )} </div> ); }; export default Chat; React Using WebSockets This code defines a basic chat interface in React using WebSockets for real-time communication. The Chat component allows a user to set a username, send messages to a WebSocket server, and receive/display incoming messages. 1. WebSocket Reference (ws) ws is defined with useRef(null) to maintain a persistent reference to the WebSocket instance across renders. 2. useEffect Hook for WebSocket Connection useEffect initializes the WebSocket connection to ws://localhost:9080/ws-blog/chat when the component mounts. This is only run once due to the empty dependency array ([]). Message handling: When a message is received, it triggers the onmessageevent, where: newMessage captures the received message text. displayMessage uses a regex to strip any unwanted random user prefix (in the format User <random-identifier>: ), displaying only the intended message content. setMessages updates the messages array with the new message. Cleanup: When the component unmounts, the WebSocket connection is closed to free up resources. 3. sendMessage Function Called when the user clicks the "Send" button Checks if the WebSocket connection is open and the input field has text Formats the message with the username and input content, then sends it to the server Clears the input field 4. handleSetUsername Function Called when the user sets a username and clicks "Start Chat" If a non-empty username is entered, isUsernameSet is set to true, hiding the username input and displaying the chat interface. 5. UI Rendering If isUsernameSet is false, the component displays an input to enter a username and a "Start Chat" button. Once the username is set, the main chat interface is shown: Messages display: A scrollable div shows each message in the messages array. Message input: An input field and "Send" button allow users to type and send messages. The entire frontend code is available in this repository. Conclusion As we wrap up, you've now unlocked the basics of WebSockets and learned how to create a simple WebSocket application with Open Liberty. Here’s a quick recap of what we covered: 1. Understanding WebSockets We explored the WebSocket protocol, which enables real-time, bidirectional communication between clients and servers. Unlike traditional HTTP requests, WebSockets maintain a persistent connection that allows data to flow freely in both directions. 2. Setting Up Open Liberty You learned how to set up your Open Liberty environment and create a basic Jakarta EE application that uses WebSockets. We covered the necessary steps to prepare your project structure and configure the server. 3. Creating a WebSocket Endpoint We walked through the creation of a WebSocket endpoint using the @ServerEndpoint annotation. This included writing the server-side logic to handle client connections, messages, and disconnections. 4. Building the Client Application You gained insights into how to build a simple client application that connects to your WebSocket server. We discussed the JavaScript code necessary to establish a connection, send messages, and receive updates in real time. By mastering these fundamental concepts, you now have the tools to build interactive and dynamic applications that can engage users in real-time. WebSockets are a powerful feature of modern web development, and with your new knowledge, you can start creating applications that leverage their capabilities.

Log-Structured Merge Trees (LSM trees) are a powerful data structure widely used in modern databases to efficiently handle write-heavy workloads. They offer significant performance benefits through batching writes and optimizing reads with sorted data structures. In this guide, we’ll walk through the implementation of an LSM tree in Golang, discuss features such as Write-Ahead Logging (WAL), block compression, and BloomFilters, and compare it with more traditional key-value storage systems and indexing strategies. We’ll also dive deeper into SSTables, MemTables, and compaction strategies for optimizing performance in high-load environments. LSM Tree Overview An LSM tree works by splitting data between an in-memory component and an on-disk component: MemTable (in-memory component): A balanced tree structure that temporarily stores recent writes SSTables (on-disk component): Sorted String tables that store data permanently, organized in levels The basic operation flow is as follows: Writes are handled by the MemTable. When the MemTable exceeds a threshold size, it is flushed to disk as a sorted SSTable. Reads first check the MemTable, and if the key is not found, it searches through the on-disk SSTables. Background processes periodically merge and compact the SSTables to improve performance and manage disk space efficiently. Simple Key-Value Store: A Comparative Approach Before we dive into the complexity of LSM trees, it’s useful to understand a simpler approach. Consider a key-value storage system implemented in Bash: Go db_set () { echo "$1,$2" >> database; } db_get () { grep "^$1," database | sed -e "s/^$1,//" | tail -n 1; } This Bash-based system appends key-value pairs to a file and retrieves the most recent value for a key. While it works for small datasets, the retrieval process (db_get) becomes increasingly inefficient as the dataset grows since it performs a linear scan through the entire file. This simplistic approach highlights the challenges of scaling databases as data increases. The primary limitation of this method is that it lacks any indexing structure, leading to O(n) search times. It also doesn’t manage updates or deletions efficiently, as old entries are retained in the file, and the entire file must be scanned for the latest version of each key. To address these issues, databases like LSM-trees introduce more sophisticated data structures and mechanisms for sorting and merging data over time. LSM Tree Golang Implementation To implement an LSM tree in Golang, we design a StorageComponent that combines an in-memory balanced tree (MemTable) with SSTables on disk. This structure allows for efficient handling of both reads and writes, as well as background processes like compaction and data merging. Java type StorageComponent struct { memTable BalancedTree ssTableFiles []*SSTable sparseIndex map[string]int config Config wal *WAL bloomFilter *BloomFilter compressor Compressor } type Config struct { MemTableSizeThreshold int CompactionInterval time.Duration TreeType string BlockSize int The StorageComponent includes the following: MemTable for fast in-memory writes SSTtables for persistent storage SparseIndex and BloomFilter to optimize read operations Write-Ahead Log (WAL) for durability Compressor to reduce disk space usage Write Operations In an LSM tree, data writes are first handled in memory by the MemTable. Before a write is applied, it is logged to the Write-Ahead Log (WAL) to ensure durability in case of crashes. Java func (sc *StorageComponent) Set(key, value string) error { if sc.wal != nil { if err := sc.wal.Log(key, value); err != nil { return err } } sc.memTable.Set(key, value) if sc.memTable.Size() > sc.config.MemTableSizeThreshold { return sc.flushMemTable() } return nil } Once the MemTable reaches a certain size, it is flushed to disk as an SSTable. This process ensures that memory usage remains within bounds, while also writing data in sorted order to disk for faster future retrievals. MemTable Flushing and SSTables MemTable flushing involves writing the current in-memory data structure to an SSTable on disk. SSTables store key-value pairs in sorted order, making subsequent reads and merges efficient. Java func (sc *StorageComponent) flushMemTable() error { ssTable := NewSSTable(sc.config.BlockSize, sc.compressor) sc.memTable.Iterate(func(key, value string) { ssTable.Add(key, value) }) if err := ssTable.Flush(); err != nil { return err } sc.updateSparseIndex(ssTable) sc.updateBloomFilter(ssTable) sc.memTable = NewBalancedTree(sc.config.TreeType) return nil } The key advantage of SSTables is their sorted structure. Sorting allows for efficient merging of multiple tables during compaction and enables range queries. A typical compaction strategy involves merging smaller SSTables into larger ones, eliminating duplicate keys and old versions of data. Write-Ahead Logging (WAL) WAL ensures data durability by logging all write operations before they are applied to the MemTable. This allows the system to recover from crashes by replaying the log and restoring the most recent writes. Java type WAL struct { file *os.File } func (w *WAL) Log(key, value string) error { entry := fmt.Sprintf("%s:%s\n", key, value) _, err := w.file.WriteString(entry) return err } By keeping a write-ahead log, we mitigate the problem of losing in-memory data that has not yet been flushed to disk in the event of a crash. Compaction and SSTables One of the key operations in an LSM tree is compaction, where multiple SSTables are merged into a single SSTable, eliminating duplicate keys and consolidating data. This process ensures that old data is removed and reduces the number of files the system must search through during reads. Java func (sc *StorageComponent) performCompaction() { // Merge SS-tables and remove obsolete entries } Compaction not only optimizes disk space usage but also improves read performance by reducing the number of SSTables that need to be scanned during a query. This concept mirrors the "upkeep" mentioned in the provided excerpt, where databases consolidate and compact logs to keep performance efficient over time. Read Operations Reading data from an LSM tree involves checking multiple sources in sequence: the MemTable first, followed by the BloomFilter, and then the SSTables. The BloomFilter helps avoid unnecessary disk reads by quickly determining whether a key might exist in the on-disk data. Java func (sc *StorageComponent) Get(key string) (string, error) { if value, found := sc.memTable.Get(key); found { return value, nil } if sc.bloomFilter != nil && !sc.bloomFilter.MightContain(key) { return "", errors.New("Key not found") } for i := len(sc.ssTableFiles) - 1; i >= 0; i-- { if value, found := sc.ssTableFiles[i].Get(key); found { return value, nil } } return "", errors.New("Key not found") } This multi-step approach ensures that reads are both fast (due to the in-memory MemTable and BloomFilter) and accurate (due to the sorted SSTables). While reading from multiple sources introduces some complexity, the use of auxiliary data structures like BloomFilters minimizes the performance hit. Block Compression Compression is another important feature of LSM trees, helping reduce disk usage and improve read performance by compressing data blocks before they are written to disk. Java type Compressor interface { Compress([]byte) []byte Decompress([]byte) []byte } Compression strikes a balance between storage efficiency and read/write performance, with larger blocks offering better compression at the expense of slightly slower point queries. This technique is commonly used in storage systems like LevelDB and RocksDB, as described in the excerpt. Indexing and Performance Considerations To optimize read performance, LSM trees often rely on a sparse index, which maps specific keys to their locations in SSTables. This index significantly improves search times by reducing the need to scan entire tables. As the excerpt discusses, efficient indexing structures, such as those derived from hash maps or balanced trees, play a crucial role in minimizing read complexity. The performance of LSM trees is governed by several factors: MemTable Size: A larger MemTable reduces the frequency of disk writes but increases memory usage and the potential for data loss in case of crashes. Compaction frequency: More frequent compaction reduces the number of SSTables, improving read performance, but it increases I/O load. Balanced tree type: The type of tree used for the MemTable (e.g., AVL, Red-Black) affects in-memory operation performance. Block size and compression: Larger blocks provide better compression ratios but may slow down queries. As noted in the excerpt, balancing the cost of frequent writes with efficient reads is essential for high-performance LSM-tree implementations. The compaction strategy used (e.g., leveled or size-tiered) also has a significant impact on both disk usage and query performance. Real-World Applications of LSM Trees in Storage Systems LSM trees are at the core of many modern database systems, providing the backbone for scalable and efficient data storage solutions. Some notable real-world applications include: Cassandra: Apache Cassandra uses LSM trees as the primary storage mechanism, enabling high throughput for write-heavy workloads. LSM trees allow Cassandra to achieve its distributed, fault-tolerant architecture by efficiently batching writes in memory before flushing to disk. LevelDB and RocksDB: Both LevelDB and its successor, RocksDB, are key-value stores that leverage LSM-trees to optimize write performance. RocksDB, in particular, is widely used in embedded databases and larger-scale systems such as Facebook’s internal infrastructure, thanks to its support for advanced features like block compression, compaction strategies, and partitioned indexes. HBase: HBase, a distributed storage system built on top of Hadoop, relies on LSM trees to manage its read and write operations. By organizing data into MemTables and SSTables, HBase ensures that both random and sequential read/write workloads are handled efficiently, even under heavy load. InnoDB (MySQL): MySQL’s InnoDB storage engine also incorporates concepts from LSM trees, particularly for handling large write loads. By separating in-memory data from persistent storage and using strategies like background compaction, InnoDB ensures both durability and performance in transactional workloads. RocksDB in Kafka: Kafka Streams uses RocksDB as a local storage engine, taking advantage of the LSM tree’s efficient write batching and compaction features to handle streaming data at scale. This allows Kafka to maintain high write throughput and minimize latency in event processing pipelines. These systems demonstrate the versatility and robustness of LSM trees, making them a popular choice for high-performance, write-optimized storage subsystems in distributed databases and data-intensive applications. Conclusion Implementing an LSM tree in Golang provides a scalable, efficient solution for handling write-heavy workloads in modern storage systems. By combining an in-memory MemTable with on-disk SSTables, and augmenting it with features like Write-Ahead Logging, block compression, and BloomFilters, this system is well-equipped to handle large volumes of data. Key takeaways include: Efficient write operations through batching in MemTable and sequential SSTable writes Durability through Write-Ahead Logging, ensuring data recovery after crashes Optimized read performance using BloomFilters and sparse indexes to minimize disk accesses. Compaction and compression to maintain storage efficiency and improve I/O performance. This LSM tree implementation provides a strong foundation for building scalable, high-performance storage systems in Golang, with potential future enhancements like range queries, concurrent access, and distributed storage.

This is the second part of the blog series “Faster Startup With Spring Boot 3.2 and CRaC," where we will learn how to warm up a Spring Boot application before the checkpoint is taken and how to provide configuration at runtime when the application is restarted from the checkpoint. Overview In the previous blog post, we learned how to use CRaC to start Spring Boot applications ten times faster using automatic checkpoints provided by Spring Boot 3.2. It, however, came with two significant drawbacks: The checkpoint is automatically taken after all non-lazy beans have been instantiated, but before the application is started, preventing us from fully warming up the application and the Java VM’s Hotspot engine, e.g., performing method inline optimizations, etc. The runtime configuration is set at build time, so we cannot specify the configuration to use when the application is restarted from the CRaC checkpoint at runtime. This also means that sensitive configuration information like credentials and secrets stored in the JVM’s memory will be serialized into the files created by the checkpoint. In this blog post, we will learn how to overcome these shortcomings by performing the checkpoint on-demand. It is a bit more complex compared to performing an automatic checkpoint, requiring a couple of extra steps in the build phase. This blog post will show you how they can be automated. The sample Spring Boot applications used in this blog post are based on Chapter 6 in my book on building microservices with Spring Boot. The system landscape consists of four microservices: three of them store data in databases, while the fourth aggregates information from the other three. The system landscape is managed using Docker Compose. Docker Compose will start the microservices from CraC-enabled Docker images, i.e. Docker images containing CRaC checkpoints, together with the databases used. This is illustrated by the following image: The blog post is divided into the following sections: Warming up an application before the checkpoint Supplying the configuration at runtime Implementing on-demand checkpoint and restore Trying out on-demand checkpoint and restore Summary Next blog post Let’s start by learning how to warm up a Spring Boot application before taking the checkpoint. 1. Warming up an Application Before the Checkpoint To warm up the applications, we need a runtime environment called a training environment. The training environment must be isolated from the production environment but configured similarly from a warmup perspective. This means that the same type of database must be used, and mocks (if used) need to behave like the actual services for the use cases used during warmup. To be able to run relevant tests during the warmup phase, we must prepare the training environment with test data. Let’s start with setting up a training environment and populating it with test data applicable to the microservices used in this blog post. 1.1 Creating the Training Environment and Populating It With Test Data In this blog post, we will reuse the development environment from the book as the training environment and prepare it with test data by using the test script from the book. The test script, test-em-all.bash, contains a section that prepares databases with test data before it runs tests to verify that the system landscape of microservices works as expected. The following image illustrates this: The training environment can be built from the source, started up, and populated with test data using the following commands: Shell ./gradlew build docker compose build docker compose up -d ./test-em-all.bash For detailed instructions, see Section 4, "Trying Out On-Demand Checkpoint and Restore," below. With the training environment established and populated with test data, let’s see how we can use it to warm up our applications before taking the CRaC checkpoint. 1.2 Creating Docker Images of Warmed-Up Applications The system landscape used in this blog post has the following requirements for the training environment to be able to warm up its microservices: The Product Composite service needs to connect to the three microservices: product, recommendation, and review. The Product and Recommendation services must be able to connect to a MongoDB database. The Review service must be able to connect to a MySQL database. From the previous blog post, we learned how to use a Dockerfile to build a Docker image containing a CRaC checkpoint of a Spring Boot Application. We will expand this approach by adding support for on-demand checkpoints. One new requirement in this blog post is that the docker build commands must be able to connect to the Docker containers in the training environment. This will be solved by adding network connectivity to the Docker Buildx builder used in this blog post. See Section 4, "Trying Out On-Demand Checkpoint and Restore," below for detailed instructions. These requirements are summarized in the following image: Note: To fully warm up the Java VM’s Hotspot engine, e.g., performing inlining of frequently used method calls, it takes tens of thousands of iterations. For the scope of this blog post, we will only perform 100 iterations for demonstration purposes. For further details on warming up the Hotspot engine, see: Azul - Analyzing and Tuning Warm-up Benchmarking the Warm-Up Performance of HotSpot VM, GraalVM and OpenJ9 Since each warmup procedure is specific to each microservice, they will have their own version of the script, checkpoint-on-demand.bash, used to warm up and take the checkpoint. The Dockerfile invoking the script, Dockerfile-crac-on-demand, is generic, so all microservices use the same Dockerfile. The most important parts of the Dockerfile look like this: Dockerfile FROM azul/zulu-openjdk:21.0.3-21.34-jdk-crac AS builder ADD build/libs/*.jar app.jar ADD crac/checkpoint-on-demand.bash checkpoint-on-demand.bash RUN --security=insecure ./checkpoint-on-demand.bash FROM azul/zulu-openjdk:21.0.3-21.34-jdk-crac AS runtime COPY --from=builder checkpoint checkpoint COPY --from=builder app.jar app.jar EXPOSE 8080 ENTRYPOINT ["java", "-XX:CRaCRestoreFrom=checkpoint"] It is a multi-stage Dockerfile where the builder part: Copies the jar file and the checkpoint script Runs the checkpoint script to perform the warmup and initiate the checkpoint process The runtime part: Copies the results of the checkpoint and the jar file from the builder Defines an entry point for the Docker image based on the Java VM performing a restore from the checkpoint folder The most important parts of one of the checkpoint scripts, checkpoint-on-demand.bash from the Product microservice, looks like this: Shell function runWarmupCalls() { assertCurl 200 "localhost:$PORT/product/1" assertEqual 1 $(echo $RESPONSE | jq ".productId") } function warmup() { waitForService localhost:$PORT/actuator for i in {1..100}; do runWarmupCalls; done } warmup && jcmd app.jar JDK.checkpoint & java -XX:CRaCCheckpointTo=$OUT_FOLDER -jar app.jar The script contains: A warmupfunction that performs calls to endpoints exposed by the application to warm up the application and the Java VM Hotspot engine. NOTE: This warmup function is supplied as a basic example, it needs to be enhanced for a real-world use case. The warmup function, together with the checkpoint command, is started up in the background. The application is started using the jar file, waiting for the warmup to be completed and the checkpoint to be taken. With that, we have covered how to perform a checkpoint after warming up the application and Java VM Hotspot engine. Next, let’s see how we can provide configuration at restart from a checkpoint, i.e., restore the application with an environment-specific configuration. 2. Supplying the Configuration at Runtime To load the runtime configuration at CRaC restore time, we can use Spring Cloud’s Context Refresh functionality. It provides the following two features: The @RefreshScope annotation can be used for properties defined in our source code, e.g., the host and port properties in the Product Composite microservice. It will look like: Java @RefreshScope @Component public class ProductCompositeIntegration implements ProductService... public ProductCompositeIntegration( @Value("${app product-service.host}") String productServiceHost, @Value("${app-product-service-port}") int productServicePort, ... For properties in libraries that we use, e.g., the properties of the SQL DataSource bean in the spring-data-jpa library, we can use the property spring.cloud.refresh.extra-refreshable to point out classes of beans that need to be refreshed. Finally, we can use the spring.config.import property to point out an external property file used in runtime to load the values for the refreshed properties. For example, the Product Composite microservice declares a separate profile, crac, to point out an external property file for its properties that depend on the runtime environment, i.e., the host and port properties. The relevant parts look like: YAML --- spring.config.activate.on-profile: crac spring.config.import: file:./runtime-configuration.yml Its runtime configuration file looks like this: YAML app.product-service: host: product-prod port: 8080 The Review microservice also uses a separate crac profile, declaring that the DataSource bean in the spring-data-jpa library needs to be refreshed: YAML spring: cloud.refresh.extra-refreshable: - javax.sql.DataSource config.import: file:./runtime-configuration.yml Its runtime configuration file looks like this: YAML spring.datasource: url: jdbc:mysql://mysql/review-db-prod username: user-prod password: pwd-prod See Section 4, "Trying Out On-Demand Checkpoint and Restore," below for details on how the runtime configuration files are used. To ensure that we can use different configurations at build time and runtime, we will use different hostnames, ports, and database credentials, as described in the following table: ENVIRONMENT BUILD TIME RUNTIME Product service localhost:7001 product-prod:8080 Recommendation service localhost:7002 recommendation-prod:8080 Review Service localhost:7003 review-prod:8080 MongoDB localhost:27017 mongodb:27017 MySQL URL localhost:3306/review-db mysql:3306/review-db-prod MySQL credentials user/pwd user-prod/pwd-prod The existing default Spring profile will be used at build time. The configuration specified by the external configuration files explained above will be used at runtime. 2.1 Problems With the MongoClient on Restore Currently, the MongoClient prevents using CRaC checkpoints; see the following error reports: Can’t create CRaC checkpoint, fails with the error message “Restarting Spring-managed lifecycle beans after JVM restore” · Issue #4708 · spring-projects/spring-data-mongodb Allow to suspend/resume MongoClient The problem can party be resolved by programmatically closing the MongoClient instance before taking the checkpoint. Unfortunately, the MongoClient uses the configuration provided at build time when restarting at restore, disregarding the configuration provided at runtime. This means that the configuration for connections to MongoDB databases must be supplied at build time. For this blog post, the problem will be resolved by using the hostname used in runtime, mongodb, also at build time. When building the Docker images for the microservices that use MongoDB, the product and recommendation services, the hostname mongodb will be used in the crac Spring profile for each service. The hostname mongodbwill be mapped to 127.0.0.1 when running the Docker build command. See Section 4, "Trying Out On-Demand Checkpoint and Restore," below for details on how the Docker build command maps mongodb to 127.0.0.1. 3. Implementing On-Demand Checkpoint and Restore To see all changes applied to implement on-demand checkpoint and restore, you can compare Chapter06 in the branch SB3.2-crac-on-demand with the branch SB3.2. To see the changes between implementing on-demand checkpoint and restore with automatic checkpoint and restore, compare the branches SB3.2-crac-on-demand and SB3.2-crac-automatic. To summarize the most significant changes: crac/Dockerfile-crac-on-demand: As mentioned above, this Docker file is used to create the CRaC-enabled Docker images. It uses the microservice’s specific checkpoint-on-demand.bash scripts to warm up the microservices before the checkpoint is taken. New config repository crac/config-repo/prod/*.yml: This folder contains the configuration used at runtime, one configuration file per microservice. crac/docker-compose-crac.yml: It is used to start up the runtime environment. It configures the microservices to start from the CRaC-enabled Docker images and uses volumes to include the external configuration files provided by the config repository mentioned above. It also defines runtime-specific hostnames of the services and credentials for the MySQL database. Source code changes in the microservices folder: Changes in all microservices: build.gradle: Added a new dependency to spring-cloud-starter, bringing in Spring Cloud’s Context Refresh functionality. New crac/checkpoint-on-demand.bash script, to warm up the microservice before taking the checkpoint resources/application.yml, added a new Spring profile, crac, configures the use of Spring Cloud’s Context Refresh functionality. Product Composite service: The RestTemplatecan’t be used during warmup with WebFlux; it needs to be switched to WebMvc. For details, see the following error report: Can’t create a CRaC checkpoint if a RestTemplate and RestClient is used during warmup, resulting in open socket exceptions · Issue #33196 · spring-projects/spring-framework. The new RestTemplateConfiguration class uses ReactorNettyRequestFactory to configure the RestTemplateaccording to the resolution described in the error report above. The class ProductCompositeIntegration added a @RefreshScope annotation to reload properties for hostnames and ports for its services. Product and Recommendation Service: The main classes ProductServiceApplication and RecommendationServiceApplication, closes its mongoClient bean in a beforeCheckpoint method to avoid open port errors at checkpoints. 4. Trying Out On-Demand Checkpoint and Restore Now, it is time to try it out! First, we need to get the source code and build the microservices. Next, we start up the training landscape. After that, we can create the Docker images using the Dockerfile Dockerfile-crac-on-demand. Finally, we can start up the runtime environment and try out the microservices. 4.1 Getting the Source Code and Building the Microservice Run the following commands to get the source code from GitHub, jump into the Chapter06 folder, check out the branch used in this blog post, SB3.2-crac-on-demand, and ensure that a Java 21 JDK is used (Eclipse Temurin is used in the blog post): Shell git clone https://github.com/PacktPublishing/Microservices-with-Spring-Boot-and-Spring-Cloud-Third-Edition.git cd Microservices-with-Spring-Boot-and-Spring-Cloud-Third-Edition/Chapter06 git checkout SB3.2-crac-on-demand sdk use java 21.0.3-tem Build the microservices and their standard Docker images, which are not based on CRaC. The training landscape will use these Docker images. Run the following commands: Shell ./gradlew build unset COMPOSE_FILE docker compose build 4.2 Starting up the Training Landscape Startup the training system landscape and populate it with test data with the following commands: Shell docker compose up -d ./test-em-all.bash Expect the test-em-all.bash script to end with the output End, all tests OK. 4.3 Building the CRaC-Enabled Docker Images As mentioned above, the docker build commands must be able to connect to the Docker containers in the training environment. In this blog post, we will use host-based networking, i.e., sharing localhost with the Docker engine. Note: To avoid a port conflict in the Docker engine’s network when building the Product Composite service, we will shut down the Product Composite service in the training landscape. Once it has been used to populate the test data, it is no longer required for building the CRaC-enabled Docker images. Bring down the Product Composite service with the command: Shell docker compose rm -fs product-composite First, we must create a Docker buildx builder with host-based networking. This can be done with the following command: Shell docker buildx create \ --name insecure-host-network-builder \ --driver-opt network=host \ --buildkitd-flags '--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host' Note: See the previous blog post for information on why we use a Docker buildx builder when building the Docker images. Now, we can build the CRaC-enabled Docker images with warmed-up microservices: Shell docker buildx --builder insecure-host-network-builder build --allow network.host --network host --allow security.insecure -f crac/Dockerfile-crac-on-demand -t product-composite-crac --progress=plain --load microservices/product-composite-service docker buildx --builder insecure-host-network-builder build --allow network.host --network host --allow security.insecure -f crac/Dockerfile-crac-on-demand -t review-crac --progress=plain --load microservices/review-service docker buildx --builder insecure-host-network-builder build --add-host=mongodb:127.0.0.1 --allow network.host --network host --allow security.insecure -f crac/Dockerfile-crac-on-demand -t product-crac --progress=plain --load microservices/product-service docker buildx --builder insecure-host-network-builder build --add-host=mongodb:127.0.0.1 --allow network.host --network host --allow security.insecure -f crac/Dockerfile-crac-on-demand -t recommendation-crac --progress=plain --load microservices/recommendation-service Note: When building the product and recommendation services, that use MongoDB, the hostname mongodb is mapped to 127.0.0.1 using the --add-host option. See Section 2.1, "Problems With the MongoClient on Restore," above for details. Verify that we got four Docker images, whose names are suffixed with -crac, with the command: Shell docker images | grep -e "-crac" We are now done with the training environment, so we can shut it down: Shell docker compose down 4.4 Running the Microservices in the Runtime Environment Start up the runtime environment with CRaC-enabled Docker images using the Docker Compose file crac/docker-compose-crac.yml, and use the test script, test-em-all.bash, to verify that it works as expected: Shell export COMPOSE_FILE=crac/docker-compose-crac.yml docker compose up -d ./test-em-all.bash Note: The Docker Compose file creates the MySQL database for the Review service using the script crac/sql-scripts/create-tables.sql. See the previous blog post for more info. Verify that the startup time (i.e., CRaC restore time) is short by running the command: Shell docker compose logs | grep "restart completed" Verify that the response contains output similar to the following: Shell recommendation-prod-1 ... restored JVM running for 133 ms product-prod-1 ... restored JVM running for 127 ms product-composite-prod-1 ... restored JVM running for 155 ms review-prod-1 ... restored JVM running for 183 ms To verify that the configuration is correctly updated in runtime at restore, run the following command: Shell docker compose logs product-composite-prod review-prod | grep "Refreshed keys" Look for the updated SQL DataSource properties used by the Review service and the updated host and portproperties used by the Product Composite service. Expect the response from the command containing the following: Shell review-prod-1 | ... Refreshed keys: [spring.datasource.username, spring.datasource.url, spring.datasource.password] product-composite-prod-1 | ... Refreshed keys: [app.product-service.host, app.product-service.port, app.recommendation-service.host, app.recommendation-service.port, app.review-service.host, app.review-service.port] Note: Due to the MongoClient problem describes above, this will not apply to the Product and Recommendation microservices. With that, we are done. Let’s wrap up by bringing down the runtime environment with the commands: Shell docker compose down unset COMPOSE_FILE 5. Summary In this blog post, we have learned how to overcome the two shortcomings of using Automatic checkpoint and restore. We have introduced On-demand checkpoint and restore that allows us to: Control when to perform the checkpoint, allowing us to warm up the application and its Java VM Hotspot engine before taking the checkpoint. This allows for optimal application performance after a restore. Use Spring Cloud’s Context Refresh functionality to reload configuration that depends on the runtime environment when the application is restored. This means we no longer need to store runtime-specific and sensitive configurations in the Docker image containing the checkpoint. 6. Next Blog Post In the next blog post, I intend to explain how to use CRaC together with reactive microservices. These microservices are built using Project Reactor, Spring WebFlux, Spring Data MongoDB Reactive, and event streaming using Spring Cloud Stream with Kafka.

Even those not particularly interested in computer technology have heard of microprocessor architectures. This is especially true with the recent news that Qualcomm is rumored to be examining the possibility of acquiring various parts of Intel and Uber is partnering with Ampere Computing. Hardware and software are evolving in parallel, and combining the best of modern software development with the latest Arm hardware can yield impressive performance, cost, and efficiency results. Why Arm? Arm is a member of the Reduced Instruction Set Computer (RISC) architecture family, a type of microprocessor architecture based on a small, highly optimized set of instructions. Other known representatives of this family are RISC-V, SPARC, and MIPS. Initially, the RISC family aimed for embedded and related markets, but soon overgrew to new potential, with Arm emerging as the most popular so far. Today, Arm competes with x86 (based on the CISC approach) in cloud computing, and here are two key differences between CISC and RISC that you need to know. The first is the way instructions are executed. Both approaches attempt to increase CPU performance, but in different ways: RISC reduces the number of cycles per instruction at the expense of the number of instructions per program. CISC minimizes the number of instructions per program, but this comes at the cost of an increased number of cycles per instruction. The second difference is the licensing approach. Both x86 and Arm are open-source architectures that are available for production through licensing. However, x86 was historically developed by Intel: the name "x86" is derived from the 8086, an early processor released by the company. Today, only three companies hold licenses to build hardware with this architecture type, and Intel remains the largest shareholder in this market. Arm licenses are easier to get, so Arm manufacturing is very competitive. In some ways, the Arm licensing approach has allowed this hardware to flourish and evolve so quickly. Originally conceived as a solution for embedded and related applications, Arm has undergone many expansions and can now be found in desktop processors like M3 and M4 chips, mobile phones, automobiles, and almost everywhere else. Modern Java Enhancements for Arm The Java community recognized Arm's potential a while ago and has successfully completed several initiatives to optimize Java for working on Arm. The first AArch64 project was implemented as part of OpenJDK, delivering a Linux/AArch64 port to JDK 9. This has had a major impact on the way Java is used today. The following Java 11 brought many optimizations to the port initiated in JEP 315: Improve Aarch64 intrinsics. Improvements of this JEP were CPU-specific and helped improve all operating systems. In the subsequent OpenJDK releases of Java 16 and Java 17, two important ports were established: Windows/AArch64 (JEP 388) and macOS/AArch64 (JEP 391), giving you full-fledged Java options on almost all popular operating systems. Today, as a developer, you can find Java on Arm in mature and modern Java releases. In addition to Java on Arm, you have the choice of small base Linux container images on Arm to further benefit from this architecture. Java on Arm presents a particular interest for enterprise development and DevOps looking to enable greater efficiency and lower costs. Migrating to Aarch64 is easy, using either an x86 emulation or an Arm native JDK. Arm Hardware Review in Different Clouds Since the software side is set up to work on Arm, with most Linux distributions and other critical projects (for the web) already available and supported, there is freedom in your choice of Arm hardware. Let's take a closer look at what Arm-based servers, which are now gaining ground in cloud services, can do for us. Arm-based servers are server machines that use processors based on the Arm architecture and are becoming increasingly popular in data centers, cloud computing, and various enterprise applications. The growing adoption of Arm-based servers is driven by several factors, including their energy efficiency, cost-effectiveness, and scalability. In addition, increasing support from operating system vendors, open-source projects, and cloud service providers is helping to make Arm server solutions more accessible and practical for enterprise applications. Arm's key players on the server side are represented by GCP, AWS, Azure, and OCI. AWS Graviton The latest AWS Graviton4, like all the other AWS Graviton processors, uses a 64-bit Arm instruction set architecture. AWS Graviton4-based Amazon EC2 R8g instances deliver up to 30% better performance than AWS Graviton3-based Amazon EC2 R7g instances. This processor is ideal for demanding workloads such as high-performance databases, in-memory caches, and real-time big data analytics. AWS Graviton is an example of Neoverse architecture, made for handling a wide range of cloud-native workloads at world-class levels of performance, efficiency, and compute density. This architecture is great for the cloud. In 2024, Arm announced Neoverse V3 with up to 128 cores, targeting the highest-performance applications. Google Axion Processors In April 2024, Google announced its new Arm-based Axion Processors, promising up to 30% better performance than the fastest general-purpose Arm-based instances available in the cloud, up to 50% better performance, and up to 60% better energy efficiency than comparable current-generation x86-based instances. Axion processors, just like AWS Graviton4, are built using the Arm Neoverse: the Neoverse V2 CPU. As many have noted, this product release puts Google in direct competition with Amazon, leading the market with Amazon Web Services (AWS) and the rest of the existing players with their Arm-based servers. Azure Cobalt 100 Arm-Based Processor In May 2024, Microsoft announced the preview of a new Azure Cobalt 100 Arm-based processor. The Cobalt 100 processor is another representative of Arm Neoverse. It uses an N-series (N2) Arm CPU design, enabling optimized performance of scale-out cloud-based applications. In general, Microsoft pays a lot of attention to Arm, in parallel investing in developer platforms and language optimization on Linux and Windows for Arm. This includes NET 8 numerous enhancements for Arm and C++, introduced in Visual Studio 17.10 SQL Server Data Tools (SSDT) for the Arm native Visual Studio. Ampere A1 Compute by Oracle In May 2021, Oracle released its first Arm-based computing product: the OCI Ampere A1 Compute. The product runs on Oracle Cloud Infrastructure (OCI). The main model is VM.Standard.A1.Flex (OCI A1), whose CPU core and memory can be flexibly configured with VM shapes scale from 1 to 156 cores and 1 to 64 GB of memory per core. The flexible approach allows unique settings for your own project needs, matching your workload requirements and saving the cost spent unnecessarily. Tests for widespread AI interference show AmpereOne A2 as a highly competitive and compelling product. Oracle also promotes Arm technology via the Arm developer ecosystem and partnerships with Ampere Computing, Arm, GitLab, Jenkins, and others. Arm Prospectus With all of the big technological names currently involved in Arm-based hardware production, bolstered by Arm's continued improvements on the software side, Arm's popularity may soon overtake x86. According to ARM CEO Rene Haas, Arm will have a 50% market share in five years. The Arm-based servers featured here offer impressive performance and efficiency for cloud-native workloads and are particularly relevant to the big data and AI industries. Moving your workloads to an Arm-based architecture is comparatively easy and guarantees significant budget reductions. Neoverse is a certain choice for servers aimed at working with large volumes of information in the cloud, with the latest examples from Microsoft, Google, and AWS. Along with the Arm rise, the Java ecosystem continues to get further enhancements for working on it. The expanding ecosystem ready for Arm includes Linux distributions, Java and OpenJDK runtimes, frameworks, and main infrastructure systems (web servers, Spark, Kafka, Cassandra, Elastic, and others). The goodwill of the Java community to enhance the ecosystem for working with Arm indicates that Arm is becoming a tier-one hardware platform. Containers — highly valued for their isolation and security, portability, and reproducibility — are adapted to Arm, so you can get small Linux containers tuned for Java today for free. Docker is investing in the Arm space by ensuring that the Docker Desktop runs natively on Windows on Arm. The latest Arm technology is well-suited for modern Java workloads. Moving an OpenJDK application to an Arm-based server is a smart way to improve Java performance and reduce resource consumption. Adding Arm-optimized Linux-based containers to your Java applications goes one step further, giving you the most complete Java on Arm solution for a sustainable and robust Java experience. As a result, enterprises are increasingly turning to the Arm architecture to reduce costs and power utilization. A future roadmap/prospectus may include an increased focus on Arm hardware with even better efficiency results.

Since Java 8, the programming workload of iterating over collections and selecting a subcollection (filtering) based on defined constraints (predicates) has been considerably simplified using the new Stream API. Despite this new feature, some scenarios can still overwhelm developers by requiring them to implement a significant amount of very specific code to meet the filtering conditions. A common example in enterprise systems is the need to filter: ". . . a collection where each object is an element of a large graph and the attributes to be considered in the filters belong to distinct objects." To illustrate the scenario, let's consider the small class diagram below. It is required to filter a collection of Post objects taking into account different attributes of the object graph. At this point, disregard the @Filterable annotation on some attributes; this will be covered later. To present a traditional Java code for filtering Post collections, let's make some assumptions: A List<Post> has been instantiated and loaded. A primitive type textFilter variable has been set as the value to be searched. The Apache Commons Lang StringUtils class is used to remove diacritics from strings. A way to filter a postsCollection by the text attribute of the Publication class is: Java postsCollection .stream() .filter(p -> Objects.isNull(p.getText()) || StringUtils.stripAccents(p.getText().trim().toLowerCase()) .contains(textFilter.trim().toLowerCase())) .toList(); If the need is to filter the collection by the review attribute of the Comment class, the code could be like this: Java postsCollection .stream() .filter(p -> Objects.isNull(p.getComments()) || p.getComments().isEmpty() || p.getComments().stream().anyMatch(c -> StringUtils.stripAccents(c.getReview().trim().toLowerCase()) .contains(textFilter.trim().toLowerCase()))) .toList(); Note that for each change in filter requirements, the code must be adjusted accordingly. And what's worse: the need to combine multiple attributes from different graph classes can lead to code hard to maintain. Addressing such scenarios, this article presents a generic approach to filtering collections by combining annotations and the Java Reflection API. Introspector Filter Algorithm The proposed approach is called Introspector Filter and its main algorithm relies on three fundaments: Traverse all relationships of each object using a Breadth-First Search strategy. Traverse the hierarchy of each object. Check if any attribute annotated with @Filterable of each object contains the given pattern. It's worth explaining that all traversing operations are supported by the Reflection API. Another relevant point is that the attributes to be considered in the filtering operation must be annotated with @Filterable. The code below presents the implementation of the Introspector Filter algorithm. Its full source code is available in the GitHub repository. Java public IntrospectorFilter() { public Boolean filter(Object value, Object filter) { if (Objects.isNull(filter)) { return true; } String textFilter = StringUtils.stripAccents(filter.toString().trim().toLowerCase()); var nodesList = new ArrayList<Node>(); nodesList.add(new Node(0, 0, value)); while (!nodesList.isEmpty()) { // BFS for relationships var node = nodesList.removeFirst(); if (node.height() > this.heightBound || node.breadth() > this.breadthBound) { continue; } var fieldValue = node.value(); var fieldValueClass = fieldValue.getClass(); int heightHop = node.height(); do { // Hierarchical traversing if (Objects.nonNull( this.searchInRelationships(node, fieldValueClass, heightHop, textFilter, nodesList))) { return true; } fieldValueClass = fieldValueClass.getSuperclass(); heightHop++; } while (isValidParentClass(fieldValueClass) && heightHop <= this.heightBound); if (isStringOrWrapper(fieldValue) && containsTextFilter(fieldValue.toString(), textFilter)) { return true; } } return false; } } Three methods have been abstracted from this code to keep the focus on the main algorithm: isValidParentClass: Verify if the parent class is valid, considering any additional annotations that have been provided to identify the class. If no additional annotation have been provided, all parent classes are considered valid. isStringOrWrapper: Check if the value of the attribute annotated with @Filterable is a String or a primitive type wrapper. When this is true, such a relationship traversing is interrupted, as there is no further way forward. containsTextFilter: Check if the attribute annotated with @Filterable contains the supplied pattern. Let's go back to the previous small class diagram. The two codes presented for filtering a postsCollection may be replaced with the following code using the Introspector Filter Algorithm. Java postsCollection.stream().filter(p -> filter.filter(p, textFilter)).toList(); Introspector Filter Project The implementation of the Introspector Filter Algorithm has been encapsulated in a library (jar). It can be incorporated into a Maven project by simply adding the following dependency to the pom.xml file. This implementation requires at least Java version 21 to work. But there is a Java 8 compatible version — change the version dependency to 0.1.0. XML <dependency> <groupId>io.github.tnas</groupId> <artifactId>introspectorfilter</artifactId> <version>1.0.0</version> </dependency> The project also has an example module detailing how to use the Introspector Filter as a global filter in JSF datatable components. Conclusion Selecting a subset of objects from a Java collection is a common task in software development. Having a flexible strategy (Introspector Filter Algorithm) or tool (Introspector Filter Library) for creating filters to be used in these selections can help developers write more concise and maintainable code. This is the proposal of Instrospector Filter Project, which is fully available in a GitHub repository.

By Thiago Nascimento CORE

Kubernetes has become the standard for container orchestration. Although APIs are a key part of most architectures, integrating API management directly into this ecosystem requires careful consideration and significant effort. Traditional API management solutions often struggle to cope with the dynamic, distributed nature of Kubernetes. This article explores these challenges, discusses solution paths, shares best practices, and proposes a reference architecture for Kubernetes-native API management. The Complexities of API Management in Kubernetes Kubernetes is a robust platform for managing containerized applications, offering self-healing, load balancing, and seamless scaling across distributed environments. This makes it ideal for microservices, especially in large, complex infrastructures where declarative configurations and automation are key. According to a 2023 CNCF survey, 84% of organizations are adopting or evaluating Kubernetes, highlighting the growing demand for Kubernetes-native API management to improve scalability and control in cloud native environments. However, API management within Kubernetes brings its own complexities. Key tasks like routing, rate limiting, authentication, authorization, and monitoring must align with the Kubernetes architecture, often involving multiple components like ingress controllers (for external traffic) and service meshes (for internal communications). The overlap between these components raises questions about when and how to use them effectively in API management. While service meshes handle internal traffic security well, additional layers of API management may be needed to manage external access, such as authentication, rate limiting, and partner access controls. Traditional API management solutions, designed for static environments, struggle to scale in Kubernetes’ dynamic, distributed environment. They often face challenges in integrating with native Kubernetes components like ingress controllers and service meshes, leading to inefficiencies, performance bottlenecks, and operational complexities. Kubernetes-native API management platforms are better suited to handle these demands, offering seamless integration and scalability. Beyond these points of confusion, there are other key challenges that make API management in Kubernetes a complex task: Configuration management: Managing API configurations across multiple Kubernetes environments is complex because API configurations often exist outside Kubernetes-native resources (kinds), requiring additional tools and processes to integrate them effectively into Kubernetes workflows. Security: Securing API communication and maintaining consistent security policies across multiple Kubernetes clusters is a complex task that requires automation. Additionally, some API-specific security policies and enforcement mechanisms are not natively supported in Kubernetes. Observability: Achieving comprehensive observability for APIs in distributed Kubernetes environments is difficult, requiring users to separately configure tools that can trace calls, monitor performance, and detect issues. Scalability: API management must scale alongside growing applications, balancing performance and resource constraints, especially in large Kubernetes deployments. Embracing Kubernetes-Native API Management As organizations modernize, many are shifting from traditional API management to Kubernetes-native solutions, which are designed to fully leverage Kubernetes' built-in features like ingress controllers, service meshes, and automated scaling. Unlike standard API management, which often requires manual configuration across clusters, Kubernetes-native platforms provide seamless integration, consistent security policies, and better resource efficiency, as a part of the Kubernetes configurations. Here are a few ways that you can embrace Kubernetes-native API management: Represent APIs and related artifacts the Kubernetes way: Custom Resource Definitions (CRDs) allow developers to define their own Kubernetes-native resources, including custom objects that represent APIs and their associated policies. This approach enables developers to manage APIs declaratively, using Kubernetes manifests, which are version-controlled and auditable. For example, a CRD could be used to define an API's rate-limiting policy or access controls, ensuring that these configurations are consistently applied across all environments. Select the right gateway for Kubernetes integration:Traditional Kubernetes ingress controllers primarily handle basic HTTP traffic management but lack the advanced features necessary for comprehensive API management, such as fine-grained security, traffic shaping, and rate limiting. Kubernetes-native API gateways, built on the Kubernetes Gateway API Specification, offer these advanced capabilities while seamlessly integrating with Kubernetes environments. These API-specific gateways can complement or replace traditional ingress controllers, providing enhanced API management features. It's important to note that the Gateway API Specification focuses mainly on routing capabilities and doesn't inherently cover all API management functionalities like business plans, subscriptions, or fine-grained permission validation. API management platforms often extend these capabilities to support features like monetization and access control. Therefore, selecting the right gateway that aligns with both the Gateway API Specification and the organization's API management needs is critical. In API management within an organization, a cell-based architecture may be needed to isolate components or domains. API gateways can be deployed within or across cells to manage communication, ensuring efficient routing and enforcement of policies between isolated components. Control plane and portals for different user personas: While gateways manage API traffic, API developers, product managers, and consumers expect more than basic traffic handling. They need features like API discovery, self-service tools, and subscription management to drive adoption and business growth. Building a robust control plane that lets users control these capabilities is crucial. This ensures a seamless experience that meets both technical and business needs. GitOps for configuration management: GitOps with CRDs extends to API management by using Git repositories for version control of configurations, policies, and security settings. This ensures that API changes are tracked, auditable, and revertible, which is essential for scaling. CI/CD tools automatically sync the desired state from Git to Kubernetes, ensuring consistent API configuration across environments. This approach integrates well with CI/CD pipelines, automating testing, reviewing, and deployment of API-related changes to maintain the desired state. Observability with OpenTelemetry: In Kubernetes environments, traditional observability tools struggle to monitor APIs interacting with distributed microservices. OpenTelemetry solves this by providing a vendor-neutral way to collect traces, metrics, and logs, offering essential end-to-end visibility. Its integration helps teams monitor API performance, identify bottlenecks, and respond to issues in real-time, addressing the unique observability challenges of Kubernetes-native environments. Scalability with Kubernetes' built-in features: Kubernetes' horizontal pod autoscaling (HPA) adjusts API gateway pods based on load, but API management platforms must integrate with metrics like CPU usage or request rates for effective scaling. API management tools should ensure that rate limiting and security policies scale with traffic and apply policies specific to each namespace, supporting multi-environment setups. This integration allows API management solutions to fully leverage Kubernetes' scalability and isolation features. Reference Architecture for Kubernetes-Native API Management To design a reference architecture for API management in a Kubernetes environment, we must first understand the key components of the Kubernetes ecosystem and their interactions. If you're already familiar with Kubernetes, feel free to move directly to the architecture details. Below is a list of the key components of the Kubernetes ecosystem and the corresponding interactions. API gateway: Deploy an API gateway as ingress or as another gateway that supports the Kubernetes Gateway API Specification. CRDs: Use CRDs to define APIs, security policies, rate limits, and observability configurations. GitOps for lifecycle management: Implement GitOps workflows to manage API configurations and policies. Observability with OpenTelemetry: Integrate OpenTelemetry to collect distributed traces, metrics, and logs. Metadata storage with etcd: Use etcd, Kubernetes’ distributed key-value store, for storing metadata such as API definitions, configuration states, and security policies. Security policies and RBAC: In Kubernetes, RBAC provides consistent access control for APIs and gateways, while network policies ensure traffic isolation between namespaces, securing API communications. Key components like the control plane, including consumer and producer portals, along with rate limiting, key and token management services, and developer tools for API and configuration design, are essential to this reference architecture. Figure: Reference architecture for Kubernetes-native API management Conclusion: Embracing Kubernetes-Native API Management for Operational Excellence Managing APIs in Kubernetes introduces unique challenges that traditional API management solutions are not equipped to handle. Kubernetes-native, declarative approaches are essential to fully leverage features like autoscaling, namespaces, and GitOps for managing API configurations and security. By adopting these native solutions, organizations can ensure efficient API management that aligns with Kubernetes' dynamic, distributed architecture. As Kubernetes adoption grows, embracing these native tools becomes critical for modern API-driven architectures. This article was shared as part of DZone's media partnership with KubeCon + CloudNativeCon.View the Event

As with past technology adoption journeys, initial experimentation costs eventually shift to a focus on ROI. In a recent post on X, Andrew Ng extensively discussed GenAI model pricing reductions. This is great news, since GenAI models are crucial for powering the latest generation of AI applications. However, model swapping is also emerging as both an innovation enabler, and a cost saving strategy, for deploying these applications. Even if you've already standardized on a specific model for your applications with reasonable costs, you might want to explore the added benefits of a multiple model approach facilitated by Kubernetes. A Multiple Model Approach to GenAI A multiple model operating approach enables developers to use the most up-to-date GenAI models throughout the lifecycle of an application. By operating in a continuous upgrade approach for GenAI models, developers can harness the specific strengths of each model as they shift over time. In addition, the introduction of specialized, or purpose-built models, enables applications to be tested and refined for optimal accuracy, performance and cost. Kubernetes, with its declarative orchestration API, is perfectly suited for rapid iteration in GenAI applications. With Kubernetes, organizations can start small and implement governance to conduct initial experiments safely and cost-effectively. Kubernetes’ seamless scaling and orchestration capabilities facilitate model swapping and infrastructure optimization while ensuring high performance of applications. Expect the Unexpected When Utilizing Models While GenAI is an extremely powerful tool for driving enhanced user experience, it's not without its challenges. Content anomalies and hallucinations are well-known concerns for GenAI models. Without proper governance, raw models—those used without an app platform to codify governance— are more likely to be led astray or even manipulated into jailbreak scenarios by malicious actors. Such vulnerabilities can result in financial loss amounting to millions in token usage and severely impact brand reputation. The financial implications of security failures are massive. A report by Cybercrime Magazine earlier this year suggests that cybercrime will cost upwards of $10 trillion annually by next year. Implementing effective governance and mitigation, such as brokering models through a middleware layer, will be critical to delivering GenAI applications safely, consistently, and at scale. Kubernetes can help with strong model isolation through separate clusters and then utilize a model proxy layer to broker the models to the application. Kubernetes' resource tagging adds another layer of value by allowing you to run a diverse range of model types or sizes, requiring different accelerators within the same infrastructure. This flexibility also helps with budget optimization, as it prevents defaulting to the largest, most expensive accelerators. Instead, you can choose a model and accelerator combo that strike a balance between excellent performance and cost-effectiveness, ensuring the application remains efficient while adhering to budget constraints. Example 1: Model curation for additional app platform governance and flexibility Moreover, role-based access controls in Kubernetes ensures that only authorized individuals or apps can initiate requests to certain models in an individual cluster. This not only prevents unnecessary expenses from unauthorized usage, but also enhances security across the board. Additionally, with the capacity to configure specific roles and permissions, organizations can better manage and allocate resources, minimize risks, and optimize operational efficiency. Rapidly evolving GenAI models benefit from these governance mechanisms while maximizing potential benefits. Scaling and Abstraction for GenAI! Oh My! The scale of the model you choose for your GenAI application can vary significantly depending on the applications’ requirements. Applications might work perfectly well with a simple, compact, purpose-built model versus a large, complex model that demands more resources. To ensure the optimal performance of your GenAI application, automating deployment and operations is crucial. Kubernetes can be made to facilitate this automation across multiple clusters and hosts using GitOps or other methodologies, enabling platform engineers to expedite GenAI app operations. One of the critical advantages of using Kubernetes for delivering GenAI apps is its ability to handle GPU and TPU accelerated workloads. Accelerators are essential for training and inferencing of complex models quickly and efficiently. With Kubernetes, you can easily deploy and manage clusters with hardware accelerators, allowing you to scale your GenAI projects as needed without worrying about performance being limited by hardware. The same can be said for models optimized for modern CPU instruction sets which helps avoid the need to schedule for more scarce GPUs and TPUs resources. In addition to handling GPU-accelerated workloads, Kubernetes also has features that make it well-suited for inferencing tasks. By utilizing capabilities like Horizontal Pod Autoscaling, Kubernetes can dynamically adjust resources based on the demand for your inferencing applications. This ensures that your applications are always running smoothly and can handle sudden spikes in traffic. On top of all this, the ML tooling ecosystem for Kubernetes is quite robust and allows for keeping data closer to the workloads. For example, JupyterHub can be used to deploy Jupyter notebooks right next to the data with GPUs auto-attached, allowing for enhanced latency and performance during the model experimentation phase. Getting Started With GenAI Apps With Kubernetes Platform engineering teams can be key enablers for GenAI application delivery. By simplifying and abstracting away complexity from developers, platform engineering can facilitate ongoing innovation with GenAI by curating models based on application needs. Developers don't need to acquire new skills in model evaluation and management; they can simply utilize the resources available in their Kubernetes-based application platform. Also, platform engineering can help with improved accuracy and cost effectiveness of GenAI apps by continuously assessing accuracy and optimizing costs through model swapping. With frequent advancements and the introduction of smaller GenAI models, applications can undergo refinements over time. Example 2: How VMware Cloud Foundation + VMware Tanzu leverage Kubernetes Kubernetes is pivotal in this continuous GenAI model upgrade approach, offering flexibility to accommodate model changes while adding access governance to the models. Kubernetes also facilitates seamless scaling and optimization of infrastructure while maintaining high-performance applications. Consequently, developers have the freedom to explore various models, and platform engineering can curate and optimize placement for those innovations. This article was shared as part of DZone's media partnership with KubeCon + CloudNativeCon.View the Event

We are living in a world where the internet is an inseparable part of our lives, and with the growth of Cloud computing and increased demand for AI/ML-based applications, the demand for network capacity is unstoppable. As networks scale exponentially, classical topologies and designs are struggling to keep in sync with the rapidly evolving demands of the modern IT infrastructure. Network management is getting complex due to the sheer amount of network infrastructure and links. AI-driven intent-based networking emerges as a potential solution, promising to reshape our approach to network management — but is it truly the solution to this problem it claims to be? Let’s dive into its details to understand how intent-based networking will be shaping the future of network management. What Is Intent-Based Networking? Traditional intent-based networking (IBN) evolved from software-defined networking (SDN). SDN is a very popular approach in network automation where software-defined controllers and APIs communicate with the physical Infrastructure. IBN is a natural progression of SDN that combines intelligence, analytics, machine learning, and orchestration to automate network management. It translates high-level business intent into network policies to configure the underlying network. IBN abstracts the complex part of underlying hardware, and network configuration to allow users to express their desired intent in natural language. AI-driven IBN brings together intelligence, analytics, machine learning, and orchestration to enhance traditional IBN capabilities. It can translate these intents into specific network configurations and policies more effectively, adapting to changing network conditions and requirements. Most modern, advanced IBN solutions do include ML and NLP to some degree making them AI-driven. Problem Statement: The user wants to balance bulk data transfer and low-latency traffic using available bandwidth. Intent: “Provision low latency for high-performance computing and GPU-accelerated database queries while supporting large dataset transfers.” Network automation with AI-driven IBN: Build and configure intelligent Quality of Service (QoS) policies and device configurations that prioritize low-latency traffic for latency-sensitive workloads over large database queries. Prioritize low latency with the high-priority QoS marking while allowing high-throughput transfers to utilize remaining bandwidth. Machine learning models continuously adjust these policies based on observed application performance. Key Components of Traditional IBN Systems To understand the advancements of AI-driven IBN, let's first examine the key components of a traditional IBN system. In a traditional IBN setup, the system consists of five main components that allow users to interact with the system and for the system to devise actions and implement changes in the network based on user intent. AI-driven Intent-Based Networking Intent Interface It’s a primary point of interaction between users and the IBN system. Network administrators and users can express their desired network configuration in natural language, eliminating its dependency on complex CLI commands and manual configurations. In traditional IBN, this interface typically relies on predefined templates and rule-based interpretation of user inputs. Intent Translation Engine This is the heart of the IBN where business intent is processed through advanced algorithms, and techniques and translated into actionable network configurations and policies. It bridges the gap between human-understandable intents and machine-executable network configurations. Traditional IBN systems use predetermined algorithms and logic trees for this translation process. Network Abstraction Layer This layer provides a unified view of the network, abstracting the underlying complexity of network infrastructure and protocols. It enables the IBN system to work seamlessly with heterogeneous network infrastructures. This abstraction in traditional IBN is often static and may require manual updates to accommodate new network elements or configurations Automation and Orchestration Engine This layer implements translated intents across network infrastructure and leverages software-defined networking to update network configuration and policies. In traditional IBN, this automation is based on predefined scripts and workflows. Continuous Validation and Assurance This feedback loop constantly monitors the network to ensure it follows the requested intent and makes necessary adjustments to maintain optimal performance. Traditional IBN systems perform this validation based on set thresholds and predefined performance metrics. The Role of AI in IBN Integration of AI with traditional Intent-Based Networking allows the system to understand, process, and execute high-level intents without reliance on complex CLI commands, or manual configurations and provides greater flexibility in network management. In this section, we will discuss how AI enhances Intent-Based Networking capabilities and automates network management tasks. The field of AI consists of various subfields such as Natural Language Processing (NLP), Machine Learning (ML), computer vision (CV), and robotics, among others. NLP allows systems to understand and process human language, while ML allows systems to learn from data without explicit programming. These and other AI subfields working with each other help us build intelligent systems. NLP and ML play a significant role in AI-driven IBN by giving the system the ability to understand and execute high-level intents. Natural Language Processing (NLP) NLP serves as the primary interface between network users and the IBN system. NLP allows users to express their intents in natural language and translate it into complex network configurations. Key applications of NLP in IIBN consist of intent translation, context understanding and processing, and automated network config generation. Machine Learning (ML) In AI-driven IBN, ML algorithms allow us to learn from the current network state, predict future states based on the topology and network changes, and make intelligent decisions for network optimization. One of the key applications of ML in IBN is traffic engineering where service providers aim to understand the network behavior, predict the future state, and adjust the network capacity and resources optimally and efficiently. AI-driven IBN is an intelligent system that incorporates both NLP, ML, and other AI subfields to provide the central framework for decision-making and problem-solving in IBN. It enables automated network design, network data analysis, intelligent troubleshooting, policy enforcement, and forecasting of potential failure scenarios. Application in High-Performance Computing Networks AI-driven IBN is a promising solution for hyper-scalar cloud providers who offer High-Performance Computing (HPC) environments, where the demands for high throughput, low latency, flexibility, and resource optimization are especially stringent. Some key applications include: Dynamic Resource Allocation In HPC, AI-driven IBN systems use algorithms such as Q-Learning and Random Forests to allocate network resources optimally by analyzing and predicting the current and future resource demand. These systems can bring flexibility and efficiency by utilizing HPC resources optimally and maximizing performance and network throughput. Workflow-Optimized Traffic Engineering AI-driven IBN systems can continuously analyze the current and future network state and demand to optimize network configurations. This is done by using Time Series Forecasting (e.g., ARIMA, Prophet) for traffic prediction, and anomaly detection algorithms for identifying unusual traffic patterns. The network configuration optimization might involve shifting traffic from a congested primary path to a secondary path, finding high-bandwidth paths for data transfer stages and low-latency paths for distributed computing stages. Fault Tolerance and Resilience IBN systems can predict and simulate potential failures for hardware resources and take proactive action to avoid catastrophic failures. It can triage, auto-mitigate, and remediate the events without interrupting network performance and service. To achieve this, IBN systems employ various algorithms and techniques. Predictive Failure Analysis using machine learning models like Random Forests or Support Vector Machines helps identify potential hardware failures before they occur. Self-healing networks leveraging reinforcement learning algorithms automatically reconfigure network paths when issues arise. These algorithms work together within the IBN framework to maintain robust network performance even in challenging conditions. AI-driven IBN in high-performance computing networks Challenges and Future Directions The availability of sufficient, good-quality data can be the first hurdle that companies have to overcome to be able to implement AI-driven IBN. The black-box nature of some AI/ ML models can lead to opaque decision-making making which needs to be overcome by making these processes transparent and understandable. Enterprise networks are complex and diverse in terms of hardware, configuration, and protocols; managing such enormous network infrastructure requires a lot of computational resources and power. Integration of IBN systems with existing network infrastructure and automation framework Complying with the security standards, polices and authentication becomes challenging with the scale and complexity. Ensuring IBN systems can make decisions and implement changes quickly enough to meet the performance requirements of modern networks As AI-driven IBN systems mature, we can expect to see increased network automation, enhanced machine learning algorithms, improved security, and greater efficiency in network management. However, realizing this future will require overcoming these challenges and addressing the skill gap in the networking industry. Conclusion AI-driven intent-based networking represents a significant advancement in how service providers can operate and manage their complex networks. With the integration of AI into IBN, users can navigate through this complexity, bring operational efficiency, get real-time visibility, and automate network management to bring the network state in sync with the business intent. The future of networking lies in the system that can analyze, interpret, process human intents, and achieve network autonomy by transforming network operations through intent-based networking.

As a Java developer, most of my focus is on the back-end side of debugging. Front-end debugging poses different challenges and has sophisticated tools of its own. Unfortunately, print-based debugging has become the norm in the front end. To be fair, it makes more sense there as the cycles are different and the problem is always a single-user problem. But even if you choose to use Console.log, there’s a lot of nuance to pick up there. Instant Debugging With the debugger Keyword A cool, yet powerful tool in JavaScript is the debugger keyword. Instead of simply printing a stack trace, we can use this keyword to launch the debugger directly at the line of interest. That is a fantastic tool that instantly brings your attention to a bug. I often use it in my debug builds of the front end instead of just printing an error log. How to Use It Place the debugger keyword within your code, particularly within error-handling methods. When the code execution hits this line, it automatically pauses, allowing you to inspect the current state, step through the code, and understand what's going wrong. Notice that while this is incredibly useful during development, we must remember to remove or conditionally exclude debugger statements in production environments. A release build should not include these calls in a production site live environment. Triggering Debugging From the Console Modern browsers allow you to invoke debugging directly from the console, adding an additional layer of flexibility to your debugging process. Example By using the debug(functionName) command in the console, you can set a breakpoint at the start of the specified function. When this function is subsequently invoked, the execution halts, sending you directly into the debugger. function hello(name) { Console.log("Hello " + name) } debug(hello) hello("Shai") This is particularly useful when you want to start debugging without modifying the source code, or when you need to inspect a function that’s only defined in the global scope. DOM Breakpoints: Monitoring DOM Changes DOM breakpoints are an advanced feature in Chrome and Firebug (Firefox plugin) that allow you to pause execution when a specific part of the DOM is altered. To use it, we can right-click on the desired DOM element, select “Break On,” and choose the specific mutation type you are interested in (e.g., subtree modifications, attribute changes, etc.). DOM breakpoints are extremely powerful for tracking down issues where DOM manipulation causes unexpected results, such as dynamic content loading or changes in the user interface that disrupt the intended layout or functionality. Think of them like the field breakpoints we discussed in the past. These breakpoints complement traditional line and conditional breakpoints, providing a more granular approach to debugging complex front-end issues. This is a great tool to use when the DOM is manipulated by an external dependency. XHR Breakpoints: Uncovering Hidden Network Calls Understanding who initiates specific network requests can be challenging, especially in large applications with multiple sources contributing to a request. XHR (XMLHttpRequest) breakpoints provide a solution to this problem. In Chrome or Firebug, set an XHR breakpoint by specifying a substring of the URI you wish to monitor. When a request matching this pattern is made, the execution stops, allowing you to investigate the source of the request. This tool is invaluable when dealing with dynamically generated URIs or complex flows where tracking the origin of a request is not straightforward. Notice that you should be selective with the filters you set: leaving the filter blank will cause the breakpoint to trigger on all XHR requests, which can become overwhelming. Simulating Environments for Debugging Sometimes, the issues you need to debug are specific to certain environments, such as mobile devices or different geographical locations. Chrome and Firefox offer several simulation tools to help you replicate these conditions on your desktop. Simulating user agents: Change the browser’s user agent to mimic different devices or operating systems. This can help you identify platform-specific issues or debug server-side content delivery that varies by user agent. Geolocation spoofing: Modify the browser’s reported location to test locale-specific features or issues. This is particularly useful for applications that deliver region-specific content or services. Touch and device orientation emulation: Simulate touch events or change the device orientation to see how your application responds to mobile-specific interactions. This is crucial for ensuring a seamless user experience across all devices. These are things that are normally very difficult to reproduce; e.g., touch-related issues are often challenging to debug on the device. By simulating them on the desktop browser we can shorten the debug cycle and use the tooling available on the desktop. Debugging Layout and Style Issues CSS and HTML bugs can be particularly tricky, often requiring a detailed examination of how elements are rendered and styled. Inspect Element The "inspect element" tool is the cornerstone of front-end debugging, allowing you to view and manipulate the DOM and CSS in real time. As you make changes, the page updates instantly, providing immediate feedback on your tweaks. Addressing Specificity Issues One common problem is CSS specificity, where a more specific selector overrides the styles you intend to apply. The inspect element view highlights overridden styles, helping you identify and resolve conflicts. Firefox vs. Chrome While both browsers offer robust tools, they have different approaches to organizing these features. Firefox’s interface may seem more straightforward, with fewer tabs, while Chrome organizes similar tools under various tabs, which can either streamline your workflow or add complexity, depending on your preference. Final Word There are many front-end tools that I want to discuss in the coming posts. I hope you picked up a couple of new debugging tricks in this first part. Front-end debugging requires a deep understanding of browser tools and JavaScript capabilities. By mastering the techniques outlined in this post — instant debugging with the debugger keyword, DOM and XHR breakpoints, environment simulation, and layout inspection — you can significantly enhance your debugging efficiency and deliver more robust, error-free web applications. Video

By Shai Almog CORE