Tutorial: Developing a Scala Application and Connecting It to ScyllaDB NoSQL

Background

Mutants have emerged from the shadows and are wreaking havoc on the earth! Increased levels of malicious mutant behavior pose a threat to national security and the general public. Luckily, some mutants have teamed up with Government agencies to help protect people against the bad ones.

To better protect the citizens and understand more about the mutants, the Government enacted the Mutant Registration Act. As required by the act, each mutant must wear a small device that reports on their actions every second. Your mission, as a new member of Division 3, is to help the Government keep the Mutants under control by building a Mutant Monitoring System (MMS). The MMS will consist of a database that will contain a Mutant Catalog and Monitoring system. As the number of mutants is on the rise, Division 3 decided that we must use more applications to connect to the mutant catalog and decided to hire developers to create powerful applications that can monitor the mutants. 

A NoSQL database is preferred for MMS because it has high availability, a scale-out data set, disaster recovery capabilities, and can accommodate IoT time-series workloads—all the requirements for the MMS database. There are many NoSQL database technologies to choose from, such as HBase, Apache Cassandra, MongoDB, InfluxDB, and ScyllaDB.

ScyllaDB was chosen for the MMS because it leverages the best from Apache Cassandra in high availability, fault tolerance, and rich ecosystem. ScyllaDB provides a dramatically higher-performing and resource-effective NoSQL database to power modern and demanding applications with low latencies and none of the Java garbage collection woes found in other NoSQL databases.

A collection of data on each mutant needs to be imported into the database. The Mutant Catalog database will keep track of basic contact information for each mutant, including the mutant’s first name, last name, address, and photo.

To monitor the mutants, we need a separate tracking database consisting of time-series data such as the mutant’s location, speed, velocity, heat, and telepathy powers. The data will be gathered from various sources and imported into the database. After the data is gathered, we will be able to analyze the statistics. 

Other tutorials can walk you through other aspects of this project. This lesson explores how to connect to a ScyllaDB cluster using the Phantom library for Scala: a Scala-idiomatic wrapper over the standard Java driver. 

When creating applications that communicate with a database such as ScyllaDB, it is crucial that the programming language being used includes support for database connectivity. Since ScyllaDB is compatible with Cassandra, we can use any of the available Cassandra libraries. For example, in Go, there is GoCQL and GoCQLX. In Node.js, there is the cassandra-driver. For the JVM, we have the standard Java driver available. Scala applications on JVM can use it, but a library tailored for Scala’s features offers a more enjoyable and type-safe development experience. Since Division 3 wants to start investing in Scala, let’s begin by writing a sample Scala application.

Creating a Sample Scala Application

The sample application that we will create connects to a ScyllaDB cluster, displays the contents of the Mutant Catalog table, inserts and deletes data, and shows the table’s contents after each action. First, we will go through each section of the code used and then explain how to run the code in a Docker container that accesses the ScyllaDB Mutant Monitoring cluster. You can see the file here:

As mentioned, we’re using the Phantom library for working with ScyllaDB. The library’s recommended usage pattern involves modeling the database, table, and data service as classes. We’ll delve into those in a future lesson. 

First, let’s focus on how we set up the connection itself. We start by importing the library: 

import com.outworkers.Phantom.dsl._

This import brings all the necessary types and implicits into scope. The main entry point for our application is the App object’s main method. In it, we set up the connection to the cluster: 

val connection: CassandraConnection =
  ContactPoints(List("scylla-node1", "scylla-node2", "scylla-node3"))
    .keySpace("catalog")

Next, in a list, we specify the DNS names through which the application contacts ScyllaDB and the keyspace to use. Finally, we instantiate the MutantsDatabase and the MutantsService classes. These classes represent, respectively, the collection of tables in use by our application and a domain-specific interface to interact with those tables. 

We’re not operating on low-level types like ResultSet or Row objects with the Phantom library, but rather on strongly-typed domain objects. Specifically, for this lesson, we will be working with a Mutant data type defined as follows: 

case class Mutant(
    firstName: String,
    lastName: String,
    address: String,
    pictureLocation: String
)

You can find the file here: 

The MutantsService class contains implementations of the functionality required for this lesson. You can find it here:

Let’s consider how we would fetch all the mutant definitions present in the table:

def getAll(): Future[List[Mutant]] =
  db.mutants.select.all().fetch()

Phantom provides a domain-specific language for interacting with ScyllaDB. Instead of threading CQL strings through our code, we’re working with a strongly-typed interface that verifies, at compile-time, that the queries we are generating are correct (according to the data type definitions). 

In this case, we are using the select method to fetch all Mutant records from ScyllaDB. 

Note: by default, Phantom uses Scala’s Future data type for representing all the ScyllaDB operations. Behind the scenes, Phantom uses a non-blocking network I/O to efficiently perform all of the queries. If you’d like to know more about Future, this is a good tutorial. 

Moving forward, the following method allows us to store an instance of the Mutant data type into ScyllaDB: 

def insertMutant(mutant: Mutant): Future[ResultSet] =
  db.mutants.store(mutant).future()

Everything is strongly typed, and as such, we’re using actual instances of our data types to interact with ScyllaDB. This gives us increased maintainability of our codebase. If we refactor the Mutant data type, this code will no longer compile, and we’ll be forced to fix it. 

And lastly, here’s a method that deletes a mutant with a specific name: 

   firstName: String,
    lastName: String
): Future[ResultSet] =
  db.mutants
    .delete()
    .where(_.firstName.eqs(firstName))
    .and(_.lastName.eqs(lastName))
    .future()

We can see that Phantom’s DSL can represent predicates in a type-safe fashion as well. We’re forced to use a matching type for comparing the firstName and lastName. 

We can sequence the calls to these methods in our main entry point using a for comprehension on the Future values returned. The code below is from the following file:

for {
  _ <- svc.getAll().andThen {
    case Success(mutants) =>
      println(s"Mutants: ${mutants.mkString(", ")}")
  }
  _ <- svc.insertMutant(
    Mutant(
      "Mike",
      "Tyson",
      "1515 Main St.",
      "https://www.facebook.com/mtyson"
    )
  )
  _ <- svc.getByName("Mike", "Tyson").andThen {
    case Success(mutant) =>
      println(s"Found mutant: ${mutant}")
  }
} yield ()

In this example, we execute all the futures sequentially (with some additional callbacks registered on them for printing out the information). 

With the coding part done, let’s set up a ScyllaDB cluster and then run the sample application in Docker.

Set up a ScyllaDB Cluster

The example requires a single DC cluster. Follow this procedure to remove previous clusters and set up a new cluster. 

Once the cluster is up, we’ll create the catalog keyspace and populate it with data. 

The first task is to create the keyspace named catalog for the mutants’ catalog.

docker exec -it mms_scylla-node1_1 cqlsh

CREATE KEYSPACE catalog WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy','DC1' : 3};

Now that the keyspace is created, it is time to create a table to hold the mutant data.

use catalog;

CREATE TABLE mutant_data (
    first_name text,
    last_name text,
    address text,
    picture_location text,
    PRIMARY KEY((first_name, last_name)));

Now let’s add a few mutants to the catalog with the following statements:

insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Bob','Loblaw','1313 Mockingbird Lane', 'http://www.facebook.com/bobloblaw');
insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Bob','Zemuda','1202 Coffman Lane', 'http://www.facebook.com/bzemuda');
insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Jim','Jeffries','1211 Hollywood Lane', 'http://www.facebook.com/jeffries');

Building the Scala Example

If you previously built the Scala Docker, you can skip directly to the section Running the Scala Example below. Otherwise, to build the application in Docker, change into the scala subdirectory in scylla-code-samples:

cd scylla-code-samples/mms/scala

Now we can build and run the container:

docker build -t scala-app .

docker run -d --net=mms_web --name some-scala-app scala-app

To connect to the shell of the container, run the following command:

docker exec -it some-scala-app sh

Running the Scala Example

Finally, the sample Scala application can be run:

java -jar scala-app/target/scala-2.13/mms-scala-app-assembly-0.1.0-SNAPSHOT.jar

The output of the application will be:

sh-4.2# java -jar scala-app/target/scala-2.13/mms-scala-app-assembly-0.1.0-SNAPSHOT.jar
Mutants: Mutant(Bob,Loblaw,1313 Mockingbird Lane,http://www.facebook.com/bobloblaw), Mutant(Jim,Jeffries,1211 Hollywood Lane,http://www.facebook.com/jeffries), Mutant(Bob,Zemuda,1202 Coffman Lane,http://www.facebook.com/bzemuda)
Found mutant: Some(Mutant(Mike,Tyson,1515 Main St.,https://www.facebook.com/mtyson))
sh-4.2# 

Conclusion

In this tutorial, we explained how to create a sample Scala application that executes a few basic CQL operations on a ScyllaDB cluster using the Phantom library. These were only the basics, and there are more exciting topics that Division 3 wants developers to explore.  

In the meantime, please be safe out there and continue to monitor the mutants!