Transfer Data With Animated QR codes

QR code is useful to transfer data across different devices. An Android device can exchange data with an iOS device through a QR code. It does not require setting up a wireless network and does not have the hassle of pairing using Bluetooth. As it is a screen-camera solution, it is also safe to use.

As discussed in the previous article, a QR code can store up to 2,953 bytes with its version set to 40 and its error correction level set to low in bytes mode, which is quite enough for transferring data like a web link, the info of driver’s licenses, etc.

What if we want to transfer more data using QR codes?

We can separate the data into several QR codes and retrieve the entire data by decoding these codes. We can print these codes on one paper or animate them on a screen. The Dynamsoft Barcode Reader (DBR) has the ability to read multiple codes in one image and can also read from the video stream.

In this article, we are going to focus on transferring data with animated QR codes.

There have been studies about this and one of them has stood out which is TXQR. It uses fountain codes to add redundancy to the data so that skipping a few frames will not affect the reading. It uses a repeated codes mode before turning to fountain codes. The program is implemented in Go.

However, using fountain codes brings extra complexity to the process, and actually, with a good barcode reading SDK, the repetition mode can also have a satisfying rate. In the following part, we are going to create an animated QR code generator and an animated QR code reader in pure JavaScript. Users can select a file to transfer and retrieve it on another device by scanning the QR codes.

Animated QR Codes Generator in JavaScript

The file to transfer is first read as bytes and divided into chunks by a fixed chunk size. Metadata is appended to each chunk’s beginning. The metadata is simple. It is the index of the chunk and the total number of chunks and if the chunk is the first one, the filename and the mime type are appended as well.

An example of chunks:

1. “1/11|example.webp|image/webp|bytes”
2. “2/11|bytes”
3. “3/11|bytes”

After the chunks are created, we use a JavaScript library to generate QR codes and animate them in a loop.

Users can adjust the chunk size and interval of QR codes to help the receiver have a better reading.

Here are the key parts of the code.

<input type="file" id="file" onchange="loadfile()"/>
<label for="name">Chunk size (bytes):</label>
<input type="text" id="chunkSize" name="chunkSize" value="2000">
<label for="name">Extra interval (ms):</label>
<input type="text" id="interval" name="interval" value="200">
<div id="placeHolder"></div>
<script>
qrcode.stringToBytes = function(data) { return data; }; //store bytes directly
function loadfile() { 
    let files = document.getElementById('file').files;
    if (files.length == 0) {
        return;
    }
    var file = files[0];
    fileReader = new FileReader();
    fileReader.onload = function(e){
        loadArrayBufferToChunks(e.target.result,file.name,file.type);
        showAnimatedQRCode();
    };
    fileReader.onerror = function () {
        console.warn('oops, something went wrong.');
    };
    fileReader.readAsArrayBuffer(file);
}

function loadArrayBufferToChunks(bytes,filename,type){
    var bytes = new Uint8Array(bytes);
    var data = concatTypedArrays(stringToBytes(encodeURIComponent(filename)+"|"+type+"|"),bytes); //The filename is encoded for non-ascii characters like Chinese.
    var chunkSize = parseInt(document.getElementById("chunkSize").value);
    var num = Math.ceil(data.length / chunkSize)
    chunks=[];
    for (var i=0;i<num;i++){
        var start = i*chunkSize;
        var chunk = data.slice(start,start+chunkSize);
        var meta = (i+1)+"/"+num+"|";
        chunk = concatTypedArrays(stringToBytes(meta),chunk);
        chunks.push(chunk);
    }
}

function showAnimatedQRCode(){
    createQRCode(chunks[currentIndex]);
    currentIndex = currentIndex + 1
    var interval = parseInt(document.getElementById("interval").value)
    setTimeout("showAnimatedQRCode()",interval);
}

function createQRCode(data){
    var typeNumber = 0;
    var errorCorrectionLevel = 'L';
    var qr = qrcode(typeNumber, errorCorrectionLevel);
    qr.addData(data);
    qr.make();
    var placeHolder = document.getElementById('placeHolder');
    placeHolder.innerHTML = qr.createSvgTag(); // or createImgTag
}

function stringToBytes(s) {
    var bytes = [];
    for (var i = 0; i < s.length; i += 1) {
        var c = s.charCodeAt(i);
        bytes.push(c & 0xff);
    }
    return bytes;
}

//https://stackoverflow.com/questions/33702838/how-to-append-bytes-multi-bytes-and-buffer-to-arraybuffer-in-javascript
function concatTypedArrays(a, b) { //array + unint8 array
    var newLength = a.length + b.byteLength;
    console.log(newLength);
    var c = new Uint8Array(newLength);
    c.set(a, 0);
    c.set(b, a.length);
    return c;
}
</script>

Animated QR Codes Reader in JavaScript

Now that we have a generator. We need a reader to read the animated QR codes. We can create native mobile applications with high performance using the Dynamsoft Barcode Reader’s mobile SDK. But for the ease of use and cross-platform concerns, here, we are gonna use the JavaScript version of DBR to create the reader.

Read QR Codes from Video Stream

The JavaScript version of DBR is easy to use. We can start reading QR codes from the video stream using the following code.

<!DOCTYPE html>
<html>
<head>
    <meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0,user-scalable=0" />
    <script src="https://cdn.jsdelivr.net/npm/dynamsoft-javascript-barcode@8.6.1/dist/dbr.js"></script>
</head>
<body>
<input type="button" value="Start Scanning" onclick="startScanning();" />
<script>
let scanner = null;
init();
async function init(){
    scanner = await Dynamsoft.DBR.BarcodeScanner.createInstance();
    scanner.onFrameRead = results => {
        //handle barcode results
    };
    scanner.onUnduplicatedRead = (txt, result) => {
    };
}

async function startScanning(){
    if (scanner==null){
        alert("Please wait for the initialization of the scanner.")
        return;
    }
    await scanner.show();
}
</script>
</body>
</html>

Read Chunks and Restore the Original Data

Now, we have to parse the data of the QR codes and try to restore the data after all the chunks are received.

var total = 0;
var code_results = {};
function processRead(result){
    var text = result["barcodeText"];
    try {
        var meta = text.split("|")[0];
        var totalOfThisOne = parseInt(meta.split("/")[1]);
        if (total!=0){
            if (total != totalOfThisOne){ //QR codes for another file
                total = totalOfThisOne;
                code_results={};
                return;
            }
        }
        
        total = totalOfThisOne;
        var index = parseInt(meta.split("/")[0]);
        code_results[index]=result;
        if (getObjectLength(code_results)==total){
            onCompleted();
        }
    
    } catch(error) {
        console.log(error);
    }
}
function onCompleted(){
    showResult(timeElapsed);
}

async function showResult(timeElapsed){
    //combine chunks of bytes
    var jointData = [];
    for (var i=0;i<getObjectLength(code_results);i++){
        var index = i+1;
        var result = code_results[index];
        var bytes = result.barcodeBytes;
        var text = result.barcodeText;
        if (index == 1){
            var filename = text.split("|")[1]; //the first one contains progress|filename|image/webp|data
            var mimeType = text.split("|")[2];
            var firstSeparatorIndex = text.indexOf("|");
            var secondSeparatorIndex = text.indexOf("|",firstSeparatorIndex+1);
            var dataStart = text.indexOf("|",secondSeparatorIndex+1)+1;
            data = bytes.slice(dataStart,bytes.length);
        }else{
            var dataStart = text.indexOf("|")+1;
            data = bytes.slice(dataStart,bytes.length);
        }
        jointData = jointData.concat(data);
    }
    //display results on the page
    //...
    resetResults();
}

function resetResults(){
    code_results={};
    total = 0;
}

The decoded results are displayed in a list on the web page.

One list item contains a link to download the file, the elapsed time, and the download speed. If the file is an image, an img element will be appended as well.

HTML:

<div id="decodedResults">
    Results:
    <ol id="decodedList">
    </ol>
</div>

JavaScript:

async function showResult(timeElapsed){
    //......
    var decodedList = document.getElementById("decodedList");
    var item = document.createElement("li");
    decodedList.appendChild(item);
    var dataURL = await ArraybufferAsDataURL(jointData,mimeType);
    appendDownloadLink(item, dataURL,filename);
    var info = document.createElement("span");
    info.innerText = " elapsed time: " + timeElapsed + "ms" +" speed: "+ (jointData.length/1024/(timeElapsed/1000)).toFixed(2) +"KB/s";
    item.appendChild(info);
    if (dataURL.indexOf("image")!=-1) {
        var img = document.createElement("img");
        img.src = dataURL;
        img.style.display = "block";
        img.style.maxHeight = "350px";
        img.style.maxWidth = "100%";
        item.appendChild(img);
    }
}

//https://stackoverflow.com/questions/12710001/how-to-convert-uint8-array-to-base64-encoded-string
const ArraybufferAsDataURL = async (data,mimeType) => {
    // Use a FileReader to generate a base64 data URI
    const dataUrl = await new Promise((r) => {
        const reader = new FileReader()
        reader.onload = () => r(reader.result)
        var array = ConvertToUInt8Array(data);
        var blob = new Blob([array],{type: mimeType});
        reader.readAsDataURL(blob)
    })
    return dataUrl;
}

function ConvertToUInt8Array(data){
    var array = new Uint8Array(data.length);
    for (var i=0;i<data.length;i++){
        array[i] = data[i];
    }
    return array;
}

function appendDownloadLink(item, dataURL,filename){
    var link = document.createElement('a');
    link.setAttribute('target', '_blank');
    link.setAttribute('href', dataURL);
    if (!filename){
        filename = "file";
    }
    link.setAttribute('download', filename);
    link.innerText=filename;
    item.appendChild(link);
}

function getObjectLength(obj){
    return getObjectKeys(obj).length;
}

function getObjectKeys(obj){
    return Object.keys(obj);
}

During the decoding process, it will also show the statistics like time elapsed, frames processed, successfully-read frames and the progress.

function updateStatistics(timeElapsed){
    var statisticsPre = document.getElementById("statisticsPre");
    statistics = "elapsed time: " + (timeElapsed)/1000 +"s";
    statistics = statistics +"\ntotal frame number: " + framesRead;
    statistics = statistics +"\nsuccessful number: " + successNum;
    statistics = statistics +"\nsuccess fps: " + (successNum/(timeElapsed/1000)).toFixed(2);
    statistics = statistics +"\nprogress: " + getObjectLength(code_results) + "/" + total;
    statisticsPre.innerHTML=statistics;
}

Define the UI

The scanner’s UI is customizable.

Create a scanner element and then bind the element with the following code:

await scanner.setUIElement(document.getElementById('scanner'));

There are several pre-defined classes: dbrScanner-video, dbrScanner-cvs-drawarea, dbrScanner-cvs-scanarea, dbrScanner-scanlight, dbrScanner-sel-camera, dbrScanner-sel-resolution. DBR JS will find and make use of them if they exist.

The scanner element:

<div id="scanner" style="display:none;">
    <video class="dbrScanner-video" playsinline="true" style="width:100%;height:100%;position:absolute;left:0;top:0;"></video>
    <canvas class="dbrScanner-cvs-drawarea" style="width:100%;height:100%;position:absolute;left:0;top:0;"></canvas>
    <div class="dbrScanner-cvs-scanarea" style="width:100%;height:100%;position:absolute;left:0;top:0;">
        <div class="dbrScanner-scanlight" style="width:100%;height:3%;position:absolute;animation:3s infinite dbrScanner-scanlight;border-radius:50%;box-shadow:0px 0px 2vw 1px #00e5ff;background:#fff;display:none;"></div>
    </div>
    <select class="dbrScanner-sel-camera" style="margin:0 auto;position:absolute;left:0;top:0;height:30px;"></select>
    <select class="dbrScanner-sel-resolution" style="position:absolute;left:0;top:30px;"></select>
</div>

If you don’t want to show some element, like the barcode overlay canvas, you can simply delete it.

Improve the Reading Speed

The transferring speed of this screen-camera solution is mainly decided by how many QR code images can the receiver capture and decode in a fixed period.

Mobile devices can capture 30 frames per second but it may take hundreds of milliseconds to decode one frame. The decoding performance affects the speed the most.

There are several ways to improve this. The Dynamsoft Barcode Reader provides rich parameters to optimize the performance for a specific usage scenario.

Scan Only QR Codes

The Dynamsoft Barcode Reader supports a multitude of barcode formats. We can update the settings to scan QR codes only so that it will not spend extra effort finding other barcode formats.

let settings = await scanner.getRuntimeSettings();
settings.barcodeFormatIds = Dynamsoft.DBR.EnumBarcodeFormat.BF_QR_CODE;
await scanner.updateRuntimeSettings(settings);

Set Up a Scan Region

The QR code is just a part of the entire video frame We can set up a scan region so that the QR code takes up most of the frame.

let settings = await scanner.getRuntimeSettings();
settings.region.regionMeasuredByPercentage = 1; //use percentage
var video = document.getElementsByTagName("video")[0];
if (video.videoHeight>video.videoWidth){
    settings.region.regionLeft = 0;
    settings.region.regionTop = 25;
    settings.region.regionRight = 100;
    settings.region.regionBottom = 75;
}else{
    settings.region.regionLeft = 25;
    settings.region.regionTop = 0;
    settings.region.regionRight = 75;
    settings.region.regionBottom = 100;
}
await scanner.updateRuntimeSettings(settings);

Use a Faster Localization Method

The Dynamsoft Barcode Reader comes with several barcodes localization methods. The scan directly mode is suitable for reading QR codes with a phone.

let settings = await scanner.getRuntimeSettings();
settings.localizationModes = [Dynamsoft.DBR.EnumLocalizationMode.LM_SCAN_DIRECTLY, 0, 0, 0, 0, 0, 0, 0, 0];
await scanner.updateRuntimeSettings(settings);

Test and Conclusion

A test is run for this combination of animated QR code generator and scanner to examine how it performs with different chunk sizes and intervals on an iOS device and an Android device. The speed results of three continuous readings are recorded (for large files, only one record).

Here are the test results on iPhone SE 2016:

• File size: 15.93KB, Chunk size: 1800, Interval: 100, Speed: 6.90KB/s, 10.68KB/s, 5.35KB/s
• File size: 15.93KB, Chunk size: 1800, Interval: 200, Speed: 7.22KB/s, 7.17KB/s, 7.47KB/s
• File size: 15.93KB, Chunk size: 1800, Interval: 400, Speed: 3.94KB/s, 3.99KB/s, 3.90KB/s
• File size: 15.93KB, Chunk size: 1800, Interval: 800, Speed: 2.09KB/s, 2.08KB/s, 2.05KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 100, Speed: 16.38KB/s, 4.26KB/s, 8.56KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 200, Speed: 11.25KB/s, 11.33KB/s, 11.32KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 400, Speed: 6.78KB/s, 6.74KB/s, 6.74KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 800, Speed: 3.56KB/s, 3.61KB/s, 3.70KB/s
• File size: 231KB, Chunk size: 1800, Interval: 400, Speed: 3.01KB/s
• File size: 231KB, Chunk size: 2900, Interval: 400, Speed: 2.12KB/s

Here are the test results on Sharp AQUOS S2 (the CPU power is weaker):

• File size: 15.93KB, Chunk size: 1800, Interval: 100, Speed: 1.15KB/s, 1.24KB/s, 0.74KB/s
• File size: 15.93KB, Chunk size: 1800, Interval: 200, Speed: 2.18KB/s, 1.38KB/s, 1.91KB/s
• File size: 15.93KB, Chunk size: 1800, Interval: 400, Speed: 3.44KB/s, 2.60KB/s, 4.06KB/s
• File size: 15.93KB, Chunk size: 1800, Interval: 800, Speed: 2.03/KB/s, 2.08KB/s, 2.07KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 100, Speed: 1.05KB/s, 1.80KB/s, 0.93KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 200, Speed: 5.42KB/s, 5.37KB/s, 9.19KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 400, Speed: 5.93KB/s, 7.10KB/s, 3.12KB/s
• File size: 15.93KB, Chunk size: 2900, Interval: 800, Speed: 3.90KB/s, 3.53KB/s, 3.68KB/s
• File size: 231KB, Chunk size: 1800, Interval: 400, Speed: 2.06KB/s
• File size: 231KB, Chunk size: 2900, Interval: 400, Speed: 1.05KB/s

We can make a conclusion based on this test heuristically.

1. The solution works great for transferring small-sized files which are under 200KB. Because the speed is limited and since more QR codes are needed to encode a large file, the chance of missing frames for large files is high.
2. The chunk size and interval should be adjusted accordingly. If the chunk size and the interval are small, the generator can generate QR codes fast, however, the receiver may not catch them in time. If the chunk size is large, more data can be transferred in the same time span, but the receiver has to spend more time decoding and it may miss frames, especially for low-end devices.

This solution may be improved in the following ways:

1. Improve the speed by using a new format, like colored QR code, which can have a larger data capacity.
2. Improve the missed frames problem by using fountain codes.
3. Improve the missed frames problem by making the current unidirectional communication to a bidirectional communication so that the generator will only show the undecoded QR codes. This can be done by using the generator to scan a QR code displayed on the receiver’s screen.
4. Use data compression. Instead of transferring JPEG files, transfer WebP files. (conversion tool)

Source Code

https://github.com/xulihang/AnimatedQRCodeReader