Multiple data streams over just one socket with mux-demux and shoe (node)

Dec 21, 2014

In a realtime app it’s common to have multiple data streams coming from the server. The mechanism to handle the different connections could include among others:

• frequent pooling to the server (possible perf issues if multiple requests are made at the same time)
• one socket per data stream ( possible perf issues in server and client)

A better alternative

Multiple data streams but just one channel (1 web socket).

This is what mux-demux refers to (multiplex-demultiplex), and can be very useful for situations when having multiple transport channels (in this case sockets) is not practical.

How to use mux-demux?

To have a reference for future projects i created a minimal sample that demostrates how to send multiple data streams over one connection. The following modules are used:

shoe: Creates a stream over a websocket so we can use the same streams Api we love.

reconnect-core : Gives us the ability to execute a method everytime we need to reconnect to the server in case connection is lost.

mux-demux : Inject multiple streams into another stream and extract them from the other side.

browserify : Use npm modules in the browser.

The example consist of 9 streams that emit random numbers. We want to send those streams to the browser and display them in realtime.

Animated gif:

The code

Example repository

The important part is here (index.js)

	var shoe = require('shoe');
	var through = require('through');
	var http = require('http');
	var ecstatic = require('ecstatic');
	var MuxDemux = require('mux-demux');
	var server,sock,sourcestreams = [];
	var samplestream = require("./samplestream.js");
	//Create an array with 100 streams and send them through one shoe connection
	for(var i = 0;i<10;i++){
	sourcestreams.push(new samplestream());
	}
	//Setup http server, use ecstatic to serve the content in public directory.
	server = http.createServer(ecstatic({root:__dirname+'/public'}));
	server.listen(7777,function(){
	console.log("listening on port 7777");
	});
	//Setup websocket connection
	sock = shoe(function(stream){
	var mx = MuxDemux();
	//Setup mux-demux "device"
	stream.pipe(mx).pipe(stream);
	//For each sample stream in sourcestreams array
	//create a write stream , mux-demux will create readable
	//streams on the other side (the browser)
	for (var i=0;i<sourcestreams.length;i++) {
	//We have to name each stream so we can get them in the browser code
	var name = "mystream"+i;
	var datasetstream = mx.createWriteStream(name);
	sourcestreams[i].pipe(datasetstream);
	};
	});
	//Create one socket in the path /numbers
	sock.install(server,'/numbers');

view raw mux-demux-sample.js hosted with ❤ by GitHub

Reconnection.

If the server goes down, we need to reconnect. For this there is a module callled reconnect-core that we can use.

Check browser.js to see how to implement it.