Manual Reference Source Test

Installation

NPM

npm install netflux

3 peer dependencies to be installed for some cases:

  • rxjs is necessary for both NodeJS and browsers if you want to take advantage of EcmaScript modules, tree-shaking etc. Otherwise for it is already included into dist/netflux.browser.es5.umd.js and dist/netflux.node.es5.cjs.js bundles.
npm install rxjs
  • uws and text-encoding if you target NodeJS (developing a bot):
npm install uws text-encoding

Netflux has an optional dependency: wrtc. This package provides WebRTC API in NodeJS, but for now it is not in use as more tests needed. Checkout wrtc repository for more info on it.

What you need

Signaling server is the only mandatory server for Netflux, but for a fully functional peer to peer network which suits all use cases you also need STUN and TURN servers.

Netflux comes with Signaling and STUN servers by default for easier quickstart.

Signaling server

Default: wss://signaling.netflux.coedit.re

The only signaling mechanism which is supported by Netflux for now is Sigver (NodeJS WebSocket server developed by us).

TIP: Your own instance of Sigver for production is recommended.

STUN server

Default: stun:stun3.l.google.com:19302

There are many other free STUN servers available in the Web.

TURN server

There are no free TURN servers available in the Web. Two solutions exist:

  • Rent one. Checkout Xirsys for example.
  • Deploy your own instance. The paragraphe below provides a guide on how to deploy and configure coturn open source TURN server. Also checkout Choosing a TURN server for a list of open source TURN servers.

How to deploy STUN/TURN servers

This guide present a basic configuration for coturn. It allow us to deploy our own STUN/TURN server in order to connect users behind a NAT or Firewall.

For more informations or options, see coturn documentation.

On most Linux Distribution, the package is in the repo: apt-get install coturn

You must have SQLite installed: sudo apt-get install sqlite3 libsqlite3-dev

Config file can be found at: /etc/turnserver.conf

Simple config file should look like this:

# you can listen ports 3478 and 5349 instead of 80/443
listening-port=80
tls-listening-port=443

listening-ip=YOUR_IP_ADDRESS
relay-ip=YOUR_IP_ADDRESS

realm=YOUR_DOMAIND.COM
server-name=OUR_DOMAIND.COM

fingerprint

# webRTC authentication method
lt-cred-mech

# WebRTC credentials
user=YOUR_USER_NAME:YOUR_PASSWORD

# Quota
total-quota=100
bps-capacity=0
stable-nonce

# Add ssl certificate for your server
cert=/etc/ssl/certificate.pem
pkey=/etc/ssl/private.key
cipher-list="ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:!DH+AES:!ECDH+3DES:!DH+3DES:!RSA+AES:!RSA+3DES:!ADH:!AECDH:!MD5"
no-loopback-peers
no-multicast-peers
no-stdout-log

If you don't have any SSL certificate, you may use Let's Encrypt.

Launch server:

`turnserver`

or in daemon:

turnserver -o

Verify that the server is up and running with Trickle ICE.

Usage

There are 4 builds (all ES5 code):

  • dist/netflux.cjs.js CommonJS format for NodeJS.
  • dist/esm/netflux.node.js ECMAScript 6 module format for NodeJS.
  • dist/esm/netflux.browser.js ECMAScript 6 module format for browsers.
  • dist/netflux.umd.js UMD format for browsers.

The package.json contains the following fields:

...
"main": "dist/netflux.cjs.js",
"module": "dist/esm/index.node.js",
"browser": "dist/esm/index.browser.js",
...

All builds are either for NodeJS or for Browser environment.

For browser environment exported members are:

  • WebGroup class
  • WebGroupState enum
  • SignalingState enum
  • Topology enum
  • DataType type
  • WebGroupOption type

For NodeJS environment exported members are the same as for browser plus:

  • Bot class
  • BotOptions type.

CommonJS

dist/netflux.cjs.js - CommonJS format, built for NodeJS.

// NodeJS
const netflux = require('netflux')
var wg = new netflux.WebGroup()

ES module

esm/index.node.js and esm/index.browser.js are suitable for Webpack, Browserify or any alike, which also undersands package.json#module and package.json#module properties respectively and can parse ES modules.

esm/index.node.js is build for NodeJS: contains all exported API members and all necessary polyfills for NodeJS environment.

esm/index.browser.js is build for browsers.

export { WebGroup, WebGroupState, Bot, BotOptions } from 'netflux'
const wg = new WebGroup()

UMD

dist/netflux.umd.js - Universal Module Definition format is compatible with AMD, CommonJS and "global" modules. Built for browser and suitable for Webpack, Browserify and any other who also understands package.json#browser property.

<!-- Browser global usage example -->
<script src="netflux.es5.umd.js">
  window.netflux !== undefined // true
  var wg = new window.netflux.WebGroup()
</script>

Configuration

For a WebGroup object all options are optional.

// Example:
const wg = new WebGroup({
  signalingServer: 'MY_SIGNALING_URL',
  rtcConfiguration: {
    iceServers: [
      { urls: 'stun:mystun.org' },
      {
        urls: ['turn:myturn.org?transport=udp', 'turn:myturn.org?transport=tcp'],
        username: 'user',
        password: 'password',
      },
    ],
  },
})

For Bot the server option is mandatory.

// Example:
const http = require('http')
const myServer = http.createServer()
const wg = new Bot({
  server: myServer,
  signalingServer: 'MY_SIGNALING_URL',
  webGroupOptions: {
    rtcConfiguration: {
      iceServers: [
        { urls: 'stun:mystun.org' },
        {
          urls: ['turn:myturn.org?transport=udp', 'turn:myturn.org?transport=tcp'],
          username: 'user',
          password: 'password',
        },
      ],
    },
  },
})

Known WebRTC obstacles

Connection establishment over WebRTC may ancounter different obstacles like NAT, Firewall, port blocking. But besides network obstacles there is also browser configuration that may prevent from connection creation.

Firefox extensions

Some extensions for Firefox (Privacy Badger for instance) modify WebRTC configuration in browser. Checkout about:config and look for peerconnection.ice. If the Status column is default then the parameter was not modified.

FAQ

How Netflux chooses between RTCDataChannel and WebSocket

Netflux operates with RTCDataChannel (WebRTC) and WebSocket connection technologies.

Technically RTCDataChannel may be established between:

  • Browser and Browser.
  • Browser and Server.
  • Server and Server.

On the other hand, a WebSocket may be created only between:

  • Browser and Server.
  • Server and Server.

But in practice:

  • All modern browsers support WebSocket (Chrome, Firefox, Edge, Safary etc.), but only some of them support RTCDataChannel (Chrome, Firefox, Safari soon and Edge one day maybe).
  • Servers may or may not listen on WebSocket or connect over WebSocket and may or may not support RTCDataChannel.

This technical diversity pushes us to try all possibilities in order to create a connection between two peers based on their capabilities.

Between two peers, there is always one who initiates the connection, while the other is in a passive state. Lets say peer _A_ wants to connect to peer _B_ (i.e. _A_ is the initiator).

The following algorithm is the same for _A_ and _B_ and is executed each time a message is received. It runs through all possiblities in order to establish a connection between these peers (i.e. return SUCCESS if the connection succeed and return FAILED otherwise). It is important that the initiator peer (i.e. A) must be notified about the algorithm's result, while for the second peer (who is in the passive state) this information has no importance.

message variable below persistes the state of the actions for both peers. It is updated by both and is exchanged between them.

// Pseudocode:
// I received a `message` from another peer

if anotherPeerIsListeningOnWebSocketAndIHaveNotTriedToConnectYet(message)
  if WebSocket connection succeed
    return SUCCESS
  else
    update(message)

if iAmListeningOnWebSocketAndAnotherPeerHasNotTriedToConnectYet(message)
  send(message)
  return

if bothPeersSupportRTCDataChannel(message)
  if iHaveNotTriedToConnectYet(message)
    if RTCDataChannel connection succeed
      return SUCCESS
    else
      update(message)
  if anotherPeerHasNotTriedToConnectYet(message)
    send(message)
    return

if iAmTheInitiator(message)
  return FAILED
else
  send(message)

State description

This section describes what trigger the WebChannel (i.e. WebGroup) state to change.

Possible values for WebChannel, Signaling and Topology states are:

WebChannel description
JOINING You are about to join the group.
JOINED You have successfully joined the group.
LEFT You have successfully left the group or couldn't join. This is also the initial value (i.e. after new WebChannel() and before calling join()).
Signaling state description
CONNECTING The connection with Signaling server is not yet open (equivalent to WebSocket.CONNECTING).
OPEN The connection with Signaling server is open (equivalent to WebSocket.OPEN).
CHECKING Signaling server is about to vefiry that you are connected to at least one group member. If so, then the state becomes CHECKED and nothing to do, otherwise it prompt you to connect to the group member. After that the state becomes CHECKED and the TopologyService should rejoin the group.
CHECKED You are connected to at least one group member or you are the only group member.
CLOSED The connection with Signaling server is closed or couldn't be opened (equivalent to WebSocket.CLOSED).
Topology state description
CONSTRUCTING Thanks to the Signaling server you are connected to one of the group member. From now the topology service is about to create the rest of connections, if necessary, in order to keep the topology structure and assure messages delivery.
CONSTRUCTED The topology is considerated as constructed.
IDLE All connections with group members were closed or topology failed to construct the peer to peer network.

Events

Member left event

Singaling Topology adjacent member not only you left in the group Result
CHECKED CONSTRUCTED true true $signalingCheck

Signaling state event

Singaling Topology connected rejoinEnabled Result
CLOSED IDLE true $rejoin
CLOSED IDLE false $leave
CLOSED CONSTRUCTED true $connectToSignaling
OPEN !CONSTRUCTING $signalingCheck
CHECKED IDLE true Topology.CONSTRUCTED
CHECKED CONSTRUCTED WebChannel.JOINED

Topology state event

Topology Singaling rejoinEnabled Result
CONSTRUCTING WebChannel.JOINING
CONSTRUCTED OPEN $signalingCheck
CONSTRUCTED CHECKED WebChannel.JOINED & $signalingCheck
CONSTRUCTED CLOSED $connectToSignaling
IDLE CLOSED true $rejoin
IDLE CLOSED false $leave
IDLE CONNECTING WebChannel.JOINING
IDLE OPEN WebChannel.JOINING & $signalingCheck
IDLE CHECKING WebChannel.JOINING
IDLE CHECKED $signalingCheck

Join method call

WebChannel onLine Visibility Result
LEFT true visible $join

Leave method call

WebChannel Result
!LEFT $leave

onLine() event (online/offline events, navigator.onLine)

WebChannel Visibility rejoinEnabled Result
LEFT visible true $leave.1 & $join

isVisible() event (Page Visibility API)

WebChannel onLine rejoinEnabled Result
LEFT true true $leave.1 & $join

$join -> consists of three steps:

  1. Initialize WebChannel properties (ids, members etc.), then
  2. Set WebChannel state to JOINING and finally
  3. Connect to Signaling server

$leave -> consists of two steps

  1. Clean WebChannel properties (ids, members, clean servicies) as if we were doing new WebChannel(), then
  2. Set WebChannel state to LEFT

$rejoin -> consists of executing the following algorithm:

  Execute $leave.1
    Execute $join.1
    Execute $join.2
    setTimeout
      if (WebChannel.LEFT && visible && onLine && rejoinEnabled) then
        $connectToSignaling

$connectToSignaling -> the third step of the $join process

$signalingCheck -> check with Signaling if still connected with the rest of the group