Mesh Chat: Messaging for AREDN mesh networks

Quick Links:

What is mesh chat?
Zones
Channels
Installing mesh chat
Using mesh chat
API Usage
Theory of Operation
Change Log

 

What is mesh chat?

 

AREDN mesh networks are awesome. AREDN mesh networks are mainly used for emergency communications. So once you have this great IP mesh network setup, what do you do with it? People have been doing VOIP, IP cameras, Team Talk, etc. In an emergency situation you need a reliable communication channel. Power, phone, Internet, etc will probably be down or not usable. In our district we had been using Team Talk to communicate. This worked ok, but has several problems:

• Single point of failure. If the raspberry pi that is running the server software or the node it is connected to goes down your toast.
• Not real easy to use.
• No history of chat logs. If you logout or get disconnected you lose your chat history.

I wrote mesh chat to overcome the above limitations and provide a reliable text messaging system for AREDN mesh networks. You use the application via a modern responsive web interface that renders well on desktop and mobile devices. The web application runs on the mesh node itself or a raspberry pi. Sending a message is as easy as typing one into a text box and clicking submit. The message database is stored on every node mesh chat is running on. There is a daemon that runs and will get a list of all nodes in the mesh running mesh chat. The daemon then polls those nodes several times a minute and gets any new messages and a list of users logged into mesh chat on that node. If any new messages are found, they are appended to the local message database.

With this design, nodes can go up and down, vary in connectivity and as long as at least 1 node stays running the message database is persisted. Once nodes come back online they will immediate catch up with the other nodes and get a full copy of the message database. If you loose connectivity from your laptop / phone to the mesh, your messages remain in the message database. It is a very reliable, distributed P2P mesh messaging system.

 

Zones

 

Meshchat is configured to run on a particular zone based on the service name it is give in the AREDN services screen. This lets you run multiple installations of Meshchat on a single mesh and separate messages. For example we have a few zones of Meshchat running with a LWMeshChat, SLCMeshChat and UTMeshChat. These are grouped based on geography of our mesh users. Meshchat will only poll nodes that have the same service name as its own.

 

Channels

 

Channels separate messages inside a zone. We have an Emergency channel for example that CERT users would monitor so they would not need to filter out other HAM communications. You can also use channels to trigger Action Scripts.

 

Installing mesh chat

 

Installation instructions are below based on which version of AREDN you are running on your node.

 

Installing packages in AREDN

To install a package in AREDN browse to http://localnode:8080 or the address of the node you want to work on. Click on Setup from the page you first get to:

Enter root for the user name and the password you entered when you setup the node. Now Click on Administration. Click on the Choose file button and select the package you downloaded to your computer. Now click upload and wait for a bit and it will upload and install the package.

For each package you install you will see some status text back. Generally you can ignore this unless you see an error. If you do put a post on the AREDN Forum to get some help.

 

AREDN v3.15.1.0

You will need to install some other packages first. Download the following packages and install them in the order listed below.

http://downloads.arednmesh.org/barrier_breaker/14.07/ar71xx/generic/packages/base/libpolarssl_1.3.9-2_ar71xx.ipk

http://downloads.arednmesh.org/barrier_breaker/14.07/ar71xx/generic/packages/base/libcurl_7.38.0-1_ar71xx.ipk

http://downloads.arednmesh.org/barrier_breaker/14.07/ar71xx/generic/packages/base/curl_7.38.0-1_ar71xx.ipk

Now you can download the mesh chat package and install it:

https://s3.amazonaws.com/aredn/meshchat_1.02_all.ipk

 

AREDN v3.16.1.0+

All the curl packages are already installed, so just download and install mesh chat:

https://s3.amazonaws.com/aredn/meshchat_1.02_all.ipk

Now you need to edit the service name that was created on installation to match the zone you want this installation to sync with. Meshchat will only poll other AREDN nodes whose service / zone name matches it’s own. Login to the node and go to Setup -> Port Forwarding, DHCP, and Services. Edit the default Meshchat-XXXX service name to the zone you want to sync with.

Raspberry Pi

For mesh chat to work correctly on a pi you need to install the API package on an AREDN node accessible to the pi. Meshchat will use that node to get the list of services on the mesh so it knows which nodes to poll for messages that match its zone.

From the command line on your raspberry pi install the required dependency packages first:

sudo apt-get install curl apache2

Download the mesh chat package and install it:

wget https://s3.amazonaws.com/aredn/meshchat_1.02_all.deb
dpkg -i meshchat_1.02_all.deb

Now we need to publish a service for the zone we want this installation in. Meshchat will poll other AREDN nodes whose service name match it’s own. Login to the node and go to Setup -> Port Forwarding, DHCP, and Services. Make sure you have a DHCP reservation for the Pi, then add a service for it with the name of the zone you want to sync with.

Save the changes.

Now on the pi edit the file /usr/lib/cgi-bin/meshchatconfig.pm to enter the zone and AREDN node that has the API package installed. Edit these two lines:

our $pi_zone = 'LWMeshChat';
our $local_meshchat_node = 'localnode';

Now browse to the ip or host name of your pi: http://<pi ip>/meshchat

 

Using mesh chat

 

Meshchat is really easy to use. When you browse the web interface for the first time you will be prompted to enter your call sign. You can put in your call sign or any other identifier you want your messages tagged as.

After clicking Login you will see the main messaging page:

 

Enter a new message in the text field choose which channel to send to and hit send. Your message is stored immediately in the local message db. Once other nodes in your zone poll your node, the new message will propagate through the mesh. The users online table shows users whose browser has pulled new messages from the node within the last 2 minutes. Between 2-4 minutes their name will be grayed out and after 4 minutes it is removed from the list.

Click on Files in the navigation to use file sharing:

 

Files are stored on the node you upload them to. The file links point to the http address of the file on that node. If you are logged into the node with the file you will see the delete button to remove the file. On an AREDN node the file storage is limited to 500 KB. The files are stored on tmpfs which is RAM. Not a lot of room. If you want to store large files, install mesh chat on a raspberry pi where the storage is limited by the size of the SD card or what ever mount point /var/www/html/meshchat/files is on. You can soft link that directory to a NFS mount or anything you want. Mesh chat will recognize any files in that directory and list them on the nodes. Only files can be in that directory, no sub directories. Great way to share VOIP phone books, images, etc for the mesh users.

 

API Usage

 

Here are a few useful API’s for mesh chat. These are subject to change between versions, but should be pretty set at this point.

Get the message db in JSON format:

curl http://localnode:8080/cgi-bin/meshchat?action=messages

[{"epoch":1455769952,"message":"Good talking to you N7IE","call_sign":"K7FPV","node":"laytonwestdistrict","platform":"pi"},{"epoch":1455769897,"message":"What IP cam is a good one?","call_sign":"N7IE","node":"n7ie-omni-15","platform":"node"},{"epoch":1455769886,"message":"You should login and check out my ip cam","call_sign":"K7FPV","node":"laytonwestdistrict","platform":"pi"},{"epoch":1455769814,"message":"Yeah mesh has been working great today.","call_sign":"N7IE","node":"n7ie-omni-15","platform":"node"}]

Get the logged in users in JSON format:

curl http://localnode:8080/cgi-bin/meshchat?action=users

[{"epoch":1455773975,"id":"LXGpq","call_sign":"K7FPV","node":"laytonwestdistrict","platform":"pi"},{"epoch":1455770201,"id":"zPXkR","call_sign":"N7IE","node":"n7ie-omni-15","platform":"node"}]

Get the files list in JSON format:

curl http://localnode:8080/cgi-bin/meshchat?action=files

{"stats":{"allowed":2097152,"files":623,"files_free":2096529,"total":14774272,"used":155648}, "files":[{"epoch":1455667822,"size":"623","file":"control.tar.gz","local":1,"node":"k7fpv","platform":"node"},{"epoch":1455051611,"size":"54003","file":"Tony Node.xlsx","local":0,"node":"laytonwestdistrict","platform":"pi"}]}

Get polling status in JSON format:

curl http://localnode:8080/cgi-bin/meshchat?action=sync_status

[{"epoch":1455775790,"node":"n7ie-omni-15"},{"epoch":1455775788,"node":"k7fpv-nsm2"},{"epoch":1455775787,"node":"kd7dho-m3"},{"epoch":1455775786,"node":"laytonwestdistrict"},{"epoch":1455744291,"node":"k7fpv-lwst"},{"epoch":1455678944,"node":"n7ie-001-bbhn-v31-bullet"},{"epoch":1455678843,"node":"kd7dho-bullet-003"}]

Send a message:

curl \
--data-urlencode "action=send_message" \
--data-urlencode "message=Hello World from API" \
--data-urlencode "call_sign=K7FPV" \
--data-urlencode "epoch=1455768943" \
http://localnode:8080/cgi-bin/meshchat

{"status":200, "response":"OK"}

Sending a message via the api could be useful in scripts or automated software on the mesh. Like sending the battery voltage in an off grid site to the message db every day, etc.

There are several others you can see by browsing the source code here.

Source code for mesh chat be on github.

 

Theory of Operation

 

Ok so you want the knit and gritty of how mesh chat works. HAM’s like to see under the covers so here ya go, deep dive.

The UI is built using skeleton css and jquery. Javascript uses AJAX to query the cgi script for messages every 5 seconds and for the list of logged in users every 5 seconds. Data is sent back from the cgi script in JSON format. Once you login, your call sign is stored in a cookie. The meshchat cgi script provides API’s to download data in JSON format or the raw tab delimited files from the file system. The UI get’s is data in JSON format and meshchatsync get the tab delimited data so it can save that right to disc with little processing. Also we don’t have a lot of room on the nodes and JSON would just bloat the message db. Pretty straight forward on the UI, the magic happens in the meshchatsync daemon.

meshchatsync is a daemon started on boot and it’s job is to poll nodes and find new messages and add those to the local message database. We get a list of nodes from OLSR. OLSR is the routing daemon / protocol that runs routing, node discovery, and service advertising on the mesh. We get a list of all services on the mesh and find all the ones that advertise a mesh chat service. Those are the nodes that are polled in the run loop.

The list of users and files is pulled every time and merged with local data. That data is fairly small. The message db is what is large and is handled more efficiently to cut down on traffic on the mesh. The messaging db is a tab delimited file that looks like this:

b26ace5a	1449275071	Testing.	n7ie	n7ie-001-bbhn-v31-bullet	pi
a07db977	1453001465	It must be the weather.	KD7DHO	kd7dho-bullet-003	node
34a93a6b	1451797154	Message from George's node	K7FPV	k7fpv	node
a3bf0947	1448048744	from dave's node	K7FPV	KD7DHO-bullet-003	node
b014f1d1	1449112251	Message from the truck	K7FPV	K7FPV	node

Fields:

1. Message ID. Random 8 character hex string generated on message creation
2. Epoch
3. Message
4. Call Sign
5. Originating node
6. Originating platform

A message db version is created after any changes are made to the message db. It is calculated by looping through all the messages and converting the hex string to decimal and summing all the id’s together. Messages are not stored in order in the db. They are sorted client side in javascript when displayed. We have no idea when we will get a new message when we poll a node and this let’s race conditions happen without concern. During the loop we ask a node for its message db version, only if our db version does not math the polled node do we download the whole db and merge in new messages. Since most of the time the db version will match, we can quickly move on and not move the whole db over the mesh. Every time a message is added to the db, we check to see if the message db is within the size limit. If it is too big, we start deleting messages until it is under the limit.

There is a global configuration file at /www/cgi-bin/meshchatconfig.pm on AREDN nodes and at /usr/lib/cgi-bin/meshchatconfig.pm on raspberry pi. Here is a description of the settings:

our $meshchat_path              = "/tmp/meshchat";                        # Working directory for meshchat to use
our $max_messages_db_size       = 512 * 1024;                             # Max size of message database. Stored in RAM on AREDN nodes so be careful.
our $max_file_storage           = 512 * 1024;                             # Max storage for shared files. Stored in RAM on AREDN nodes so be careful.
our $lock_file                  = $meshchat_path . '/lock';               # Lock file use to get exclusive access to files in working directory
our $messages_db_file           = $meshchat_path . '/messages';           # Message database
our $sync_status_file           = $meshchat_path . '/sync_status';        # Polling sync status
our $local_users_status_file    = $meshchat_path . '/users_local';        # Users logged into the local node
our $remote_users_status_file   = $meshchat_path . '/users_remote';       # Users logged into other nodes
our $remote_files_file          = $meshchat_path . '/files_remote';       # Files stored on other nodes
our $messages_version_file      = $meshchat_path . '/messages_version';   # Message database version
our $local_files_dir            = $meshchat_path . '/files';              # Path to store shared files for this node
our $pi_nodes_file              = $meshchat_path . '/pi';                 # List of raspberry pi nodes
our $tmp_upload_dir             = '/tmp/web/upload';                      # Temp dir to upload files to
our $poll_interval              = 10;                                     # How long to sleep in the polling loop between polling all nodes
our $non_meshchat_poll_interval = 600;                                    # How long to poll nodes who have an advertised meshchat service, but it is returning 404
our $connect_timeout            = 5;                                      # curl timeout on polling requests
our $platform                   = 'node';                                 # Platform mesh chat is installed on, node or pi
our $debug                      = 1;                                      # Enable debug messages
our $extra_nodes                = [];                                     # Array of extra nodes to poll not from OLSR

our $pi_zone                    = 'LWMeshChat';                           # Zone to poll messages for
our $local_meshchat_node        = 'localnode';                            # Host name of AREDN node that has the API package installed

That’s it for mesh chat. Any questions please leave a comment below or on the AREDN forum.

 

Change Log

v1.02
Alphabetical file list
File list column width changes
Fixes for long channel names
Stretch support

v1.0
First production release.

v0.7b7 Release Notes:
Fixes special characters not encoding into JSON
Adds random number to default service announcement to prevent message database problems on first install
Layout fixes
File download and url fixes

v0.7b5 Release Notes:
Use proper port number for file sharing links
Fixed race condition with OLSR and zone discovery, this was causing sync issues
Fix init.d stop to not use killall
Mobile layout fixes
Item counts at top of tables

v0.7b4 Release Notes:
UI queries the message db version and only downloads when it does not match what it has loaded. This dramatically reduces the network traffic and load on the node.
Tighten up white space to bring messages above the fold
Added message search
Change file list sync to use one file per node vs a merged db

v0.7b3 Release Notes:

v0.7b2 Release Notes:

v0.6
Mobile and tablet style fixes
Get list of nodes running meshchat from OLSR vs polling every node
Change from wget-nossl to curl
Added config setting $tmp_upload_dir for USB drives on nodes
Loading spinner on Send button and better send message error handling and notification

v0.4
Initial production version