Planet IM

September 24, 2020

Publishing and Subscribing with Halcyon

Publishing and Subscribing with Halcyon

As you recall, Halcyon is multiplatform XMPP library written in Kotlin. In a previous article: “A look at Halcyon” we had a look at basic concepts in library and we created a simple client.

This time we will dive into more complex stuff. We will create simple solution to monitoring temperature at home :-) In this article we will not focus on measuring temperature. We will create a command-line tool to publish temperature provided as parameter.

First letter in XMPP acronym is from the word “eXtensible”. There is a lot of extensions for the XMPP protocol. One of them is XEP-0060: Publish-Subscribe - specification for publish-subscribe functionality. We will use it to create our temperature monitor.

You need to use XMPP Server with PubSub component. You can use your deployment (for example Tigase XMPP Server or use one of the publicly available servers, for example sure.im and its PubSub component pubsub.sure.im. A PubSub node with unique name (to avoid conflicts) will have to be created in the PubSub component. Please note that node created with default configuration is open, which means that everyone can subscribe to it (but only you will be able to publish data there).

Data structure

First of all we have to create data structure. In our case, it will be as simple as possible:

<temperature timestamp="1597946187562">23.8</temperature>

timestamp is time represented as a number of milliseconds after January 1, 1970 00:00:00 GMT.

We can use DSL (defined in Halcyon) to create such XML fragment:

val payload = element("temperature") {
    attributes["timestamp"] = (Date()).time.toString()
	+temperature.toString()
}

Publisher

Publisher is a simple XMPP client that connects to the server, sends information to PubSub component and immediately disconnects.

First of all, lets define global values to keep node name and PubSUB JID:

val PUBSUB_JID = "pubsub.tigase.org".toJID()
val PUBSUB_NODE = "temperature_in_my_house"

It cannot be called a good practice, but is good enough for us right now :-)

In the previous article we explained how to create a simple client. Now we will focus on PubSubModule. This module allows publishing and receiving events as well as managing PubSub nodes and subscriptions.

This is the main code that publishes events:

pubSubModule.publish(PUBSUB_JID, PUBSUB_NODE, null, payload).handle {
    success { request, iq, result ->
        println("YAY! Published with id=${result!!.id}")
    }
    error { request, iq, errorCondition, s ->
        System.err.println("ERROR $errorCondition! $s")
    }
}.send()

But what if the PubSub node doesn’t exist (e.g. it wasn’t created yet)? It’s simple: we have to create it using method create():

pubSubModule.create(PUBSUB_JID, PUBSUB_NODE).handle {
    success { _: IQRequest<Unit>, _: IQ, _: Unit? -> println("Got it! Node created!") }
    error { _: IQRequest<PubSubModule.PublishingInfo>, _: IQ?, errorCondition: ErrorCondition, msgs: String? ->
        println(
            "OOPS! Cannot create node $errorCondition $msgs"
        )
    }
}.send()

The question is: under what conditions we should call this part of code and automatically create the node? One of the possibilities would be moment when item publishing fails with error item-not-found.

pubSubModule.publish(PUBSUB_JID, PUBSUB_NODE, null, payload).handle {
    success { request, iq, result ->
        println("YAY! Published with id=${result!!.id}")
    }
    error { request, iq, errorCondition, s ->
        if (errorCondition == ErrorCondition.ItemNotFound) {
            println("Node not found! We need to create it!")
            pubSubModule.create(PUBSUB_JID, PUBSUB_NODE).handle {
                success { _: IQRequest<Unit>, _: IQ, _: Unit? -> println("Got it! Node created!") }
                error { _: IQRequest<PubSubModule.PublishingInfo>, _: IQ?, errorCondition: ErrorCondition, msgs: String? ->
                    println(
                        "OOPS! Cannot create node $errorCondition $msgs"
                    )
                }
            }.send()
        } else System.err.println("ERROR $errorCondition! $s")
    }
}.send()

To simplify the code, publishing will not be repeated after node creation.

It is good to use client.waitForAllResponses() before disconnect(), to not break connection before all responses comes back.

Listener

Listener is also a client (it should works on different account) that subscribes to receiving events from specific nodes of PubSub component. PubSub items received by PubSubModule are distributed in the client as PubSubEventReceivedEvent in Event Bus. To receive those events you have to register an events listener:

client.eventBus.register<PubSubEventReceivedEvent>(PubSubEventReceivedEvent.TYPE) {
    if (it.pubSubJID == PUBSUB_JID && it.nodeName == PUBSUB_NODE) {
        it.items.forEach { item ->
            val publishedContent = item.getFirstChild("temperature")!!
            val date = Date(publishedContent.attributes["timestamp"]!!.toLong())
            val value = publishedContent.value!!
            println("Received update: $date :: $value°C")
        }
    }
}

Note, that this listener will be called on every received PubSub event (like OMEMO keys distribution, PEP events, etc). That’s why you need to check node name and JabberID of PubSub component.

Your client will not receive anything from PubSub if it does not subscribe to specific node. Because subscription is persistent (at least with default node configuration), client doesn’t need to subscribe every time it connects to the server. Though, it should be able to check if it’s subscribed to the specific node or not. For that, you need to retrieve list of subscribers and see if the JabberID of the client is on the list:

val myOwnJID = client.getModule<BindModule>(BindModule.TYPE)!!.boundJID!!
pubSubModule.retrieveSubscriptions(PUBSUB_JID, PUBSUB_NODE).response {
    if (!it.get()!!.any { subscription -> subscription.jid.bareJID == myOwnJID.bareJID }) {
        println("We have to subscribe")
        pubSubModule.subscribe(PUBSUB_JID, PUBSUB_NODE, myOwnJID).send()
    }
}.send()

NOTE: In this example we intentionally skipped checking response errors.

PubSub component can keep some history of published elements. We can retrieve that list easily:

pubSubModule.retrieveItem(PUBSUB_JID, PUBSUB_NODE).response {
    when (it) {
        is IQResult.Success -> {
            println("Previously published temperatures:")
            it.get()!!.items.forEach {
                val date = Date(it.content!!.attributes["timestamp"]!!.toLong())
                val value = it.content!!.value!!
                println(" - $date :: $value°C")
            }
        }
        is IQResult.Error -> println("OOPS! Error " + it.error)
    }
}.send()

Length of the history is defined in node configuration.

Sample output

Submitting new temperature in Publisher…: publishing

yields receiving notifications in Listener: listening

Summary

We presented a simple way to create a PubSub publisher and consumer. You can extend it: for example you can run publisher on Raspberry Pi connected to some meteo-sensors. Possible applications of PubSub component are limited only by your imagination.

All source codes for this article can be found in GitHub repository.

September 15, 2020

Using STUN & TURN server with Tigase XMPP Server with XEP-0215 (External Service Discovery)

Communication with your family and friends is not only about instant chats. Audio and Video calls are quite important and sometimes, under unfavourable network configurations establishing a call may prove difficult. Luckily, with the help of STUN (Session Traversal Utilities for NAT) and TURN (Traversal Using Relays around NAT ) servers it’s no longer a problem

In the following guide we will show how to setup TURN and STUN servers with Tigase XMPP Server, so that compatible XMPP clients will be able to use them. Our xmpp.cloud installation supports not only them, but also XMPP MIX

Assumptions

We are assuming that you have installed your preferred TURN server and created an account on the TURN server for use by your XMPP server users and that you have installed and configured Tigase XMPP Server.

At the end of the article there is a short guide hot to quickly setup CoTURN server.

Enabling external service discovery (required only for Tigase XMPP Server 8.1.0 and earlier)

First you need to edit etc/config.tdsl file and:

  1. Add following line in the main section of the file:

    'ext-disco' () {}
  2. Add following line in the sess-man section of the file:

    'urn:xmpp:extdisco:2' () {}

so that your config file would look like this:

'ext-disco' () {}
'sess-man' () {
    'urn:xmpp:extdisco:2' () {}
    …
}
…

Start Tigase XMPP Server

After applying changes mentioned above, you need to start Tigase XMPP Server or, in case if it was running, restart it.

Open Admin UI

Open web browser and head to http://<your-xmpp-server-and-port>/admin/ (for example: https://localhost:8080). When promped, log in by providing admin user credentials: bare JID (i.e.: user@domain) as the user and related password. Afterwards you’ll see main Web AdminUI screen:

web admin main page

and on that screen open Configuration group on the left by clicking on it.

Add external TURN service

After opening Configuration group (1) click on Add New Item (2) position which has ext-disco@… in its subtitle.

In the opened form you need to provide following detail: web admin add new turn item

  • Service - ID of the service which will be used for identification by Tigase XMPP Server (ie. turn@example.com)
  • Service name - name of the service which may be presented to the user (ie. TURN server)
  • Host - fully qualified domain name of the TURN server or its IP address (ie. turn.example.com)
  • Port - port at which TURN server listens (ie. 3478)
  • Type - type of the server, enter turn
  • Transport - type of transport used for communication with the server udp or tcp (usually udp)
  • Requires username and password - for notifying XMPP client that this service requires its username and password for XMPP service (leave unchecked)
  • Username - username required for authentication for TURN server (ie. turn-user)
  • Password - password required for authentication for TURN server (ie. turn-password)

After filling out the form, press Submit button (3) to send form and add a TURN server to external services for your server. Admin UI will confirm that service was added with the following result web admin add new item confirmation

Add external STUN service

While adding a TURN server is usually all what you need, in some cases you may want to allow your users to use also STUN. Steps are quite similar like on TURN server - after opening Configuration group (1) click on Add New Item (2) position which has ext-disco@… in its subtitle and in the opened form you need to provide following detail: web admin add new stun item

  • Service - ID of the service which will be used for identification by Tigase XMPP Server (ie. stun@example.com)
  • Service name - name of the service which may be presented to the user (ie. STUN server)
  • Host - fully qualified domain name of the STUN server or its IP address (ie. stun.example.com)
  • Port - port at which TURN server listens (ie. 3478)
  • Type - type of the server, enter stun
  • Transport - type of transport used for communication with the server udp or tcp (usually udp)
  • Requires username and password - for notifying XMPP client that this service requires its username and password for XMPP service (leave unchecked)
  • Username - username required for authentication for STUN server (if required)
  • Password - password required for authentication for STUN server (if required)

Note

If you are using the same server for STUN and TURN (you usually will as TURN servers usually contain STUN functionality) you will fill the following form with almost the same details *(only use different Service field value, Type will be stun and most likely you will skip passing Username and Password - leaving them empty, the rest of the field values will be the same).

After filling out the form, press Submit button (3) to send form and add a STUN server to external services for your server. Admin UI will confirm that service was added with the following result web admin add new item confirmation

And now what?

Now you have fully configured your STUN/TURN server for usage with Tigase XMPP Server allowing XMPP clients connected to your server and compatible with XEP-0215: External Service Discovery to take full advantage of your STUN/TURN server ie. by providing better VoIP experience.

CoTURN installation

You can quickly setup CoTURN server using Docker. Please follow Docker installation on your operating system and then install CoTURN using Docker Hub (instrumentisto/coturn). The bare minimum required to run it looks like that (please update realm with your domain and external-ip with IP on which server should be accessible):

sudo docker run --name coturn -d --network=host --restart always  instrumentisto/coturn -n --log-file=stdout --min-port=49160 --max-port=49200 --realm=awesomexmpp.net --external-ip=<external_ip> -a'

Subsequently, add user to CoTURN with password and domain:

sudo docker exec -i -t coturn turnadmin -a -u tigase -r awesomexmpp.net -p Ajbk7Ck38nIobLVl
Last updated: October 25, 2020 04:00 AM