paddatrapper

I am one of the maintainers of LibreTime, an open source, online radio scheduling and automation platform. One of the features LibreTime offers is live shows via Icecast mounts. These allow DJs or admins to stream content live to LibreTime that will then be played out instead of the scheduled media in real time to listeners.

Icecast mounts are great for a single person connecting and streaming content. They do not support multiple sources connecting at the same time. This makes bringing in a second DJ or guest difficult, as the group connection needs to happen before the Icecast source connection.

Mumble is an open source group audio chat application that allows users to connect and interact in rooms that can be linked together in a variety of ways. It is available on every major computing platform – Windows, OSX, Linux, Android and iOS and can be configured to imitate a studio environment so that DJs and guests are comfortable in some of the aspects of what is going on. Mumble also supports bot accounts, these bots can connect to the server and do a variety of things, for example play audio into a room or stream audio from a room to somewhere else.

I spent the last month or so developing a system that would allow people to stream a Mumble room to Icecast so that DJs and guests could participate remotely in a Mumble show, while streaming out to Icecast. The implementation includes a Mumble bot for playing out media from the LibreTime library, allowing radio stations to control what is going out on air.

Implementation

This implementation uses the following components:

• Mumble – used for presenting the show
• Botamusique – used to play media from LibreTime's library into Mumble
• MumbleIce – used to mix together the Mumble audio streams and to stream that to Icecast

The Mumble server is configured to require a server password and has 2 rooms under root: Studio and Off-Air under that. Studio and Off-Air are linked, with ACLs preventing people in Off-Air from speaking in Studio. This allows DJs and guests to talk to each other during music breaks, while still being able to hear what is going out on air. Both Botamusique and MumbleIce bots are configured to join the Studio room.

Botamusique is setup to run on the same server as LibreTime. I use LXC containers, so I mounted the LibreTime library into both containers. In the Botamusique configuration, I disabled the ability to delete media, disabled the ability to upload media and changed the command symbol to #. The last option was to avoid conflicts between botamusique and MumbleIce.

MumbleIce runs on my Kubernetes cluster and the configuration is mostly default. The Icecast credentials are those required to connect to the master mount in LibreTime.

To present a show, the DJ simply joins studio, types #web into chat to get the botamusique website URL with access token, types !connect to connect to LibreTime and starts to stream the show. At the end of the show, they type !disconnect to stop streaming. The stream automatically stops streaming to LibreTime if no one speaks for more than 30 seconds. This is to protect against streaming silence after a show if the DJ forgets to disconnect the stream before leaving.

I have spent the last week getting Jitsi setup for streaming to RTMP endpoints. While the directions in the Jibri docs are fairly straight forward, I found it very difficult to troubleshoot when things went wrong.

Jitsi has multiple moving parts, and the streaming/recording section is no different. It is done through a separate app, Jibri, which launches a Chrome instance, controlled through Selenium, captures the output and streams/records it with FFmpeg.

If there are issues with the recorder authenticating against the Prosody XMPP server while Jibri is connecting to the call, it will timeout and the error

SEVERE: [117] org.jitsi.jibri.selenium.pageobjects.CallPage.visit() Timed out waiting for call page

and the prosody log will show an incoming connection, but no corresponding authentication success.

This issue is fixed by running the following on the Jitsi server:

prosodyctl register recorder recorder.domain.org $password

The other issue I encountered is that Jibri will start streaming and stop with

 timeout: no media received

if there are any NAT issues trying to negotiate the WebRTC connection. I found no other errors in the logs, only that it timed out after 30s. The 30s is defined in the Jibri code. I even saw the live stream correctly determine the number of people in the call and show their avatars, but it failed to show video, even when it was enabled. I fixed this by moving the Jibri service to run on the same server as Jitsi, with no NAT between it and the Internet.

One final note is that Jibri, Chrome and FFmpeg require a fair amount of memory and CPU. So small VPS setups with 1 vCPU and under 4GB of memory will fail to work, again with nothing in the logs. top is your friend in diagnosing this issue.

With the COVID-19 outbreak, may conferences have moved to be fully virtual. This has been achieved by streaming talks and using IRC or other channels for audience participation. Libreplanet used Jitsi to connect the moderators and presenters and IRC to facilitate discussion between attendees. The talks were streamed to their website, where people could watch them.

DebConf 19 also experimented with remote participation for two talks: What's new in the Linux kernel and Anti-Harassment Bof. These used RTMP and Jitsi respectively.

In both cases, I have observed several limitations that prevent the experience from feeling fully engaging and from flowing smoothly. Jitsi provided real-time communication, but this does not scale to include the audience. It lacks the moderation features required for a presenter to give a talk and take questions without allowing people, even accidentally, to take over the stream and the output video. RTMP provided this flexibility, but requires specific configuration to prevent the delay between presenter and attendee to be large enough that real-time Q&A is impossible. Further, as seen at Libreplanet and DC19, the Jitsi solutions ended up running through an RTMP CDN before reaching attendees anyway. This incurs additional delays.

Proposal

While these existing solutions can be shoe-horned into working, I would like to take the opportunity to develop a better solution, one that ideally integrates the strengths highlighted above, while mitigating some of the weaknesses. From my point of view, I see the following list of requirements:

1. Configurable rooms – conference organisers need to be able to set up different rooms, similar to how a physical conference does. A room can have talks, BoFs or could be a 'hallway track' (where attendees can have discussions between themselves).
2. A talk can have 1 or more presenters and some moderators. The presenter is responsible for displaying their own slides (OBS, screen sharing, etc), while the moderator is responsible for managing the room. This includes time remaining, questions and introducing the presenter.
3. Attendees can join and watch anonymously, but are required to identify themselves in order to ask questions or participate in discussion. This reduces abuse and allows moderators to ensure that the conference Code of Conduct is adhered to.
4. Discussion can occur in an embedded IRC client in the software. Thus existing IRC workflows are maintained and low-bandwidth remote participation is possible.
5. Q&A can be handled through the web-interface. Attendees can signal that they have a question, and a moderator or the presenter can allow them to ask. This audio question is relayed to everyone in the room and once the question is asked, the audio is switched back to the presenter/moderator.
6. Attendees can mute their audio and/or video without issue.

There are several opportunities for bad actors to abuse this system. Questions will need to be able to be cut short by moderators if necessary. In presentation talks, attendee audio needs to be muted unless explicit permission is given for the attendee to talk. Attendee registration for inclusion in the discussion is important, as this allows moderators to ban bad actors more easily.

In BoF sessions, the audio policy needs to be slightly different from presentation talks. In a BoF, many people give input and the discussion is more of a round-robin style. Here, people should be able to indicate that they intend to be part of the discussion, thus having their audio and video unmuted. The discussion then takes the style adopted by most chat clients, such as Jitsi, where the active speaker's video is shown. This should be able to be overridden by a constant video source such as slides or some other relevant information.

I envision this system would use RTMP on the backend, allowing people to connect to end-points mapped to rooms. This would allow people to use OBS or other streaming software to integrate slides, transitions and other media into their presentations. I would like to leverage existing software such as Jitsi and IRC if possible, as this would make implementation easier (hopefully). Development is on GitLab. If you are interesting in getting involved, or have comments: @paddatrapper@pleroma.debian.social

More concrete planning is happening here

Discourse is a community discussion platform that is touted to replace email for project discussions. I am active on two instances for software projects I am involved with. I mainly interact with the web-interface, but I have been meaning to try the email interface for a while. The recent discussion in Debian about the future of mailing lists has finally pushed me to actually give it a try.

Discourse has categories and topics. A category can be thought of as a mailing list with topics as the individual threads.

I like the email support Discourse has, but I have found it lacking in two major respects:

1. I am unable to create topics in a category via email. This is apparently possible, but I was unable to find a way of doing it in either of the two instances. Perhaps it requires additional configuration from the instance administrators for it to work.
2. I am unable to reply to a topic via email if I have not been emailed about it by the system. The mailing list archive has the option to reply to an email by clicking on a link on the page. This composes an email with the Subject and To fields populated with the correct values. This is impossible in discourse, but it does allow you to reply through the web-interface. Future replies are then sent to me as emails, which I can reply to as normal.

Both of these issues have work-arounds, but they make email a second-class citizen on the discourse platform for now.

EDIT 12 March 2020: When configured correctly, new threads can be created by emailing the topic-name@instance.url (for example site-feedback@example.com)

Before I found KDE and fell in love with the flexibility it provides, I enjoyed tiling window managers for their simplicity and the fact that I could do much of my work without needing to mess around with a mouse or trackpad. I am familiar with i3, and decided to try get it running as the WM for my KDE Plasma desktop.

KDE actually provides a tutorial for doing just that. However, I run into a few stumbling blocks along the way.

Stumbling Block 1

I did not read the full tutorial properly. Replacing the WM is only supported on XOrg, using Xsessions. Wayland users are unable to replace KWin. When I first tried this, I decided to give sway a go, but this is a Wayland-only WM. This short-coming is noted in the tutorial I was following, but I did not read it until after I pulled half my hair out.

Stumbling Block 2

Xsessions with a = in the TryExec line will be silently hidden from the SDDM session selector list. In order to add a new Xsession, you copy one of the existing Xsessions under /usr/share/xsessions/ and change the Exec that it does. Changing the WM requires using env KDEWM=/usr/bin/i3 /usr/bin/startplasma-x11. The environment variable is what tells Plasma which WM to use. At first, I tried to change both the Exec and TryExec lines and then got very confused when the new session would not be available. After removing the TryExec line, everything worked like a dream.

Conclusions

I value the simplicity and speed of a tiling WM, but I have also enjoyed being able to select my wifi network from a popup menu and unmount drives with a click of a button. Replacing KWin with i3 has allowed me to get the best of both worlds – my windows sit nicely next to one another, I have independent workspaces on different monitors and I can change my wifi network without needing to mess around in nmcli.