Introduction
Hello! 😎
In this advanced WebRTC tutorial I will show you how to stream your camera to a HTML page using WebRTC, GStreamer and C++. We will be using boost to handle the signaling. By the end of this tutorial you should have a simple understanding on WebRTC GStreamer. 👀
Requirements
GStreamer and its development libraries
Boost libraries
CMake for building the project
A C++ compiler
Basic C++ Knowledge
Creating the Project
First we need a place to house our projects files, create a new directory like so:
mkdir webrtc-stream && cd webrtc-stream
First we need to create a build file in order to build the completed project, create a new file called "CMakeLists.txt" and populate it with the following:
cmake_minimum_required(VERSION 3.10)
# Set the project name and version
project(webrtc_server VERSION 1.0)
# Specify the C++ standard
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)
# Find required packages
find_package(PkgConfig REQUIRED)
pkg_check_modules(GST REQUIRED gstreamer-1.0 gstreamer-webrtc-1.0 gstreamer-sdp-1.0)
find_package(Boost 1.65 REQUIRED COMPONENTS system filesystem json)
# Include directories
include_directories(${GST_INCLUDE_DIRS} ${Boost_INCLUDE_DIRS})
# Add the executable
add_executable(webrtc_server main.cpp)
# Link libraries
target_link_libraries(webrtc_server ${GST_LIBRARIES} Boost::system Boost::filesystem Boost::json)
# Set properties
set_target_properties(webrtc_server PROPERTIES
CXX_STANDARD 14
CXX_STANDARD_REQUIRED ON
)
# Specify additional directories for the linker
link_directories(${GST_LIBRARY_DIRS})
# Print project info
message(STATUS "Project: ${PROJECT_NAME}")
message(STATUS "Version: ${PROJECT_VERSION}")
message(STATUS "C++ Standard: ${CMAKE_CXX_STANDARD}")
message(STATUS "Boost Libraries: ${Boost_LIBRARIES}")
message(STATUS "GStreamer Libraries: ${GST_LIBRARIES}")
The above links all the required libraries together in order to build the code into an executable that can be executed.
Now we can get on to coding the project. 🥸
Coding the Project
Now we can start coding the source code for the project, create a new file called "main.cpp", we will start by importing the necessary headers for GStreamer, WebRTC, Boost and standard libraries:
#define GST_USE_UNSTABLE_API
#include <gst/gst.h>
#include <gst/webrtc/webrtc.h>
#include <boost/beast.hpp>
#include <boost/asio.hpp>
#include <boost/json.hpp>
#include <iostream>
#include <thread>
namespace beast = boost::beast;
namespace http = beast::http;
namespace websocket = beast::websocket;
namespace net = boost::asio;
using tcp = net::ip::tcp;
using namespace boost::json;
Next we will be define constants that will be used later, mainly the STUN server and port that the server will listen on:
#define STUN_SERVER "stun://stun.l.google.com:19302"
#define SERVER_PORT 8000
Now we will declare global variables for the GStreamer main loop and pipeline elements:
GMainLoop *loop;
GstElement *pipeline, *webrtcbin;
Next we will create the functions to handle each of the events. First one being a function that sends ICE candidates to the WebSocket client:
void send_ice_candidate_message(websocket::stream<tcp::socket>& ws, guint mlineindex, gchar *candidate)
{
std::cout << "Sending ICE candidate: mlineindex=" << mlineindex << ", candidate=" << candidate << std::endl;
object ice_json;
ice_json["candidate"] = candidate;
ice_json["sdpMLineIndex"] = mlineindex;
object msg_json;
msg_json["type"] = "candidate";
msg_json["ice"] = ice_json;
std::string text = serialize(msg_json);
ws.write(net::buffer(text));
std::cout << "ICE candidate sent" << std::endl;
}
The next "on_answer_created" function handles the creation of a WebRTC answer and sends it back to the client:
void on_answer_created(GstPromise *promise, gpointer user_data)
{
std::cout << "Answer created" << std::endl;
websocket::stream<tcp::socket>* ws = static_cast<websocket::stream<tcp::socket>*>(user_data);
GstWebRTCSessionDescription *answer = NULL;
const GstStructure *reply = gst_promise_get_reply(promise);
gst_structure_get(reply, "answer", GST_TYPE_WEBRTC_SESSION_DESCRIPTION, &answer, NULL);
GstPromise *local_promise = gst_promise_new();
g_signal_emit_by_name(webrtcbin, "set-local-description", answer, local_promise);
object sdp_json;
sdp_json["type"] = "answer";
sdp_json["sdp"] = gst_sdp_message_as_text(answer->sdp);
std::string text = serialize(sdp_json);
ws->write(net::buffer(text));
std::cout << "Local description set and answer sent: " << text << std::endl;
gst_webrtc_session_description_free(answer);
}
The next function is just a placeholder for handling negotiation events, this event is not needed in this example:
void on_negotiation_needed(GstElement *webrtc, gpointer user_data)
{
std::cout << "Negotiation needed" << std::endl;
}
The "on_set_remote_description" function sets the remote description and creates an answer:
void on_set_remote_description(GstPromise *promise, gpointer user_data)
{
std::cout << "Remote description set, creating answer" << std::endl;
websocket::stream<tcp::socket>* ws = static_cast<websocket::stream<tcp::socket>*>(user_data);
GstPromise *answer_promise = gst_promise_new_with_change_func(on_answer_created, ws, NULL);
g_signal_emit_by_name(webrtcbin, "create-answer", NULL, answer_promise);
}
The "on_ice_candidate" function handles ICE candidate events and sends them to the WebSocket client:
void on_ice_candidate(GstElement *webrtc, guint mlineindex, gchar *candidate, gpointer user_data)
{
std::cout << "ICE candidate generated: mlineindex=" << mlineindex << ", candidate=" << candidate << std::endl;
websocket::stream<tcp::socket>* ws = static_cast<websocket::stream<tcp::socket>*>(user_data);
send_ice_candidate_message(*ws, mlineindex, candidate);
}
The "handle_websocket_session" function manages the WebSocket connection, setting up the GStreamer pipeline and handling both SDP and ICE messages:
void handle_websocket_session(tcp::socket socket)
{
try
{
websocket::stream<tcp::socket> ws{std::move(socket)};
ws.accept();
std::cout << "WebSocket connection accepted" << std::endl;
GstStateChangeReturn ret;
GError *error = NULL;
pipeline = gst_pipeline_new("pipeline");
GstElement *v4l2src = gst_element_factory_make("v4l2src", "source");
GstElement *videoconvert = gst_element_factory_make("videoconvert", "convert");
GstElement *queue = gst_element_factory_make("queue", "queue");
GstElement *vp8enc = gst_element_factory_make("vp8enc", "encoder");
GstElement *rtpvp8pay = gst_element_factory_make("rtpvp8pay", "pay");
webrtcbin = gst_element_factory_make("webrtcbin", "sendrecv");
if (!pipeline || !v4l2src || !videoconvert || !queue || !vp8enc || !rtpvp8pay || !webrtcbin)
{
g_printerr("Not all elements could be created.\n");
return;
}
g_object_set(v4l2src, "device", "/dev/video0", NULL);
g_object_set(vp8enc, "deadline", 1, NULL);
gst_bin_add_many(GST_BIN(pipeline), v4l2src, videoconvert, queue, vp8enc, rtpvp8pay, webrtcbin, NULL);
if (!gst_element_link_many(v4l2src, videoconvert, queue, vp8enc, rtpvp8pay, NULL))
{
g_printerr("Elements could not be linked.\n");
gst_object_unref(pipeline);
return;
}
GstPad *rtp_src_pad = gst_element_get_static_pad(rtpvp8pay, "src");
GstPad *webrtc_sink_pad = gst_element_get_request_pad(webrtcbin, "sink_%u");
gst_pad_link(rtp_src_pad, webrtc_sink_pad);
gst_object_unref(rtp_src_pad);
gst_object_unref(webrtc_sink_pad);
g_signal_connect(webrtcbin, "on-negotiation-needed", G_CALLBACK(on_negotiation_needed), &ws);
g_signal_connect(webrtcbin, "on-ice-candidate", G_CALLBACK(on_ice_candidate), &ws);
ret = gst_element_set_state(pipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE)
{
g_printerr("Unable to set the pipeline to the playing state.\n");
gst_object_unref(pipeline);
return;
}
std::cout << "GStreamer pipeline set to playing" << std::endl;
for (;;)
{
beast::flat_buffer buffer;
ws.read(buffer);
auto text = beast::buffers_to_string(buffer.data());
value jv = parse(text);
object obj = jv.as_object();
std::string type = obj["type"].as_string().c_str();
if (type == "offer")
{
std::cout << "Received offer: " << text << std::endl;
std::string sdp = obj["sdp"].as_string().c_str();
GstSDPMessage *sdp_message;
gst_sdp_message_new_from_text(sdp.c_str(), &sdp_message);
GstWebRTCSessionDescription *offer = gst_webrtc_session_description_new(GST_WEBRTC_SDP_TYPE_OFFER, sdp_message);
GstPromise *promise = gst_promise_new_with_change_func(on_set_remote_description, &ws, NULL);
g_signal_emit_by_name(webrtcbin, "set-remote-description", offer, promise);
gst_webrtc_session_description_free(offer);
std::cout << "Setting remote description" << std::endl;
}
else if (type == "candidate")
{
std::cout << "Received ICE candidate: " << text << std::endl;
object ice = obj["ice"].as_object();
std::string candidate = ice["candidate"].as_string().c_str();
guint sdpMLineIndex = ice["sdpMLineIndex"].as_int64();
g_signal_emit_by_name(webrtcbin, "add-ice-candidate", sdpMLineIndex, candidate.c_str());
std::cout << "Added ICE candidate" << std::endl;
}
}
}
catch (beast::system_error const& se)
{
if (se.code() != websocket::error::closed)
{
std::cerr << "Error: " << se.code().message() << std::endl;
}
}
catch (std::exception const& e)
{
std::cerr << "Exception: " << e.what() << std::endl;
}
}
The next "start_server" function initializes the server, sccepting TCP connections and spawning new threads to handle each connection:
void start_server()
{
try
{
net::io_context ioc{1};
tcp::acceptor acceptor{ioc, tcp::endpoint{tcp::v4(), SERVER_PORT}};
for (;;)
{
tcp::socket socket{ioc};
acceptor.accept(socket);
std::cout << "Accepted new TCP connection" << std::endl;
std::thread{handle_websocket_session, std::move(socket)}.detach();
}
}
catch (std::exception const& e)
{
std::cerr << "Exception: " << e.what() << std::endl;
}
}
Finally we just need to create the final main function to initialize GStreamer, start the server and run the main loop:
int main(int argc, char *argv[])
{
gst_init(&argc, &argv);
loop = g_main_loop_new(NULL, FALSE);
std::cout << "Starting WebRTC server" << std::endl;
std::thread server_thread(start_server);
g_main_loop_run(loop);
server_thread.join();
gst_element_set_state(pipeline, GST_STATE_NULL);
gst_object_unref(pipeline);
g_main_loop_unref(loop);
std::cout << "WebRTC server stopped" << std::endl;
return 0;
}
Done, now we can finally build the project! 😄
Building the Project
To build the above source code into an executable first create a new directory called build:
mkdir build && cd build
Build the project:
cmake ..
make
If all goes well the project should be built successfully and you should have an executable.
Next we need to create a page to view the stream. 😸
Creating the Frontend
Create a new directory called "public" and in it create a new html file called "index.html" and populate it with the following code:
<!DOCTYPE html>
<html>
<head>
<title>WebRTC Stream</title>
</head>
<body>
<video id="video" autoplay playsinline muted></video>
<script>
const video = document.getElementById('video');
const signaling = new WebSocket('ws://localhost:8000/ws');
let pc = new RTCPeerConnection({
iceServers: [{urls: 'stun:stun.l.google.com:19302'}]
});
signaling.onmessage = async (event) => {
const data = JSON.parse(event.data);
console.log('Received signaling message:', data);
if (data.type === 'answer') {
console.log('Setting remote description with answer');
await pc.setRemoteDescription(new RTCSessionDescription(data));
} else if (data.type === 'candidate') {
console.log('Adding ICE candidate:', data.ice);
await pc.addIceCandidate(new RTCIceCandidate(data.ice));
}
};
pc.onicecandidate = (event) => {
if (event.candidate) {
console.log('Sending ICE candidate:', event.candidate);
signaling.send(JSON.stringify({
type: 'candidate',
ice: event.candidate
}));
}
};
pc.ontrack = (event) => {
console.log('Received track:', event);
if (event.track.kind === 'video') {
console.log('Attaching video track to video element');
video.srcObject = event.streams[0];
video.play().catch(error => {
console.error('Error playing video:', error);
});
video.load();
}
};
pc.oniceconnectionstatechange = () => {
console.log('ICE connection state:', pc.iceConnectionState);
};
pc.onicegatheringstatechange = () => {
console.log('ICE gathering state:', pc.iceGatheringState);
};
pc.onsignalingstatechange = () => {
console.log('Signaling state:', pc.signalingState);
};
async function start() {
pc.addTransceiver('video', {direction: 'recvonly'});
const offer = await pc.createOffer();
console.log('Created offer:', offer);
await pc.setLocalDescription(offer);
console.log('Set local description with offer');
signaling.send(JSON.stringify({type: 'offer', sdp: pc.localDescription.sdp}));
}
start();
</script>
</body>
</html>
The above can be explained in my other WebRTC tutorials, but it simple communicates with the signalling server and when a remote stream is received plays the video in the video HTML element.
Done now we can actually run the project! 👍
Running the Project
To run the project simple execute the following command:
./webrtc_server
To run the html page we will use a python module:
python3 -m http.server 9999
Navigate your browser to http://localhost:9999 and on load you should see your camera showing in the video element like so:
Done! 😁
Considerations
In order to improve the above, I would like to implement the following:
Handle multiple viewers
Handle receiving a stream from HTML
Creating an SFU
Recording
Conclusion
In this tutorial I have shown you how to stream your camera using native C++, GStreamer and view the stream in a HTML page. I hope this tutorial has taught you something, I certainly had a lot of fun creating it.
As always you can find the source code for the project on my Github: https://github.com/ethand91/webrtc-gstreamer
Happy Coding! 😎
Like my work? I post about a variety of topics, if you would like to see more please like and follow me. Also I love coffee.
If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch