Let’s say two people want to video chat using our app. One solution would be for the first person to stream their camera feed to our server. Then, our application would pass this data to the other caller. Unfortunately, our application acting as a middleman could introduce a significant lag, especially if our server is far from the call participants. Fortunately, we can use WebRTC to connect our users directly without sending all of the data through our server. This improves performance and helps us achieve a smooth experience.
Even if the video and audio are not streamed through our application, we still need to build a server to allow the users to establish a connection with each other. In this article, we create a NestJS application to connect our users. Besides that, we also create a frontend app using React that manages and displays the video stream.
For the code of our backend application, check out this repository. If you want to take a look at the frontend code, open this repository.
Capturing the camera
In this article, we gradually build both the frontend and backend applications. For the frontend part, we use Vite.
If you want to know more about Vite, check out React with Vite and TypeScript and its common challenges
Let’s start by capturing a stream from the local camera. To do that, we need to use the mediaDevices object.
1import { useEffect, useState } from 'react';
2
3export function useLocalCameraStream() {
4 const [localStream, setLocalStream] = useState<MediaStream | null>(null);
5
6 useEffect(() => {
7 navigator.mediaDevices
8 .getUserMedia({ video: true, audio: true })
9 .then((stream) => {
10 setLocalStream(stream);
11 });
12 }, []);
13
14 return {
15 localStream,
16 };
17}To display it, we need to use the <video> element. Since we need to attach the media stream to its srcObject property, we must use a React reference.
1import { FunctionComponent } from 'react';
2
3interface Props {
4 mediaStream: MediaStream;
5 isMuted?: boolean;
6}
7
8export const VideoFeed: FunctionComponent<Props> = ({
9 mediaStream,
10 isMuted = false,
11}) => {
12 return (
13 <video
14 ref={(ref) => {
15 if (ref) {
16 ref.srcObject = mediaStream;
17 }
18 }}
19 autoPlay={true}
20 muted={isMuted}
21 />
22 );
23};The last step is to combine our hook with the VideoFeed component.
1import { useLocalCameraStream } from './useLocalCameraStream';
2import { VideoFeed } from './VideoFeed';
3
4export function App() {
5 const { localStream } = useLocalCameraStream();
6
7 if (!localStream) {
8 return null;
9 }
10
11 return <VideoFeed mediaStream={localStream} isMuted={true} />;
12}Creating video chat rooms
We want our users to be able to join a particular chat room and communicate in real time with the other person in the room. To do that, we need to use WebSockets.
If you want to know more about websockets, check out API with NestJS #26. Real-time chat with WebSockets
To use WebSockets in our NestJS application, we need to install a few additional libraries.
1npm install @nestjs/websockets @nestjs/platform-socket.ioIn this article, we use Socket.IO, because it is the most popular implementation of WebSockets for NestJS.
Let’s start with allowing the users to join a chat room with a particular name. In this straightforward implementation, we only handle a maximum of two people at once in the room.
1import {
2 ConnectedSocket,
3 MessageBody,
4 SubscribeMessage,
5 WebSocketGateway,
6 WebSocketServer,
7} from '@nestjs/websockets';
8import { Server, Socket } from 'socket.io';
9
10@WebSocketGateway()
11export class ChatGateway {
12 @WebSocketServer()
13 server: Server;
14
15 @SubscribeMessage('join_room')
16 async joinRoom(
17 @MessageBody() roomName: string,
18 @ConnectedSocket() socket: Socket,
19 ) {
20 const room = this.server.in(roomName);
21
22 const roomSockets = await room.fetchSockets();
23 const numberOfPeopleInRoom = roomSockets.length;
24
25 // a maximum of 2 people in a room
26 if (numberOfPeopleInRoom > 1) {
27 room.emit('too_many_people');
28 return;
29 }
30
31 socket.join(roomName);
32 }
33}Thanks to the above code, the user joins a room with the provided name as soon as the frontend application emits the join_room message.
Dealing with Cross-Origin Resource Sharing
There is a solid chance that our frontend and backend applications are deployed to different origins. This can cause issues with Cross-Origin Resource Sharing when our frontend application tries to talk with the API.
If you want to know more about CORS, check out API with NestJS #117. CORS – Cross-Origin Resource Sharing
To deal with this, we need to define the URL of our frontend application on the backend side. A good way to do that is through environment variables.
1FRONTEND_URL=http://localhost:51731import { Module } from '@nestjs/common';
2import { AppController } from './app.controller';
3import { AppService } from './app.service';
4import { ChatModule } from './chat/chat.module';
5import * as Joi from 'joi';
6import { ConfigModule } from '@nestjs/config';
7
8@Module({
9 imports: [
10 ChatModule,
11 ConfigModule.forRoot({
12 validationSchema: Joi.object({
13 FRONTEND_URL: Joi.string().required(),
14 }),
15 }),
16 ],
17 controllers: [AppController],
18 providers: [AppService],
19})
20export class AppModule {}To be able to provide the frontend URL dynamically, we should extend the IoAdapter built into NestJS.
1import { IoAdapter } from '@nestjs/platform-socket.io';
2import { INestApplicationContext } from '@nestjs/common';
3import { Server, ServerOptions } from 'socket.io';
4import { CorsOptions } from 'cors';
5
6export class WebsocketAdapter extends IoAdapter {
7 constructor(
8 appOrHttpServer: INestApplicationContext,
9 private readonly corsOptions: CorsOptions,
10 ) {
11 super(appOrHttpServer);
12 }
13
14 create(port: number, options?: ServerOptions): Server {
15 return super.create(port, {
16 ...options,
17 cors: this.corsOptions,
18 });
19 }
20}We can now use it in our bootstrap function.
1import { NestFactory } from '@nestjs/core';
2import { AppModule } from './app.module';
3import { ConfigService } from '@nestjs/config';
4import { WebsocketAdapter } from './chat/websocket-adapter';
5
6async function bootstrap() {
7 const app = await NestFactory.create(AppModule);
8
9 const configService = app.get(ConfigService);
10
11 app.enableCors({
12 origin: configService.get('FRONTEND_URL'),
13 });
14
15 app.useWebSocketAdapter(
16 new WebsocketAdapter(app, {
17 origin: configService.get('FRONTEND_URL'),
18 }),
19 );
20
21 await app.listen(3000);
22}
23bootstrap();Joining a room
One way of letting the user join a particular room is through URL path parameters. For example, if the users enter the /video-chat-room/important-meeting URL, we assume that the room name is the important-meeting.
The React Router is the most straightforward way to achieve that with React.
1npm install react-router-dom1import { BrowserRouter, Route, Routes } from 'react-router-dom';
2import { VideoChatRoom } from './VideoChatRoom';
3import React from 'react';
4import { useLocalCameraStream } from './useLocalCameraStream';
5
6export const App = () => {
7 const { localStream } = useLocalCameraStream();
8
9 return (
10 <BrowserRouter>
11 <Routes>
12 <Route
13 path="video-chat-room/:roomName"
14 element={localStream && <VideoChatRoom localStream={localStream} />}
15 />
16 </Routes>
17 </BrowserRouter>
18 );
19};To connect to our API, we need its URL. The best way to store it is through environment variables.
1VITE_API_URL=http://localhost:30001/// <reference types="vite/client" />
2
3interface ImportMetaEnv {
4 readonly VITE_API_URL: string;
5}We need to use the socket.io-client library to establish the WebSocket connection.
1npm install socket.io-client1import { io } from 'socket.io-client';
2
3export const socket = io(import.meta.env.VITE_API_URL, {
4 autoConnect: false,
5});We can now connect to our NestJS application and join a room.
1import { useParams } from 'react-router-dom';
2import { useCallback, useEffect } from 'react';
3import { socket } from '../socket';
4
5export function useChatConnection() {
6 const { roomName } = useParams();
7
8 const handleConnection = useCallback(() => {
9 socket.emit('join_room', roomName);
10 }, [roomName]);
11
12 useEffect(() => {
13 socket.connect();
14 socket.on('connect', handleConnection);
15
16 return () => {
17 socket.off('connect', handleConnection);
18 };
19 }, [roomName, handleConnection, roomName]);
20}Above, we emit the join_room event as soon as we connect to the WebSocket and provide the name of the chat based on the URL.
1import { VideoFeed } from './VideoFeed';
2import { FunctionComponent } from 'react';
3import { useChatConnection } from './useChatConnection';
4
5interface Props {
6 localStream: MediaStream;
7}
8
9export const VideoChatRoom: FunctionComponent<Props> = ({ localStream }) => {
10 useChatConnection();
11
12 return <VideoFeed mediaStream={localStream} isMuted={true} />;
13};Connecting the people in the room
Once we have precisely two people in our chat room, we want one of them to initiate a connection. To be able to do that, we need to let one of the people know that the other person is ready. To do that, we can emit the another_person_ready event.
1import {
2 ConnectedSocket,
3 MessageBody,
4 SubscribeMessage,
5 WebSocketGateway,
6 WebSocketServer,
7} from '@nestjs/websockets';
8import { Server, Socket } from 'socket.io';
9
10@WebSocketGateway()
11export class ChatGateway {
12 @WebSocketServer()
13 server: Server;
14
15 @SubscribeMessage('join_room')
16 async joinRoom(
17 @MessageBody() roomName: string,
18 @ConnectedSocket() socket: Socket,
19 ) {
20 const room = this.server.in(roomName);
21
22 const roomSockets = await room.fetchSockets();
23 const numberOfPeopleInRoom = roomSockets.length;
24
25 // a maximum of 2 people in a room
26 if (numberOfPeopleInRoom > 1) {
27 room.emit('too_many_people');
28 return;
29 }
30
31 if (numberOfPeopleInRoom === 1) {
32 room.emit('another_person_ready');
33 }
34
35 socket.join(roomName);
36 }
37}We need to start by creating an RTC Peer Connection. To do that, we need to provide the ICE server. In WebRTC, ICE is a framework used to facilitate the connection between the users. It helps us find the best path to connect two peers online. A part of the ICE framework is the STUN protocol used to discover the public address we need to establish the connection.
Fortunately, Google provides a public STUN server we can use.
1import { useMemo } from 'react';
2
3export function usePeerConnection(localStream: MediaStream) {
4 const peerConnection = useMemo(() => {
5 const connection = new RTCPeerConnection({
6 iceServers: [{ urls: 'stun:stun2.1.google.com:19302' }],
7 });
8
9 localStream.getTracks().forEach((track) => {
10 connection.addTrack(track, localStream);
11 });
12
13 return connection;
14 }, []);
15
16 return {
17 peerConnection,
18 };
19}Above, we create an instance of the RTCPeerConnection and add video and audio tracks from the local stream created thanks to our webcam.
Let’s pass the peerConnection object to the useChatConnection hook we created before.
1import { VideoFeed } from './VideoFeed';
2import { FunctionComponent } from 'react';
3import { useChatConnection } from './useChatConnection';
4import { usePeerConnection } from './usePeerConnection.tsx';
5
6interface Props {
7 localStream: MediaStream;
8}
9
10export const VideoChatRoom: FunctionComponent<Props> = ({ localStream }) => {
11 const { peerConnection } = usePeerConnection(localStream);
12 useChatConnection(peerConnection);
13
14 return <VideoFeed mediaStream={localStream} isMuted={true} />;
15};Sending the connection offer
The first step in establishing the WebRTC connection is sending the connection offer. To do that, we need to use the createOffer and setLocalDescription methods.
Once the offer is created, we send it to our NestJS server through the send_connection_offer event.
1import { useCallback } from 'react';
2import { socket } from '../socket.tsx';
3import { useParams } from 'react-router-dom';
4
5export function useOfferSending(peerConnection: RTCPeerConnection) {
6 const { roomName } = useParams();
7
8 const sendOffer = useCallback(async () => {
9 const offer = await peerConnection.createOffer();
10 await peerConnection.setLocalDescription(offer);
11
12 socket.emit('send_connection_offer', {
13 roomName,
14 offer,
15 });
16 }, [roomName]);
17
18 return { sendOffer };
19}We should send the offer as soon as we receive the another_person_ready event indicating that the other person in the chat room is ready.
1import { useParams } from 'react-router-dom';
2import { useCallback, useEffect } from 'react';
3import { socket } from '../socket';
4import {useOfferSending} from "./useOfferSending.tsx";
5
6export function useChatConnection(peerConnection: RTCPeerConnection) {
7 const { roomName } = useParams();
8
9 const { sendOffer } = useOfferSending(peerConnection);
10
11 const handleConnection = useCallback(() => {
12 socket.emit('join_room', roomName);
13 }, [roomName]);
14
15 useEffect(() => {
16 socket.connect();
17 socket.on('connect', handleConnection);
18 socket.on('another_person_ready', sendOffer);
19 return () => {
20 socket.off('connect', handleConnection);
21 socket.off('another_person_ready', sendOffer);
22 };
23 }, [roomName, handleConnection, roomName]);
24}We now need to adjust our NestJS server to handle this event. Fortunately, it is very straightforward. As soon as our WebSocket server gets the send_connection_offer, it sends it to the other person in the room.
1import {
2 ConnectedSocket,
3 MessageBody,
4 SubscribeMessage,
5 WebSocketGateway,
6 WebSocketServer,
7} from '@nestjs/websockets';
8import { Server, Socket } from 'socket.io';
9
10@WebSocketGateway()
11export class ChatGateway {
12 @WebSocketServer()
13 server: Server;
14
15 // ...
16
17 @SubscribeMessage('send_connection_offer')
18 async sendConnectionOffer(
19 @MessageBody()
20 {
21 offer,
22 roomName,
23 }: {
24 offer: RTCSessionDescriptionInit;
25 roomName: string;
26 },
27 @ConnectedSocket() socket: Socket,
28 ) {
29 this.server.in(roomName).except(socket.id).emit('send_connection_offer', {
30 offer,
31 roomName,
32 });
33 }
34}Sending the answer
The other person in the room must prepare the answer as soon as they receive the offer. First, we need to call the setRemoteDescription function to acknowledge the offer. Then, we must create the answer and send it to our NestJS server through the WebSocket.
1import { useCallback } from 'react';
2import { socket } from '../socket.tsx';
3import { useParams } from 'react-router-dom';
4
5export function useOffersListening(peerConnection: RTCPeerConnection) {
6 const { roomName } = useParams();
7
8 const handleConnectionOffer = useCallback(
9 async ({ offer }: { offer: RTCSessionDescriptionInit }) => {
10 await peerConnection.setRemoteDescription(offer);
11 const answer = await peerConnection.createAnswer();
12 await peerConnection.setLocalDescription(answer);
13
14 socket.emit('answer', { answer, roomName });
15 },
16 [roomName],
17 );
18
19 return {
20 handleConnectionOffer,
21 };
22}We should configure our frontend application to send the answer once it receives the offer.
1import { useParams } from 'react-router-dom';
2import { useCallback, useEffect } from 'react';
3import { socket } from '../socket';
4import { useOfferSending } from './useOfferSending.tsx';
5import { useSendingAnswer } from './useSendingAnswer.tsx';
6
7export function useChatConnection(peerConnection: RTCPeerConnection) {
8 const { roomName } = useParams();
9
10 const { sendOffer } = useOfferSending(peerConnection);
11
12 const { handleConnectionOffer } = useSendingAnswer(peerConnection);
13
14 const handleConnection = useCallback(() => {
15 socket.emit('join_room', roomName);
16 }, [roomName]);
17
18 useEffect(() => {
19 socket.connect();
20 socket.on('connect', handleConnection);
21 socket.on('another_person_ready', sendOffer);
22 socket.on('send_connection_offer', handleConnectionOffer);
23 return () => {
24 socket.off('connect', handleConnection);
25 socket.off('another_person_ready', sendOffer);
26 socket.off('send_connection_offer', handleConnectionOffer);
27 };
28 }, [roomName, handleConnection, roomName, handleConnectionOffer]);
29}We should configure our server to get the answer and pass it to the other person in the chat room.
1import {
2 ConnectedSocket,
3 MessageBody,
4 SubscribeMessage,
5 WebSocketGateway,
6 WebSocketServer,
7} from '@nestjs/websockets';
8import { Server, Socket } from 'socket.io';
9
10@WebSocketGateway()
11export class ChatGateway {
12 @WebSocketServer()
13 server: Server;
14
15 // ...
16
17 @SubscribeMessage('answer')
18 async answer(
19 @MessageBody()
20 {
21 answer,
22 roomName,
23 }: {
24 answer: RTCSessionDescriptionInit;
25 roomName: string;
26 },
27 @ConnectedSocket() socket: Socket,
28 ) {
29 this.server.in(roomName).except(socket.id).emit('answer', {
30 answer,
31 roomName,
32 });
33 }
34}Processing the answer
Finally, we need to process the answer to the offer. We need to start by acknowledging the answer through the setRemoteDescription method.
1import { useCallback } from 'react';
2
3export function useAnswerProcessing(peerConnection: RTCPeerConnection) {
4 const handleOfferAnswer = useCallback(
5 ({ answer }: { answer: RTCSessionDescriptionInit }) => {
6 peerConnection.setRemoteDescription(answer);
7 },
8 [peerConnection],
9 );
10
11 return {
12 handleOfferAnswer,
13 };
14}We need to do that as soon as we receive the answer event.
1import { useParams } from 'react-router-dom';
2import { useCallback, useEffect } from 'react';
3import { socket } from '../socket';
4import { useOfferSending } from './useOfferSending.tsx';
5import { useSendingAnswer } from './useSendingAnswer.tsx';
6import { useAnswerProcessing } from './useAnswerProcessing.tsx';
7
8export function useChatConnection(peerConnection: RTCPeerConnection) {
9 const { roomName } = useParams();
10
11 const { sendOffer } = useOfferSending(peerConnection);
12
13 const { handleConnectionOffer } = useSendingAnswer(peerConnection);
14
15 const { handleOfferAnswer } = useAnswerProcessing(peerConnection);
16
17 const handleConnection = useCallback(() => {
18 socket.emit('join_room', roomName);
19 }, [roomName]);
20
21 useEffect(() => {
22 socket.connect();
23 socket.on('answer', handleOfferAnswer);
24 socket.on('connect', handleConnection);
25 socket.on('another_person_ready', sendOffer);
26 socket.on('send_connection_offer', handleConnectionOffer);
27 return () => {
28 socket.off('connect', handleConnection);
29 socket.off('another_person_ready', sendOffer);
30 socket.off('send_connection_offer', handleConnectionOffer);
31 socket.off('answer', handleOfferAnswer);
32 };
33 }, [
34 roomName,
35 handleConnection,
36 roomName,
37 handleConnectionOffer,
38 handleOfferAnswer,
39 sendOffer,
40 ]);
41}When we receive the stream from the other person in the room, it causes the track event to be dispatched. Let’s listen to it and store it in the state so that we can display it later.
1import { useMemo, useState } from 'react';
2import { socket } from '../socket.tsx';
3import { useParams } from 'react-router-dom';
4
5export function usePeerConnection(localStream: MediaStream) {
6 const { roomName } = useParams();
7 const [guestStream, setGuestStream] = useState<MediaStream | null>(null);
8
9 const peerConnection = useMemo(() => {
10 const connection = new RTCPeerConnection({
11 iceServers: [{ urls: 'stun:stun2.1.google.com:19302' }],
12 });
13
14 connection.addEventListener('track', ({ streams }) => {
15 setGuestStream(streams[0]);
16 });
17
18 localStream.getTracks().forEach((track) => {
19 connection.addTrack(track, localStream);
20 });
21
22 return connection;
23 }, [localStream, roomName]);
24
25 return {
26 peerConnection,
27 guestStream,
28 };
29}The icecandidate event
The icecandidate event in WebRTC is triggered when the local ICE agent discovers a new network candidate for establishing a peer-to-peer connection. This event provides the candidate’s information, which includes details like the IP address and port number. We must send this candidate to the peer we want to connect to. To do that, we need to send an event to our NestJS server.
1import { useMemo, useState } from 'react';
2import { socket } from '../socket.tsx';
3import { useParams } from 'react-router-dom';
4
5export function usePeerConnection(localStream: MediaStream) {
6 const { roomName } = useParams();
7 const [guestStream, setGuestStream] = useState<MediaStream | null>(null);
8
9 const peerConnection = useMemo(() => {
10 const connection = new RTCPeerConnection({
11 iceServers: [{ urls: 'stun:stun2.1.google.com:19302' }],
12 });
13
14 connection.addEventListener('icecandidate', ({ candidate }) => {
15 socket.emit('send_candidate', { candidate, roomName });
16 });
17
18 // ...
19
20 return connection;
21 }, [localStream, roomName]);
22
23 return {
24 peerConnection,
25 guestStream,
26 };
27}Now, let’s handle the send_candidate event by passing the data to the other person in the chat room.
1import {
2 ConnectedSocket,
3 MessageBody,
4 SubscribeMessage,
5 WebSocketGateway,
6 WebSocketServer,
7} from '@nestjs/websockets';
8import { Server, Socket } from 'socket.io';
9
10@WebSocketGateway()
11export class ChatGateway {
12 @WebSocketServer()
13 server: Server;
14
15 // ...
16
17 @SubscribeMessage('send_candidate')
18 async sendCandidate(
19 @MessageBody()
20 {
21 candidate,
22 roomName,
23 }: {
24 candidate: unknown;
25 roomName: string;
26 },
27 @ConnectedSocket() socket: Socket,
28 ) {
29 this.server.in(roomName).except(socket.id).emit('send_candidate', {
30 candidate,
31 roomName,
32 });
33 }
34}The last step is to handle receiving the candidate.
1import { useParams } from 'react-router-dom';
2import { useCallback, useEffect } from 'react';
3import { socket } from '../socket';
4import { useOfferSending } from './useOfferSending.tsx';
5import { useSendingAnswer } from './useSendingAnswer.tsx';
6import { useAnswerProcessing } from './useAnswerProcessing.tsx';
7
8export function useChatConnection(peerConnection: RTCPeerConnection) {
9 const { roomName } = useParams();
10
11 const { sendOffer } = useOfferSending(peerConnection);
12
13 const { handleConnectionOffer } = useSendingAnswer(peerConnection);
14
15 const { handleOfferAnswer } = useAnswerProcessing(peerConnection);
16
17 const handleConnection = useCallback(() => {
18 socket.emit('join_room', roomName);
19 }, [roomName]);
20
21 const handleReceiveCandidate = useCallback(
22 ({ candidate }: { candidate: RTCIceCandidate }) => {
23 peerConnection.addIceCandidate(candidate);
24 },
25 [peerConnection],
26 );
27
28 useEffect(() => {
29 socket.connect();
30 socket.on('answer', handleOfferAnswer);
31 socket.on('send_connection_offer', handleConnectionOffer);
32 socket.on('another_person_ready', sendOffer);
33 socket.on('connect', handleConnection);
34 socket.on('send_candidate', handleReceiveCandidate);
35 return () => {
36 socket.off('answer', handleOfferAnswer);
37 socket.off('send_connection_offer', handleConnectionOffer);
38 socket.off('another_person_ready', sendOffer);
39 socket.off('connect', handleConnection);
40 socket.off('send_candidate', handleReceiveCandidate);
41 };
42 }, [
43 roomName,
44 handleConnection,
45 roomName,
46 handleConnectionOffer,
47 handleOfferAnswer,
48 sendOffer,
49 handleReceiveCandidate
50 ]);
51}Displaying both video streams
We now have access to the video stream of the other person in the room through the guestStream variable. Let’s use it to render both videos.
1import { VideoFeed } from './VideoFeed';
2import { FunctionComponent } from 'react';
3import { useChatConnection } from './useChatConnection';
4import { usePeerConnection } from './usePeerConnection.tsx';
5
6interface Props {
7 localStream: MediaStream;
8}
9
10export const VideoChatRoom: FunctionComponent<Props> = ({ localStream }) => {
11 const { peerConnection, guestStream } = usePeerConnection(localStream);
12 useChatConnection(peerConnection);
13
14 return (
15 <div>
16 <VideoFeed mediaStream={localStream} isMuted={true} />
17 {guestStream && (
18 <div>
19 guest
20 <VideoFeed mediaStream={guestStream} />
21 </div>
22 )}
23 </div>
24 );
25};Summary
In this article, we’ve gone into detail about what Web Real-Time Communication (WebRTC) is and what problem it solves. We explained how it allows people to communicate in real-time directly through web browsers and keeps the role of the server to a minimum. We implemented a WebSocket server using NestJS and a frontend application with React to show how it works.
By implementing a real-life example, we got from the basics and progressed into more advanced aspects. Thanks to that, we established a solid understanding of WebRTC and used this knowledge in our code.