代码改变世界
[登录 · 注册]
  • webrtc笔记(5): 基于kurento media server的多人视频聊天示例
  • 这是kurento tutorial中的一个例子(groupCall),用于多人音视频通话,效果如下:

    登录界面:

     聊天界面: 

    运行方法:

    1、本地用docker把kurento server跑起来

    2、idea里启用这个项目

    3、浏览器里输入https://localhost:8443/ 输入用户名、房间号,然后再开一个浏览器tab页,输入一个不同的用户名,房间号与第1个tab相同,正常情况下,这2个tab页就能聊上了,还可以再加更多tab模拟多人视频(注:docker容器性能有限,mac本上实测,越过4个人,就很不稳定了)

    下面是该项目的一些代码和逻辑分析:

    一、主要模型的类图如下:

    点击查看原图

    UserSession类:代表每个连接进来的用户会话信息。

    Room类:即房间,1个房间可能有多个UserSession实例。

    RoomManager类:房间管理,用于创建或销毁房间。

    UserRegistry类:用户注册类,即管理用户。

    二、主要代码逻辑:

    1、创建房间入口

      public Room getRoom(String roomName) {
        log.debug("Searching for room {}", roomName);
        Room room = rooms.get(roomName);
    
        if (room == null) {
          log.debug("Room {} not existent. Will create now!", roomName);
          room = new Room(roomName, kurento.createMediaPipeline());
          rooms.put(roomName, room);
        }
        log.debug("Room {} found!", roomName);
        return room;
      }
    

    注:第7行,每个房间实例创建时,都绑定了一个对应的MediaPipeline(用于隔离不同房间的媒体信息等)

    2、创建用户实例入口

        public UserSession(final String name, String roomName, final WebSocketSession session,
                           MediaPipeline pipeline) {
    
            this.pipeline = pipeline;
            this.name = name;
            this.session = session;
            this.roomName = roomName;
            this.outgoingMedia = new WebRtcEndpoint.Builder(pipeline).build();
    
            this.outgoingMedia.addIceCandidateFoundListener(event -> {
                JsonObject response = new JsonObject();
                response.addProperty("id", "iceCandidate");
                response.addProperty("name", name);
                response.add("candidate", JsonUtils.toJsonObject(event.getCandidate()));
                try {
                    synchronized (session) {
                        session.sendMessage(new TextMessage(response.toString()));
                    }
                } catch (IOException e) {
                    log.debug(e.getMessage());
                }
            });
        }
    

    UserSession的构造函数上,把房间实例的pipeline做为入参传进来,然后上行传输的WebRtcEndPoint实例outgoingMedia又跟pipeline绑定(第8行)。这样:"用户实例--pipeline实例--房间实例" 就串起来了。

    用户加入房间的代码:

        public UserSession join(String userName, WebSocketSession session) throws IOException {
            log.info("ROOM {}: adding participant {}", this.name, userName);
            final UserSession participant = new UserSession(userName, this.name, session, this.pipeline);
    
            //示例工程上,没考虑“相同用户名”的人进入同1个房间的情况,这里加上了“用户名重名”检测
            if (participants.containsKey(userName)) {
                final JsonObject jsonFailMsg = new JsonObject();
                final JsonArray jsonFailArray = new JsonArray();
                jsonFailArray.add(userName + " exist!");
                jsonFailMsg.addProperty("id", "joinFail");
                jsonFailMsg.add("data", jsonFailArray);
                participant.sendMessage(jsonFailMsg);
                participant.close();
                return null;
            }
    
            joinRoom(participant);
            participants.put(participant.getName(), participant);
            sendParticipantNames(participant);
            return participant;
        }
    

    原代码没考虑到用户名重名的问题,我加上了这段检测,倒数第2行代码,sendParticipantNames在加入成功后,给房间里的其它人发通知。

    3、SDP交换的入口

    kurento-group-call/src/main/resources/static/js/conferenceroom.js 中有一段监听websocket的代码:

    ws.onmessage = function (message) {
        let parsedMessage = JSON.parse(message.data);
        console.info('Received message: ' + message.data);
    
        switch (parsedMessage.id) {
            case 'existingParticipants':
                onExistingParticipants(parsedMessage);
                break;
            case 'newParticipantArrived':
                onNewParticipant(parsedMessage);
                break;
            case 'participantLeft':
                onParticipantLeft(parsedMessage);
                break;
            case 'receiveVideoAnswer':
                receiveVideoResponse(parsedMessage);
                break;
            case 'iceCandidate':
                participants[parsedMessage.name].rtcPeer.addIceCandidate(parsedMessage.candidate, function (error) {
                    if (error) {
                        console.error("Error adding candidate: " + error);
                        return;
                    }
                });
                break;
            case 'joinFail':
                alert(parsedMessage.data[0]);
                window.location.reload();
                break;
            default:
                console.error('Unrecognized message', parsedMessage);
        }
    }
    

    服务端在刚才提到的sendParticipantNames后,会给js发送各种消息,existingParticipants(其它人加入)、newParticipantArrived(新人加入) 这二类消息,就会触发generateOffer,开始向服务端发送SDP

    function onExistingParticipants(msg) {
        const constraints = {
            audio: true,
            video: {
                mandatory: {
                    maxWidth: 320,
                    maxFrameRate: 15,
                    minFrameRate: 15
                }
            }
        };
        console.log(name + " registered in room " + room);
        let participant = new Participant(name);
        participants[name] = participant;
        let video = participant.getVideoElement();
    
        const options = {
            localVideo: video,
            mediaConstraints: constraints,
            onicecandidate: participant.onIceCandidate.bind(participant)
        };
        participant.rtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendonly(options,
            function (error) {
                if (error) {
                    return console.error(error);
                }
                this.generateOffer(participant.offerToReceiveVideo.bind(participant));
            });
    
        msg.data.forEach(receiveVideo);
    }
    

    4、服务端回应各种websocket消息

    org.kurento.tutorial.groupcall.CallHandler#handleTextMessage 信令处理的主要逻辑,就在这里:

        @Override
        public void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
            final JsonObject jsonMessage = gson.fromJson(message.getPayload(), JsonObject.class);
    
            final UserSession user = registry.getBySession(session);
    
            if (user != null) {
                log.debug("Incoming message from user '{}': {}", user.getName(), jsonMessage);
            } else {
                log.debug("Incoming message from new user: {}", jsonMessage);
            }
    
            switch (jsonMessage.get("id").getAsString()) {
                case "joinRoom":
                    joinRoom(jsonMessage, session);
                    break;
                case "receiveVideoFrom":
                    final String senderName = jsonMessage.get("sender").getAsString();
                    final UserSession sender = registry.getByName(senderName);
                    final String sdpOffer = jsonMessage.get("sdpOffer").getAsString();
                    user.receiveVideoFrom(sender, sdpOffer);
                    break;
                case "leaveRoom":
                    leaveRoom(user);
                    break;
                case "onIceCandidate":
                    JsonObject candidate = jsonMessage.get("candidate").getAsJsonObject();
    
                    if (user != null) {
                        IceCandidate cand = new IceCandidate(candidate.get("candidate").getAsString(),
                                candidate.get("sdpMid").getAsString(), candidate.get("sdpMLineIndex").getAsInt());
                        user.addCandidate(cand, jsonMessage.get("name").getAsString());
                    }
                    break;
                default:
                    break;
            }
        }
    

     其中user.receiveVideoFrom方法,就会回应SDP

        public void receiveVideoFrom(UserSession sender, String sdpOffer) throws IOException {
            log.info("USER {}: connecting with {} in room {}", this.name, sender.getName(), this.roomName);
    
            log.trace("USER {}: SdpOffer for {} is {}", this.name, sender.getName(), sdpOffer);
    
            final String ipSdpAnswer = this.getEndpointForUser(sender).processOffer(sdpOffer);
            final JsonObject scParams = new JsonObject();
            scParams.addProperty("id", "receiveVideoAnswer");
            scParams.addProperty("name", sender.getName());
            scParams.addProperty("sdpAnswer", ipSdpAnswer);
    
            log.trace("USER {}: SdpAnswer for {} is {}", this.name, sender.getName(), ipSdpAnswer);
            this.sendMessage(scParams);
            log.debug("gather candidates");
            this.getEndpointForUser(sender).gatherCandidates();
        }
    

    SDP和ICE信息交换完成,就开始视频通讯了。

    参考文章:

    https://doc-kurento.readthedocs.io/en/6.10.0/tutorials/java/tutorial-groupcall.html

  • 【推广】 阿里云小站-上云优惠聚集地(新老客户同享)更有每天限时秒杀!
    【推广】 云服务器低至0.95折 1核2G ECS云服务器8.1元/月
    【推广】 阿里云老用户升级四重礼遇享6.5折限时折扣!
  • 原文:https://www.cnblogs.com/yjmyzz/p/webrtc-groupcall-using-kurento.html
走看看 - 开发者的网上家园