Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

Commit

Permalink
Support Multithread
Browse files Browse the repository at this point in the history
Use subthreads to process data, avoid lags in GUI.
  • Loading branch information
zty199 committed Mar 21, 2021
1 parent 95f837d commit abf8e99
Show file tree
Hide file tree
Showing 19 changed files with 787 additions and 351 deletions.
26 changes: 18 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,30 @@
A simple network classroom Demo based on UDP Multicast, including basic multimedia transmission. Only works in LAN.

## Realized Features
* Support basic multimedia transmission based on UDP Multicast , including cam, screen share, mic, stereo mixer, etc.
* Support basic multimedia transmission based on **UDP Multicast** , including cam, screen share, mic, stereo mixer, etc.

* Support 1080p video transmission. (Temporarily limit to 720p because of HIGH CPU and Network consumption)
* Support 1080p video transmission. (Temporarily limit to 720p because of HIGH **CPU** and **Network** consumption)

* Support dynamic switch between different devices (cam / audio IO).

* Support cam resolution adjustment.

* Support volume adjustment. (Only current audio stream affected)
* Support volume adjustment. (Only **current audio stream** affected, global device volume is controlled by system)

* Support rescan available devices.
* Support available devices rescan.

*Support white board and screen mark when sharing screen.
* Support white board and screen mark when sharing screen.

* Support multithread data processing to avoid lags in GUI.

## Existing Issues
* When using wireless cards, UDP Multicast is extremely Unreliable, both sending and receiving ends meet serious packet loss. Ethernet cards recommended.
* When using wireless cards, UDP Multicast is extremely **Unreliable**, both sending and receiving ends meet serious packet loss. **Ethernet cards** recommended.

* Video frames captured from cam and screen are directly encoded into JPEG by CPU(?), source resolution higher than 720p can cause serious lags on GUI.
* Video frames captured from cam and screen are directly encoded into **JPEG** by CPU(?), source resolution higher than **720p** can cause serious lags on GUI.

* Cam / Audio IO devices' names are Inaccurate on Linux (based on device drivers), most of audio IO devices shown in combobox don't work (not physically connected to any devices, just ports reserved).

* Available devices list can't refresh in real time. Choosing one that is removed may cause crash, and new devices connected won't show up.
* Available devices list can't refresh in real time. Choosing one that is removed may cause crash, and new devices connected won't show up unless corresponding function button is clicked.

## Future Plan
File Transfer, Text Message Transmission, Student Sign-in, etc.
Expand All @@ -47,5 +49,13 @@ File Transfer, Text Message Transmission, Student Sign-in, etc.

* [海天鹰屏幕笔](https://github.com/sonichy/HTYScreenPen)

* [QT多线程的使用](https://www.cnblogs.com/coolcpp/p/qt-thread.html)

* [Qt QRunnable的使用](https://blog.csdn.net/qq_43711348/article/details/103983857)

* [QT编程问题小结(编译、多线程、UDP Socket等)](https://blog.csdn.net/rabbitjerry/article/details/70947807)

* [QT基于UDP通信的多线程编程问题](https://blog.csdn.net/kamereon/article/details/49582617)

## Clarification
This project is my graduation design for undergraduates. Only for study and communication.
12 changes: 11 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,16 @@

* 支持屏幕共享时使用白板和屏幕标注。

* 支持多线程收发数据,避免单线程造成 GUI 卡顿。

## 目前存在的问题
* 使用无线网卡时,UDP 组播极为 **不稳定** ,收发数据包均存在严重丢包问题。推荐使用 **有线网卡**

* 目前摄像头和桌面画面由 CPU(?)直接编码为 **JPEG** 格式进行传输,原始画面分辨率高于 **720p** 会造成卡顿。

* Linux 下获取的音频/视频设备名称 **不明确** (由设备驱动决定),声音设备列表中大多数设备不可用。(多数是声卡硬件原始预留的端口,未连接任何物理设备)

* 可用设备列表不能实时刷新,使用设备过程中移除设备可能造成程序崩溃,新插入设备不会自动识别。
* 可用设备列表不能实时刷新,使用设备过程中移除设备可能造成程序崩溃,新插入设备不会自动识别,需要关闭对应功能后才能刷新设备列表

## 后续计划实现的功能
文件传输,文字通信,学生签到等。
Expand All @@ -47,5 +49,13 @@

* [海天鹰屏幕笔](https://github.com/sonichy/HTYScreenPen)

* [QT多线程的使用](https://www.cnblogs.com/coolcpp/p/qt-thread.html)

* [Qt QRunnable的使用](https://blog.csdn.net/qq_43711348/article/details/103983857)

* [QT编程问题小结(编译、多线程、UDP Socket等)](https://blog.csdn.net/rabbitjerry/article/details/70947807)

* [QT基于UDP通信的多线程编程问题](https://blog.csdn.net/kamereon/article/details/49582617)

## 声明
该项目为大学本科毕业设计,仅供学习交流使用。
4 changes: 4 additions & 0 deletions src/UDPMulticast/UDPMulticast.pro
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,18 @@ DEFINES += QT_DEPRECATED_WARNINGS
CONFIG += c++11

SOURCES += \
audiopacksender.cpp \
main.cpp \
mainwindow.cpp \
screenpen.cpp \
videoframesender.cpp \
videosurface.cpp

HEADERS += \
audiopacksender.h \
mainwindow.h \
screenpen.h \
videoframesender.h \
videosurface.h

FORMS += \
Expand Down
98 changes: 98 additions & 0 deletions src/UDPMulticast/audiopacksender.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
#include "audiopacksender.h"

AudioPackSender::AudioPackSender(char *ap)
{
this->ap = new AudioPack;
memcpy(this->ap, ap, sizeof(*(this->ap)));
setAutoDelete(true);
}

AudioPackSender::~AudioPackSender()
{
delete audio_socket;
delete ap;
}

void AudioPackSender::run()
{
groupAddress = QHostAddress("239.0.0.1");
audio_port = 8889;

audio_socket = new QUdpSocket;
audio_socket->setSocketOption(QAbstractSocket::MulticastTtlOption, 1); // 设置套接字属性
audio_socket->setSocketOption(QAbstractSocket::SendBufferSizeSocketOption, 1024 * 64); // 缓冲区最大存储 4个 数据包(单个 16K)

qint32 res;
qint32 dataLength = ap->len;
uchar *dataBuffer = (uchar *)ap->data;

qint32 packetNum = dataLength / UDP_MAX_SIZE;
qint32 lastPaketSize = dataLength % UDP_MAX_SIZE;
qint32 currentPacketIndex = 0;
if(lastPaketSize != 0)
{
packetNum++;
}

PackageHeader packageHead;
packageHead.uTransPackageHdrSize = sizeof(packageHead);
packageHead.uDataSize = dataLength;
packageHead.uDataPackageNum = packetNum;

uchar frameBuffer[sizeof(packageHead) + UDP_MAX_SIZE];
memset(frameBuffer, 0, sizeof(packageHead) + UDP_MAX_SIZE);

packageHead.uTransPackageSize = packageHead.uTransPackageHdrSize + UDP_MAX_SIZE;
packageHead.uDataPackageCurrIndex = 0;
packageHead.uDataPackageOffset = 0;
memcpy(frameBuffer, &packageHead, packageHead.uTransPackageHdrSize);
res = audio_socket->writeDatagram(
(const char *)frameBuffer, packageHead.uTransPackageSize,
groupAddress, audio_port);
if(res < 0)
{
qDebug() << "audio_socket: Audio Pack Size Send Failed!";
}

while(currentPacketIndex < packetNum)
{
if(currentPacketIndex < (packetNum - 1))
{
packageHead.uTransPackageSize = packageHead.uTransPackageHdrSize + UDP_MAX_SIZE;
packageHead.uDataPackageCurrIndex = currentPacketIndex + 1;
packageHead.uDataPackageOffset = currentPacketIndex * UDP_MAX_SIZE;
memcpy(frameBuffer, &packageHead, packageHead.uTransPackageHdrSize);
memcpy(frameBuffer + packageHead.uTransPackageHdrSize, dataBuffer + packageHead.uDataPackageOffset, UDP_MAX_SIZE);

res = audio_socket->writeDatagram(
(const char *)frameBuffer, packageHead.uTransPackageSize,
groupAddress, audio_port);

if(res < 0)
{
qDebug() << "audio_socket: Packet Send Failed!";
}

currentPacketIndex++;
}
else
{
packageHead.uTransPackageSize = packageHead.uTransPackageHdrSize + (dataLength - currentPacketIndex * UDP_MAX_SIZE);
packageHead.uDataPackageCurrIndex = currentPacketIndex + 1;
packageHead.uDataPackageOffset = currentPacketIndex * UDP_MAX_SIZE;
memcpy(frameBuffer, &packageHead, packageHead.uTransPackageHdrSize);
memcpy(frameBuffer + packageHead.uTransPackageHdrSize, dataBuffer + packageHead.uDataPackageOffset, dataLength - currentPacketIndex * UDP_MAX_SIZE);

res = audio_socket->writeDatagram(
(const char *)frameBuffer, packageHead.uTransPackageSize,
groupAddress, audio_port);

if(res < 0)
{
qDebug() << "video_socket: Packet Send Failed!";
}

currentPacketIndex++;
}
}
}
41 changes: 41 additions & 0 deletions src/UDPMulticast/audiopacksender.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#ifndef AUDIOPACKSENDER_H
#define AUDIOPACKSENDER_H

#include <QRunnable>
#include <QUdpSocket>

#define UDP_MAX_SIZE 1200 // UDP 数据包最大长度 * MTU = 1500,故数据包大小 1500 - 20(IP头)- 8(UDP头)

class AudioPackSender : public QRunnable
{
public:
explicit AudioPackSender(char *ap);
~AudioPackSender();

protected:
void run() override;

private:
QUdpSocket *audio_socket;
QHostAddress groupAddress;
quint16 audio_port;

struct PackageHeader
{
quint32 uTransPackageHdrSize;
quint32 uTransPackageSize;
quint32 uDataSize;
quint32 uDataPackageNum;
quint32 uDataPackageCurrIndex;
quint32 uDataPackageOffset;
};

struct AudioPack
{
char data[1024 * 16]; // 单个音频数据包大小设为 16K,音质 44K/128Kbps(?)
int len;
} *ap;

};

#endif // AUDIOPACKSENDER_H
Loading

0 comments on commit abf8e99

Please sign in to comment.