How to streaming IP cctv camera using gstreamer and ffmpeg

https://learncctv.com/what-is-rtsp-protocol-for-ip-cameras/

maybe some people are confused about doing cctv restreamers or other broadcasts to make better quality or just want to reduce the quality. streamers also can make rtsp can be accessed from one source to many. before we discuss further, we should first understand what a streamer is.

streamer is a platform used to broadcast activities live and share them with online audiences. while for gstreamer and ffmpeg according to wikipedia, GStreamer is a development framework for creating applications such as media players, video editors, streaming media broadcasters, and so on. GStreamer provides a media player that comes with the Gst-editor. As the name implies, Gst-editor functions as a pipeline construction and manipulation program. The gst-editor uses Gnome Canvas as its foundation. Apart from the two, there is also Gst-recorder which is a video recording program, which can perfectly record synchronized audio and video into the formats supported by GStreamer. FFmpeg is a computer program that can record, convert and stream digital audio and video in a variety of formats. FFmpeg is a command line application which consists of a library of free / open source software. Includes libavcodec, a library for audio / video codecs used by several other projects, and libavformat, a library for audio / video container mux and demux container. The project name comes from the MPEG standard video group, append “FF” for “fast forward”.

In this article, I will share some of the experiments I have done for the development of a project related to streaming a CCTV camera.

• client — server gstreamer (webcam)
  client — server here is referred to as sender — receiver where the server is the sender. This experiment is done using a webcam to see whether the configuration that has been made is working properly or not. on gstreamer we can also choose to send data using UDP or TCP. gstreamer can also send one source to many using multiudp so that the client can receive streams simultaneously. In this experiment using 2 computers, where the computer as the sender (server) is installed a webcam with the port device = /dev/video0. then on another computer as a receiver or client.

— sender (server)
to send one UDP client can be done with

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "image/jpeg,width=320, height=240,framerate=30/1" ! rtpjpegpay ! udpsink host=192.168.18.178 port = 5006

for multi client UDP is done with

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "image/jpeg,width=320, height=240,framerate=30/1" ! rtpjpegpay ! multiudpsink clients=192.168.18.178:5006,192.168.18.177:5006,192.168.18.176:5006

— receiver (client)
on the receiver of each client, adjust the IP to the port that has been determined on the sender

gst-launch-1.0 -e -v udpsrc port=5006 ! application/x-rtp, encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegparse ! jpegdec ! autovideosink

After running, a window will appear showing the image that has been streamed from two different computers, namely the sender as the server and the receiver as the client.

streaming gstreamer using webcam

• Restreamer using gstreamer and opencv (python)
  in this experiment aims to perform a restreamer. when a cctv is streamed directly, packet loss or loss of information often occurs when sending data. this causes the image received will change or cannot be seen clearly. Images that experience packet loss usually turn green, gray, boxes and even freeze for a while. To avoid this, it is necessary to do a restreamer or re-code the data so that the data displayed is better. this process can also be developed as desired. if you want to add an artificial intelligence, the step that needs to be done is to plant a model in this section. The output in this experiment uses opencv cv2.imshow so that it is easier to modify and if you want to send data, you can combine it with other tools such as kafka, rabbitmq etc. in this experiment I used a script from sahil parekh. before running this script it is necessary to install a few tools first

- opencv
 pip install opencv-python
 - gstreamer
 apt-get install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio
 - gi
 sudo apt-get install python3-gi
For virtualenv users - The vext way
 pip install vext
 pip install vext.gi
The pure python developer way:
 Install a bunch of developer stuff:
 sudo apt-get install pkg-config libcairo2-dev gcc python3-dev libgirepository1.0-dev
Install the python packages:
 pip install gobject PyGObject

After installing the necessary tools or libraries, don’t forget to change the cameralink source on main_prg.py with the ip of your camera.

import multiprocessing as mp
import time
import vid_streamv3 as vs
import cv2
import sys

'''
Main class
'''
class mainStreamClass:
    def __init__(self):

        #Current Cam
        self.camProcess = None
        self.cam_queue = None
        self.stopbit = None
        self.camlink = '' #Add your RTSP cam link
        self.framerate = 6
    
    def startMain(self):

        #set  queue size
        self.cam_queue = mp.Queue(maxsize=100)

        #get all cams
        time.sleep(3)

        self.stopbit = mp.Event()
        self.camProcess = vs.StreamCapture(self.camlink,
                             self.stopbit,
                             self.cam_queue,
                            self.framerate)
        self.camProcess.start()

        # calculate FPS
        lastFTime = time.time()

        try:
            while True:

                if not self.cam_queue.empty():
                    # print('Got frame')
                    cmd, val = self.cam_queue.get()

                    '''
                    #calculate FPS
                    diffTime = time.time() - lastFTime`
                    fps = 1 / diffTime
                    # print(fps)
                    
                    '''
                    lastFTime = time.time()

                    # if cmd == vs.StreamCommands.RESOLUTION:
                    #     pass #print(val)

                    if cmd == vs.StreamCommands.FRAME:
                        if val is not None:
                            cv2.imshow('Cam: ' + self.camlink, val)
                            cv2.waitKey(1)

        except KeyboardInterrupt:
            print('Caught Keyboard interrupt')

        except:
            e = sys.exc_info()
            print('Caught Main Exception')
            print(e)

        self.stopCamStream()
        cv2.destroyAllWindows()


    def stopCamStream(self):
        print('in stopCamStream')

        if self.stopbit is not None:
            self.stopbit.set()
            while not self.cam_queue.empty():
                try:
                    _ = self.cam_queue.get()
                except:
                    break
                self.cam_queue.close()

            self.camProcess.join()


if __name__ == "__main__":
    mc = mainStreamClass()
    mc.startMain()

• client — server gstreamer (rtsp)
  client — server here is referred to as sender — receiver where the server is the sender. This experiment was carried out using a CCTV IP Camera. Before we use this script, make sure that IP CCTV is already known and when we try to run it with VLC or Mplayer IP, it can display CCTV video in realtime. on gstreamer we can also choose to send data using UDP or TSP. gstreamer can also send one source to many using multiudp so that the client can receive streams simultaneously. In this experiment using 2 computers, where the computer as the sender (server) is installed a CCTV IP with port rtsp: // user: password@192.168.18.209: 554. then on another computer as a receiver or client.

— sender (server)
to send one UDP client can be done with

gst-launch-1.0 -e -v rtspsrc location=rtsp://user:password@192.168.18.209:554 ! rtph264pay ! multiudpsink clients=192.168.18.178:5006

for multi client UDP is done with

gst-launch-1.0 -e -v rtspsrc location=rtsp://user:password@192.168.18.209:554 ! rtph264pay ! udpsink host=192.168.18.178 port=5006

— receiver (client)
on the receiver of each client, adjust the IP to the port that has been determined on the sender

st-launch-1.0 -v udpsrc port=5006 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

In this experiment, the sender or server has successfully encoded the frame and sent it to the client, but on the client side it has not succeeded in decoding it so that the frame image does not appear.

• Restreamer using ffmpeg (webcam)
  in this experiment aims to perform a restreamer. when a cctv is streamed directly, packet loss or loss of information often occurs when sending data. this causes the image received will not be seen clearly. The image that contains packet loss usually turns green, gray, boxes and even freezes for a while. To avoid this, it is necessary to do a restreamer or re-code the data so that the data displayed is better. ffmpeg is easier to use and there are a variety of parameters you can use or try to improve the quality of your streaming. This experiment uses a webcam with a simple stream as follows:

ffmpeg -i /dev/video0 -f matroska -vcodec libx264 pipe:1|ffplay -i pipe:0

• Restreamer using ffmpeg (rtsp)
  To restream (rtsp) using the IP from a CCTV camera, the method is almost the same as using a webcam, we just need to change the input source from / dev / video0 to “rtsp: // user: password@192.168.18.209: 554”

ffmpeg -i "rtsp://user:password@192.168.18.209:554" -f matroska -vcodec libx264 pipe:1|ffplay -i pipe:0

• client — server ffmpeg (rtsp)
  client — server here is referred to as sender — receiver where the server is the sender. This experiment was carried out using a CCTV IP Camera. before we use this script make sure that ip cctv is already known and when we try to run it with ffplay. many parameters can be changed to get the best configuration like fps, send mode etc.

— sender (server)
to send one UDP client can be done with

ffmpeg -i rtsp://user:password@192.168.18.209:554 -framerate 15 -s 720x480 -c:v libx264 -preset slower -crf 17 -async 1 -vsync 1 -f mpegts udp://192.168.18.178:5006

— receiver (client)
on the receiver of each client, adjust the IP to the port that has been determined on the sender

ffplay -i udp://192.168.18.178:5006

In this experiment the video has been successfully encoded and decoded. the image is still a little noise so parameter settings such as fps, mode, etc. are needed.

• stream rtsp ffmpeg using youtube
  ffmpeg can also be used to do live streaming using YouTube. the method is quite easy, you only need to prepare an rtsp link and a youtube account that has activated the streaming menu. as for the method is as follows:

fmpeg -f lavfi -i anullsrc -rtsp_transport udp -i {link rtsp camera atau lainnya} -tune zerolatency -vcodec libx264 -t 12:00:00 -pix_fmt + -c:v copy -c:a aac -strict experimental -f flv rtmp://a.rtmp.youtube.com/live2/{streaming id}

{link rtsp camera atau lainnya} is rtsp link like rtsp://user:password@192.168.18.209:554

{streaming id} is an id that has been generated by YouTube as many as 20 digits xxxx-xxxx-xxxx-xxxx-xxxx