Android Question Video Streming - Drone DJI Tello

[Star-Dust]

Finally I produced my App for driving the DJI Tello Drone, thanks to the precious indications of @inakigarm (see here).
Now I moved on to the next step, to receive streaming images.

The drone sends images captured by the camera with wifi in udp protocol in 720p format

Private Sub StreamVideo_PacketArrived (RicevedPacket As UDPPacket)
    ' 1280 * 720
    Log( ":::" & RicevedPacket.Data.Length)
End Sub

I get data on the UDP socket, now how can I transform byte data into images?
I receive packages of 65k and I have to recompose the image, but how many Kb takes up the total image considering it is 720p?

Maybe there are too many questions

 

[walterf25]

Finally I produced my App for driving the DJI Tello Drone, thanks to the precious indications of @inakigarm (see here).
Now I moved on to the next step, to receive streaming images.

The drone sends images captured by the camera with wifi in udp protocol in 720p format

Private Sub StreamVideo_PacketArrived (RicevedPacket As UDPPacket)
    ' 1280 * 720
    Log( ":::" & RicevedPacket.Data.Length)
End Sub

I get data on the UDP socket, now how can I transform byte data into images?
I receive packages of 65k and I have to recompose the image, but how many Kb takes up the total image considering it is 720p?

Maybe there are too many questions

I'm not familiar with the DJI Tello library, but i would assume you can do something like:

    Dim out As OutputStream
    out.InitializeToBytesArray(1000)
    Dim udp As UDPPacket
    out.WriteBytes(udp.Data, 0, udp.Data.Length)
    
    Dim bmp As Bitmap
    bmp.WriteToStream(out, 100, "JPEG")

Or something similar?

Walter

 

[Star-Dust]

I'm not familiar with the DJI Tello library, but i would assume you can do something like:

    Dim out As OutputStream
    out.InitializeToBytesArray(1000)
    Dim udp As UDPPacket
    out.WriteBytes(udp.Data, 0, udp.Data.Length)
   
    Dim bmp As Bitmap
    bmp.WriteToStream(out, 100, "JPEG")

Or something similar?

Walter

This would be the opposite operation. Transfer from Bitmap to the stream. Perhaps we are closer to this.

Private Sub StreamVideo_PacketArrived (RicevedPacket As UDPPacket)
    ' 1280 * 720
    Log( ":::" & RicevedPacket.Data.Length)
    Dim bmp As Bitmap
    Dim Str As InputStream
    
    Str.InitializeFromBytesArray(RicevedPacket,0,RicevedPacket.Data.Length)
    bmp.Initialize2(Str)
End Sub

But I will receive a series of frames one after the other, I have to understand where one starts and ends the other.

 

[DonManfred]

What is the size of the packages?

log(RicevedPacket.data.length)

Maybe the package always include one image?

 

[Star-Dust]

What is the size of the packages?

log(RicevedPacket.data.length)

Maybe the package always include one image?

I could get it if in the initialization of UDP I would insert the size (in byte) of the image. But I do not know it

Knowing the size of the image I would do so

StreamVideo.Initialize ("StreamVideo", 11111,ImageSize)

Private Sub StreamVideo_PacketArrived (RicevedPacket As UDPPacket)
    ' 1280 * 720
    Log( ":::" & RicevedPacket.Data.Length)
    File.WriteBytes(Path,"image.png",RicevedPacket)
End Sub

At the moment, ImageSize is 65k, and I receive all 65k packages.
I would hazard that the data format is similar to what is received from the camera of the android devices.

 

[DonManfred]

Try to hexencode one of the packages and write it to a textfile. you may find - probably - a "end-marker" inside the data at the end of the image. After that i guess you only get 0 bytes (hex 00). Worth a try if you dont know exactly.

Does the documentation provide this info? Or an example-code?

 

[Star-Dust]

Try to hexencode one of the packages and write it to a textfile. you may find - probably - a "end-marker" inside the data at the end of the image. After that i guess you only get 0 bytes (hex 00). Worth a try if you dont know exactly.

Does the documentation provide this info? Or an example-code?

Unfortunately the documentation alone that the image(720p) is received in streming on port 11111 in UDP protocol.

He does not say anything else.

I will try to see if there is a hexadecimal character of the end of the image, even if I think it is not, because in the 720p protocol the dimensions should be the same. The letter p indicates that the whole image is sent, if it had the final i only the differences from the previous image. (I think )

 

[DonManfred]

i found this. Maybe it helps?

Hello Tello

Video packets
The streaming video is also sent to the ground station on a different UDP port. Unfortunately the Tello does not just use one of the existing standards for streaming video such as RTSP. Instead, the raw video packets are sent via UDP and need to be reassembled and decoded before they can be viewed.

Since the size of a single video frame is larger than the size of a single UDP packet, the Tello breaks apart each frame, and sends the packets with header of 2 bytes to indicate how they need to be reassembled.


Position Usage
0 Sequence number
1 Sub-sequence number
2-n Video data
The video data itself is just H264 encoded YUV420p. Using this information, it is possible to decode the video using standard tools such as ffmpeg, once you remove the header bytes.

PD i just used this google search
and tried the first result ;-)

 

[Star-Dust]

There are many forums for software developers for tello. An interesting one is indicated in the @inakigarm thread.

Unfortunately what about the streaming I could not find it.

I have seen that a developer, also successful thanks to the API, causes the drone tello to follow the objects (or a person with facial recognition)
What in other drones is native here is recreated thanks to the App.

I will continue to search on google. in the meantime, if some idea comes to mind, share it.

PS. The link you posted to me is about the GOBOT language, which already has its integrated commands for DJI Tello and also the streaming decoder

 

[Erel]

You cannot treat a video stream as a series of images unless the encoding used is a simple one such as MJPEG. It is most probably not the case here.

 

[Star-Dust]

Thank's @DonManfred and @Erel.

 

[Star-Dust]

Something I have is Pyton. It is a source to control the drone and the source piece you find is about streaming reception.
I understand that as Erel thought, is a compressed video in some format (h264 or MPEG-4 AVC). Unfortunately I can understand little from this code, you?

###############################################################################
# VideoRX Thread
###############################################################################
    def _threadVideoRX(self, stop_event, arg):
        #print '_threadVideoRX started !!!'

        sockVideo = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        addrVideo = ('192.168.10.2', self.TELLO_PORT_VIDEO)
        sockVideo.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        sockVideo.settimeout(.5)
        sockVideo.bind(addrVideo)
      
        data = bytearray(4096)
        fileVideo = open('video.h264', 'wb')
        isSPSRcvd = False
      
        while not stop_event.is_set():
            try:
                size, addr = sockVideo.recvfrom_into(data)
            except socket.timeout, e:
                time.sleep(.5)
                continue
            except socket.error, e:
                print e
                break
            else:
                if size > 6 and data[2] == 0x00 and data[3] == 0x00 and data[4] == 0x00 and data[5] == 0x01:
                    nal_type = data[6] & 0x1f
                    #print 'NAL=', nal_type
                    if nal_type == 7:
                        isSPSRcvd = True;

                # drop 2 bytes
                if isSPSRcvd:
                    fileVideo.write(data[2:size])

         fileVideo.close()
        #print '_threadVideoRX terminated !!!'

 

[Erel]

You should save all the incoming data to a file and then copy it to your PC. Try to play it with VLC or a similar tool.

 

[DonManfred]

You should save all the incoming data to a file and then copy it to your PC

Addition:

Make sure not to save the HEADER-Bytes to the stream

unless the encoding used is a simple one such as MJPEG

Since the size of a single video frame is larger than the size of a single UDP packet, the Tello breaks apart each frame, and sends the packets with header of 2 bytes to indicate how they need to be reassembled.


Position Usage
0 Sequence number
1 Sub-sequence number
2-n Video data
The video data itself is just H264 encoded YUV420p.

Don´t know if that helps. I´m not familar with Videocompression...

 

[Hodi]

Hi @Star-Dust ,

Did you managed the convert the video stream. I´m currently trying to create an B4J app but can´t get the stream like I planned to do.

Br,
Tobi