問題描述

假設(shè)我有一臺無線攝像機(jī)，我想將鏡頭實(shí)時(shí)傳輸?shù)?Unity.有沒有辦法做到這一點(diǎn)?

Let's say I have a wireless camera that I want to stream footage from to unity in real-time. Is there a way to achieve this?

額外問題:

• 如果相機(jī)的角度更廣(180 度，甚至 360 度)
• 如果這是我想與之互動的片段，延遲會有多大的問題
• 除了常規(guī)鏡頭之外，是否可以發(fā)送更多數(shù)據(jù)，例如深度感知(使用深度感知相機(jī))?
• 我是瘋了還是已經(jīng)這樣做了?

提前致謝

推薦答案

我假設(shè)這是一個(gè)帶有以太網(wǎng)端口或 Wi-Fi 的相機(jī)，您可以連接到它并從中實(shí)時(shí)流式傳輸圖像.

I assume this is a camera with Ethernet port or Wi-Fi that you can connect to and stream images from it live.

如果是這樣，那么是的，可以使用 Unity 完成.

If so, then yes, it can be done with Unity.

如何在沒有外部庫的情況下完成:

連接到相機(jī):

1.連接到與相機(jī)相同的本地網(wǎng)絡(luò)，或者如果支持unpn，您也可以通過互聯(lián)網(wǎng)連接到它.通常，您需要攝像機(jī)的 IP 和端口來執(zhí)行此操作.假設(shè)攝像機(jī) IP 地址為 192.168.1.5，端口號為 900.要連接的 url 是 http://192.168.1.5:900.

1.Connect to the-same local network with the camera or if unpn is supported, you can also connect to it through internet. Usually, you need the IP and the port of the camera to do this. Let's say that the Camera IP Address is 192.168.1.5 and the port number is 900. The url to connect to is http://192.168.1.5:900.

有時(shí)，它只是一個(gè)以 .mjpg 或 .bin 結(jié)尾的 url，例如 http://192.168.1.5/mjpg/video.mjpg 和 http://192.168.1.5/mjpg/video.bin

Sometimes, it is simply an url that ends with .mjpg or .bin such as http://192.168.1.5/mjpg/video.mjpg and http://192.168.1.5/mjpg/video.bin

每臺相機(jī)都是不同的.找到該網(wǎng)址的唯一方法是閱讀其手冊.如果該手冊不可用，請使用其官方應(yīng)用程序連接到它，然后使用 Wireshark 發(fā)現(xiàn)相機(jī)圖像的 url.username 和 password(如果需要)也可以在手冊中找到.如果沒有，請?jiān)?Google 上搜索型號，您需要的所有東西都應(yīng)找到.

Every camera is different. The only way to find the url is to read its manual. If the manual is not available, connect to it with its official Application then use Wireshark to discover the url of the camera Image. The username and the password(if required) can also be found in the manual. If not available, google the model number and everything you need should be found.

從相機(jī)中提取 JPEG:

連接到相機(jī)后，相機(jī)會不斷地向您發(fā)送數(shù)據(jù).您可以掃描這些數(shù)據(jù)并從中檢索圖像.

When connected to the camera, the camera will be sending endless data to you. This data you can scan and retrieve the image from it.

2.搜索 0xFF 后跟 0xD8 的 JPEG 標(biāo)頭.如果這兩個(gè)字節(jié)彼此相鄰，則開始讀取字節(jié)并繼續(xù)將它們保存到數(shù)組中.您可以使用 index(int) 變量來計(jì)算收到的字節(jié)數(shù).

2.Search for the JPEG header which is 0xFF followed by 0xD8. If these two bytes are next to each other then start reading the bytes and continue to save them to an array. You can use an index(int) variable to keep count of how many bytes you have received.

int counter = 0;
byte[] completeImageByte = new byte[500000];
byte[] receivedBytes = new byte[500000];
receivedBytes[counter] = byteFromCamera;
counter++;

3.從攝像頭讀取數(shù)據(jù)時(shí)，檢查接下來的兩個(gè)字節(jié)是否是JPEG頁腳，即0xFF后跟0xD9.如果這是真的，那么您已經(jīng)收到了完整的圖像(1 幀).

3.While reading the data from the camera, check if the next two bytes is the JPEG footer which is 0xFF followed by 0xD9. If this is true then you have received the complete image(1 frame).

您的圖像字節(jié)應(yīng)如下所示:

Your image bytes should look something like:

0xFF 0xD8 someotherbytes(數(shù)千個(gè))..... 然后是 0xFF 0xD9

0xFF 0xD8 someotherbytes(thousands of them)..... then 0xFF 0xD9

將 receivedBytes 復(fù)制到 completeImageByte 變量中，以便以后可以使用它來顯示圖像.將 counter 變量重置為 0.

Copy receivedBytes to the completeImageByte variable so that it can be used to display the image later on. Reset counter variable to 0.

Buffer.BlockCopy(receivedBytes, 0, completeImageByte, 0, counter);
counter = 0;

在屏幕上顯示 JPEG 圖像:

4.將圖像顯示到屏幕

由于您每秒將收到許多圖像，因此我發(fā)現(xiàn)顯示此圖像的最有效方式是使用RawImage 組件.因此，如果您希望它在移動設(shè)備上運(yùn)行，請不要為此使用 Image 或 Sprite Renderer.

Since you will be receiving many images per second, the most efficient way I found to display this is to use RawImage Component. So, don't use Image or Sprite Renderer for this if you want this to run on mobile devices.

public RawImage screenDisplay;
if(updateFrame){
Texture2D camTexture = new Texture2D(2, 2);
camTexture.LoadImage(completeImageByte);
screenDisplay.texture = camTexture;
}

camTexture = new Texture2D(2, 2); 只需在 Start() 函數(shù)中執(zhí)行一次即可.

You only need to do camTexture = new Texture2D(2, 2); once in the Start() function.

5.跳回步驟2并繼續(xù)執(zhí)行，直到您想要完全為止.

5.Jump back to step 2 and continue doing it until you want to quite.

用于連接相機(jī)的 API:.

如果相機(jī)需要身份驗(yàn)證(用戶名和密碼)，請使用 HttpWebRequest.

Use HttpWebRequest if the camera requires authentication (username and password).

對于不需要身份驗(yàn)證的用戶，請使用 UnityWebRequest.使用 UnityWebRequest 時(shí)，您必須從 派生自己的類DownloadHandlerScript 否則您的應(yīng)用程序?qū)⒈罎?，因?yàn)槟鷮⒉煌５亟邮諗?shù)據(jù).

For those that does not require authentication, use UnityWebRequest. When using UnityWebRequest, you must derive your own classes from DownloadHandlerScript or your app will crash as you will be receiving data non stop.

從DownloadHandlerScript派生你自己的類的例子:

Example of deriving your own class from DownloadHandlerScript:

using UnityEngine;
using System.Collections;
using UnityEngine.Networking;

public class CustomWebRequest : DownloadHandlerScript
{    
    // Standard scripted download handler - will allocate memory on each ReceiveData callback
    public CustomWebRequest()
        : base()
    {
    }

    // Pre-allocated scripted download handler
    // Will reuse the supplied byte array to deliver data.
    // Eliminates memory allocation.
    public CustomWebRequest(byte[] buffer)
        : base(buffer)
    {
    }

    // Required by DownloadHandler base class. Called when you address the 'bytes' property.
    protected override byte[] GetData() { return null; }

    // Called once per frame when data has been received from the network.
    protected override bool ReceiveData(byte[] byteFromCamera, int dataLength)
    {
        if (byteFromCamera == null || byteFromCamera.Length < 1)
        {
            //Debug.Log("CustomWebRequest :: ReceiveData - received a null/empty buffer");
            return false;
        }

        //Search of JPEG Image here

        return true;
    }

    // Called when all data has been received from the server and delivered via ReceiveData
    protected override void CompleteContent()
    {
        //Debug.Log("CustomWebRequest :: CompleteContent - DOWNLOAD COMPLETE!");
    }

    // Called when a Content-Length header is received from the server.
    protected override void ReceiveContentLength(int contentLength)
    {
        //Debug.Log(string.Format("CustomWebRequest :: ReceiveContentLength - length {0}", contentLength));
    }
}

用法:

using UnityEngine;
using System.Collections;
using UnityEngine.Networking;

public class Test : MonoBehaviour
{

    CustomWebRequest camImage;
    UnityWebRequest webRequest;
    byte[] bytes = new byte[90000];

    void Start()
    {
        string url = "http://camUrl/mjpg/video.mjpg";
        webRequest = new UnityWebRequest(url);
        webRequest.downloadHandler = new CustomWebRequest(bytes);
        webRequest.Send();
    }
}

您可以讓他們在ReceiveData<中執(zhí)行步驟2、3、4和5CustomWebRequest 腳本中的/code> 函數(shù).