//
you're reading...
Computer vision, How-to's

Webcam Streaming

2014-09-04 08.17.54

Following on from Andy’s post on Webcam support, here is how to use your Webcam to stream color video to your PC.

The leJOS program is:

package mypackage;

import java.io.BufferedOutputStream;
import java.io.IOException;
import java.net.Socket;

import lejos.hardware.BrickFinder;
import lejos.hardware.Button;
import lejos.hardware.ev3.EV3;
import lejos.hardware.lcd.LCD;
import lejos.hardware.video.Video;

public class Stream {
	private static final int WIDTH = 160;
	private static final int HEIGHT = 120;
	private static final String HOST = "192.168.0.2";
	private static final int PORT = 55555;

	public static void main(String[] args) throws IOException  {

		EV3 ev3 = (EV3) BrickFinder.getLocal();
		Video video = ev3.getVideo();
		video.open(WIDTH, HEIGHT);
		byte[] frame = video.createFrame();

		Socket sock = new Socket(HOST, PORT);
		BufferedOutputStream bos = new BufferedOutputStream(sock.getOutputStream());
		long start = System.currentTimeMillis();
		int frames = 0;
		LCD.drawString("fps:", 0, 2);

		while(Button.ESCAPE.isUp()) {
			try {
				video.grabFrame(frame);
				LCD.drawString("" + (++frames * 1000f/(System.currentTimeMillis() - start)), 5,2);

				bos.write(frame);
				bos.flush();
			} catch (IOException e) {
				break;
			}
		}

		bos.close();
		sock.close();
		video.close();
	}
}

And the PC program is:

package mypackage;

import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.awt.image.BufferedImage;
import java.io.BufferedInputStream;
import java.net.ServerSocket;
import java.net.Socket;

import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;

public class CameraFrame {
	private static final int WIDTH = 160;
	private static final int HEIGHT = 120;
	private static final int NUM_PIXELS = WIDTH * HEIGHT;
	private static final int BUFFER_SIZE = NUM_PIXELS * 2;
	private static final int PORT = 55555;

	private ServerSocket ss;
	private Socket sock;
	private byte[] buffer = new byte[BUFFER_SIZE];
	private BufferedInputStream bis;
	private BufferedImage image;
	private CameraPanel panel = new CameraPanel();
	private JFrame frame;

	public CameraFrame() {	
		try {
			ss = new ServerSocket(PORT);
			sock = ss.accept();
			bis = new BufferedInputStream(sock.getInputStream());
		} catch (Exception e) {
			System.err.println("Failed to connect: " + e);
			System.exit(1);
		}

		image = new BufferedImage(WIDTH, HEIGHT, BufferedImage.TYPE_INT_RGB);
	}

	public void createAndShowGUI() {
		frame = new JFrame("EV3 Camera View");

		frame.getContentPane().add(panel);
		frame.setPreferredSize(new Dimension(WIDTH, HEIGHT));

		frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
	    frame.addWindowListener(new WindowAdapter() {
	        @Override
	        public void windowClosing(WindowEvent e) {
	        	close();
	        }
	    });

		frame.pack();
		frame.setVisible(true);
	}

	public void close() {
    	try {
    		if (bis != null) bis.close();
    		if (sock != null) sock.close();
			if (ss != null) ss.close();
		} catch (Exception e1) {
			System.err.println("Exception closing window: " + e1);
		}
	}

	private int convertYUVtoARGB(int y, int u, int v) {
		int c = y - 16;
		int d = u - 128;
		int e = v - 128;
		int r = (298*c+409*e+128)/256;
		int g = (298*c-100*d-208*e+128)/256;
		int b = (298*c+516*d+128)/256;
	    r = r>255? 255 : r<0 ? 0 : r;
	    g = g>255? 255 : g<0 ? 0 : g;
	    b = b>255? 255 : b<0 ? 0 : b;
	    return 0xff000000 | (r<<16) | (g<<8) | b;
	}

	public void run() {
		while(true) {
			synchronized (this) {
				try {
					int offset = 0;
					while (offset < BUFFER_SIZE) {
						offset += bis.read(buffer, offset, BUFFER_SIZE - offset);
					}
					for(int i=0;i<BUFFER_SIZE;i+=4) {
						int y1 = buffer[i] & 0xFF;
						int y2 = buffer[i+2] & 0xFF;
						int u = buffer[i+1] & 0xFF;
						int v = buffer[i+3] & 0xFF;
						int rgb1 = convertYUVtoARGB(y1,u,v);
						int rgb2 = convertYUVtoARGB(y2,u,v);
						image.setRGB((i % (WIDTH * 2)) / 2, i / (WIDTH * 2), rgb1);
						image.setRGB((i % (WIDTH * 2)) / 2 + 1, i / (WIDTH * 2), rgb2);
					}
				} catch (Exception e) {
					break;
				}
			}
			panel.repaint(1);
		}
	}

	class CameraPanel extends JPanel {
		private static final long serialVersionUID = 1L;

	    @Override
	    protected void paintComponent(Graphics g) {
	        super.paintComponent(g);
	        // Ensure that we don't paint while the image is being refreshed
	        synchronized(CameraFrame.this) {
	        	g.drawImage(image, 0, 0, null);
	        }
	    }	
	}

	public static void main(String[] args) {	
		final CameraFrame cameraFrame = new CameraFrame();
		SwingUtilities.invokeLater(new Runnable() {
			@Override
			public void run() {
				cameraFrame.createAndShowGUI();	
			}
		});
		cameraFrame.run();
	}
}
Advertisements

Discussion

16 thoughts on “Webcam Streaming

  1. Thanks a lot for this article – the possibilities this enables now are quite staggering.

    I have a question though: how do you achieve communication over IP by WiFi and camera usage whereas there is only one usb port on the brick? Do you use some kind of usb hub to plug a wifi dongle and the camera? Any specific brand/model?

    Same for the webcam: do you know what are the models supported?

    Thanks you again very much for this article that is of great help to me! I’ll continue following your post closely 🙂

    Posted by Parapampa | 2015/01/18, 15:31
  2. No space left on device Error

    thanks for sharing this very good stuff 🙂

    I tried it my self by using usb hub with 4 ports, I used one for netgear adapter, and other for usb camera.

    when I installed program on EV3, I got the following error:
    VIDIOC_STREAMON error 28, No space left on device

    I googled to find a solution, but they are all talking about 2 web camers on the same bus to produce the problem, which is not my case.

    Would u please help me on that? I am thinking that the usb camera is the problem as it is cheap china one.

    Posted by Islam Elsharkawy | 2015/02/13, 08:12
    • I am sorry, I can’t really help, as I have never seen that error. It might be either the camera or the hub. You could try without the hub, but you would need to use USB or Bluetooth for streaming. Or you could try a different camera. The lsusb output might give a clue to the problem.

      Posted by Geek Grandad | 2015/02/13, 10:44
  3. Hi

    I tried to run the code in my lejos 0.9.0 beta. Runned the pc program as a java aplication, and runned the Lejos code as a lejos program. Lejos gives me the “Connection refused” error and some more errors. I tested the SSH, TELNET and Ping, it’s all ok.
    What i’m a doing wrong?

    Thanks

    Posted by Johnh | 2015/02/18, 18:14
    • Did you change HOST to the IP address of your PC? You also need your firewall (e.g. Windows firewall) to let the PC program run as a server on port 55555 – you will normally be prompted for this. You can change the port number if it clashes with anything else.

      Posted by Geek Grandad | 2015/02/18, 21:51
      • Do i put the external IP of the host PC? I have to do some port forwarding right? For it to be acessible from any network…

        Posted by Johnh | 2015/02/18, 23:58
      • Its the local IP of the PC you need as long as the PC and the EV3 are on the same network. You don’t need port forwarding.

        Posted by Geek Grandad | 2015/02/19, 09:04
  4. But if i want to make it online, the only thing to do is to change the ip to external ip right?

    Posted by Johnh | 2015/02/19, 10:26
    • I don’t know what you mean by you want it online. The EV3 program makes an outward call to a Java server on a PC that shows the video stream from the camera. If you used an external IP address and did port forwarding, then you could have the EV3 in one house and connect over the internet to a PC in another house (for example). That is not the normal definition of online. If you wanted the video stream accessed via a web browser, you would need to do http streaming. That is possible – I might have a go at a blog post on http video streaming. With http streaming, the server would be on the EV3, so you would need port forwarding on the network where the EV3 is running,

      Posted by Geek Grandad | 2015/02/19, 10:51
      • Yes that’s what i want, you’re right. I struggling with another problem now, the video size and the fps. Is it possible to increase the image size and/or the fps?

        Thanks for your support!

        Posted by Johnh | 2015/02/19, 10:55
      • The EV3 processor is not very fast. People seem to get fps ranging from 5 to 10 – I got about 5. If you increased the video size you would get less. I think the video size supported may depend on your camera – you will have to experiment to see what works.

        Posted by Geek Grandad | 2015/02/19, 11:41
      • I’ve written the post on HTTP streaming – https://lejosnews.wordpress.com/2015/02/20/webcam-http-streaming/

        Posted by Geek Grandad | 2015/02/20, 15:42
  5. Thanks a lot for this article. could you please tell me why I have problem with video library
    import lejos.hardware.video.Video;
    it seems not included in eclipse as well refers to this line by red color

    Posted by Ayad | 2015/04/10, 12:30
  6. hi, how can I get a bigger picture?

    thanks

    Posted by angelo | 2016/05/14, 15:26

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

About leJOS News

leJOS News keeps you up-to-date with leJOS. It features latest news, explains cool features, shows advanced techniques and highlights amazing projects. Be sure to subscribe to leJOS News and never miss an article again. Best of all, subscription is free!
Follow leJOS News on WordPress.com
%d bloggers like this: