Following on from Andy’s post on Webcam support, here is how to use your Webcam to stream color video to your PC.
The leJOS program is:
package mypackage;
import java.io.BufferedOutputStream;
import java.io.IOException;
import java.net.Socket;
import lejos.hardware.BrickFinder;
import lejos.hardware.Button;
import lejos.hardware.ev3.EV3;
import lejos.hardware.lcd.LCD;
import lejos.hardware.video.Video;
public class Stream {
private static final int WIDTH = 160;
private static final int HEIGHT = 120;
private static final String HOST = "192.168.0.2";
private static final int PORT = 55555;
public static void main(String[] args) throws IOException {
EV3 ev3 = (EV3) BrickFinder.getLocal();
Video video = ev3.getVideo();
video.open(WIDTH, HEIGHT);
byte[] frame = video.createFrame();
Socket sock = new Socket(HOST, PORT);
BufferedOutputStream bos = new BufferedOutputStream(sock.getOutputStream());
long start = System.currentTimeMillis();
int frames = 0;
LCD.drawString("fps:", 0, 2);
while(Button.ESCAPE.isUp()) {
try {
video.grabFrame(frame);
LCD.drawString("" + (++frames * 1000f/(System.currentTimeMillis() - start)), 5,2);
bos.write(frame);
bos.flush();
} catch (IOException e) {
break;
}
}
bos.close();
sock.close();
video.close();
}
}
And the PC program is:
package mypackage;
import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
import java.awt.image.BufferedImage;
import java.io.BufferedInputStream;
import java.net.ServerSocket;
import java.net.Socket;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;
public class CameraFrame {
private static final int WIDTH = 160;
private static final int HEIGHT = 120;
private static final int NUM_PIXELS = WIDTH * HEIGHT;
private static final int BUFFER_SIZE = NUM_PIXELS * 2;
private static final int PORT = 55555;
private ServerSocket ss;
private Socket sock;
private byte[] buffer = new byte[BUFFER_SIZE];
private BufferedInputStream bis;
private BufferedImage image;
private CameraPanel panel = new CameraPanel();
private JFrame frame;
public CameraFrame() {
try {
ss = new ServerSocket(PORT);
sock = ss.accept();
bis = new BufferedInputStream(sock.getInputStream());
} catch (Exception e) {
System.err.println("Failed to connect: " + e);
System.exit(1);
}
image = new BufferedImage(WIDTH, HEIGHT, BufferedImage.TYPE_INT_RGB);
}
public void createAndShowGUI() {
frame = new JFrame("EV3 Camera View");
frame.getContentPane().add(panel);
frame.setPreferredSize(new Dimension(WIDTH, HEIGHT));
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.addWindowListener(new WindowAdapter() {
@Override
public void windowClosing(WindowEvent e) {
close();
}
});
frame.pack();
frame.setVisible(true);
}
public void close() {
try {
if (bis != null) bis.close();
if (sock != null) sock.close();
if (ss != null) ss.close();
} catch (Exception e1) {
System.err.println("Exception closing window: " + e1);
}
}
private int convertYUVtoARGB(int y, int u, int v) {
int c = y - 16;
int d = u - 128;
int e = v - 128;
int r = (298*c+409*e+128)/256;
int g = (298*c-100*d-208*e+128)/256;
int b = (298*c+516*d+128)/256;
r = r>255? 255 : r<0 ? 0 : r;
g = g>255? 255 : g<0 ? 0 : g;
b = b>255? 255 : b<0 ? 0 : b;
return 0xff000000 | (r<<16) | (g<<8) | b;
}
public void run() {
while(true) {
synchronized (this) {
try {
int offset = 0;
while (offset < BUFFER_SIZE) {
offset += bis.read(buffer, offset, BUFFER_SIZE - offset);
}
for(int i=0;i<BUFFER_SIZE;i+=4) {
int y1 = buffer[i] & 0xFF;
int y2 = buffer[i+2] & 0xFF;
int u = buffer[i+1] & 0xFF;
int v = buffer[i+3] & 0xFF;
int rgb1 = convertYUVtoARGB(y1,u,v);
int rgb2 = convertYUVtoARGB(y2,u,v);
image.setRGB((i % (WIDTH * 2)) / 2, i / (WIDTH * 2), rgb1);
image.setRGB((i % (WIDTH * 2)) / 2 + 1, i / (WIDTH * 2), rgb2);
}
} catch (Exception e) {
break;
}
}
panel.repaint(1);
}
}
class CameraPanel extends JPanel {
private static final long serialVersionUID = 1L;
@Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
// Ensure that we don't paint while the image is being refreshed
synchronized(CameraFrame.this) {
g.drawImage(image, 0, 0, null);
}
}
}
public static void main(String[] args) {
final CameraFrame cameraFrame = new CameraFrame();
SwingUtilities.invokeLater(new Runnable() {
@Override
public void run() {
cameraFrame.createAndShowGUI();
}
});
cameraFrame.run();
}
}
leJOS News 
Thanks a lot for this article – the possibilities this enables now are quite staggering.
I have a question though: how do you achieve communication over IP by WiFi and camera usage whereas there is only one usb port on the brick? Do you use some kind of usb hub to plug a wifi dongle and the camera? Any specific brand/model?
Same for the webcam: do you know what are the models supported?
Thanks you again very much for this article that is of great help to me! I’ll continue following your post closely 🙂
Posted by Parapampa | 2015/01/18, 15:31Yes, I use a very tiny USB hub – http://www.ebay.co.uk/itm/Mini-3-Ports-High-Speed-USB-2-0-Hub-Rotate-Multi-Splitter-Expansion-PC-Laptop-/181449268300?pt=UK_Computing_USB_Cables&hash=item2a3f381c4c.
I think all UVC webcams are supported – see http://www.ideasonboard.org/uvc/. I use one similar to this – http://www.ebay.co.uk/itm/50-Mega-Pixels-6-LED-HD-Webcam-Camera-Web-Cam-with-MIC-for-Computer-PC-UK-Stock-/350927764714?pt=UK_Computing_ComputerComponents_Webcams&hash=item51b4ecbcea. They seem to be a copy of a Microsoft webcam and are very cheap.
Posted by Geek Grandad | 2015/01/18, 22:50Great, many thanks for this comprehensive answer! I’ll try those out, this is much appreciated.
Posted by Parapampa | 2015/01/20, 10:15No space left on device Error
thanks for sharing this very good stuff 🙂
I tried it my self by using usb hub with 4 ports, I used one for netgear adapter, and other for usb camera.
when I installed program on EV3, I got the following error:
VIDIOC_STREAMON error 28, No space left on device
I googled to find a solution, but they are all talking about 2 web camers on the same bus to produce the problem, which is not my case.
Would u please help me on that? I am thinking that the usb camera is the problem as it is cheap china one.
Posted by Islam Elsharkawy | 2015/02/13, 08:12I am sorry, I can’t really help, as I have never seen that error. It might be either the camera or the hub. You could try without the hub, but you would need to use USB or Bluetooth for streaming. Or you could try a different camera. The lsusb output might give a clue to the problem.
Posted by Geek Grandad | 2015/02/13, 10:44Hi
I tried to run the code in my lejos 0.9.0 beta. Runned the pc program as a java aplication, and runned the Lejos code as a lejos program. Lejos gives me the “Connection refused” error and some more errors. I tested the SSH, TELNET and Ping, it’s all ok.
What i’m a doing wrong?
Thanks
Posted by Johnh | 2015/02/18, 18:14Did you change HOST to the IP address of your PC? You also need your firewall (e.g. Windows firewall) to let the PC program run as a server on port 55555 – you will normally be prompted for this. You can change the port number if it clashes with anything else.
Posted by Geek Grandad | 2015/02/18, 21:51Do i put the external IP of the host PC? I have to do some port forwarding right? For it to be acessible from any network…
Posted by Johnh | 2015/02/18, 23:58Its the local IP of the PC you need as long as the PC and the EV3 are on the same network. You don’t need port forwarding.
Posted by Geek Grandad | 2015/02/19, 09:04But if i want to make it online, the only thing to do is to change the ip to external ip right?
Posted by Johnh | 2015/02/19, 10:26I don’t know what you mean by you want it online. The EV3 program makes an outward call to a Java server on a PC that shows the video stream from the camera. If you used an external IP address and did port forwarding, then you could have the EV3 in one house and connect over the internet to a PC in another house (for example). That is not the normal definition of online. If you wanted the video stream accessed via a web browser, you would need to do http streaming. That is possible – I might have a go at a blog post on http video streaming. With http streaming, the server would be on the EV3, so you would need port forwarding on the network where the EV3 is running,
Posted by Geek Grandad | 2015/02/19, 10:51Yes that’s what i want, you’re right. I struggling with another problem now, the video size and the fps. Is it possible to increase the image size and/or the fps?
Thanks for your support!
Posted by Johnh | 2015/02/19, 10:55The EV3 processor is not very fast. People seem to get fps ranging from 5 to 10 – I got about 5. If you increased the video size you would get less. I think the video size supported may depend on your camera – you will have to experiment to see what works.
Posted by Geek Grandad | 2015/02/19, 11:41I’ve written the post on HTTP streaming – https://lejosnews.wordpress.com/2015/02/20/webcam-http-streaming/
Posted by Geek Grandad | 2015/02/20, 15:42Thanks a lot for this article. could you please tell me why I have problem with video library
import lejos.hardware.video.Video;
it seems not included in eclipse as well refers to this line by red color
Posted by Ayad | 2015/04/10, 12:30hi, how can I get a bigger picture?
thanks
Posted by angelo | 2016/05/14, 15:26Hello, i am trying to set up a communication, but i am still getting connection timed out error. Can you please give me some advice?
Posted by Hrabě Česnek | 2018/01/09, 23:40Will this code also work with Bluetooth?
Posted by Mark | 2018/03/16, 22:25Hi, i don’t get why the code is separated into lejos program and pc program, do i need to run them separately?
Posted by jia yung | 2019/04/02, 16:11