There has been a lot of speculation on what near mode is since we announced it. As I mentioned in the original post, the Kinect for Windows device has new firmware which enables the depth camera to see objects as close as 50 centimeters in front of the device without losing accuracy or precision, with graceful degradation down to 40 centimeters.
The lenses on the Kinect for Windows sensor are the same as the Kinect for Xbox 360 sensor, so near mode does not change the field of view as some people have been speculating. As some have observed, the Kinect for Xbox 360 sensor was already technically capable of seeing down 50 centimeters – but with the caveat “as long as the light is right”.
That caveat turned out to be a pretty big caveat. The Kinect for Windows team spent many months developing a way to overcome this so the sensor would properly detect close up objects in more general lighting conditions. This resulted not only in the need for new firmware, but changes to the way the devices are tested on the manufacturing line. In addition to allowing the sensor to see objects as close as 40 centimeters, these changes make the sensor less sensitive to more distant objects: when the sensor is in near mode, it has full accuracy and precision for objects 2 meters away, with graceful degradation out to 3 meters. Here is a handy chart one of our engineers made that shows the types of depth values returned by the runtime:
In Beta 2, for an object 800-4000 millimeters from the sensor the runtime would return the depth value, and the runtime returned a 0 regardless if the detected depth was unknown, too near or too far. Our version 1.0 runtime will return depth values if an object is in the above cyan zone. If the object is in the purple, brown or white zones, the runtime will return a distinct value indicating the appropriate zone.
Additionally, in version 1.0 of the runtime, near mode will have some skeletal support, although not full 20-joint skeletal tracking (ST). The below table outlines the differences between the default mode and near mode:
We believe that near mode, with its operational envelope of 40 centimeters to 3 meters, will enable many new classes of applications. While full 20-joint ST will not be supported in near mode with version 1.0 of the runtime, we will be working hard to support ST in near mode in the future!
General Manager, Kinect for Windows
This is great news. I believe, if possible, that hand/wrist/elbow tracking would be very powerful, if it could be pulled off. This may allow for sign language type apps, or hand gestures. I don't know if that would be a stretch feature or not, but I can think of quite a few uses for it ðŸ™‚
Will we able to upgrade our current kinect for 360 to this new firmware?
Hi Craig. Does that mean the commercial Kinect for Windows will have both Default and Near Modes? Is Near Mode something we can flip on and off. I actually prefer the Default Mode; don't have much use for Near Mode in my current apps. Thanks, Craig.
Kinect Fan, yes, you will be able to switch between near mode and default mode on the Kinect for Windows hardware, using the Kinect for Windows SDK 1.0.
Woohoo! That's awesome! Thanks, Josh. By the way, I attended your Kinect SDK seminar at Carnegie Mellon last October. Great stuff. ðŸ™‚
@Andrea: No, this firware is not designed for the XBox Kinect.
Will you be considering supporting multiple monitors or will the application have to run on the screen where the Kinect is running?
It would be an interesting experiment to see if Kinect could calibrate between the placement of monitors.
@Matthew: Not sure I understand your question. The Kinect sensor can be placed wherever is appropriate for the specific application. It doesn't need to be associated with a particular monitor, or with any monitor at all, for that matter.
Is it possible for kinect windows device to capture gestures of a person when sitting down behind a desk (i.e. Kinect working in Near mode)? I understood that the currently available kinect device is not capable of partial skeleton detection. Is that true?
Hi! May I know whether the sdk has support for eyebrow and lip detection stated by the rumors?
Kinect for Windows SDK 1.5 released on 5/21/12 includes the Face Tracking SDK.
The Face Tracking component tracks face position, orientation and facial features in real time.
A 3D mesh of the tracked face along with eye-brow position and mouth shape is animated in real time.
Multiple faces can be tracked simultaneously.
Face Tracking components can be used in native C++ code and a managed wrapper is provide for C# and VB projects.
Please confirm whether the hardware (not just firmware) is different between Kinect XBOX and PC (except the USB cable of course, but you can buy a cheap adapter for that).
Reading between the lines, the XBOX Kinect is still the same hardware but logistical problems (worldwide manufacturing, testing and distribution) made it just too big a change to risk right now.
If the hardware is the same, could the firmware upgrade come some time in the future. Or are all improvements just being withheld to help sell a Kinect 2, released with perhaps even built into an XBOX 720?
Found the answer myself here…
comment trouver le profondeur d'une troux dans la rue