For various reasons, I need to do multiple marker tracking in processing with NyARToolkit. However, with the default NyAR4psg layer between these two, multiple marker tracking is downright hard, and when you get it working, it’s not quite what you expect. After a few days of Java hacking, during which I was very pleasantly surprised with eclipse, I am now pleased to present to you my modifications to the NyAR4psg that makes multiple marker tracking easy! See here:
Standard hiro and kanji markers tracked simultaneously with augmented reality sphere and cube. In the background some artwork by my daughter!
I’ve called it NyARMultiBoard, and you can use it instead of the default NyARBoard if you want to track multiple markers.
Download a ZIP file containing everything (source code, jar files) from this directory. If you unpack this into your processing sketchbook/libraries directory, it should work out of the box. It’s a drop-in replacement for NyAR4psg, so you don’t need to have that installed as well. There is an example to get you started in NyAR2/example/NyARMultiTest. Note: This uses the GSVideo capturing stack as I explain here, you should easily be able to change it back to processing defaults (just change GSCapture to Capture).
Please let me know in the comments if this works (or doesn’t) for you!
I made this screencast to demonstrate the multiple marker tracking, assisted by TNR:
I also made this really bad screencast (old webcam + night time lighting + transcoding):
If you’re really into the details
I’ve just added two new classes NyARMultiBoard and NyARMultiBoardMarker to the default NyAR4psg distribution. Very importantly, NyARToolkit itself needs to be patched with one extra method in NyARDetectMarker, see the NyARMultiBoard comments.
Update on 20110304
I’ve fixed the problematic frame bug in gsvideo that many of you have been running into. See this post.
Update on 20110305
I’ve updated NyAR2 so it works with the P3D renderer as well, which is often faster for blitting the webcam image onto the display. The updated zip file is named NyAR2-20110305.zip, and it can be downloaded from the usual directory. My changes are based on NyAR4psg 0.3.0 and NyARToolkit 2.5.2.
Related posts:
Pingback: Charl Botha
Pingback: やった
Step 1) Poke out your eyes and replace them with cameras running this software.
Step 2) ????
Step 3) Profit!
Great stuff! The soundtrack really turns it in to James Bond stuff.
what is the music playing in the two screencasts?
Hello, i ve benn trying to use your library, i m using mac osx 10.5.8 leopard, and i m getting the following error:
This version of Processing only supports libraries and JAR files compiled for Java 1.5.
A library used by this sketch was compiled for Java 1.6 or later,
and needs to be recompiled to be compatible with Java 1.5.
Exception in thread “Animation Thread” java.lang.UnsupportedClassVersionError: Bad version number in .class file
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:676)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:317)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:375)
at NyARMultiTest2.setup(NyARMultiTest2.java:58)
at processing.core.PApplet.handleDraw(PApplet.java:1571)
at processing.core.PApplet.run(PApplet.java:1496)
at java.lang.Thread.run(Thread.java:613)
Could u help me with this???
My processing setup is based on a java 1.6 setup, whilst yours is based on 1.5 (probably the default).
To get this working, you have the following options:
1. Get your processing to use java 1.6. I have no idea how to do this on OS-X.
2. Rebuild my nyar2 code (the thing in the zip file) on your system.
3. Wait for me to have time to rebuild and package NyAR2 with java 1.5.
Option 2, if you are able to do this, will be the quickest. Option 3 would also be easy, but I’m not sure when I’m going to have time to do this.
Are options 1 or 2 possible for you?
Hello, i ve benn trying to use your library, i m using mac osx 10.5.8 leopard, and i m getting the following error:
processing.app.debug.RunnerException: IllegalAccessError: tried to access class com.sun.opengl.impl.Project from class javax.media.opengl.glu.GLU
at processing.app.Sketch.placeException(Sketch.java:1565)
at processing.app.debug.Runner.findException(Runner.java:568)
at processing.app.debug.Runner.reportException(Runner.java:543)
at processing.app.debug.Runner.exception(Runner.java:498)
at processing.app.debug.EventThread.exceptionEvent(EventThread.java:367)
at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
at processing.app.debug.EventThread.run(EventThread.java:89)
java.lang.IllegalAccessError: tried to access class com.sun.opengl.impl.Project from class javax.media.opengl.glu.GLU
at javax.media.opengl.glu.GLU.(GLU.java:218)
at processing.opengl.PGraphicsOpenGL.(PGraphicsOpenGL.java:113)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:501)
at processing.core.PApplet.makeGraphics(PApplet.java:1316)
at processing.core.PApplet.size(PApplet.java:1142)
at processing.core.PApplet.size(PApplet.java:1102)
at NyARTest.setup(NyARTest.java:80)
at processing.core.PApplet.handleDraw(PApplet.java:1571)
at processing.core.PApplet.run(PApplet.java:1496)
at java.lang.Thread.run(Thread.java:613)
java.lang.IllegalAccessError: tried to access class com.sun.opengl.impl.Project from class javax.media.opengl.glu.GLU
at javax.media.opengl.glu.GLU.(GLU.java:218)
at processing.opengl.PGraphicsOpenGL.(PGraphicsOpenGL.java:113)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:501)
at processing.core.PApplet.makeGraphics(PApplet.java:1316)
at processing.core.PApplet.size(PApplet.java:1142)
at processing.core.PApplet.size(PApplet.java:1102)
at NyARTest.setup(NyARTest.java:80)
at processing.core.PApplet.handleDraw(PApplet.java:1571)
at processing.core.PApplet.run(PApplet.java:1496)
at java.lang.Thread.run(Thread.java:613)
processing.app.debug.RunnerException: RuntimeException: tried to access class com.sun.opengl.impl.Project from class javax.media.opengl.glu.GLU
at processing.app.Sketch.placeException(Sketch.java:1565)
at processing.app.debug.Runner.findException(Runner.java:568)
at processing.app.debug.Runner.reportException(Runner.java:543)
at processing.app.debug.Runner.exception(Runner.java:498)
at processing.app.debug.EventThread.exceptionEvent(EventThread.java:367)
at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
at processing.app.debug.EventThread.run(EventThread.java:89)
Exception in thread “Animation Thread” java.lang.RuntimeException: tried to access class com.sun.opengl.impl.Project from class javax.media.opengl.glu.GLU
at processing.core.PApplet.makeGraphics(PApplet.java:1341)
at processing.core.PApplet.size(PApplet.java:1142)
at processing.core.PApplet.size(PApplet.java:1102)
at NyARTest.setup(NyARTest.java:80)
at processing.core.PApplet.handleDraw(PApplet.java:1571)
at processing.core.PApplet.run(PApplet.java:1496)
at java.lang.Thread.run(Thread.java:613)
Could you first double-check for me that you can run all the OpenGL examples that ship with processing?
(PDE menu | File | Examples | Libraries | OpenGL)
Pingback: Tutorial: Multimaker + Nyartoolkit « Emiliusvgs Projects
hi, thanks for your guide.
I get an error message from processing:
NyAR4psg/0.3.0;NyARToolkit for java/2.5.0+;ARToolKit/2.72.1
do_wait: drmWaitVBlank returned -1, IRQs don’t seem to be working correctly.
Try adjusting the vblank_mode configuration parameter.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xb77c9136, pid=2861, tid=2683689840
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) Client VM (16.3-b01 mixed mode linux-x86 )
# Problematic frame:
# C [libc.so.6+0x75136] memcpy+0×46
#
# An error report file with more information is saved as:
# /home/francesco/processing-1.0.9/hs_err_pid2861.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
what do you think about it?
I’m using ubuntu 9.10, i installed all the packages requested i think.
Ah, important thing: the patch worked just one time in ten or twenty attempts, and for about 10 seconds.
Multitracking was working :(
drmWaitVBlank — indicates a problem with your graphics drivers.
First thing you could do, is disable compiz (System | Preferences | Appearance | Visual Effects — then choose “none”).
Let me know how it goes (also after a reboot).
thank you for your reply. It seems that the problem was the size in void setup() that didn’t matched with my webcam. Once I put my values there, the patch started working :)
I still get the error I described you, but I don’t care until it works. thanks again for your guide!
Hi Franz,
The problematic frame crash only occurs in the more recent releases of GSVideo. Using the 0.4.5 release ‘fixes’ this. For more info: http://forum.processing.org/topic/gsvideo-0-6-crash-problem
Kind regards,
Noeska
Thanks for you reply. Too bad I think the problem isn’t there (i tried the 0.4.5 gsvideo but didn’t changed), in fact the sketch works, but only in 352×288 resolution.
I tried to adjust in ubuntu the parameter processing gave error, but without result.
the same error:
do_wait: drmWaitVBlank returned -1, IRQs don’t seem to be working correctly.
Try adjusting the vblank_mode configuration parameter.
any other suggestion?
Thanks in advance
I think I have the same problem, not sure though.
here the NyARMultiTest.pde works for a few seconds, but then the image freezes.
I tried downgrading to GSVideo 0.4.5 version, but no change. It also only works at 352×288 resolution.
also, as suggested, visual effects in system>preferences>appearance are set to ‘none’…
the GSVideo example ‘asciivideo’ does not freeze…
anything else I could try?
thanks!
Hi, just download and test… awesome. :)
Two things (Mac os X 10.6):
FIRST:
- I add this few lines after setup to avoid the freeze gray screen (as in the NyARToolKit)
try {
quicktime.QTSession.open();
}
catch (quicktime.QTException qte) {
qte.printStackTrace();
}
SECOND:
- I’d like to analize the capture at lower resolution than the size of applet
… I mean…
1) size (800,600,OPENGL)
2) create a new cam object (640,480)
… get error (out of array) on nya.detect
#####
hint(DISABLE_DEPTH_TEST);
image(cam,0,0,800,600);
hint(ENABLE_DEPTH_TEST);
if (nya.detect(cam))
{
#####
anyway… thanks. :)
Big thankies for all the great work you have done. I’ve studied Proce55ing and ARToolkit and love the idea of merging the two platforms to make better contraptions.
However, I’ve had a couple of crashes (after lots of compiles and camera device switching etc.) and not sure who to send the log to (Sun/Oracle, Processing, GSVideo or OpenGL) and if I should bother at all. First few lines are quothed below;
———————————————————-
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x7c342eee, pid=3860, tid=4920
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) Client VM (16.3-b01 mixed mode windows-x86 )
# Problematic frame:
# C [msvcr71.dll+0x2eee]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
————— T H R E A D —————
Current thread (0×18803000): JavaThread “Animation Thread” [_thread_in_native, id=4920, stack(0x1bee0000,0x1bf30000)]
siginfo: ExceptionCode=0xc0000005, reading address 0×22860020
Registers:
EAX=0×22960020, EBX=0×00100000, ECX=0×00040000, EDX=0×00000000
ESP=0x1bf2f810, EBP=0x1bf2f818, ESI=0×22860020, EDI=0x09ad1284
EIP=0x7c342eee, EFLAGS=0×00010212
Top of Stack: (sp=0x1bf2f810)
0x1bf2f810: 09ad1284 18803110 1bf2f838 6d3e728d
0x1bf2f820: 09ad1284 22860020 00100000 1bf2f86c
0x1bf2f830: 00000000 18803000 1bf2f878 023aa7fe
0x1bf2f840: 18803110 1bf2f86c 22860020 00000000
0x1bf2f850: 1bf2f864 00000000 00000000 0012c000
0x1bf2f860: 00000000 09ad1278 00004000 143b1610
0x1bf2f870: 6d8b1086 18803000 1bf2f8e0 02292e21
0x1bf2f880: 22860020 00000000 00000000 00000000
Instructions: (pc=0x7c342eee)
0x7c342ede: 00 00 00 75 14 c1 e9 02 83 e2 03 83 f9 08 72 28
0x7c342eee: f3 a5 ff 24 95 0c 30 34 7c 8b c7 ba 03 00 00 00
Stack: [0x1bee0000,0x1bf30000], sp=0x1bf2f810, free space=13e1bf2f344k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [msvcr71.dll+0x2eee]
C
J java.nio.Bits.copyToByteArray(JLjava/lang/Object;JJ)V
j java.nio.DirectIntBufferU.get([III)Ljava/nio/IntBuffer;+126
j java.nio.IntBuffer.get([I)Ljava/nio/IntBuffer;+5
j codeanticode.gsvideo.GSCapture.read()V+24
J NyARMultiTest.draw()V
J processing.core.PApplet.handleDraw()V
j processing.core.PApplet.run()V+51
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [jvm.dll+0xf049c]
V [jvm.dll+0x17fcf1]
V [jvm.dll+0xf0667]
V [jvm.dll+0xf06dd]
V [jvm.dll+0x11a2a0]
V [jvm.dll+0x1ddb14]
V [jvm.dll+0x17f96c]
C [msvcr71.dll+0x9565]
C [kernel32.dll+0x4d0e9]
C [ntdll.dll+0x419bb]
C [ntdll.dll+0x4198e]
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J java.nio.Bits.copyToByteArray(JLjava/lang/Object;JJ)V
j java.nio.DirectIntBufferU.get([III)Ljava/nio/IntBuffer;+126
j java.nio.IntBuffer.get([I)Ljava/nio/IntBuffer;+5
j codeanticode.gsvideo.GSCapture.read()V+24
J NyARMultiTest.draw()V
J processing.core.PApplet.handleDraw()V
j processing.core.PApplet.run()V+51
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x18a71800 JavaThread "gstreamer service thread 1" daemon [_thread_blocked, id=3684, stack(0x21490000,0x214e0000)]
0x0090b400 JavaThread “DestroyJavaVM” [_thread_blocked, id=5412, stack(0x000d0000,0x00120000)]
0x02224c00 JavaThread “D3D Screen Updater” daemon [_thread_blocked, id=4684, stack(0x1c040000,0x1c090000)]
=>0×18803000 JavaThread “Animation Thread” [_thread_in_native, id=4920, stack(0x1bee0000,0x1bf30000)]
0x1879cc00 JavaThread “AWT-EventQueue-0″ [_thread_blocked, id=3428, stack(0x196b0000,0x19700000)]
0×18710800 JavaThread “AWT-Windows” daemon [_thread_in_native, id=2204, stack(0x191a0000,0x191f0000)]
0×18710000 JavaThread “AWT-Shutdown” [_thread_blocked, id=2228, stack(0x19150000,0x191a0000)]
0×18744800 JavaThread “Java2D Disposer” daemon [_thread_blocked, id=5876, stack(0x18bb0000,0x18c00000)]
0x02273c00 JavaThread “Low Memory Detector” daemon [_thread_blocked, id=4628, stack(0x18510000,0x18560000)]
0×02270800 JavaThread “CompilerThread0″ daemon [_thread_blocked, id=4536, stack(0x184c0000,0x18510000)]
0x0225dc00 JavaThread “JDWP Command Reader” daemon [_thread_in_native, id=4412, stack(0x18470000,0x184c0000)]
0x0225cc00 JavaThread “JDWP Event Helper Thread” daemon [_thread_blocked, id=5084, stack(0x18420000,0x18470000)]
0x0225a000 JavaThread “JDWP Transport Listener: dt_shmem” daemon [_thread_blocked, id=3700, stack(0x183d0000,0x18420000)]
0x0224f800 JavaThread “Attach Listener” daemon [_thread_blocked, id=3440, stack(0x18380000,0x183d0000)]
0x0224c400 JavaThread “Signal Dispatcher” daemon [_thread_blocked, id=3400, stack(0x18330000,0x18380000)]
0x0221f800 JavaThread “Finalizer” daemon [_thread_blocked, id=5944, stack(0x182e0000,0x18330000)]
0x0221a800 JavaThread “Reference Handler” daemon [_thread_blocked, id=3560, stack(0x18290000,0x182e0000)]
……………….
OMG, I’ve posted before I read the other peoples’ errors, looks like it is the current GSVideo thinggie. I may put up with the errors unless I find an older or new version that allows for camera source selection, my default device is a built-in TV tuner analogue input. BTW is there a version of NyARToolkit with English comments?
As you’ve seen, you have to downgrade to 0.4.5 or to 0.5.1 to get rid of the problematic frame error. I’m not sure if these versions support camera selection.
The author is aware of the frame error, but we don’t know when an update is due.
NyARToolkit == Japanese comments. I had to deal with this whilst making NyARMultiBoard and NyARMultiBoardMarker. It helps to read my English comments. :) (inside the zip of course)
Good news – GSVideo 0.5.1 is OK, so far no crashes and the input stream selection works fine too.
Google Translate did a decent job on comments. Sometimes I just ‘have to’ read the code and while I like guessing I spent a few minutes translating the remaining Japanese comments in the src files. Thanks again, I bet this will keep me off the streets for some time.
Another small issue with GSVideo. Java(TM) Platform SE binary is always stopped by MS DEP (data Execution prevention) upon exit (not a problem using the regular processing video instead of GSVideo). Could it just be the attempt to delete the temp file, I’ve noticed that many sketches fail to delete temp files, but don’t throw the DEP error?
MS ERROR REPORT DIALOG
———————-
Problem signature:
Problem Event Name: BEX
Application Name: java.exe
Application Version: 6.0.200.2
Application Timestamp: 4bc398af
Fault Module Name: jna5292602199824696819.tmp_unloaded
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 4a140bf1
Exception Offset: 1f84e3b6
Exception Code: c0000005
Exception Data: 00000008
OS Version: 6.0.6002.2.2.0.768.3
Locale ID: 3081
Additional Information 1: 4c0d
Additional Information 2: 47838752725dbac0a892abe924026338
Additional Information 3: 6a72
Additional Information 4: b0efcf739a041b60b8e73fa8f263e1a6
———————–
I haven’t seen *that* one before. At least it’s only happening at exit…
I hope that GSVideo stabilises during the coming year. We just gave a course to 124 CS first years with processing + GSVideo, and one of the biggest complaints was the stability. Hopefully much better in a year’s time.
Still, In comparison with working in the original ARToolkit (free version) the NyARToolkit is a dream, I used it to prove a hypothesis that proprioception allows for intuitive manipulation (navigation) of 3D CAD models during critique by untrained users (clients etc). I also completed a studio unit, where some of the students never managed to get the Wii Remote working with Proce55ing via Wrj4P5 and BlueCove Bluetooth stack, many of the projects could have benefited from video tracking instead of the Wii. Most were just too scared of bricking their laptops using a hack I helped to deploy using the Windows XP Broadcom BT enumerator instead of the joke that Vista forces upon the users. See our blog at http://xii9190.wordpress.com/ if you are curious (BTW my name on the blog is infophile and the video is worth a look http://www.youtube.com/watch?v=zXYTJdifoNk shameless promotion haha).
Hi ,
Thanks for your working , I have a question , are running your program using Windows or Linux??
Thanks a lot
I did most of this development on Linux. Everything has been tested on Windows as well and works fine there .
can you teach me how to get all of this stuff in ubuntu 10.04? i want to develop an AR in ubuntu. help please??
Hi , first of all thanks for replying to me . I managed to get it work on windows. But I have another important question which relates to my project , what I want is , when the program recognizes the markers it should be able to register the positions of these markers, is there any function can do this ?.
Thanks.
This is where it gets interesting. :)
1. You can easily get the 2d positions of all four corners. See how pos2d is used in the sample code.
2. More flexible: You have to learn how to use transformation matrices, as you get a transformation matrix for each marker containing its 3D position and orientation. See here for a start: http://graphics.tudelft.nl/Courses/TI1100A/MatrixFun (your chrome should be able to translate the Dutch to English).
Good luck!
Hi ,
How could I get the rank number of each marker ? for example I put the markers in front of the camera in an order, from the left hand side to the right hand side . So if I put hiro.patt in the first position from the left hand side it will have rank number =1 , and kanji.patt will have rank number =2, then if I will change the position of the hiro.patt to the second or third or n position it will take the rank number depends on its position in the order , and this will be implemented to the other marker . Is that possible to be implemented??Cheers
Hi ,
in the codes of the programs, there is a for loop for the nya.markers.length.
Where is the array of the markers ? there is no array of markers ,there is only an array of patts.
nya.markers is created internally by NYA based on the number of patterns that you give it (IIRC). Please take a look at the sample code again.
This is awesome! Thanks so much for making this available.
Hi there. I’ve been trying to get this to work but, after getting around some problems, there’s one that i can’t solve.
I’ve been using the example from http://diariowm.wordpress.com.
The error trace i get: http://pastebin.com/XC1a8Dh1
The code (sketch): http://pastebin.com/7yxW44dE
Any help would be very much appreciated :)
Hi sir,
I’m very appreciate for your work, which gives me opportunity to achieve multiple marker tracking on Mac OS X 10.5.8.
Unfortunately, the same as Tomas Ferraro’s question, the code doesn’t work properly on my computer. I’ve tried “Solution 2″, that is to say, using JVM1.5 to rebuild the jar file “NyAR2.jar” by Eclipses. However, it still shows the annoying messages – UnsupportedClassVersionError: A library is using code compiled with an unsupported library of Java.
Here’s my procedures of rebuilding the jar file:
1. Open a Java Project NyARBuild
2. Import src/*.java into the NyARBuild/src
3. Solve all the dependency problems by including all necessary libraries until there’s no error.
4. Choose J2SE 1.5 as the compiler
5. Export NyARBuild/src/*.java as NyAR2.jar for replacing the original one in the NyARMultiTest example.
Would you kindly point me out where’s the erroneous procedure and how to fix it? Appreciates a lot!!
With step 2: Why are you importing the src dir? The parent directory is already an Eclipse file. Use “Import existing project”.
Also do make sure that library/NyARToolkit.jar (this is a dependency of NyAR2) has been built with the correct JDK. Get this from the NyARToolkit website.
In step 5: Why are you exporting *.java files? You should export the whole built library.
Good luck!
Hi cpbotha,
Thanks for your help, I’ve successfully built the eclipse project in a correct procedure (sorry for the previous newbie questions :P) and substitute the new NyAR2.jar to the library folder of the NyARMultiTest sketch. I’ve also make sure to copy/paste the NyARToolkit.jar from nyar4psg-0.3.0 (I downloaded it from the official website and able to execute its sample application properly). However, it still doesn’t work. The UnsupportedClassVerionError message raises again and again making me frustrated. I’ll probably turn to windows for further development. However, still wondering that will you release the probably Mac-compatible version in the future (built with JVM1.5)? Cause it’s a pity that if there’s something wrong in my process to crush it. Thanks a lot :)
Hi sir,
I got something weird in the debug console by trying the processes again.
Although I’ve rebuilt the jar file by JVM1.5,
The error message still shows that NyARMultiBoard.class was compiled for Java 1.6 or later.
Am I miss-replacing the files?
FYI, by excuting /example/NyARMultiTest/NyARMultiTest.pde as an example, I replaced /library/NyAR2.jar to the re-built one.
Is there any alternative ways for trying?
Thanks!
Pingback: Carlin M. Wragg
The part I understand most: is your daughters artwork in the background ;)
I’ve now fixed the gsvideo problematic frame crash, see: http://cpbotha.net/2011/03/04/i-crushed-the-gsvideo-problematic-frame-error/
(this comment is for the people who are subscribed to this post)
Another updated for the comment subscribers: I’ve just release a new version of NyAR2 that supports the P3D renderer (sometimes faster for image blitting) as well as OPENGL.
hello,
I’m trying to run your NyAR2 example with video input coming from Webcamstudio for linux, which creates a virtual device at /dev/video1 .
I can access it by:
cam=new GSCapture(this,width,height, “v4lsrc”,”/dev/video1″);
it works most of the time, but it crashes quite often. Not sure if it’s related to Webcamstudio at all, but I haven’t seen it when using /dev/video0 directly…:
NyAR4psg/0.3.0;NyARToolkit for java/2.5.0+;ARToolKit/2.72.1jp.nyatla.nyartoolkit.NyARException
at jp.nyatla.nyartoolkit.detector.NyARDetectMarker$DetectSquareCB.onSquareDetect(NyARDetectMarker.java:173)
at jp.nyatla.nyartoolkit.core.squaredetect.NyARSquareContourDetector_Rle.detectMarkerCB(NyARSquareContourDetector_Rle.java:139)
at jp.nyatla.nyartoolkit.detector.NyARDetectMarker.detectMarkerLite(NyARDetectMarker.java:275)
at jp.nyatla.nyar4psg.NyARMultiBoard.detect(NyARMultiBoard.java:163)
at NyARMultiTest.draw(NyARMultiTest.java:114)
at processing.core.PApplet.handleDraw(Unknown Source)
at processing.core.PApplet.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)
processing.app.debug.RunnerException: RuntimeException: Error while marker detecting up NyARToolkit for java
at processing.app.Sketch.placeException(Sketch.java:1543)
at processing.app.debug.Runner.findException(Runner.java:582)
at processing.app.debug.Runner.reportException(Runner.java:558)
at processing.app.debug.Runner.exception(Runner.java:498)
at processing.app.debug.EventThread.exceptionEvent(EventThread.java:367)
at processing.app.debug.EventThread.handleEvent(EventThread.java:255)
at processing.app.debug.EventThread.run(EventThread.java:89)
Exception in thread “Animation Thread” java.lang.RuntimeException: Error while marker detecting up NyARToolkit for java
at processing.core.PApplet.die(Unknown Source)
at processing.core.PApplet.die(Unknown Source)
at jp.nyatla.nyar4psg.NyARMultiBoard.detect(NyARMultiBoard.java:199)
at NyARMultiTest.draw(NyARMultiTest.java:114)
at processing.core.PApplet.handleDraw(Unknown Source)
at processing.core.PApplet.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)
what can I do about this ?
thanks,
Tim
Your exception is happening deep in NyARToolkit, at NyARDetectMarker.java line 173. I checked the code:
if(!NyARLinear.crossPos(sq.line[i],sq.line[(i + 3) % 4],sq.sqvertex[i])){
throw new NyARException();//ここのエラー復帰するならダブルバッファにすればOK
}
… my lack of Japanese is not helping here. :)
Bottom line: Either futz with webcamstudio settings, or start studying the NyARToolkit source and try to figure out what NyARLinear.cross() is doing and what it means when it returns false.
If you do figure it out, please report back!
(I took a quick look in NyARLinear: your video is causing some algorithmic inconsistency in NyARToolkit.
You could try the new NyARToolkit 3 release and see if that helps.)
ok thanks for the reply.
it’s unlikely that i’d figure it out any time soon…
btw, according to google translate, that japanese line says:
“If you double-buffered OK to return this error”
I have another question:
I’m sending the translation and rotation parameters to another program (pd/Gem), but I’m having a hard time interpreting the angles. It seems that they invert when z-rotation is > 180 or some other wierd thing happens to them.
I’m not sure if I’m doing it correctly.
I multiply the values with “float radiantoangle=180/PI”, to get degrees, and then sending the numbers coming from
radiantoangle*nya.markers[i].angle.array()[0]
radiantoangle*nya.markers[i].angle.array()[1]
radiantoangle*nya.markers[i].angle.array()[2]
to pd/Gem, where i need 0- to 360° values to rotate an object…
is this correct ?
I’ve burnt myself using the angles and translations. The transformation matrices are your best bet. Rather use each maker’s transmat (OpenGL-style) or transmatP3D (added in my yesterday’s release of NyAR2: PMatrix3D-style) 4×4 homogeneous transformation matrices. These are all relative to board space.
See this for more hints: http://graphics.tudelft.nl/Courses/TI1100A/MatrixFun
thanks for the tip.
I’ve been staring at that page before, but it’s beyond my comprehension. (and I do read dutch)
I have no idea how to use this…
an example would be great :)
@tim167:
Two tips:
1. you need to dust off that linear algebra text book (or find a good primer on the internet). ;) Once you get transformation matrices, this will be easy.
2. instead of using the OpenGL conversion function on transmat, you can just use transmatP3D directly.
Good luck!
I’m beginning to study AR, and very interested by nyartoolkit. But so far everything I found on installing the toolkit was in Japanese. Can anyone help me, my email is robson.f16@gmail.com
You install it like any other processing library: Unpack the zip file in your sketchbook in the libraries folder. You should finally have sketchbook/libraries/NyAR2, with in that directory the two folders “example” and “library”. When you startup processing again, you’ll be able to run the examples.
I’d like to thank the maintainer of this blog. The code you posted was crucial for the project I was working at, and I’m going to publish my work soon so that others can possibly benefit from it.
Bye!
Hello,
I’m testing NyAR2′s example, but i can’t run NyARMultiTest .
Processing error: Cannot find a class or type named “NyARMultiBoard” from the line: NyARMultiBoard nya;.
I’ve installed NyAR2-20110305 library
in processing 1.2.1 Windows XP
May you help me ?
thanks
Alain
Make sure you’ve unpacked the NyAR2*.zip file in your sketchbook/libraries folder, and that you’ve restarted processing.
That error message means that it’s NOT finding the library.
Thanks for your answer.
i download again your library
I copy NyAR2 in
D:\Documents and Settings\parisala\Mes documents\Processing\libraries
in NyAR2 there are:
bin
data
example
library
– NyAR2.jar (40ko)
– NyARToolkit.jar (552 ko)
reference
src
…
it’s correct for all examples except NyARMultiTest.pde
I ‘ve installed only NyAR2 and GSVideo libraries
I do the same test with V20100817 and have the same error
so i reboot my PC and make a new test
Alain
And now the answer !
in my libraries directoty i have a NyARToolkit directory for an other test.
I have renamed this library and now the example run perfectly
Thanks for your works
Best regard
Alain
hi,
Thanks for this, it’s been quite useful already.
I would like to add more markers.
I made *.patt files using this http://flash.tarotaro.org/blog/2008/12/14/artoolkit-marker-generator-online-released/
I checked in gedit that the patt files have exactly the same structure as the example markers, and placed them in the /data directory along with patt.kanji and patt.hiro. I added them to the sketch like so:
String[] patts = {
“patt.hiro”, “patt.kanji”, “marker8.patt”, “marker9.patt”, “marker10.patt”
};
If I run it I get:
“Error while setting up NyARToolkit for java”
does it have to do with “camera_para.dat” maybe?
how should I add more markers?
thanks
Tim
Right below the patts line, there is a widths array that should have the same number of entries as patts. It contains the corresponding widths of the markers, as is also indicated in the comments.
ah, right, thanks for curing my selective blindness ;)
hi tim!
have you manage to make your custom markers?
i’ve tried with no sucssess…
Hello,
it didn’t work out for me yet…
it fails at the line:
” nya=new NyARMultiBoard(this,width,height,”camera_para.dat”,patts,widths);”
what i did was: add “patt.marker1″ to the patts array, add one entry (80) to the widths array, and copied my marker file “patt.marker1″ to /libraries/NyAR2/data.
any suggestions?
thanks
no….the same for me…. i think is because the file generated is a “.pat” not a “.patt”.
let’s stay in touch… if i find how to i’ll tell you and you do the same…ok?.
cheers
Hi Joao,
I also tried these:
1)
rename my own custom marker to patt.hiro, replacing the original patt.hiro…that works.
2)
rename “patt.hiro” to “patt.hiro_” and change the patts array entry accordingly…that does _not_ work.
This makes me think that the list of filenames is also stored in another place or so and should be changed there too, but where…?
Hi there, is there any size/resolution limitation?
I’ve been trying to run the multimarker example in 960*544 (allowed by my lifeCam) using P3D, but the resulting image is stretched and the tracked objects appear traslated or something. Any idea on this? Should I change something?
Thanks in advance!
No limitations that spring to mind at this moment, but there’s a lot of software in that stack that could be having trouble with a strange non 4×3 aspect ratio.
The fact that the resulting image is stretched means that the problem already starts with your video capturing and has nothing to do with nyartoolkit. Start debugging there, I would think.
Thanks for replying, i’ll be testing that. I got another small question: I wanted to track 3D models (like the .DAEs on FLARtoolkit), but the libraries I’ve found works only with the OPENGL renderer… Have you got any experience tracking 3D models with P3D? (as you say, performance is great on P3D). Thanks, again.
Jon
I’ve found this lib http://code.google.com/p/saitoobjloader/
which actually works on P3D, works nicely but couln’t map a jpg texture correctly yet… not sure if it’s a limitation or maybe a map setting from 3ds max.
double-check the documentation, it could be a P3D limitation.
Hi Jon…
I’ve been folowing this tutorial : http://diariowm.wordpress.com/2010/12/20/animar-md2-realidad-aumentada/
that explains how to load m2d files to be tracked on the markers…
it actualy works preaty well, but the only thing is that .m2d format is obsolete and i can’t find a way to export 3d models to this format.
have you manage to load sucssesfully 3d objects on the markers yet?
Joao
@tim167:
You have to put the patt files (they can be named in any way, as long as you specify correctly in your code) in the SKETCH’s data dir, NOT in the library’s data dir. :)
doh! ;)
it works. many thanks.
Hello, you really saved my work. I’m from Argentina and i’m in an artistic project with 2 teammates, and i’ve been having trouble with almost everything i tried before to make multiAR work and to have access to parameters like distance to the camera, etc.
You are a genious! You saved my day.
In my project i have a camera looking from up to a table where there are 6 separated patterns. The problem i have is that i need to know when one of the six if over the rest (someone took it up), and it is nearer to the webcam. I have already build the 6 patterns and put them in your multi example and it detects the 6 perfectly. Do you have any clue how can i do to check when one or more are closer to the camera?
Thanks you very much! I really appreciate your work. Sorry if my english is not good enough.
Returned to your sketch testing Processing v1.5 and that motivated me to try the new 20110205 version for P3D rendering. However, I get:
Console error:
The method beginTransform(PGraphicsOpenGL) in the type NyARTMultiMarker is not applicable for the arguments ()
Editor hilites:
nya.markers[i].beginTransform();
The only way I can get it to run is switching back to OpenGL, including the line “PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;” and the “pgl” argument in “beginTransform()”.
I get 53 to 56 FPS which is a nice addition to the display window in context of a testbed app (thankies).
Any ideas???
OOPS after a few tests in PS 1.2.1 and 1.5 I get 11-13 FPS, not sure where the diff came fromm it looks slower too. It is way too late (in AUS) to restart the laptop now, I may report another time if I find anything.
It really looks like you used the old version by accident. The new version of beginTransform() should take no parameters. Solved this yet?
Yup, thanks to your hint I did. The new sketch/library folder structure confused me. Since the libraries folder isn’t visible in sketches any longer (that is where I had the original 2010 version of NyAR2 was) I thought the libraries subfolder of the new NyAR2 was active. Having moved the new libs to libraries and the .pde to examples everything works as expected. Indeed the P3D rendering engine is faster, reporting 59,7 – 60.6 FPS, however, the lag of video tracking seems to be about the same as before. As expected, the CPU load is higher using P3D which could be a drawback with more complex 3D models. Thanks again.
eep, I do rush to submit without checking twice – the CPU load is higher with OpenGL (approx 70% vs 40%) and the sketch drops to 12 FPS.
I’d love to try it in Mobile Processing, but the chances are so slim I might have some sleep first. HAHA
I’m trying to use NyARToolkit on HD footage (1280×720 or 1920×1080), however it doesn’t seem to work at higher resolutions. Can you confirm this? Or should I change some settings to make this work?
Something else. I’d like to use NyARToolkit with either the GSVideo + GLGraphics combo for HD videos or the latest SVN Processing’s OPENGL2. However both say the transform functions only works under P3D and OPENGL. Is there a way to make it run under either GLGraphics or OPENGL2?
Thanks!
1. I can’t confirm this behaviour. Will test when I have time. If you dig up more info, please let me know. What would help a lot is succesively deactivating parts sof the demo so that you can determine where the problem lies: GSVideo, NyARToolkit, or a subsystem of one of these two.
2. I haven’t had time to check the new renderers in processing. Once I’ve been able to check, I’ll be able to say how easy / complicated this will be. If you have some time, you could always check already. For example adding P3D support was trivial thanks to the APIs being essentially the same.
Thanks for the reply.
1. It’s definitely not GSVideo, because the problem arises even when using an image. I’ve tested it at different sizes and with different renderers with an image matching those dimensions. The results:
(640,480,P3D), (640,480,OPENGL) and (720,360,OPENGL) work correctly.
(720,360,P3D) works but has incorrect placement (due to the aspect ratio?)
(960,720,P3D) and (960,720,OPENGL) or above don’t work at all (due to high resolution?)
The really weird thing is that higher resolutions don’t return any error, they just don’t work. As in: nothing AR-related shows up on screen.
2. Found the code, but having some trouble to recompile or temporarily override, so I’ll just wait it out ;-)
This is developed using Java right? Is there a method for this to be done using .NET?
Hi, can i run the sample project in my Windows??
Yes, everything runs on Windows too. Last year we tested the Linux version on about 125 CS first years, this year we tested the Windows version on 130 CS first years.
After putting the NyAR2 in the libraries folder when I try to run NyARMultiTest from the examples in processing it doesnt give me any error but just shows a blank grey screen, with the message shown below.
GSVideo version: 20110709
NyAR4psg/0.3.0;NyARToolkit for java/2.5.0+;ARToolKit/2.72.
First get the straight GSVideo examples to work, then try again!
Hi.. I hope you’re still active in this topic ><. Can you add a tutorial how to use 3D model on nyartoolkit? Because I can only use the example ( load cubic ).. And I want to load 3D model like 3ds , obj file with texture.. I think you really understand this topic, so I ask you this ^^.. Please reply ^^..
Sorry, no time to do that right now.
You might find this FAQ I wrote for an AR course helpful: http://graphics.tudelft.nl/Courses/TI1100A/FAQ (it’s in Dutch ,google translate is your friend! :)
Hi!! I’m a student and I’m working with NyArtoolkit. I have two doubts with my project. I have to use diferents markers and models thats came in Nyartoolkit by default. Do you know any web to download some markers or any way to create my own markers ??? The same think with the 3D models. Thank for your time!!!!!!
You can create your own patt files from your own image markers using this site: http://flash.tarotaro.org/blog/2009/07/12/mgo2/ — because it downsamples to 16×16 for the patt files, you should choose images which are easy to differentiate even at that resolution.
Use Google Translate on this Dutch FAQ that I wrote for a class: http://graphics.tudelft.nl/Courses/TI1100A/FAQ to get more pointers on the use of other 3D models.
Hello,
I too am having the grey-screen problem, when I try to run the MultiTest example, with the following appearing below:
”
GSVideo version: 1.0.0
NyAR4psg/0.3.0;NyARToolkit for java/2.5.0+;ARToolKit/2.72.”
All of the other examples seem to work fine, and I have already double checked that all of the GS Video examples work as well.
Any suggestions?
Thank you kindly.
I have no idea how to go about debugging that remotely, I can’t reproduce the problem on my side (and neither could 130 students this year :)
So I feel rather silly for not having noticed it in the first place, but in the MultiTest example, the camera is never started, which wasn’t something that I thought to look for after successfully running the previous examples. To anyone else with a similar unsolvable grey-screen issue, be sure to include “cam.start();”
Thank you for this wonderful contribution!
Pingback: AR « automatedChaos
Hello Charl,
Firstofall thanks for your marker, it enabled me to run a small prototype of 3D AR with two markers in no time. (much easier that with the NyAR4psg:))
A small question though I try to have more than two markers, so I just added the .patt files to the sketch and added their names to the String[]patts. Then I added ‘else if’ to the draw function like that :
if (i==0){ /*draw code*/;}
else if (i==1){ /*other draw code*/;}
else if (i==2){ /*yet another draw code*/;} etc.
But when I run this i get this error message :
“ArrayIndexOutOfBoundsException : 2″
highlightening the following line in my code :
nya=new NyARMultiBoard(this,width,height,camPara,patts,widths);
(btw I’ve changed the cam_para.dat to point to a string camPara in which is stored a path to the file)
It seems it has to do with the number of markers available for detection ? how can I track more than 2 markers ? That could be great if you could point me out to ressources or give me a nudge in the right direction
Cheers and congrats for the great examples.
Well it wasn’t long after the post, but I stayed on this a little before being able to realise what was wrong… silly me …
for thos we might be wondering
I just needed to add the width values corresponding to the new patterns :
double[] widths = {80,80,80,80};
the error came from the fact that both lists didn’t match in length, if you declare four patterns, you should declare four widths :)