Creating Augmented Reality Cards with Unity and ARToolkit

This first tutorial on augmented reality (AR) will cover the basics of making a personalised (digital) birthday card.

Download the source code

You can download all of the files associated with this tutorial from here.

Augmented reality

AR is the process of putting digital content (images, videos, 3D models, …) on top of your surroundings seen through a device (such as a computer or mobile phone for example). What it basically does is use a camera as input and augment the stream from it with content not available in real life. As you can see in this image:

AR example

For this tutorial I’m assuming you have already installed Unity3D and downloaded the ARToolkit for Unity package.
I am using Unity 5.3.8 and the ARToolkit package downloaded from the website. More recent versions of Unity (5.4.x onward) gave errors when using the ARToolkit package. Doing the steps from this post on the ARToolkit github might resolve the issue but I did not try it so I cannot assure you it will work.
I will make an Android app at the end so make sure you have installed the ‘Android support build’ if you want to be able to digitally send the (birthday) card.

Did you come across any errors in this tutorial? Please let us know by completing this form and we’ll look into it!

FREE COURSES
Python Blog Image

FINAL DAYS: Unlock coding courses in Unity, Godot, Unreal, Python and more.

Making AR birthday cards

When starting Unity you will have to create a new unity project. You can delete the two objects that every new project starts with: ‘Main Camera’ and ‘Directional Light’ (right click and delete) since we will not be needing these objects. To use the ARToolkit package downloaded from the webpage you will have to import the package to your project: choose Assets -> ‘Import Package’ -> ‘Custom Package…’.

import package

Next go the folder where you downloaded the ARToolkit to and double click on it. In the next window click on import to import the whole package.

import package2

If everything is imported correctly there should be no errors in the console window.
Once everything is imported the next step is to populate the empty scene. We will need three empty objects and one camera object. First create a new empty object and rename it to ‘ARToolkit’. Make a new empty object as a child of the ARToolkit object and rename it to ‘Scene root’. As a child of the ‘Scene root’ object make a new empty object and rename it to ‘Marker’. The last object needed is a camera object, create this object as child of the ‘Scene root’ object.
If done correctly the Hierarchy window in Unity should look like this:

Hierarchy

After populating the scene in Unity we will start using the imported ARToolkit package. We will use the scripts in the package (under ARToolKit5-Unity->Scripts) and add them to the empty objects.
To the (empty) object ‘ARToolkit’ we will add the scripts ‘ARcontroller’ and ‘ARMarker’. To the (empty) object ‘Scene root’ we will add the script ‘AROrigin’. The (empty) object ‘Marker’ needs the script ‘ARTrackedObject’ added and the ‘Camera’ object needs the script ‘ARCamera’ added.

add scripts

In the inspector window we will have to make some adjustments to the added scripts to get them working.
Select the ‘ARToolkit’ object, and if everything was added correctly the inspector window will have the components ‘AR Controller (script)’ and ‘AR Marker (script)’. In the ‘AR Marker (script)’ component write in the input box ‘Marker1’ (this is just a name so you can choose whatever you like). Make sure ‘patt.hiro’ is chosen at the pattern file and the width box says ‘0.08’.
Now select the ‘Marker’ object, if the script was added correctly the ‘AR Tracked Object (script)’ component should be visible in the inspector window. In the ‘Marker tag’ box write the same name as you did in the ‘ARToolkit’ object, in my case ‘Marker1’ (if you chose a different name use that one). If the name is the same in both under the ‘Marker tag’ box ‘Got marker’ should say ‘yes’ (if not make sure both names are identical).
The third adjustment we will have to make is in the ‘Camera’ object so select it. If you added the script correctly the ‘inspector window’ should have a component ‘AR Camera (script)’. In this component we will not have to change anything. In the (standard) ‘Camera’ component we will have to make a change though. In the ‘Culling mask’ dropdown box we must make sure we only have selected ‘AR foreground’ (click the dropdown box, click on nothing, then click the dropdown box again and chose AR foreground).
The last adjustments we will have to make are to the layers of the objects. The layer of the ‘ARToolkit’ object must be ‘AR background’ (when you choose this layer make sure you only change this layer, so click the ‘No, this object only’ button). Next click on the ‘Scene root’ object and change the layer to ‘AR foreground’. Be sure to change all its children’s layer to ‘AR foreground’ as well (click the button ‘yes, change children’).
If everything is done correctly this is how the ‘inspector window’ of the different objects should look like:

Camera inspector Marker inspector ARToolkit inspector

At this point everything is ready to make an AR app (be it for the pc or mobile phone).
Building the app at this point will look like it is not working. But what it does is put an empty object (the ‘Marker’ object) on top of the recognized pattern (patt.hiro: a black square with the word ‘hiro’ in it, you can find it here. If you want the other pattern ‘patt.kanji’ go up one step to the patterns folder of the github repository and choose ‘Kanji pattern.pdf’). The only thing we need is, to add an object as a child to the ‘Marker’ object. This is the object that will appear on the screen. As making (beautiful intricate) objects in Unity is not a part of this tutorial we will add a simple cube or sphere. Click the right mouse button and add a 3D object (sphere) as a child of ‘Marker’ object to the scene. Change the scale in X, Y and Z to 0.08 (the same size as in the width box of the ‘AR Marker (script)’ added to ‘ARToolkit’ object). You can always change the size to whatever you prefer later. Now zoom in to the object. If everything went correctly you will see a square (with the word ‘Marker1’, or whatever you named the marker, under it) and a sphere (or whatever object/model you added).

object with marker

To make sure the object is projected above the marker move the marker in the negative Z direction to about -0.1 (or just type -0.1 in the Z box next to ‘Position’ in the ‘inspector window’). It is always possible to change this to the amount you want, but after some tests I found -0.1 to be a nice amount.
We are finished and ready to test it! If we push the play button the object will appear on top of the marker when we put the pattern in front of the camera.

test play

The last thing we need to do is make a nice mobile app. As I said at the beginning, make sure you have installed ‘Android build support’ in Unity, if not do it now! And of course don’t forget to save the scene!

Make an Android app

To make an Android app of the project we just created you could just try to build it and choose Android as the Platform. When doing this myself I did not get the camera stream as soon as I started the app. After some tweaking I found that the settings in the Plugins of the ARToolkit package needed changes from its initial settings. It may work for you with these initial settings, but I had to change them. I tested it on a OnePlus 3 with OxygenOS Open Beta 16 (version OP3_O2_Open_16).
To change the settings you will have to choose the Plugins folder in the Assets. You will see 6 folders (Android, ARWrapper.bundle, IOS, Metro, x86 and x86_64), each with settings for a different operating system.
In the Android folder click on the ‘libARWrapper’ file. In the inspector window make sure under ‘select platform for plugin’ Android is selected and under ‘Platform settings’ ARMv7 is selected.
In the IOS folder click on the ‘libARWrapper’ file. In the inspector window make sure under ‘select platform for plugin’ nothing is selected (so no checks in the check boxes).
The Metro folder has two subfolders, Win81 and WindowsPhone81. Which have multiple subfolders themselves. In subfolder arm of Win81 click on the ARWrapper file and make sure nothing is selected. Do the same in the x86 and x86_64 subfolders. The WindowsPhone81 folder has two subfolders, arm and x86. Make sure that nothing is selected in the ARWrapper file in both folders.
In the x86 folder click on the ‘libARWrapper’ file. In the inspector window make sure under ‘select platform for plugin’ Standalone is selected and under ‘Platform settings’ x86 is selected. I had to check two checkboxes, one for Linux and one for Mac OSX. It maybe that on windows machines other operating systems appear, I would make sure that x86 is checked for every appearing operating system.
In the libARWrapper file in the x86_64 folder do the same as in the folder x86 but now make sure all x86_64 boxes are checked.
To summarize:
Plugins->Android->libARWrapper à ‘select platform for plugin’=Android & ‘Platform seetings’=ARMv7
Plugins->iOS->libARWrapper à ‘select platform for plugin’=nothing
Plugins->Metro->Win81->arm->ARWrapper à ‘select platform for plugin’=nothing
Plugins->Metro->Win81->x86->ARWrapper à ‘select platform for plugin’=nothing
Plugins->Metro->Win81->x86_64->ARWrapper à ‘select platform for plugin’=nothing
Plugins->Metro->WindowsPhone81->arm->ARWrapper à ‘select platform for plugin’=nothing
Plugins->Metro->WindowsPhone81->x86->ARWrapper à ‘select platform for plugin’=nothing
Plugins->x86->libARWrapper à ‘select platform for plugin’=standalone & ‘Platform seetings’=x86 (all)
Plugins->x8-64->libARWrapper à ‘select platform for plugin’=standalone & ‘Platform seetings’=x86_64 (all)
Now it is time to build the apk file. Go the build settings (File à Build settings…). Under Scenes in Build click the ‘Add Open Scenes’ button. Now make sure only the checkbox for your created scene is selected. Under Platform select Android and click on the Player Settings button. It will open some settings you can change in the inspector window. The only thing I changed here was the name of the app in the Product Name textbox. This is the name that will appear on your mobile phone, so choose a nice one. You can add a default icon or change the company name to make the app more personal. Then click the build button and save it wherever you prefer. After the apk file is created copy it to your phone and install it. Make sure install form unknown sources is enabled on your phone (settings à security à unknown sources) to install it. After installation the first time you run the app, it will ask for permission to use the camera. Of course this is needed for augmented reality so allow it.

Making a (birthday) card

The only thing left is to make a (birthday) card. You can make any card you imagine as long it has the marker somewhere hidden. Do not make the marker too small, otherwise it will not recognize it. Also make sure the black outer border of the marker is visible. As you can see in the video it works on a black and white (printed) ‘card’ and a full color card (although this one was on the screen). Also visible in the video is that it is not very stable if the marker is small. The closer you put the camera to the marker, the more stable the object appears.

Now it is time to make your own object and add it to the project instead of the ‘simple’ sphere in this tutorial. Make a nice (digital) card and sent it to whomever you want (and don’t forget to send the apk file as well ;) ).

Files for the AR project (including test.apk)

Comments or questions?

Do you have any questions about this article? Do you have any suggestions for a new AR tutorial? Let me know in the comments section below.

Happy augmenting reality!