Make an AR Drawing App - Part 3

Make an AR Drawing App – Part 3

by

Introduction

Check out the other parts of this series:

In this tutorial, we are going to wrap up our project from Part 1 and Part 2 by giving it two incredible new features. The first is the ability to draw on a detected surface (as I’m sure you already know, if you have seen the past two tutorials). This employs raycasting and updating pose position for our stroke and pen point. The second feature is the ability to adjust the color of our stroke, because it wouldn’t be art if you were not able to do that. In this part, we will look at how to edit materials from within a script and where the material is stored in the class hierarchy.

There are lots of great things we are going to be doing, so let us get started!

Source Code files

You can download the tutorial source code files here.

Did you come across any errors in this tutorial? Please let us know by completing this form and we’ll look into it!

FREE COURSES
Python Blog Image

FINAL DAYS: Unlock coding courses in Unity, Godot, Unreal, Python and more.

Raycasting in AR Foundation

What is Raycasting?

When we say “raycast” we mean the act of sending a beam into 3D space from a specified origin (this would be the camera in our case). This beam can collect data about what it hits. Some examples would be components attached to the hit object, transform, name, tag, and pretty much anything else the object possesses. The syntax for raycasting usually looks like this:

Physics.Raycast (rayOrigin, direction, out whereHitInformationIsStored, howFarToShootTheRay);

However, this only works if we want to detect something in virtual 3D space, we are working in AR so things are a little bit different. When we raycast in augmented reality, we need to check if there is a flat surface or tracking point detected first. This requires a different module and different syntax.

Raycasting in Augmented Reality

The very first thing we need if we want to raycast in AR Foundation is an “AR Raycast Manager” component.

AR Raycast Manager within Unity Inspector

This component will exist on our AR Session Origin and will be responsible for raycasting and storing hit information. Now, on your SurfacePenPoint game object, open up the “DrawOnSurface” script. In this script, the first thing that we will need to do is to gain access to the ARFoundation and ARSubsystem classes:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;

Then, we need to gain access to the ARRaycastManager and create a public variable that will determine what we want to ARFoundation to detect:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;

public class DrawOnSurface : MonoBehaviour
{
    public TrackableType surfaceToDetect;

    private ARRaycastManager arOrigin;

    // Start is called before the first frame update
    void Start()
    {
        arOrigin = FindObjectOfType<ARRaycastManager>();
    }

    // Update is called once per frame
    void Update()
    {
    }
}

And then, in the update function, we send out a ray from the center of the camera into AR detected space. Then we use the raycast method on the ARRaycastManager to see if we’ve hit the surface we wanted to detect. And finally, we update the position of our pen point to be the where the ray collides with our detected surface.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;

public class DrawOnSurface : MonoBehaviour
{
    public TrackableType surfaceToDetect;

    private ARRaycastManager arOrigin;

    // Start is called before the first frame update
    void Start()
    {
        arOrigin = FindObjectOfType<ARRaycastManager>();
    }

    // Update is called once per frame
    void Update()
    {
        Vector3 centerPoint = Camera.current.ViewportToScreenPoint(new Vector3(0.5f, 0.5f, 0));

        List<ARRaycastHit> validHits = new List<ARRaycastHit>();
        arOrigin.Raycast(centerPoint, validHits, surfaceToDetect);

        gameObject.transform.position = validHits[0].pose.position;
        gameObject.transform.rotation = validHits[0].pose.rotation;
    }
}

Now, let’s test this out to see if it works. Set the “Surface To Detect” variable to “Planes”

Draw on Surface Script component in Unity with Planes for Surface To Detect

and then hit “Build and Run.”

Unity drawing application with surfaces detected

When we switch it to “Draw On Surface”, we can see that the position of our pen point is tracking with our camera and only shows up when ARFoundation detects a plane! Congratulations! You now know how to raycast in ARFoundation!

Combining the pen points

If you’ll notice with our project, however, when we try and draw on a surface it doesn’t do anything. Right now, only the position of the pen point is tracking to the surface. This is because you’ll notice that in the Stroke script, we are looking for a game object named “PenPoint”, however, no such game object exists. This only works if there is only one pen point. We need to be able to use whatever pen point is active at the moment. We do this by going to the draw script and creating a public transform variable called “penPoint.” We aren’t going to edit this in the inspector so we can tell Unity to hide this in the inspector.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Draw : MonoBehaviour
{

    public bool mouseLookTesting;
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;

    [HideInInspector]
    public Transform penPoint;

Now we assign the transform of the active pen point to this variable and tell the Draw script to instantiate a stroke at this variable’s transform.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Draw : MonoBehaviour
{

    public bool mouseLookTesting;
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;

    [HideInInspector]
    public Transform penPoint;

    public static bool drawing = false;

    private float pitch = 0;
    private float yaw = 0;

    // Start is called before the first frame update 
    void Start()
    {
    }

    void Update()
    {
        if (mouseLookTesting)
        {
            yaw += 2 * Input.GetAxis("Mouse X");
            pitch -= 2 * Input.GetAxis("Mouse Y");

            transform.eulerAngles = new Vector3(pitch, yaw, 0.0f);
        }
        if (PenManager.drawingOnSurface)
        {
            penPoint = surfacePenPoint.transform; // <- New line 

            spacePenPoint.GetComponent<MeshRenderer>().enabled = false;
            surfacePenPoint.GetComponent<MeshRenderer>().enabled = true;
        }
        else
        {
            penPoint = spacePenPoint.transform; // <- New line

            surfacePenPoint.GetComponent<MeshRenderer>().enabled = false;
            spacePenPoint.GetComponent<MeshRenderer>().enabled = true;

        }

    }

    public void StartStroke()
    {
        GameObject currentStroke;
        drawing = true;
        currentStroke = Instantiate(stroke, penPoint.transform.position, penPoint.transform.rotation) as GameObject; // <- New line
    }

Now, we can tell the Stroke script to use this variable instead of searching for a specific game object.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Stroke : MonoBehaviour
{
    private Transform penPoint;

    // Start is called before the first frame update
    void Start()
    {
        penPoint = GameObject.FindObjectOfType<Draw>().penPoint;
    }

    // Update is called once per frame
    void Update()
    {
        penPoint = GameObject.FindObjectOfType<Draw>().penPoint;

        if (Draw.drawing)
        {
            this.transform.position = penPoint.transform.position;
            this.transform.rotation = penPoint.transform.rotation;
        }
        else
        {
            this.enabled = false;
        }

    }
}

The pen point that the player uses will randomly change, therefore, we have to constantly check to see if the pen point changes. This is why we assign the penPoint variable in the Start and Update functions. We are now ready to test it on our device. Before I do that, however, I’m just going to change the size of the buttons.

Unity UI buttons in Scene and Game view

They were pretty small on my screen so I’ll just make them a little bigger. Now, when we build to our device, you’ll see all the problems are fixed!

Unity drawing application with yes drawn out

Colors!

The final thing we are going to do to this is project is to be able to change the color of our paint strokes. The obvious place we need to start is in the “Stroke” script. We need to create a public Color variable called “strokeColor.”

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Stroke : MonoBehaviour
{
    private GameObject penPoint;

    public Color strokeColor;

We will change this value externally so all we need to do is assign this color to the material attached to this game object.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Stroke : MonoBehaviour
{
    private Transform penPoint;

    public Color strokeColor;
    
    // Start is called before the first frame update
    void Start()
    {
        penPoint = GameObject.FindObjectOfType<Draw>().penPoint;
    }

    // Update is called once per frame
    void Update()
    {
        penPoint = GameObject.FindObjectOfType<Draw>().penPoint;
        GetComponent<Renderer>().material.color = strokeColor; // <- this is the new line

        if (Draw.drawing)
        {
            this.transform.position = penPoint.transform.position;
            this.transform.rotation = penPoint.transform.rotation;
        }
        else {
            this.enabled = false;
        }

    }
}

Notice that the material is stored on the Renderer component. It is important to remember that a material is not a separate component but is rather housed on the Renderer. And, like the pen point value, this is changes randomly as the player specifies a different color so we have to keep it in the update function.

Now we can change this color externally through some UI. Go to the canvas and create three new slider elements. Name them, “RedSlider,” “Green Slider,” and “Blue Slider” with colors that match the name.

Unity Slider objects with Anchors circled in Inspector

I anchored them to the bottom of the screen. Now, we need to make these sliders do something. Let’s start by going to the Draw script, grabbing the UI module, and creating a public array, of type Slider, called “colorSliders.”

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Draw : MonoBehaviour
{

    public bool mouseLookTesting;
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;
    public Slider[] colorSliders;

We use an array because it doesn’t clutter the inspector window. Set the length to three and drag in each of our color sliders with red first, then green, then blue (in the RGB order).

Unity Draw Script Component with Slider objects added

Now, we create a private color variable called “colorFromUI” and assign it in the update function.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Draw : MonoBehaviour
{

    public bool mouseLookTesting;
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;
    public Slider[] colorSliders;

    [HideInInspector]
    public Transform penPoint;

    public static bool drawing = false;

    private float pitch = 0;
    private float yaw = 0;
    private Color colorFromUI;

    // Start is called before the first frame update 
    void Start()
    {
    }

    void Update()
    {
        colorFromUI = new Color(colorSliders[0].value * 5, colorSliders[1].value * 5, colorSliders[2].value * 5); // <- this is the new line

Notice, we are using each slider value as each RGB value.

Now, we simply set the “strokeColor” variable to this “colorFromUI” value in the “StartStroke” method.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Draw : MonoBehaviour
{

    public bool mouseLookTesting;
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;
    public Slider[] colorSliders;

    [HideInInspector]
    public Transform penPoint;

    public static bool drawing = false;

    private float pitch = 0;
    private float yaw = 0;
    private Color colorFromUI;

    // Start is called before the first frame update 
    void Start()
    {
    }

    void Update()
    {
        colorFromUI = new Color(colorSliders[0].value * 2, colorSliders[1].value * 2, colorSliders[2].value * 2);

        if (mouseLookTesting)
        {
            yaw += 2 * Input.GetAxis("Mouse X");
            pitch -= 2 * Input.GetAxis("Mouse Y");

            transform.eulerAngles = new Vector3(pitch, yaw, 0.0f);
        }
        if (PenManager.drawingOnSurface)
        {
            penPoint = surfacePenPoint.transform; 

            spacePenPoint.GetComponent<MeshRenderer>().enabled = false;
            surfacePenPoint.GetComponent<MeshRenderer>().enabled = true;
        }
        else
        {
            penPoint = spacePenPoint.transform;

            surfacePenPoint.GetComponent<MeshRenderer>().enabled = false;
            spacePenPoint.GetComponent<MeshRenderer>().enabled = true;

        }

    }

    public void StartStroke()
    {
        GameObject currentStroke;
        drawing = true;
        currentStroke = Instantiate(stroke, penPoint.transform.position, penPoint.transform.rotation) as GameObject;
        currentStroke.GetComponent<Stroke>().strokeColor = colorFromUI; // <- this is the new line
    }

    public void EndStroke()
    {
        drawing = false;
    }
}

And now, if we enable mouse look testing, we can use the sliders to change the color of our stroke! Yes!

Unity drawing application with color selectors

Polishing up our project

Everything is working really well right now, there are just a few things small things that would really add a lot to our project. The first one is the material on our pen point. This should change color as we move the sliders. To do this, we can assign the “Stroke” material to our pen point.

PenColor objects in Unity Scene view

Next, we simply assign the “colorFromUI” value to the material on the current active pen point.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Draw : MonoBehaviour
{
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;
    public bool mouseLookTesting;
    [HideInInspector]
    public Transform penPoint;

    public Slider[] colorSliders;

    public static bool drawing = false;

    private float pitch = 0;
    private float yaw = 0;
    private Color colorFromUI;

    // Start is called before the first frame update
    void Start()
    {
       
    }

    // Update is called once per frame
    void Update()
    {
        if (mouseLookTesting)
        {
            yaw += 2 * Input.GetAxis("Mouse X");
            pitch -= 2 * Input.GetAxis("Mouse Y");

            transform.eulerAngles = new Vector3(pitch, yaw, 0.0f);
        }
        if (PenManager.drawingOnSurface)
        {
            penPoint = surfacePenPoint.transform;

            spacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = false;
            surfacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = true;
            surfacePenPoint.GetComponentInChildren<Renderer>().material.color = colorFromUI; // <- this is the new line
        }
        else
        {
            penPoint = spacePenPoint.transform;

            surfacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = false;
            spacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = true;
            spacePenPoint.GetComponentInChildren<Renderer>().material.color = colorFromUI; // <- this is the new line
        }

Now, looking at the logic statements that we just put this new line of code into, we can see a lot of repeating code and it is cluttering our script. Let’s create two new methods to replace this code. The first one will be called “EnableSurfacePenPoint” and we will simply copy the “if” portion of our logic statement into this method. The second one will be called “Enable3DSpacePenPoint” and it will contain the “else” portion. Now, we can just call these methods instead of having this ugly code occupy our logic statement.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class Draw : MonoBehaviour
{
    public GameObject stroke;
    public GameObject spacePenPoint;
    public GameObject surfacePenPoint;
    public bool mouseLookTesting;
    [HideInInspector]
    public Transform penPoint;

    public Slider[] colorSliders;

    public static bool drawing = false;

    private float pitch = 0;
    private float yaw = 0;
    private Color colorFromUI;

    // Start is called before the first frame update
    void Start()
    {
       
    }

    // Update is called once per frame
    void Update()
    {
        if (mouseLookTesting)
        {
            yaw += 2 * Input.GetAxis("Mouse X");
            pitch -= 2 * Input.GetAxis("Mouse Y");

            transform.eulerAngles = new Vector3(pitch, yaw, 0.0f);
        }
        if (PenManager.drawingOnSurface)
        {
            EnableSurfacePenPoint();
        }
        else
        {
            Enable3DSpacePenPoint();
        }

        colorFromUI = new Color(colorSliders[0].value * 2, colorSliders[1].value * 2, colorSliders[2].value * 2);
    }

    void EnableSurfacePenPoint()
    {
        penPoint = surfacePenPoint.transform;

        spacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = false;
        surfacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = true;
        surfacePenPoint.GetComponentInChildren<Renderer>().material.color = colorFromUI;
    }

    void Enable3DSpacePenPoint()
    {
        penPoint = spacePenPoint.transform;

        surfacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = false;
        spacePenPoint.GetComponentInChildren<MeshRenderer>().enabled = true;
        spacePenPoint.GetComponentInChildren<Renderer>().material.color = colorFromUI;
    }

And now, our pen point will change to the appropriate color, and our script is now cleaned up!

The last tidying task that I suggest you do is to make null the “Plane Prefab” on the AR Plane Manager component and remove the Point Cloud Prefab on the AR Point Cloud Manager.

AR Point Cloud Manager Component in Unity Inspector

This just makes your AR app look much cleaner on your device.

Unity AR drawing application with smiley face drawn

Conclusion

And thus ends our exploration into AR Foundation. I hope that throughout this series your mind has been exploding with the millions of possibilities that AR Foundation affords. There are so many things we can do with this tool, and I hope your use of AR Foundation will not stop at this tutorial series. I am confident that you will use AR Foundation to

Keep making great games!

Continue Learning

File Download Link

Send me a download link for the files of .

You authorize us to send you information about our products. To learn more please refer to our Privacy Policy.