Low floors and high ceilings are important design principles for creativity support tools. This means it should be easy to start, but there should also be room for advanced development. Examples of tools that follow this principle include Seymour Papert’s Logo, its contemporary equivalent, Scratch, and the artist and designer-focused Processing.
As a young person studying programming, I struggled with graphic programming when I attempted to create the “Hello Triangle” program with OpenGL. I quickly concluded that this was not for me until I discovered Processing. The simplified syntax and intuitive primitives made it easy to “just get things done.” I found it natural and intuitive, and I couldn’t understand why this wasn’t the standard way to teach visual coding for beginners.
Although starting to work in 2D is pretty easy, it is a different story for 3D. Before we can see the final result on the screen, various elements must be considered, such as lights, cameras, geometry, materials, transformations, etc. This can be challenging, particularly in a pure code environment, as choosing parameters for the camera or the appropriate light intensity can be daunting and time-consuming.
The frustration of creating digital content arises from the fact that spatial dimensions, lighting, and a sense of scale are intrinsic to the physical world we inhabit every day. Our tools should facilitate creation by leveraging our understanding of the physical world. Instead of relying on indirect methods like the keyboard and mouse, we should be able to manipulate objects directly with our hands. Similarly, instead of setting up and positioning virtual cameras, we should be able to look at them directly through our eyes. Rather than simulating complex lights, we should first leverage the lighting of our environment as a starting point. All this should be done in everyday units like meters instead of arbitrary values. Mixed Reality can make all this possible today.
Various tools have emerged based on these ideas in 3D asset creation. These tools range from drawing tools like Tilt Brush or Quill to full-fledged computer-aided design tools like Gravity Sketch. There are also immersive authoring tools like Editor XR for the Unity game engine. These tools simplify existing workflows but have limitations since they are designed for a specific purpose and output, much like Illustrator or Photoshop for 2D graphics.
To overcome these limitations, we need to create our own software. The current workflow for developing Mixed Reality applications in game engines like Unity involves coding on a computer, testing in simulated environments, deploying, and finally testing on headsets. It is inefficient and daunting, especially for novices.
Furthermore, sharing work often involves navigating proprietary app stores, dealing with large compilation tools, or encountering long download times for web-published projects, which also lack editability. This bias towards consumer applications over experiments often leads to discovery only through online video documentation instead of the actual experience.
In the early 2000s, Processing carved out a space that Flash and Director couldn’t fill, offering a more straightforward path into graphics programming. It advocated the idea of sketching, borrowing from the tradition of painters who draft numerous sketches before their final piece.
Today, there’s a need for a tool in spatial computing that embodies this sketch-first philosophy. I believe this is what is needed to make it happen:
I am building upon the p5.xr by Stalgia Grigg using WebXR to make this tool a reality.