1984's Spatial Computing Work
Minimap is an ongoing internal research project exploring spatial mapping of
a location. This initial demo understands the structure of a room using RoomPlan
and paints object textures using ARKit & RealityKit.
Minimap uses:
- RoomPlan
- Metal Shaders
- RealityKit
- ARKit
- SwiftUI
Project Type: 1984 internal research
Watch the Demo with Sound
Reality Synthesizer is an open-source, augmented reality app that visualizes
MIDI notes played on a paired synthesizer. The MIDI notes are interpreted by
Metal shaders to product augmented reality visualizations. Reality Synthesizer
utilizes LiDAR capabilities introduced in iOS 15.4.
Reality Synthesizer uses:
- LiDAR
- Metal Shaders
- AVFoundation
- AudioKit
- Linnstrument Helper
Project Type: Open-source
Get the Code Watch the Demo with Sound
Rock Paper Checkmate is an augmented reality auto-chess game played
against a computer by playing Rock Paper Scissors. Rock Paper Checkmate includes
people occlusion with depth, a custom machine learning model, and recognizes hand
gestures shown to the camera so anyone can play augmented reality auto-chess.
Rock Paper Checkmate uses:
- ARKit 5
- RealityKit 2 and Reality Composer
- Create ML for Hand Pose Classification
- SwiftUI
- Vision Framework for Human Hand Pose Estimations
Project Type: 1984 product
Download
Mask Me is an ecommerce app which uses face landmarks,
face anchors, and saliency analysis to enable anyone to visualize,
create, and purchase custom face masks in real-time using augmented reality.
Mask Me uses:
- ARKit 4
- Vision Framework to Detect Face Landmarks
- Vision Framework for Objectness Saliency Analysis
- ARKit Face Anchors
- WooCommerce and Stripe for Ecommerce
Project Type: Client work for Mask Market
CyberWave uses the new LiDAR sensor to visualize your music
in augmented reality. Play your music to change your
surroundings in real-time.
CyberWave uses:
- ARKit 3.5
- LiDAR for ARMeshAnchors
- Auto-Renewable Subscriptions for In-App Purchases
- Person Segmentation with Depth
- GLSL Shaders
Project Type: 1984 product
Download Watch the Demo with Sound
Primer helps you visualize your home with new
wallpaper, tiling, and paint. Primer utilizes Core Animation to display
intuitive, animated, augmented reality tooltips to visually onboard users.
Primer uses:
- ARKit 3
- Core Animation
- Person Segmentation with Depth
Project Type: Client work for Primer
Acquired by Shopify
WebXR Viewer is an open-source, augmented reality, WebXR browser for iOS.
WebXR Viewer has a fine-grained AR permissioning system that puts privacy & ethical design first
and is built on Metal to render WebXR websites at 60 fps.
WebXR Viewer uses:
- ARKit 2
- WebXR
- Metal
- SceneKit
Project Type: Open-source client work for Mozilla
Get the Latest Code DownloadiScape
1984 developed innovative AR interactions and controls for iScape's collaborative landscaping app.
iScape utilizes shared world maps, persistent worlds, 3D models, and the novel ability to create
custom geometry for visualizing AR textures in the user's world.
My work on iScape was featured by HBO, Axios, and Apple at both WWDC 2019 & in the App Store.
iScape uses:
- ARKit 2
- SceneKit
- AR World Maps
- Collaborative, Multi-User Sessions
Project Type: Client work for iScape
Watch a Demo Video Download
NudgeTV released an AR basketball minigame to accompany their machine vision
technology. The basketball game uses Multipeer Connectivity and Collaborative
Sessions to seamlessly connect up to 4 players. SceneKit's real-time physics
are employed to create realistic, yet otherwise-impossible, gameplay.
NudgeTV's basketball minigame uses:
- ARKit 3
- Collaborative Sessions
- Multipeer Connectivity
- Person Segmentation with Depth
Project Type: Client work for NudgeTV
AR Guitar is an open-source, augmented reality exploration
of ARKit 3's body tracking functionality. AR Guitar watches for hand placement
and movement in relation to hips. The left hand controls which sound is selected
and the right hand controls when sounds are played.
AR Guitar uses:
- ARKit 3
- RealityKit
- Body Tracking
- Reality Composer
Project Type: Open-source
Get the Code Watch with SoundJanksy utilizes the iOS Vision APIs to detect rectangles in real-time.
After a user taps a rectangle found by the computer vision algorithms, an ARKit reference image
is created on the fly and animated via SceneKit to receive the
Banksy treatment.
Janksy uses:
- ARKit 2
- SceneKit
- Vision
- AR Image Anchors
Project Type: 1984 product
Real World Ad Blocker uses tracked AR Image Anchors to replace irrelevant
billboards with user-selected video content. Real World Ad Blocker demonstrates augmented reality's
ability to fundamentally change the world around us.
Real World Ad Blocker uses:
- ARKit 2
- SceneKit
- AR Image Anchors
Project Type: Proof-of-concept internal research
Check Out the TweetFloat addresses a crucial hurdle for any AR experience: Users don't have
access to custom 3D models. Instead, Float taps into content everyone has: Photos and videos.
Float enables remixing media in AR and augmented reality captioning of the real world.
Float uses:
- ARKit
- SceneKit
- ReplayKit
Project Type: 1984 product
DownloadThis GitHub repo serves as a starting point for people interested in
image tracking ARKit development. The simple code and examples demonstrate how easy it is to
use ARKit to create Harry-Potter-style moving pictures, living movie posters, video postcards,
paper-thin displays, and lots more augmented reality fun.
Image Tracking Demo uses:
- ARKit 2
- SceneKit
- GitHub
Project Type: Open-source
Get the Code