Apple's Innovations in AR/VR Technology

Head-mounted displays for virtual and mixed reality applications that have facial interfaces designed to provide proper positioning and comfort for the displays while accommodating facial movements. The interfaces engage regions above and below the user's eyes. They have different vertical and fore-aft stiffnesses in those regions to distribute loading and prevent excessive restriction of facial movements. The upper facial support has greater vertical and fore-aft stiffnesses than the lower support. This allows the display to move more vertically and fore-aft when the user smiles or looks down without excessive movement restriction.

Source

Populating XR environments with variations of objects to make them look more realistic and reduce the need for manually customizing numerous copies. The method involves generating variations of an object in a XR environment by assigning values for visual properties based on distribution criteria rather than copying and manually editing multiple instances. This provides a more realistic appearance when populating XR environments with objects like books or cars, reduces memory usage compared to storing multiple variations, and avoids repetitive user customization.

Source

Guided stretching session system that uses 3D environments, user profiles, and head pose tracking to provide personalized and interactive stretching exercises. The system presents visual cues and a direction indicator in a 3D environment based on user profiles and head pose. As the user moves, the indicator updates and feedback is provided when cues are completed. This adaptive feedback helps users maintain proper form and progress through the session.

Source

Head-mounted display device with compact optical modules for VR/AR headsets. The modules have a lens, display, and camera all connected to a housing that can be adjusted to vary the interpupillary distance. This allows the modules to be moved closer together to reduce overall size. The cameras also move with the modules. This compact design enables more compact headsets by reducing the distance between the modules and cameras. The adjustable interpupillary distance also accommodates different user head sizes.

Source

Electronic devices like head-mounted displays that merge real-world and virtual images to create augmented reality experiences. The devices have displays to show virtual content and optical systems to combine real-world and virtual images. The optical systems include components that merge the two types of light to create a single view for the user. This allows virtual objects to appear overlaid on the real world. The devices can have transparent displays, projectors, or holographic systems to create the mixed reality experience.

Source

Head-mounted devices like VR/AR headsets that resist motion with adjustable tension to improve comfort and prevent blurring. The devices have sensors to detect user head movement, and a tension controller analyzes the data to predict future motion. It then sends tension commands to an adjuster that applies force to resist head movement. This prevents the device from moving relative to the user's head, improving visual clarity and comfort compared to passive headstraps. The tension can be actively adjusted based on user behavior profiles and motion predictions.

Source

Electronic devices with optical components like head-mounted displays that have improved lens support structures to prevent detachment during drops. The lens support structure has an inner frame around the lens that is attached to the device housing. The lens is then mounted on the inner frame using a flexible lens retention mechanism. This allows the lens to flex and deform slightly during impact to absorb shock and prevent detachment.

Source

Wearable electronic devices like head-mounted displays with adjustable optical components that can improve user experience by dynamically enhancing visibility of virtual content overlaid on real-world objects. The devices have adjustable optical components like spatially addressable light modulators or lenses that can selectively darken parts of the user's field of view. This allows bright real-world objects to be dimmed when overlapped by virtual content, making the virtual content more visible. The adjustable optical components can be integrated into the device to avoid bulkiness and weight issues.

Source

Modifying virtual content presented on opaque displays to simulate how it would look on see-through displays. The system takes virtual content and modifies it based on simulation characteristics of see-through displays to more accurately replicate how it would appear on an actual see-through display. It also evaluates placement of virtual objects to avoid user perception issues when presented on a simulated see-through display.

Source

Enhancing user interactions with electronic devices by generating tactile outputs in response to device orientation and proximity to objects. The device generates tactile feedback when oriented within a range that changes with distance. It also modifies feedback based on device orientation. This reduces cognitive load and power usage compared to redundant user inputs. The device also generates tactile outputs indicative of AR planes, data sharing, and objects hovering over the display.

Source

Accurately positioning virtual content in mixed reality scenes relative to the physical environment and across multiple users and sessions. The technique involves identifying content from multiple sources based on device location, then accurately placing it in the scene using a combination of coarse and fine localization. This allows seamless integration of virtual and physical objects. The fine localization uses techniques like analyzing images of physical objects and matching 3D models.

Source

Changing display rendering modes based on multiple regions of a physical environment to improve user experience in augmented reality and other extended reality applications. The display adapts color characteristics of virtual content and the physical environment to provide a more immersive experience. It detects color characteristics of both the virtual content and the surrounding physical environment, and changes the display mode to match them when certain conditions are met. This prevents color conflicts between the virtual and physical elements.

Source

Improving efficiency of scanning and modeling environments using augmented and virtual reality by providing better user interfaces and scanning methods. The techniques include: 1. Previewing a 3D model of the environment being scanned while scanning, with the preview rotatable independently of the camera view. This allows users to easily view and manipulate the model without constantly realigning it to match the camera view. 2. Displaying visual cues over the camera feed to indicate unscanned areas between scanned sections. This helps users efficiently fill in gaps and complete the scan. 3. Changing the visual properties of graphical objects representing scanned features based on predicted accuracy, to provide feedback to the user. 4. Rotating the preview of the 3D model in response to user input, without requiring them to also rotate the camera view. This allows users to quickly reorient the preview to match their perspective.

Source

Displaying indicators in extended reality views that show content being used on other devices, positioned based on the other device's location, to provide intuitive access to cross-device content. For example, if a user views a website on their phone, an HMD view may contain an indicator representing the website near the phone's depiction, to indicate the website is open there.

Source

Compact and efficient image projector for AR/VR applications that uses a 1D scanning system to project a sequence of vertical line images. A scanning mirror reflects the lines as they are generated by a 1D image source. A pupil expander then expands the reflected lines into parallel replicas that fill the eye box. This allows a compact projector with high resolution, efficiency, and low power by focusing on vertical lines instead of a full 2D image. The lines are generated by emitters on a photonic integrated circuit that can modulate color and intensity.

Source

Head-mounted device with adjustable optical module positions for accommodating different user interpupillary distances. The device has actuators and optical module guide structures to allow the optical module positions to be adjusted. This allows customizing the headset to fit different users' eye spacing without needing separate headset models.

Source

Refining 3D surface representations of physical environments using a second 3D surface representation with geometric primitives to improve the accuracy and smoothness of the initial 3D point cloud or mesh. The method involves adjusting points in the initial representation based on corresponding points on the geometric primitives. This reduces noise and provides a better representation of flat surfaces like tables.

Source

Enhancing online shopping and customer support experiences by providing real-time communication with a remote salesperson through video chat, allowing the salesperson to demonstrate products and guide the user through the shopping experience. The user's device can also show a view of their environment with products added to see how they would look in the user's space. This provides an interactive, guided consumer experience with computer-generated reality capability.

Source

Head-mounted display (HMD) system that provides content optimized for low-light environments using onboard sensors like IR, depth, and ultrasonic. The HMD senses the environment in low light conditions using these sensors and generates customized content like enhanced graphics, overlays, and indicators based on the sensor data. This allows the display to adapt and enhance the user experience in dark environments where visibility is limited. The sensors also allow the HMD to detect objects and provide augmented reality features even in low light.

Source

Detecting eye gaze direction of users of electronic devices like head-mounted displays using low-power sensors to initiate actions like launching experiences when the user's gaze is approximately oriented towards a target area. An illuminator directs light towards the eye to produce a reflection. A sensor aligned with the gaze direction detects the reflection's properties. If the gaze is approximately towards the target area, it indicates the user's gaze and initiates the action. This allows efficient gaze detection without continual full eye tracking and reduces power.

Source