What does the next iPhone look like.....without the screen?
Exploring the future of UX with AI-powered devices
Let’s be honest, the modern smartphone has pretty much peaked at this point - innovating in the confines of a metal prism is pretty minuscule. To create the next big thing in mobile computing - you have to literally think outside of the prism and that is exactly what humane is doing
Humane is giving you the power of a cell phone with a wearable so tiny that it gets attached to your clothes, projects all the necessary information in the palm of your hand, and is controlled by your 🗣️ voice and 🖐️ hands.
The way we interact with devices is changing. From a User Experience where we currently tap 2D screens to complete an action will change into more immersive experiences (ex: Projected Dashboards for Cars, AI jewelry) that require our bodies to be the controllers. To get to that future, that requires three major changes:
(1) AI-powered backend
(2) New Front-End Design
(3) New Hardware
🤖 AI-powered backend
To power these new devices, AI needs to become the foundational model at the back end. Here are some AI models that are heading towards this future 👇 .
ChatGPT Vision: An AI model built on top of GPT4 that allows users to plug in voice and images for it to spit back some information. In terms of UX design, engineers could train this model to understand design and give some tips on how to improve a designer’s work.
https://twitter.com/mckaywrigley/status/1707801301093068880
Deep Mind RT-2: An AI model built for robots to understand images which helps turn them into actions the robot can carry out. It is built off of Google’s Pathways Language Model Embodies (PaLM-E) and the Pathways Language and Image Model (PaLI-X) to serve as the basis for DeepMind RT-2. For UXers, this changes the entirety of normal design principles we are taught because of such innovative hardware at the forefront.
UI-AI Glyphs & Layers: This is an AI model developed by a company called Diagram that builds out AI models specifically for designers. The Glyph model focuses on creating text-inputted commands to create different types of icons, and different types of typography to better match a design. The UI-AI model is currently being integrated at Figma which is sick!
📲 New Front-End Design
With AI as the back end, the front end needs to change as well. Here are the three things that are going to power these new devices 👇
Voice: using our natural speech to interact with AI
Touch: using our fingers and hands to interact with AI
Eye: using eye tracking to interact with screenless tech and AI.
⚙️ New Hardware
Ray-Ban’s Smart Glasses
Sticking with wearables, Meta and Ray-Ban recently announced smart glasses that allow users to effortlessly navigate through their day, capture moments, stay connected through calls and texts, and explore a hands-free digital realm all through touch gestures, voice controls, and a unique UI that overlays over the real world.
This really disrupts the role of the UI designer as we now have to think about secondary inputs like voice and touch becoming the primary controllers. It also challenges designers on how to design for a screen-less medium. but there is a company that is actively integrating the design community in their wearable & screenless future - Apple.
Apple’s VisionOS on Figma
At this year’s Worldwide Developer’s Conference (WWDC), Apple unveiled the Vision Pro - their answer to the VR headset. Alongside the new product line, they also released some amazing features for engineers and designers!
(1) - An entirely new operating system called visionOS where engineers can start developing apps for the headset!
(2) - A partnership with Figma that allows designers to tap into Apple’s design ecosystem from iOS, iPadOS, macOS, and now visionOS. You can access the visionOS library here.
Why does this matter for designers? - This is huge because with such a design-centered company like Apple, partnering with Figma allows designers to really push Apple’s design into new and innovative territories. They are also the first company to open source a design library for a VR headset which means that designers can start creating UI designs for hardware that isn’t coming out till 2024.
Diagram.com
Speaking of Figma, they recently acquired a start-up called Diagram. This is what they do 👇
text into a prototype design. 🤯HOLY SHIT.
The day Jordan Singer, CEO of Diagram, tweeted this video - the world of design changed for the better and proved that AI is the tool that anchors the next wave of UI designers, UX researchers, and product designers.
From this small plug-in that Singer developed, he built Diagram which is building more AI tools for the design workflow through Figma. You can learn more about the acquisition here and the tweet that changed the design world here.
Adobe’s new AI projects
One of the best companies in the design space showed off its AI advancements at its conference, Adobe Max. This was an hour-long keynote where Adobe engineers showed off some personal projects they were developing for the Adobe Creative Suite.
Here are some of my favorite projects:
(1) Project Glyph Ease
This uses AI to create a model of certain typographies to create AI-generated fonts that are purely unique to whatever the user desires. It takes different types of inputs such as handwritten text, pre-designed texts, and already existing fonts.
(2) Project ResUp
This one really piqued my interest! As a movie lover, seeing old movies be tuned up to a higher resolution with just a click of a button was sick!
While these advancements are showing the future of design, to start practically designing UIs can be quite overwhelming because there are a ton of resources to choose from. To make your job easier, I have created a curated list of the best resources to help designers get started today!
🛠️ Tools
Paper + Pencil/Pens: When you start designing a UI, you can’t load up software on your laptop and start designing - you need to draw everything out - the layout of each screen, how screens connect with each other, and understanding the user flow from Point A to B. Here is one of the basic wireframes I drew to understand the basic layout of the AMC movie app. 👇
Figma is the best when you are ready to digitize your designs. It is considered to be the best for designing, wire-framing, and prototyping. It also has a slew of features such as converting your Figma designs into code, adding different textures for typography, and importing design libraries that allow you to access design assets from companies like Apple and Google. You can learn about Figma here.
Sketch
📚 Courses/Walkthroughs
Google’s UX Design Professional Certificate: The Google professional certification is the best way to get yourself started with the space. It drills down the basic foundations needed to be a designer and creates three main portfolio projects that every junior designer will have. This 7-course certificate also explains the day-to-day lives of designers at Google and gives what industry life is like. You can learn more about them here.
(p.s. I’m currently taking this and going to write a future blog post on my first project, stay tuned for that!)
Kickass UX: Co-founders Colton Schweitzer and Ludovic Delmas are re-inventing UX education with their free course. They analyzed the current problems for aspiring UXers and came up with two jaw-dropping conclusions that needed to be addressed :
(1) “UX education is expensive, impersonal, and inconsistent”
(2) “Hundreds of lookalike portfolios flood the UX market every year.”
With this research in mind, they created Kickass UX which allows students to dive deeper and create innovative portfolios that allow them to get more impactful jobs and do stuff that they are excited about! Here’s a preview of one of their lessons 👇
You can learn more about Kickass UX here.
Interaction Design Foundation - The Interaction Design Foundation is the biggest online design school with the best courses, including designing for AI, designing UX for AR, job hunting techniques for UX, and a ton more in between! They also have amazing partnerships with the best people in design. Don Norman, who is considered to be the godfather of design has a course called “Design for a Better World”. The IxDF is always pushing out amazing resources to keep up with the future of design. You can check them out here.
Relab Studios - This YouTube channel is the best when it comes to two main things: walkthroughs and explainers. They make amazing explainer videos on understanding pivotal job differences between the various roles in UX. They also make amazing walkthrough videos. For example, if you were having trouble understanding what a user flow is, they have a video walking you through on what a great user flow looks like. Check them out here.
👥 Accounts to Follow
With so many accounts creating UI/UX content, figuring out what’s legit can be overwhelming. Here my recommendation:
Jordan Singer’s Twitter: CEO of Diagram - he is always posting projects, takes, and the future of AI & design.
Figma’s Blog - Shortcut: They post the biggest updates to Figma and discuss the future of design!
UX Collective Newsletter on Medium: A collection of UX designer-written articles for you to learn more about the role and life of all types of UXers.
Kickass UX’s Newsletter: Updates to their learning platform, but also giving you constant tips for your UX education.
Zander Whitehurst’s IG: - A UI/UX designer who is known for his 1-minute Figma tutorials. He also gives great insight of the day-to-day life of a designer.