Developers building cross-platform software need a deep understanding of each device's capabilities and limitations to deliver an optimal experience. An app must adapt to diverse screen sizes, input methods, hardware features, and usage contexts.
For users, understanding device capabilities helps them select the right devices to meet their needs across different contexts, such as work, entertainment, health, and personal productivity.
I've divided this post into six parts: devices, factors, connectivity, operating system, interactions, and use cases. First up, computing, input, display, and storage devices.
Computing devices
Electronic devices designed to process, store, and manage data.
Wearables
- Ring
- Band
- Watch
- Contacts
- Glasses
- Fitness tracker
- Smart clothing
- Medical wearable
- Camera
- Smart patch
- Tattoo/sticker
- Sleep tracker
Personal
- Mobile
- Tablet
- Laptop
- Mini PC
- Graphics tablet
- Single-board computer (SBC)
- Handheld gaming console
- E-reader
Stationary
- Desktop
- Mini PC
- Kiosk
- Home hub
Audio
- Speakers
- Earbuds
- Earphones
- Headphones
- Headset
Media
- TV
Internet of Things (IoT)
- Speaker
- Thermostat
- Camera
- Motion detector
- Doorbell
- Scales
- Refrigerator
- TV
- Robot
- Toy
- Lighting
- ...
Input devices
Any hardware device used to send data or control signals to a computer or other electronic system.
Key
- Keyboard
- Keypad
Pointing
- Mouse
- Touchpad
- Trackball
- Joystick
Touch
- Light pen
- Stylus
- Graphics tablet
- Electronic whiteboard
- Touchscreen
Games
- Gamepad
- Game controller
- Joystick
- Motion controller
Audio
- Microphone
Scanning
- Barcode reader
- Card reader
Wearables
- EMG wristband
- Motion controller
Gestures
- Motion
- Voice
- Touch
- Eye
- Facial
- Body
Display devices
Hardware devices that visually present information, images, or video.
Screen
- Monitor
- TV
- 3D display
- Electronic paper
- Mobile
- Tablet
- Laptop
- Graphics tablet
- Watch
- Kiosk
Modern screens can be flat or flexible – able to roll, twist, bend, stretch, or transform. They may also feature haptic feedback, transparency, 3D visualisation, dynamic colour changes, and touch sensitivity.
Projection
- Projector
Wearables
- Head-mounted display (HMD)
- Glasses
- Contacts
Public
- Digital signage
Storage devices
Hardware components used to store, retrieve, and manage digital data.
Magnetic
- Hard disk drive (HDD)
Solid state
- Solid-state drive (SSD)
- USB flash drive
- Memory card
Optical
- DVD
- Blu-ray
Form factor
The physical characteristics of a device, for example, the physical size, shape, layout, weight, materials, and screen size. Form factor is crucial for developers as it affects key aspects such as UI design, functionality, and performance optimisation.
Usability factors
The characteristics of a device that determine how easy, efficient, and enjoyable it is for users to interact with. These factors influence the overall user experience. Example factors:
- Efficiency
- Ease of use
- Convenience
- Learnability
- Accessibility
- Portability
- Consistency
- Memorability
- Feedback
- Error handling
- Satisfaction
- Adaptability
- Battery life
- Flexibility
- Aesthetics
- Connectivity
- Interoperability
- Durability
- Ergonomics
- Eye strain
- Voice strain
- Voice fatigue
- Voice privacy
- One-handed use
- Hands-free use
- Interaction speed
- Cognitive load
- Physical effort
- Physical environment
- Response time
- Privacy
- Safety
- Security
Social factors
The social, cultural, and environmental influences that affect how a device is perceived, used, and accepted by individuals and groups. For example, in certain contexts or cultures, wearing smart glasses (with a camera) might be seen as socially acceptable, while in others, it could be viewed as intrusive or inappropriate.
Connectivity
Connectivity refers to the capability of devices to establish and maintain a connection, enabling communication between them. It enables sharing of information, resources, and access to services. Devices can connect either through a network (wireless or wired) or via direct physical connections, such as Ethernet or USB cables.
Wired
- USB-C
- HDMI
- Ethernet
- VGA
- Audio jack
- DisplayPort
- SATA
Wireless
- Wi-Fi
- Bluetooth
- Infrared
- Cellular networks
- NFC
- Satellite
- Zigbee
- Z-Wave
Device detection
The operating system can detect new devices either on the network or via direct physical connection.
Combinations
Example combos
The operating system can detect when a device is connected to a new display, and adapt the UI accordingly. For example, connect a watch to a 30-inch monitor – the monitor displays the UI for that screen size. Users can optionally sign into a web account, eg to sync data.
- Watch + monitor + keyboard + mouse
- Tablet + monitor + keyboard + mouse
- Mobile + monitor + keyboard + mouse
- Mobile + laptop
- Mobile + projector
- Glasses + earbuds
- Laptop + watch + watch
Operating system
One operating used across any device. An adaptive, modular OS where the installation automatically adjusts to the device type. One installer, one operating system name, eg Fossa. OS versions – Fossa 1, Fossa 2...
An OS is not part of this project, but it's something I would like to see. A solid open source base which others could build on.
Screen time
An OS can provide the capability to customise the device's functionality and UI. Users should have the ability to control the availability of native apps and websites/web apps. Native apps can help monitor and limit device usage.
Parents can customise app and website availability, with customisation settings secured with password and biometric authentication. Reinstalls can also require authentication.
Sync
Data can be synced to the web and then to other devices, or synced locally between devices, such as over a Wi-Fi network.
Interactions
Users can interact with devices featuring displays in various ways. Users can provide input directly through the display using touch, ultrasonic waves, or other contactless technologies. Users can also input data through devices, such as a mouse, keyboard, or microphone (either onboard or external), which can capture both voice and non-voice sounds.
Finally, users can provide input through eye, body, or facial movements, enabling more intuitive and hands-free interaction.
No display
With devices that lack a display, users can interact through sound, and if the device is equipped with a camera, they can also use movement for input – provided the software supports it.
Interaction factors
How you choose to interact with a device depends on a number of factors – environment, device capabilities, input requirements, output requirements, hands-free, privacy, efficiency, and preference.
Environment
Physical environment, mobile network strength, GPS signal, ambient noise, weather conditions, user movement, and social/cultural factors.
Device capabilities
Display, input method support, performance, connectivity, hardware and software capabilities, and form factor.
Input requirements
Do you need to type long passages of text? Need to provide a short command? Are the interactions simple, or do they require complex, multi-step actions? Will you be manipulating the user interface in a detailed way?
Output requirements
Require real-time information? Visual controls? Need to view text, images, or video? Voice output sufficient?
Hands-free
Need to multi-task? More efficient to use voice versus using a display?
Privacy
Do you mind the device always listening for voice commands? Do you prefer to keep your input private, such as avoiding voice commands in certain public contexts?
Efficiency
How much time and effort does it take to complete the task using your chosen input method? How easy is the process?
Preference
Individuals might simply prefer to use one input method over another, even if it's less efficient or less private.
Voice
Voice input is fine for short commands and hands-free operation. Issues include: privacy, social context, background noise, voice strain, voice fatigue, and effort – users have to enunciate clearly and might have to repeat themselves due to misunderstandings or background noise.
Displays
Displays are crucial for many tasks – viewing text, images, and videos, real-time data visualisations and feedback, multi-step and complex actions. Displays can offer privacy in situations where voice can't, ie when others can overhear what you're saying.
Use cases
Let's categorise device use cases – sensors, service bookings and orders, product orders, wayfinding, camera, communication, websites, and apps.
Health sensors
Sensors designed to monitor and measure various physiological and biological parameters of a person's body.
- Heart rate
- Blood oxygen
- Temperature
- ECG
- Accelerometers
- Blood pressure
- Galvanic skin response
- Blood glucose
- Respiratory rate
- Skin temperature
- Hydration
- EMG
- UV
- EEG
- Fall detection
Biometric sensors
Devices that capture and measure unique physical or behavioural characteristics of an individual, which can be used for identification, authentication, or health monitoring.
- Fingerprint
- Facial recognition
- Iris scanners
- Voice recognition
- Heart rate variability
- Palm
- Vein
- Gait
- Signature
Environment sensors
Devices that measure and monitor various physical, chemical, or biological conditions in the surrounding environment.
- Temperature
- Humidity
- Air quality
- Light
- Gas
- Motion
- Air pressure
- Sound
- Magnetic
Service bookings and orders
- Rideshare/taxi
- Restaurant
- Hotel
- Flight
- Car rental
- Food delivery
- Event ticket
- Tour
- ...
Product orders
- Books
- Clothing
- Footwear
- Electronics
- Home decor
- Food
- Toys and games
- Luggage
- ...
Wayfinding
- Maps
- GPS navigation
- Indoor navigation
- AR navigation
- Pedestrian navigation
- Real-time location sharing
- Geospatial data integration
- Parking guidance
Camera
- Photography
- Video recording
- Translate text
- Augmented reality
- Object identification
- Image search
- QR code scanning
- Barcode scanning
- Document scanning
- Video calling
- Video messaging
- Live streaming
- Virtual meetings
- Security camera
- Motion detection
- Facial recognition
- Game control
- Visual shopping
- Gesture recognition
Communication
- Phone call
- Video call
- Text message
- Video message
- Messaging app
- Notification
- Push-to-talk
- Social network
- File and document sharing
- Collaborative software
- Virtual meeting
Websites
- Information gathering
- Entertainment
- Communication
- Shopping
- Reservations
- Work purposes
- Banking
- News
- Alerts
- Government services
- Customer support
- Real-time updates
- Reviews
- Learning
Apps
- Entertainment
- Work
- Personal
Summary
A display unlocks a wide range of capabilities. With operating system support, a device can connect to one or more large displays, enhancing its functionality and enabling the user to interact with apps on a much more expansive interface.
Voice input is fine for short commands and hands-free operation, but has issues regarding privacy, social context, background noise, and effort.