Beyond the Smart Speaker: Building a Desktop Companion That Can Talk, Blink, and Move

This community-built robot explores how voice, movement, and visual expression can make AI hardware feel more present in everyday life.

Most voice assistants live inside speakers, phones, or screens. They can answer a question, but they rarely feel physically present.

What changes when a voice assistant can also blink, move its ears, and react like a small character sitting beside you?

That question inspired Maker Science to build Joy, a DIY desktop companion robot featured in the Hackster community project, “This DIY Desktop Companion Robot Actually Talks.”

The idea began with a teddy bear sitting on the maker’s desk. Rather than recreate the toy, the creator imagined a new kind of companion—one that could speak while using animated eyes and physical movement to make the interaction feel more natural.

Giving a voice assistant a physical presence

Joy combines three forms of expression:

• Voice for conversation
• LED matrix eyes for visual feedback
• Servo-powered ears for physical movement

Two MAX7219 8×8 LED matrix modules form the robot’s eyes. Compared with the small OLED used in an earlier version of Joy, the separate matrices give the robot a more recognizable face and allow it to blink while speaking.

Two MG90S servo motors control a pair of rabbit-inspired ears. By coordinating movement with speech and eye animation, the robot responds more like a character or pet—and less like a conventional smart speaker.

This is a simple design choice, but an important one. People do not experience an interactive device only through its words. Timing, motion, sound, and visual feedback all influence whether an interaction feels responsive and understandable.

The hardware behind the interaction

At the center of the project is a Seeed Studio ReSpeaker XVF3800 audio platform with a XIAO board, accompanied by an external speaker and amplifier.

The ReSpeaker XVF3800 uses a circular four-microphone array for voice capture. Its audio-processing capabilities include acoustic echo cancellation, automatic gain control, beamforming, voice activity detection, noise suppression, dereverberation, and direction-of-arrival detection. These features help prepare voice input before it reaches the rest of an application. More information is available in the official ReSpeaker XVF3800 documentation.

This matters because a conversational device first needs to hear reliably. A strong language model cannot compensate for every problem caused by noisy, distant, or unclear audio.

The project’s remaining components include:

• Two LED matrix displays
• Two metal-gear servo motors
• An external speaker
• An audio amplifier
• A rechargeable battery and charging module
• ESP-IDF as the development environment

The result is not simply an audio device. It is a compact physical system that can listen, speak, display expressions, and move.

What Joy tells us about AI hardware

AI hardware is often discussed in terms of processors, model size, memory, and inference performance. Those factors matter, but Joy highlights another part of the experience: embodiment.

Once AI moves beyond a browser or phone, product designers must consider questions such as:

• How does the device show that it is listening?
• How does a user know that a request was understood?
• Should every response be verbal?
• Can movement communicate attention or emotion?
• How can the device feel expressive without becoming distracting?

LED eyes and moving ears are not intelligence by themselves. However, they give an intelligent system additional ways to communicate its state.

A blink could indicate that the device is active. A change in expression could show that it is processing a request. Ear movement could acknowledge the direction of a voice or signal the beginning of a response.

This connection between intelligence and physical feedback is what separates an embodied companion from a voice assistant placed inside a decorative enclosure.

From a maker project to future companions

Joy is a community prototype, not a finished consumer product. The project page demonstrates its form, movement, and speaking behavior, but does not document the complete conversational software pipeline or identify a specific language model.

Future versions could explore:

• Direction-aware movement using sound-source localization
• On-device keyword spotting for private, low-latency activation
• Different expressions for listening, thinking, speaking, and errors
• Local processing for selected commands
• Cloud-based conversational AI for more open-ended dialogue
• Sensors that allow the robot to respond to its environment

The ReSpeaker platform already supports audio features such as voice activity detection and direction of arrival. Seeed also provides examples for building TinyML keyword-spotting applications and connecting the hardware to conversational AI pipelines.

The important lesson is not that every desktop device needs a personality. It is that AI hardware becomes more understandable when users can see and hear what the system is doing.

Small hardware, more human interaction

Joy began with an ordinary object on a desk and a maker’s question: what if a companion robot could feel less like a speaker and more like a responsive character?

By combining voice capture, speech output, animated eyes, and moving ears, Maker Science created a compact example of embodied interaction. The project shows how makers can experiment not only with what an AI system says, but also with how a physical device listens, responds, and expresses itself.

As conversational AI moves into homes, desks, toys, and robots, these details will become increasingly important. The future of AI hardware will not be defined only by intelligence inside the device, but also by the experience built around it.

Explore the original Joy desktop companion project on Hackster.io.

Community Project Note: Joy is an independent community project created by Maker Science and is shared here with attribution. It is not an official Seeed Studio product.

About Author