AI Robotics Developer — Autonomous AI Agent, Personality, Memory & Real-Time Interaction (Jetson Orin NX)

$10-30 USD

Posted 1 minute ago

•

Ends in 6 days

$10-30 USD

Paid on delivery

I'm looking for an experienced AI/robotics developer to help complete an autonomous AI-powered robot running on an NVIDIA Jetson Orin NX 16GB. This is not a project to build a simple voice assistant or chatbot. The goal is to create a robot with a consistent personality, long-term memory, emotional behavior, independent decision-making, and natural interactions, capable of reacting to its environment in real time. The robot is a character with an established personality, history, and relationship with another humanoid robot. Its behavior should reflect that identity through dialogue, voice, visual expressions, and physical movements. Current Development Status The project is already partially implemented. The following components are installed and operational: Whisper (STT): Speech recognition is implemented and working. Qwen (LLM): A local language model is installed and capable of generating conversational responses. Piper (TTS): Local text-to-speech is implemented. Basic conversational pipeline: The robot can already recognize speech, generate an answer, and speak it aloud. The main issue is that the responses are not yet consistently aligned with the robot's intended personality. The current implementation also needs further development to become a truly autonomous, responsive character rather than a basic question-and-answer system. The goal is to improve and extend the existing system, not unnecessarily rebuild components that already work. Main Features to Develop 1. Personality and Autonomous Intelligence Implement a persistent personality system based on an existing character profile. Maintain consistent personality traits, speaking style, opinions, and emotional reactions. Develop an internal emotional state that influences speech and physical behavior. Enable independent reasoning and decision-making. Allow the robot to decide when to speak, react, initiate conversations, or remain silent. Generate context-aware humor and sarcasm naturally, without repetitive scripted responses. Maintain consistent behavior throughout long conversations and across restarts. 2. Long-Term Memory and Character History Implement persistent short-term and long-term memory. Import a pre-existing fictional character history containing events that occurred before the robot's first activation. Differentiate between objective story history and the robot's subjective knowledge and memories. Remember conversations, events, relationships, and significant experiences. Retrieve relevant memories without overloading the LLM context. Support manual memory editing, deletion, and protection. Implement memory consolidation and a dream-generation system during charging or inactivity, keeping simulated dreams separate from factual memories. 3. Autonomous Conversation and Interruption Handling Implement natural conversational turn-taking. Enable the robot to detect and respond to interruptions while speaking. Develop an interruption-priority system: the robot should not necessarily stop speaking every time someone interrupts. Allow the robot to pause, resume, continue, or interrupt based on context, emotion, and conversational importance. Support independent conversational initiative rather than requiring a question or wake word for every interaction. 4. Vision, Hearing, and Attention Integrate camera-based face, person, and object tracking. Integrate the ReSpeaker XVF3800 four-microphone array for sound detection and direction. Develop an attention manager combining visual and audio information. Enable the robot to automatically orient toward speakers or relevant visual events. Prioritize competing stimuli and avoid unnecessary attention switching. Implement fast reactions independently of the LLM. 5. Physical Movement and Emotional Expression Integrate the Pimoroni Servo 2040 controller. Support eight servos controlling the quadruped's legs, with additional servos planned for eye movement. Implement basic locomotion, including standing, sitting, walking, and turning. Integrate an animated eye and physical mechanical iris for expressive behavior. Synchronize eye, iris, and body movements with speech, emotions, and attention. Ensure smooth movements, safe servo limits, and appropriate failure recovery. 6. Voice and Low-Latency Optimization Optimize the existing STT → LLM → TTS pipeline. Reduce the delay between the end of the user's speech and the robot's spoken response. Explore streaming LLM output and streaming TTS. Improve natural speech delivery, emotion, timing, and expressiveness. Maintain responsiveness while vision, audio, reasoning, and motor control operate simultaneously. 7. Cloud AI Integration and Comparison In addition to the fully local system, I want to implement an optional cloud-based AI pipeline for comparison. Requirements: Integrate a cloud LLM as an alternative to the local Qwen model. Evaluate cloud-based TTS as an alternative to the local voice system. Allow switching between local and cloud processing. Maintain the same personality, character memory, and behavior logic regardless of which backend is used. Compare response latency, conversational quality, personality consistency, voice quality, resource usage, and operating costs. Benchmark both solutions under realistic conditions. The cloud system is intended for testing and comparison. The robot must remain fully functional locally without internet access. 8. Architecture, Reliability, and Testing Work with or extend the existing ROS 2 architecture. Keep hardware reflexes separate from slower AI reasoning. Ensure the language model generates high-level behavioral intentions rather than directly controlling servos. Implement appropriate watchdogs, safety limits, and failure handling. Maintain a modular and maintainable codebase. Measure system performance, including STT latency, LLM first-token latency, TTS startup latency, and total conversational response time. Provide documentation, source code, and reproducible performance benchmarks. Hardware NVIDIA Jetson Orin NX 16GB ReSpeaker XVF3800 four-microphone array Pimoroni Servo 2040 controller Eight leg servos, with additional eye/iris actuation Camera LCD eye display and mechanical iris Speaker/audio output Required Experience I'm looking for someone with practical experience in: Python and Linux development on NVIDIA Jetson devices Local LLM inference and optimization AI agents, autonomous behavior, and persistent memory architectures Whisper, Qwen, Piper, or comparable speech/AI technologies ROS 2 and robotics integration Real-time audio processing and conversational AI Computer vision and servo control Cloud LLM/TTS APIs and latency benchmarking Experience creating expressive robotic characters or socially interactive robots would be particularly valuable. Expected Outcome The finished system should behave like an independent, intelligent robotic character, not a conventional AI assistant. It should be capable of listening, observing, reasoning, remembering, developing opinions, expressing emotions, initiating interactions, and coordinating speech with physical behavior. Personality consistency, natural interaction, and low latency are the most important priorities. Please include examples of relevant work, especially projects involving local AI agents, robotics, persistent memory, or real-time conversational systems. I would also appreciate a brief explanation of how you would approach improving the existing system rather than rebuilding it from scratch.

Project ID: 40808402

About the project

4 proposals

Open for bidding

Remote project

Active 11 secs ago

Place your bid

Bid amount Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

It's free to sign up and bid on jobs

4 freelancers are bidding on average $20 USD for this job

User Avatar
Flag of INDONESIA

Hi there, I'm excited by the depth and ambition of your project. With extensive experience in Python, ROS 2, and AI/robotics development on NVIDIA Jetson platforms, I've built expressive robots and conversational AI agents with nuanced personalities, persistent memory, and real-time multimodal interaction. My work includes integrating local LLMs (Qwen, Llama), Whisper for STT, Piper TTS, and robust servo/vision control, ensuring low latency and natural, emotion-driven behavior. To elevate your robot from a Q&A assistant to an autonomous character, I'd extend your current pipeline by layering a modular personality/emotion module above the LLM, guiding both dialogue and physical expression. Persistent memory would be managed via a hybrid system, balancing fast context retrieval and long-term storage, with manual editing and dream-generation during idle cycles. The attention manager would fuse audio/vision for real-time orientation, and interruption handling would blend priority logic with emotional context. I'd focus on streaming and parallelization to minimize response delays, and implement a switchable cloud/local LLM/TTS system for benchmarking. All improvements would build on your existing ROS 2 structure, maximizing modularity and reliability, with detailed performance metrics and full documentation. You can see relevant examples of my work in expressive social robots and multimodal AI agents upon request. I look forward to helping your robot truly come alive. Best Regards

User Avatar
Flag of INDIA

Hi, I would improve your current Jetson-based stack by keeping Whisper, Qwen, and Piper, then adding a persistent character layer for personality, emotional state, memory, and behavioral rules. I’d separate fast ROS2 reflexes from slower reasoning, integrate vision/audio attention, interruption handling, and servo/eye expressions, then benchmark local vs cloud backends while preserving the same character behavior. Start the chat so we can discuss the architecture and next steps. Regards, Sai Bhaskar

User Avatar
Flag of UNITED STATES

Hi This project calls for developing an autonomous AI agent equipped with a personality, memory, and real-time interaction on the Jetson Orin NX platform. The key technical challenge here is integrating various elements like Natural Language Processing and Reinforcement Learning into a cohesive system that enables interactive AI behavior. I'll address this by leveraging libraries such as TensorFlow for deep learning models, and using ROS (Robot Operating System) for orchestrating the robotics side, ensuring seamless communication between hardware and AI components. With over 6 years in full-stack development and hands-on experience using NVIDIA Jetson platforms for enhancing interactive AIs, I've built projects that incorporated LLMs and real-time robotics with effective results. The idea of merging AI memory functionality for your system is intriguing, and I'm eager to contribute to its success. I can help with this project and would be happy to walk you through my approach or dive into specifics of your current setup. Let's chat! Thanks, Shammi

内容来自 Freelancer News,版权归原作者所有。

阅读原文 ↗

把喜欢的内容,留在你的新标签页。

打开我的订阅