Multimodal LLM

Cove Sensory MCP is a local stdio MCP server that hands text-only LLMs like Claude Code and Codex their missing senses: image, video, audio, and music understanding. It routes media files through configurable multimodal providers—Gemini, MiniMax-M3, or OpenAI-compatible endpoints—while keeping everything on-device and privacy-first. Rather than a full assistant, it is a composable sensory layer you bolt onto your existing agent. H2: What Is Cove Sensory MCP and Why a “Sensory Layer”? Cove Sensory MCP is a Model Context Protocol server that deliberately limits itself to a single job: giving text-only language models the ability to perceive media files. The concept comes from Anthropic’s Model Context Protocol, a universal open standard that lets AI applications connect to external tools and context. Instead of trying to build chat, memory, personality, playback, monitoring, or call policies, Cove’s creators carved out one focused capability—sensing—and made it reusable across agents. ...

What Is JARVIS-1 and Why Does It Matter for Open-World AI Agents? JARVIS-1 is an open-world multi-task agent that combines multimodal large language models with a growing memory system to complete over 200 different tasks in Minecraft using human-like control and observation spaces. Developed by Team CraftJarvis at Peking University and BIGAI, JARVIS-1 achieves a 5x reliability improvement over previous state-of-the-art agents on the challenging ObtainDiamondPickaxe task and near-perfect performance on short-horizon tasks like chopping trees. Its key innovation is a multimodal memory that blends pre-trained knowledge from the LLM with actual game survival experiences, enabling self-improvement without retraining. ...