Node based
- Rivet (ironcladapp.com) is a Interfaces for Infrastructure Hardware and Edge based Agents using Large language models
- Node based visual interfaces is a Interfaces for Large language models which is open source and locally hosted. also see Langflow
- Langflow Interfaces for Large language models builder, with slightly more features than Node based visual interfaces which is another one I need to try soon from their github
- ComfyUI is a Interfaces for Stable Diffusion. It allows very high levels of control over Diffusion Models by leveraging open source extensions and a vibrant developer and creator community. The tool’s flexibility and efficiency comes from its innovative design philosophy, that prioritizes user customization and interaction.
LLM and multimodal local
Key Frontend Options
Open WebUI
- Description: Web-based UI inspired by ChatGPT, designed for high extensibility.
- Features:
- Workspaces for personalised assistants (similar to GPT’s custom setups).
- OpenAI-compatible endpoints for streamlined backend integration.
- Optimised for responsiveness, especially on touchscreen devices.
- Use Cases: General-purpose use, roleplay (RP), and advanced configuration.
- Limitations: Lack of comprehensive documentation remains a significant barrier.
- Link: Open WebUI GitHub
LibreChat
- Description: Web-based interface focusing on customisation and versatility.
- Features:
- Presets for reusable configurations.
- YAML and
.envsupport for endpoint and key management.
- Limitations:
- Export/migration tools are cumbersome.
- Slower responsiveness compared to Open WebUI.
- Link: LibreChat GitHub
SillyTavern
- Description: Universal adapter popular for roleplay.
- Features:
- Wide compatibility with backends like Koboldcpp.
- Growing focus on general use beyond RP.
- Limitations: Outdated UI design limits appeal.
- Link: SillyTavern GitHub
LM Studio
- Description: Standalone desktop application for local inference.
- Features:
- Easy to use with a modern UI.
- Suitable for new users and traditional “Windows-style” workflows.
- Limitations: Closed source; outbound connections for updates raise privacy concerns.
- Link: LM Studio
Msty
- Description: Native macOS application with extensive feature sets.
- Features:
- Integrates advanced tools like text-to-speech (TTS).
- Highly optimised for macOS environments.
- Link: Msty App
ChatterUI
-
Description: Lightweight Android app for local models.
-
Features:
- Mobile-focused with offline support.
- Works as a “SillyTavern Lite” alternative.
-
Link: ChatterUI GitHub
Backend Integration and Performance
3.1 Koboldcpp
- Strengths:
- Lightweight backend with robust feature support (e.g., roleplay, text-to-speech).
- Integrates well with Open WebUI and SillyTavern.
- Limitations: UI is functional but lacks visual polish.
- Link: Koboldcpp GitHub
3.2 Llama.cpp
- Strengths:
- Minimalist server UI with OpenAI-compatible API.
- Excellent for developers due to fast updates.
- Limitations: Limited feature set compared to Oobabooga and Open WebUI.
- Link: Llama.cpp GitHub
Oobabooga
-
Strengths:
- Broad feature set including image generation and voice capabilities.
- Stable for solo usage.
-
Limitations: Slower performance compared to newer backends like TabbyAPI or vLLM.
-
Link: Oobabooga GitHub
Multimodal Support and Advanced Features
Multimodal Capabilities
- Open WebUI: Can integrate vision models, image generation, TTS, and more with third-party tools like Azure or Together.ai.
- Koboldcpp: Supports some multimodal backends but lacks native syntax highlighting.
Developer-Oriented Tools
-
LM Studio: Quick model mounting for API integrations.
-
AnythingLLM: Flexible for agent development and experimentation.
-
For Beginners: LM Studio or Msty provide ease of use with minimal setup.
-
For Advanced Users: Open WebUI offers extensive customisation and backend compatibility.
-
For Roleplay: SillyTavern excels in flexibility with multiple backends.
-
For Multimodal Needs: Combine Open WebUI with specific vision or TTS backends.
-
For Developers: Use Llama.cpp for rapid updates or Koboldcpp for lightweight integration.