Self-Hosted LLM for Sensitive Data: Hosting Is Not Security
A self-hosted LLM for sensitive data is not secure just because it is on-prem. vLLM's --api-key skipped /invocations and /metrics in my test, and an nginx allowlist closed both.
#vLLM
Posts tagged vLLM on abchaudary.me.