tdpeuter/open-webui
tdpeuter/open-webui
1
Fork 0
forked from open-webui/open-webui
Code Issues Pull requests 1 Releases 2 Packages Activity Actions

Merge branch 'main' into bun

Browse source
This commit is contained in:
ThatOneCalculator 2024-01-04 13:30:30 -08:00
commit 8480a9b418
No known key found for this signature in database
GPG key ID: 8703CACD01000000
16 changed files with 543 additions and 445 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
Dockerfile
View file

@ -2,12 +2,6 @@
FROM imbios/bun-node:alpine as build
ARG OLLAMA_API_BASE_URL='/ollama/api'
RUN echo $OLLAMA_API_BASE_URL
ENV PUBLIC_API_BASE_URL $OLLAMA_API_BASE_URL
RUN echo $PUBLIC_API_BASE_URL
WORKDIR /app
COPY package.json package-lock.json ./

2
README.md
View file

@ -35,6 +35,8 @@ Also check our sibling project, [OllamaHub](https://ollamahub.com/), where you c
- 📜 **Prompt Preset Support**: Instantly access preset prompts using the '/' command in the chat input. Load predefined conversation starters effortlessly and expedite your interactions. Effortlessly import prompts through [OllamaHub](https://ollamahub.com/) integration.
- 👍👎 **RLHF Annotation**: Empower your messages by rating them with thumbs up and thumbs down, facilitating the creation of datasets for Reinforcement Learning from Human Feedback (RLHF). Utilize your messages to train or fine-tune models, all while ensuring the confidentiality of locally saved data.
- 📥🗑️ **Download/Delete Models**: Easily download or remove models directly from the web UI.
- ⬆️ **GGUF File Model Creation**: Effortlessly create Ollama models by uploading GGUF files directly from the web UI. Streamlined process with options to upload from your machine or download GGUF files from Hugging Face.

4
TROUBLESHOOTING.md
View file

@ -4,7 +4,7 @@
The Ollama WebUI system is designed to streamline interactions between the client (your browser) and the Ollama API. At the heart of this design is a backend reverse proxy, enhancing security and resolving CORS issues.
- **How it Works**: When you make a request (like `/ollama/api/tags`) from the Ollama WebUI, it doesn’t go directly to the Ollama API. Instead, it first reaches the Ollama WebUI backend. The backend then forwards this request to the Ollama API via the route you define in the `OLLAMA_API_BASE_URL` environment variable. For instance, a request to `/ollama/api/tags` in the WebUI is equivalent to `OLLAMA_API_BASE_URL/tags` in the backend.
- **How it Works**: The Ollama WebUI is designed to interact with the Ollama API through a specific route. When a request is made from the WebUI to Ollama, it is not directly sent to the Ollama API. Initially, the request is sent to the Ollama WebUI backend via `/ollama/api` route. From there, the backend is responsible for forwarding the request to the Ollama API. This forwarding is accomplished by using the route specified in the `OLLAMA_API_BASE_URL` environment variable. Therefore, a request made to `/ollama/api` in the WebUI is effectively the same as making a request to `OLLAMA_API_BASE_URL` in the backend. For instance, a request to `/ollama/api/tags` in the WebUI is equivalent to `OLLAMA_API_BASE_URL/tags` in the backend.
- **Security Benefits**: This design prevents direct exposure of the Ollama API to the frontend, safeguarding against potential CORS (Cross-Origin Resource Sharing) issues and unauthorized access. Requiring authentication to access the Ollama API further enhances this security layer.
@ -27,6 +27,6 @@ docker run -d --network=host -v ollama-webui:/app/backend/data -e OLLAMA_API_BAS
1. **Verify Ollama URL Format**:
- When running the Web UI container, ensure the `OLLAMA_API_BASE_URL` is correctly set, including the `/api` suffix. (e.g., `http://192.168.1.1:11434/api` for different host setups).
- In the Ollama WebUI, navigate to "Settings" > "General".
- Confirm that the Ollama Server URL is correctly set to `/ollama/api`, including the `/api` suffix.
- Confirm that the Ollama Server URL is correctly set to `[OLLAMA URL]/api` (e.g., `http://localhost:11434/api`), including the `/api` suffix.
By following these enhanced troubleshooting steps, connection issues should be effectively resolved. For further assistance or queries, feel free to reach out to us on our community Discord.

152
backend/apps/ollama/main.py
View file

@ -1,73 +1,68 @@
from flask import Flask, request, Response, jsonify
from flask_cors import CORS
from fastapi import FastAPI, Request, Response, HTTPException, Depends
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
import requests
import json
from pydantic import BaseModel
from apps.web.models.users import Users
from constants import ERROR_MESSAGES
from utils.utils import decode_token
from utils.utils import decode_token, get_current_user
from config import OLLAMA_API_BASE_URL, WEBUI_AUTH
app = Flask(__name__)
CORS(
app
) # Enable Cross-Origin Resource Sharing (CORS) to allow requests from different domains
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Define the target server URL
TARGET_SERVER_URL = OLLAMA_API_BASE_URL
app.state.OLLAMA_API_BASE_URL = OLLAMA_API_BASE_URL
# TARGET_SERVER_URL = OLLAMA_API_BASE_URL
@app.route("/",
defaults={"path": ""},
methods=["GET", "POST", "PUT", "DELETE"])
@app.route("/<path:path>", methods=["GET", "POST", "PUT", "DELETE"])
def proxy(path):
# Combine the base URL of the target server with the requested path
target_url = f"{TARGET_SERVER_URL}/{path}"
print(target_url)
@app.get("/url")
async def get_ollama_api_url(user=Depends(get_current_user)):
if user and user.role == "admin":
return {"OLLAMA_API_BASE_URL": app.state.OLLAMA_API_BASE_URL}
else:
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
# Get data from the original request
data = request.get_data()
class UrlUpdateForm(BaseModel):
url: str
@app.post("/url/update")
async def update_ollama_api_url(
form_data: UrlUpdateForm, user=Depends(get_current_user)
):
if user and user.role == "admin":
app.state.OLLAMA_API_BASE_URL = form_data.url
return {"OLLAMA_API_BASE_URL": app.state.OLLAMA_API_BASE_URL}
else:
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
async def proxy(path: str, request: Request, user=Depends(get_current_user)):
target_url = f"{app.state.OLLAMA_API_BASE_URL}/{path}"
body = await request.body()
headers = dict(request.headers)
# Basic RBAC support
if WEBUI_AUTH:
if "Authorization" in headers:
_, credentials = headers["Authorization"].split()
token_data = decode_token(credentials)
if token_data is None or "email" not in token_data:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
user = Users.get_user_by_email(token_data["email"])
if user:
# Only user and admin roles can access
if user.role in ["user", "admin"]:
if path in ["pull", "delete", "push", "copy", "create"]:
# Only admin role can perform actions above
if user.role == "admin":
pass
else:
return (
jsonify({
"detail":
ERROR_MESSAGES.ACCESS_PROHIBITED
}),
401,
)
else:
pass
else:
return jsonify(
{"detail": ERROR_MESSAGES.ACCESS_PROHIBITED}), 401
else:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
else:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
if user.role in ["user", "admin"]:
if path in ["pull", "delete", "push", "copy", "create"]:
if user.role != "admin":
raise HTTPException(
status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED
)
else:
pass
r = None
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
headers.pop("Host", None)
headers.pop("Authorization", None)
@ -75,47 +70,30 @@ def proxy(path):
headers.pop("Referer", None)
try:
# Make a request to the target server
r = requests.request(
method=request.method,
url=target_url,
data=data,
data=body,
headers=headers,
stream=True, # Enable streaming for server-sent events
stream=True,
)
r.raise_for_status()
# Proxy the target server's response to the client
def generate():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
response = Response(generate(), status=r.status_code)
# Copy headers from the target server's response to the client's response
for key, value in r.headers.items():
response.headers[key] = value
return response
return StreamingResponse(
r.iter_content(chunk_size=8192),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
print(e)
error_detail = "Ollama WebUI: Server Connection Error"
if r != None:
print(r.text)
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
print(res)
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
return (
jsonify({
"detail": error_detail,
"message": str(e),
}),
400,
)
if __name__ == "__main__":
app.run(debug=True)
raise HTTPException(status_code=r.status_code, detail=error_detail)

176
backend/apps/ollama/old_main.py Normal file
View file

@ -0,0 +1,176 @@
from flask import Flask, request, Response, jsonify
from flask_cors import CORS
import requests
import json
from apps.web.models.users import Users
from constants import ERROR_MESSAGES
from utils.utils import decode_token
from config import OLLAMA_API_BASE_URL, WEBUI_AUTH
app = Flask(__name__)
CORS(
app
) # Enable Cross-Origin Resource Sharing (CORS) to allow requests from different domains
# Define the target server URL
TARGET_SERVER_URL = OLLAMA_API_BASE_URL
@app.route("/url", methods=["GET"])
def get_ollama_api_url():
headers = dict(request.headers)
if "Authorization" in headers:
_, credentials = headers["Authorization"].split()
token_data = decode_token(credentials)
if token_data is None or "email" not in token_data:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
user = Users.get_user_by_email(token_data["email"])
if user and user.role == "admin":
return (
jsonify({"OLLAMA_API_BASE_URL": TARGET_SERVER_URL}),
200,
)
else:
return (
jsonify({"detail": ERROR_MESSAGES.ACCESS_PROHIBITED}),
401,
)
else:
return (
jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}),
401,
)
@app.route("/url/update", methods=["POST"])
def update_ollama_api_url():
headers = dict(request.headers)
data = request.get_json(force=True)
if "Authorization" in headers:
_, credentials = headers["Authorization"].split()
token_data = decode_token(credentials)
if token_data is None or "email" not in token_data:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
user = Users.get_user_by_email(token_data["email"])
if user and user.role == "admin":
TARGET_SERVER_URL = data["url"]
return (
jsonify({"OLLAMA_API_BASE_URL": TARGET_SERVER_URL}),
200,
)
else:
return (
jsonify({"detail": ERROR_MESSAGES.ACCESS_PROHIBITED}),
401,
)
else:
return (
jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}),
401,
)
@app.route("/", defaults={"path": ""}, methods=["GET", "POST", "PUT", "DELETE"])
@app.route("/<path:path>", methods=["GET", "POST", "PUT", "DELETE"])
def proxy(path):
# Combine the base URL of the target server with the requested path
target_url = f"{TARGET_SERVER_URL}/{path}"
print(target_url)
# Get data from the original request
data = request.get_data()
headers = dict(request.headers)
# Basic RBAC support
if WEBUI_AUTH:
if "Authorization" in headers:
_, credentials = headers["Authorization"].split()
token_data = decode_token(credentials)
if token_data is None or "email" not in token_data:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
user = Users.get_user_by_email(token_data["email"])
if user:
# Only user and admin roles can access
if user.role in ["user", "admin"]:
if path in ["pull", "delete", "push", "copy", "create"]:
# Only admin role can perform actions above
if user.role == "admin":
pass
else:
return (
jsonify({"detail": ERROR_MESSAGES.ACCESS_PROHIBITED}),
401,
)
else:
pass
else:
return jsonify({"detail": ERROR_MESSAGES.ACCESS_PROHIBITED}), 401
else:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
else:
return jsonify({"detail": ERROR_MESSAGES.UNAUTHORIZED}), 401
else:
pass
r = None
headers.pop("Host", None)
headers.pop("Authorization", None)
headers.pop("Origin", None)
headers.pop("Referer", None)
try:
# Make a request to the target server
r = requests.request(
method=request.method,
url=target_url,
data=data,
headers=headers,
stream=True, # Enable streaming for server-sent events
)
r.raise_for_status()
# Proxy the target server's response to the client
def generate():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
response = Response(generate(), status=r.status_code)
# Copy headers from the target server's response to the client's response
for key, value in r.headers.items():
response.headers[key] = value
return response
except Exception as e:
print(e)
error_detail = "Ollama WebUI: Server Connection Error"
if r != None:
print(r.text)
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
print(res)
return (
jsonify(
{
"detail": error_detail,
"message": str(e),
}
),
400,
)
if __name__ == "__main__":
app.run(debug=True)

8
backend/main.py
View file

@ -47,7 +47,7 @@ async def check_url(request: Request, call_next):
app.mount("/api/v1", webui_app)
app.mount("/ollama/api", WSGIMiddleware(ollama_app))
app.mount("/",
SPAStaticFiles(directory="../build", html=True),
name="spa-static-files")
# app.mount("/ollama/api", WSGIMiddleware(ollama_app))
app.mount("/ollama/api", ollama_app)
app.mount("/", SPAStaticFiles(directory="../build", html=True), name="spa-static-files")

143
src/lib/apis/ollama/index.ts
View file

@ -1,12 +1,76 @@
import { OLLAMA_API_BASE_URL } from '$lib/constants';
export const getOllamaVersion = async (
base_url: string = OLLAMA_API_BASE_URL,
token: string = ''
) => {
export const getOllamaAPIUrl = async (token: string = '') => {
let error = null;
const res = await fetch(`${base_url}/version`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/url`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...(token && { authorization: `Bearer ${token}` })
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
if ('detail' in err) {
error = err.detail;
} else {
error = 'Server connection failed';
}
return null;
});
if (error) {
throw error;
}
return res.OLLAMA_API_BASE_URL;
};
export const updateOllamaAPIUrl = async (token: string = '', url: string) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/url/update`, {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...(token && { authorization: `Bearer ${token}` })
},
body: JSON.stringify({
url: url
})
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((err) => {
console.log(err);
if ('detail' in err) {
error = err.detail;
} else {
error = 'Server connection failed';
}
return null;
});
if (error) {
throw error;
}
return res.OLLAMA_API_BASE_URL;
};
export const getOllamaVersion = async (token: string = '') => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/version`, {
method: 'GET',
headers: {
Accept: 'application/json',
@ -35,13 +99,10 @@ export const getOllamaVersion = async (
return res?.version ?? '';
};
export const getOllamaModels = async (
base_url: string = OLLAMA_API_BASE_URL,
token: string = ''
) => {
export const getOllamaModels = async (token: string = '') => {
let error = null;
const res = await fetch(`${base_url}/tags`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/tags`, {
method: 'GET',
headers: {
Accept: 'application/json',
@ -67,18 +128,15 @@ export const getOllamaModels = async (
throw error;
}
return res?.models ?? [];
return (res?.models ?? []).sort((a, b) => {
return a.name.localeCompare(b.name);
});
};
export const generateTitle = async (
base_url: string = OLLAMA_API_BASE_URL,
token: string = '',
model: string,
prompt: string
) => {
export const generateTitle = async (token: string = '', model: string, prompt: string) => {
let error = null;
const res = await fetch(`${base_url}/generate`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/generate`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
@ -109,14 +167,10 @@ export const generateTitle = async (
return res?.response ?? 'New Chat';
};
export const generateChatCompletion = async (
base_url: string = OLLAMA_API_BASE_URL,
token: string = '',
body: object
) => {
export const generateChatCompletion = async (token: string = '', body: object) => {
let error = null;
const res = await fetch(`${base_url}/chat`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/chat`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
@ -135,15 +189,10 @@ export const generateChatCompletion = async (
return res;
};
export const createModel = async (
base_url: string = OLLAMA_API_BASE_URL,
token: string,
tagName: string,
content: string
) => {
export const createModel = async (token: string, tagName: string, content: string) => {
let error = null;
const res = await fetch(`${base_url}/create`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/create`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
@ -165,14 +214,10 @@ export const createModel = async (
return res;
};
export const deleteModel = async (
base_url: string = OLLAMA_API_BASE_URL,
token: string,
tagName: string
) => {
export const deleteModel = async (token: string, tagName: string) => {
let error = null;
const res = await fetch(`${base_url}/delete`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/delete`, {
method: 'DELETE',
headers: {
'Content-Type': 'text/event-stream',
@ -202,3 +247,27 @@ export const deleteModel = async (
return res;
};
export const pullModel = async (token: string, tagName: string) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/pull`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
name: tagName
})
}).catch((err) => {
error = err;
return null;
});
if (error) {
throw error;
}
return res;
};

5
src/lib/apis/openai/index.ts
View file

@ -29,5 +29,8 @@ export const getOpenAIModels = async (
return models
.map((model) => ({ name: model.id, external: true }))
.filter((model) => (base_url.includes('openai') ? model.name.includes('gpt') : true));
.filter((model) => (base_url.includes('openai') ? model.name.includes('gpt') : true))
.sort((a, b) => {
return a.name.localeCompare(b.name);
});
};

325
src/lib/components/chat/SettingsModal.svelte
View file

@ -7,19 +7,23 @@
import { config, models, settings, user, chats } from '$lib/stores';
import { splitStream, getGravatarURL } from '$lib/utils';
import { getOllamaVersion } from '$lib/apis/ollama';
import { createNewChat, deleteAllChats, getAllChats, getChatList } from '$lib/apis/chats';
import {
WEB_UI_VERSION,
OLLAMA_API_BASE_URL,
WEBUI_API_BASE_URL,
WEBUI_BASE_URL
} from '$lib/constants';
getOllamaVersion,
getOllamaModels,
getOllamaAPIUrl,
updateOllamaAPIUrl,
pullModel,
createModel,
deleteModel
} from '$lib/apis/ollama';
import { createNewChat, deleteAllChats, getAllChats, getChatList } from '$lib/apis/chats';
import { WEB_UI_VERSION, WEBUI_API_BASE_URL } from '$lib/constants';
import Advanced from './Settings/Advanced.svelte';
import Modal from '../common/Modal.svelte';
import { updateUserPassword } from '$lib/apis/auths';
import { goto } from '$app/navigation';
import Page from '../../../routes/(app)/+page.svelte';
export let show = false;
@ -33,7 +37,7 @@
let selectedTab = 'general';
// General
let API_BASE_URL = OLLAMA_API_BASE_URL;
let API_BASE_URL = '';
let themes = ['dark', 'light', 'rose-pine dark', 'rose-pine-dawn light'];
let theme = 'dark';
let notificationEnabled = false;
@ -139,19 +143,13 @@
// About
let ollamaVersion = '';
const checkOllamaConnection = async () => {
if (API_BASE_URL === '') {
API_BASE_URL = OLLAMA_API_BASE_URL;
}
const _models = await getModels(API_BASE_URL, 'ollama');
const updateOllamaAPIUrlHandler = async () => {
API_BASE_URL = await updateOllamaAPIUrl(localStorage.token, API_BASE_URL);
const _models = await getModels('ollama');
if (_models.length > 0) {
toast.success('Server connection verified');
await models.set(_models);
saveSettings({
API_BASE_URL: API_BASE_URL
});
}
};
@ -229,67 +227,60 @@
const pullModelHandler = async () => {
modelTransferring = true;
const res = await fetch(`${API_BASE_URL}/pull`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
...($settings.authHeader && { Authorization: $settings.authHeader }),
...($user && { Authorization: `Bearer ${localStorage.token}` })
},
body: JSON.stringify({
name: modelTag
})
});
const reader = res.body
.pipeThrough(new TextDecoderStream())
.pipeThrough(splitStream('\n'))
.getReader();
const res = await pullModel(localStorage.token, modelTag);
while (true) {
const { value, done } = await reader.read();
if (done) break;
if (res) {
const reader = res.body
.pipeThrough(new TextDecoderStream())
.pipeThrough(splitStream('\n'))
.getReader();
try {
let lines = value.split('\n');
while (true) {
const { value, done } = await reader.read();
if (done) break;
for (const line of lines) {
if (line !== '') {
console.log(line);
let data = JSON.parse(line);
console.log(data);
try {
let lines = value.split('\n');
if (data.error) {
throw data.error;
}
for (const line of lines) {
if (line !== '') {
console.log(line);
let data = JSON.parse(line);
console.log(data);
if (data.detail) {
throw data.detail;
}
if (data.status) {
if (!data.digest) {
toast.success(data.status);
if (data.error) {
throw data.error;
}
if (data.status === 'success') {
const notification = new Notification(`Ollama`, {
body: `Model '${modelTag}' has been successfully downloaded.`,
icon: '/favicon.png'
});
}
} else {
digest = data.digest;
if (data.completed) {
pullProgress = Math.round((data.completed / data.total) * 1000) / 10;
if (data.detail) {
throw data.detail;
}
if (data.status) {
if (!data.digest) {
toast.success(data.status);
if (data.status === 'success') {
const notification = new Notification(`Ollama`, {
body: `Model '${modelTag}' has been successfully downloaded.`,
icon: '/favicon.png'
});
}
} else {
pullProgress = 100;
digest = data.digest;
if (data.completed) {
pullProgress = Math.round((data.completed / data.total) * 1000) / 10;
} else {
pullProgress = 100;
}
}
}
}
}
} catch (error) {
console.log(error);
toast.error(error);
}
} catch (error) {
console.log(error);
toast.error(error);
}
}
@ -410,21 +401,11 @@
}
if (uploaded) {
const res = await fetch(`${$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL}/create`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
...($settings.authHeader && { Authorization: $settings.authHeader }),
...($user && { Authorization: `Bearer ${localStorage.token}` })
},
body: JSON.stringify({
name: `${name}:latest`,
modelfile: `FROM @${modelFileDigest}\n${modelFileContent}`
})
}).catch((err) => {
console.log(err);
return null;
});
const res = await createModel(
localStorage.token,
`${name}:latest`,
`FROM @${modelFileDigest}\n${modelFileContent}`
);
if (res && res.ok) {
const reader = res.body
@ -490,92 +471,33 @@
};
const deleteModelHandler = async () => {
const res = await fetch(`${API_BASE_URL}/delete`, {
method: 'DELETE',
headers: {
'Content-Type': 'text/event-stream',
...($settings.authHeader && { Authorization: $settings.authHeader }),
...($user && { Authorization: `Bearer ${localStorage.token}` })
},
body: JSON.stringify({
name: deleteModelTag
})
const res = await deleteModel(localStorage.token, deleteModelTag).catch((error) => {
toast.error(error);
});
const reader = res.body
.pipeThrough(new TextDecoderStream())
.pipeThrough(splitStream('\n'))
.getReader();
while (true) {
const { value, done } = await reader.read();
if (done) break;
try {
let lines = value.split('\n');
for (const line of lines) {
if (line !== '' && line !== 'null') {
console.log(line);
let data = JSON.parse(line);
console.log(data);
if (data.error) {
throw data.error;
}
if (data.detail) {
throw data.detail;
}
if (data.status) {
}
} else {
toast.success(`Deleted ${deleteModelTag}`);
}
}
} catch (error) {
console.log(error);
toast.error(error);
}
if (res) {
toast.success(`Deleted ${deleteModelTag}`);
}
deleteModelTag = '';
models.set(await getModels());
};
const getModels = async (url = '', type = 'all') => {
const getModels = async (type = 'all') => {
let models = [];
const res = await fetch(`${url ? url : $settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL}/tags`, {
method: 'GET',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
...($settings.authHeader && { Authorization: $settings.authHeader }),
...($user && { Authorization: `Bearer ${localStorage.token}` })
}
})
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
})
.catch((error) => {
console.log(error);
if ('detail' in error) {
toast.error(error.detail);
} else {
toast.error('Server connection failed');
}
return null;
});
console.log(res);
models.push(...(res?.models ?? []));
models.push(
...(await getOllamaModels(localStorage.token).catch((error) => {
toast.error(error);
return [];
}))
);
// If OpenAI API Key exists
if (type === 'all' && $settings.OPENAI_API_KEY) {
const API_BASE_URL = $settings.OPENAI_API_BASE_URL ?? 'https://api.openai.com/v1';
const OPENAI_API_BASE_URL = $settings.OPENAI_API_BASE_URL ?? 'https://api.openai.com/v1';
// Validate OPENAI_API_KEY
const openaiModelRes = await fetch(`${API_BASE_URL}/models`, {
const openaiModelRes = await fetch(`${OPENAI_API_BASE_URL}/models`, {
method: 'GET',
headers: {
'Content-Type': 'application/json',
@ -603,7 +525,7 @@
...openAIModels
.map((model) => ({ name: model.id, external: true }))
.filter((model) =>
API_BASE_URL.includes('openai') ? model.name.includes('gpt') : true
OPENAI_API_BASE_URL.includes('openai') ? model.name.includes('gpt') : true
)
]
: [])
@ -639,15 +561,18 @@
};
onMount(async () => {
console.log('settings', $user.role === 'admin');
if ($user.role === 'admin') {
API_BASE_URL = await getOllamaAPIUrl(localStorage.token);
}
let settings = JSON.parse(localStorage.getItem('settings') ?? '{}');
console.log(settings);
theme = localStorage.theme ?? 'dark';
notificationEnabled = settings.notificationEnabled ?? false;
API_BASE_URL = settings.API_BASE_URL ?? OLLAMA_API_BASE_URL;
system = settings.system ?? '';
requestFormat = settings.requestFormat ?? '';
options.seed = settings.seed ?? 0;
@ -674,10 +599,7 @@
authContent = settings.authHeader.split(' ')[1];
}
ollamaVersion = await getOllamaVersion(
API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token
).catch((error) => {
ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => {
return '';
});
});
@ -1041,51 +963,51 @@
</div>
</div>
<hr class=" dark:border-gray-700" />
<div>
<div class=" mb-2.5 text-sm font-medium">Ollama API URL</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none"
placeholder="Enter URL (e.g. http://localhost:8080/ollama/api)"
bind:value={API_BASE_URL}
/>
</div>
<button
class="px-3 bg-gray-200 hover:bg-gray-300 dark:bg-gray-600 dark:hover:bg-gray-700 rounded transition"
on:click={() => {
checkOllamaConnection();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M15.312 11.424a5.5 5.5 0 01-9.201 2.466l-.312-.311h2.433a.75.75 0 000-1.5H3.989a.75.75 0 00-.75.75v4.242a.75.75 0 001.5 0v-2.43l.31.31a7 7 0 0011.712-3.138.75.75 0 00-1.449-.39zm1.23-3.723a.75.75 0 00.219-.53V2.929a.75.75 0 00-1.5 0V5.36l-.31-.31A7 7 0 003.239 8.188a.75.75 0 101.448.389A5.5 5.5 0 0113.89 6.11l.311.31h-2.432a.75.75 0 000 1.5h4.243a.75.75 0 00.53-.219z"
clip-rule="evenodd"
{#if $user.role === 'admin'}
<hr class=" dark:border-gray-700" />
<div>
<div class=" mb-2.5 text-sm font-medium">Ollama API URL</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none"
placeholder="Enter URL (e.g. http://localhost:11434/api)"
bind:value={API_BASE_URL}
/>
</svg>
</button>
</div>
</div>
<button
class="px-3 bg-gray-200 hover:bg-gray-300 dark:bg-gray-600 dark:hover:bg-gray-700 rounded transition"
on:click={() => {
updateOllamaAPIUrlHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M15.312 11.424a5.5 5.5 0 01-9.201 2.466l-.312-.311h2.433a.75.75 0 000-1.5H3.989a.75.75 0 00-.75.75v4.242a.75.75 0 001.5 0v-2.43l.31.31a7 7 0 0011.712-3.138.75.75 0 00-1.449-.39zm1.23-3.723a.75.75 0 00.219-.53V2.929a.75.75 0 00-1.5 0V5.36l-.31-.31A7 7 0 003.239 8.188a.75.75 0 101.448.389A5.5 5.5 0 0113.89 6.11l.311.31h-2.432a.75.75 0 000 1.5h4.243a.75.75 0 00.53-.219z"
clip-rule="evenodd"
/>
</svg>
</button>
</div>
<div class="mt-2 text-xs text-gray-400 dark:text-gray-500">
The field above should be set to <span
class=" text-gray-500 dark:text-gray-300 font-medium">'/ollama/api'</span
>;
<a
class=" text-gray-500 dark:text-gray-300 font-medium"
href="https://github.com/ollama-webui/ollama-webui#troubleshooting"
target="_blank"
>
Click here for help.
</a>
<div class="mt-2 text-xs text-gray-400 dark:text-gray-500">
Trouble accessing Ollama?
<a
class=" text-gray-300 font-medium"
href="https://github.com/ollama-webui/ollama-webui#troubleshooting"
target="_blank"
>
Click here for help.
</a>
</div>
</div>
</div>
{/if}
<hr class=" dark:border-gray-700" />
@ -1103,7 +1025,6 @@
class=" px-4 py-2 bg-emerald-600 hover:bg-emerald-700 text-gray-100 transition rounded"
on:click={() => {
saveSettings({
API_BASE_URL: API_BASE_URL === '' ? OLLAMA_API_BASE_URL : API_BASE_URL,
system: system !== '' ? system : undefined
});
show = false;

9
src/lib/constants.ts
View file

@ -1,13 +1,8 @@
import { dev, browser } from '$app/environment';
import { PUBLIC_API_BASE_URL } from '$env/static/public';
import { dev } from '$app/environment';
export const OLLAMA_API_BASE_URL = dev
? `http://${location.hostname}:8080/ollama/api`
: PUBLIC_API_BASE_URL === ''
? browser
? `http://${location.hostname}:11434/api`
: `http://localhost:11434/api`
: PUBLIC_API_BASE_URL;
: '/ollama/api';
export const WEBUI_BASE_URL = dev ? `http://${location.hostname}:8080` : ``;
export const WEBUI_API_BASE_URL = `${WEBUI_BASE_URL}/api/v1`;

12
src/routes/(app)/+layout.svelte
View file

@ -14,7 +14,7 @@
import { getOpenAIModels } from '$lib/apis/openai';
import { user, showSettings, settings, models, modelfiles, prompts } from '$lib/stores';
import { OLLAMA_API_BASE_URL, REQUIRED_OLLAMA_VERSION, WEBUI_API_BASE_URL } from '$lib/constants';
import { REQUIRED_OLLAMA_VERSION, WEBUI_API_BASE_URL } from '$lib/constants';
import SettingsModal from '$lib/components/chat/SettingsModal.svelte';
import Sidebar from '$lib/components/layout/Sidebar.svelte';
@ -32,10 +32,7 @@
const getModels = async () => {
let models = [];
models.push(
...(await getOllamaModels(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token
).catch((error) => {
...(await getOllamaModels(localStorage.token).catch((error) => {
toast.error(error);
return [];
}))
@ -58,10 +55,7 @@
const setOllamaVersion = async (version: string = '') => {
if (version === '') {
version = await getOllamaVersion(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token
).catch((error) => {
version = await getOllamaVersion(localStorage.token).catch((error) => {
return '';
});
}

58
src/routes/(app)/+page.svelte
View file

@ -7,7 +7,6 @@
import { page } from '$app/stores';
import { models, modelfiles, user, settings, chats, chatId, config } from '$lib/stores';
import { OLLAMA_API_BASE_URL } from '$lib/constants';
import { generateChatCompletion, generateTitle } from '$lib/apis/ollama';
import { copyToClipboard, splitStream } from '$lib/utils';
@ -163,36 +162,32 @@
// Scroll down
window.scrollTo({ top: document.body.scrollHeight });
const res = await generateChatCompletion(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
{
model: model,
messages: [
$settings.system
? {
role: 'system',
content: $settings.system
}
: undefined,
...messages
]
.filter((message) => message)
.map((message) => ({
role: message.role,
content: message.content,
...(message.files && {
images: message.files
.filter((file) => file.type === 'image')
.map((file) => file.url.slice(file.url.indexOf(',') + 1))
})
})),
options: {
...($settings.options ?? {})
},
format: $settings.requestFormat ?? undefined
}
);
const res = await generateChatCompletion(localStorage.token, {
model: model,
messages: [
$settings.system
? {
role: 'system',
content: $settings.system
}
: undefined,
...messages
]
.filter((message) => message)
.map((message) => ({
role: message.role,
content: message.content,
...(message.files && {
images: message.files
.filter((file) => file.type === 'image')
.map((file) => file.url.slice(file.url.indexOf(',') + 1))
})
})),
options: {
...($settings.options ?? {})
},
format: $settings.requestFormat ?? undefined
});
if (res && res.ok) {
const reader = res.body
@ -595,7 +590,6 @@
const generateChatTitle = async (_chatId, userPrompt) => {
if ($settings.titleAutoGenerate ?? true) {
const title = await generateTitle(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
$settings?.titleAutoGenerateModel ?? selectedModels[0],
userPrompt

64
src/routes/(app)/c/[id]/+page.svelte
View file

@ -7,7 +7,6 @@
import { page } from '$app/stores';
import { models, modelfiles, user, settings, chats, chatId } from '$lib/stores';
import { OLLAMA_API_BASE_URL } from '$lib/constants';
import { generateChatCompletion, generateTitle } from '$lib/apis/ollama';
import { copyToClipboard, splitStream } from '$lib/utils';
@ -180,36 +179,32 @@
// Scroll down
window.scrollTo({ top: document.body.scrollHeight });
const res = await generateChatCompletion(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
{
model: model,
messages: [
$settings.system
? {
role: 'system',
content: $settings.system
}
: undefined,
...messages
]
.filter((message) => message)
.map((message) => ({
role: message.role,
content: message.content,
...(message.files && {
images: message.files
.filter((file) => file.type === 'image')
.map((file) => file.url.slice(file.url.indexOf(',') + 1))
})
})),
options: {
...($settings.options ?? {})
},
format: $settings.requestFormat ?? undefined
}
);
const res = await generateChatCompletion(localStorage.token, {
model: model,
messages: [
$settings.system
? {
role: 'system',
content: $settings.system
}
: undefined,
...messages
]
.filter((message) => message)
.map((message) => ({
role: message.role,
content: message.content,
...(message.files && {
images: message.files
.filter((file) => file.type === 'image')
.map((file) => file.url.slice(file.url.indexOf(',') + 1))
})
})),
options: {
...($settings.options ?? {})
},
format: $settings.requestFormat ?? undefined
});
if (res && res.ok) {
const reader = res.body
@ -611,12 +606,7 @@
const generateChatTitle = async (_chatId, userPrompt) => {
if ($settings.titleAutoGenerate ?? true) {
const title = await generateTitle(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
selectedModels[0],
userPrompt
);
const title = await generateTitle(localStorage.token, selectedModels[0], userPrompt);
if (title) {
await setChatTitle(_chatId, title);

7
src/routes/(app)/modelfiles/+page.svelte
View file

@ -6,7 +6,6 @@
import { onMount } from 'svelte';
import { modelfiles, settings, user } from '$lib/stores';
import { OLLAMA_API_BASE_URL } from '$lib/constants';
import { createModel, deleteModel } from '$lib/apis/ollama';
import {
createNewModelfile,
@ -20,11 +19,7 @@
const deleteModelHandler = async (tagName) => {
let success = null;
success = await deleteModel(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
tagName
);
success = await deleteModel(localStorage.token, tagName);
if (success) {
toast.success(`Deleted ${tagName}`);

8
src/routes/(app)/modelfiles/create/+page.svelte
View file

@ -2,7 +2,6 @@
import { v4 as uuidv4 } from 'uuid';
import { toast } from 'svelte-french-toast';
import { goto } from '$app/navigation';
import { OLLAMA_API_BASE_URL } from '$lib/constants';
import { settings, user, config, modelfiles, models } from '$lib/stores';
import Advanced from '$lib/components/chat/Settings/Advanced.svelte';
@ -132,12 +131,7 @@ SYSTEM """${system}"""`.replace(/^\s*\n/gm, '');
Object.keys(categories).filter((category) => categories[category]).length > 0 &&
!$models.includes(tagName)
) {
const res = await createModel(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
tagName,
content
);
const res = await createModel(localStorage.token, tagName, content);
if (res) {
const reader = res.body

9
src/routes/(app)/modelfiles/edit/+page.svelte
View file

@ -7,8 +7,6 @@
import { page } from '$app/stores';
import { settings, user, config, modelfiles } from '$lib/stores';
import { OLLAMA_API_BASE_URL } from '$lib/constants';
import { splitStream } from '$lib/utils';
import { createModel } from '$lib/apis/ollama';
@ -104,12 +102,7 @@
content !== '' &&
Object.keys(categories).filter((category) => categories[category]).length > 0
) {
const res = await createModel(
$settings?.API_BASE_URL ?? OLLAMA_API_BASE_URL,
localStorage.token,
tagName,
content
);
const res = await createModel(localStorage.token, tagName, content);
if (res) {
const reader = res.body