Merge pull request #1033 from open-webui/dev

0.1.109
This commit is contained in:
Timothy Jaeryang Baek 2024-03-06 15:40:28 -05:00 committed by GitHub
commit 92e21acb4c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
39 changed files with 1962 additions and 1075 deletions

View file

@ -1,6 +1,6 @@
# Ollama URL for the backend to connect
# The path '/ollama/api' will be redirected to the specified backend URL
OLLAMA_API_BASE_URL='http://localhost:11434/api'
# The path '/ollama' will be redirected to the specified backend URL
OLLAMA_BASE_URL='http://localhost:11434'
OPENAI_API_BASE_URL=''
OPENAI_API_KEY=''

View file

@ -32,7 +32,7 @@ assignees: ''
**Confirmation:**
- [ ] I have read and followed all the instructions provided in the README.md.
- [ ] I have reviewed the troubleshooting.md document.
- [ ] I am on the latest version of both Open WebUI and Ollama.
- [ ] I have included the browser console logs.
- [ ] I have included the Docker container logs.

View file

@ -26,17 +26,27 @@ jobs:
VERSION=$(jq -r '.version' package.json)
echo "::set-output name=version::$VERSION"
- name: Extract latest CHANGELOG entry
id: changelog
run: |
CHANGELOG_CONTENT=$(awk '/^## \[/{n++} n==1' CHANGELOG.md)
echo "CHANGELOG_CONTENT<<EOF"
echo "$CHANGELOG_CONTENT"
echo "EOF"
echo "::set-output name=content::${CHANGELOG_CONTENT}"
- name: Create GitHub release
uses: actions/github-script@v5
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const changelog = `${{ steps.changelog.outputs.content }}`;
const release = await github.rest.repos.createRelease({
owner: context.repo.owner,
repo: context.repo.repo,
tag_name: `v${{ steps.get_version.outputs.version }}`,
name: `v${{ steps.get_version.outputs.version }}`,
body: 'Automatically created new release',
body: changelog,
})
console.log(`Created release ${release.data.html_url}`)

View file

@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.1.109] - 2024-03-06
### Added
- **🔄 Multiple Ollama Servers Support**: Enjoy enhanced scalability and performance with support for multiple Ollama servers in a single WebUI. Load balancing features are now available, providing improved efficiency (#788, #278).
- **🔧 Support for Claude 3 and Gemini**: Responding to user requests, we've expanded our toolset to include Claude 3 and Gemini, offering a wider range of functionalities within our platform (#1064).
- **🔍 OCR Functionality for PDF Loader**: We've augmented our PDF loader with Optical Character Recognition (OCR) capabilities. Now, extract text from scanned documents and images within PDFs, broadening the scope of content processing (#1050).
### Fixed
- **🛠️ RAG Collection**: Implemented a dynamic mechanism to recreate RAG collections, ensuring users have up-to-date and accurate data (#1031).
- **📝 User Agent Headers**: Fixed issue of RAG web requests being sent with empty user_agent headers, reducing rejections from certain websites. Realistic headers are now utilized for these requests (#1024).
- **⏹️ Playground Cancel Functionality**: Introducing a new "Cancel" option for stopping Ollama generation in the Playground, enhancing user control and usability (#1006).
- **🔤 Typographical Error in 'ASSISTANT' Field**: Corrected a typographical error in the 'ASSISTANT' field within the GGUF model upload template for accuracy and consistency (#1061).
### Changed
- **🔄 Refactored Message Deletion Logic**: Streamlined message deletion process for improved efficiency and user experience, simplifying interactions within the platform (#1004).
- **⚠️ Deprecation of `OLLAMA_API_BASE_URL`**: Deprecated `OLLAMA_API_BASE_URL` environment variable; recommend using `OLLAMA_BASE_URL` instead. Refer to our documentation for further details.
## [0.1.108] - 2024-03-02
### Added

View file

@ -20,7 +20,7 @@ FROM python:3.11-slim-bookworm as base
ENV ENV=prod
ENV PORT ""
ENV OLLAMA_API_BASE_URL "/ollama/api"
ENV OLLAMA_BASE_URL "/ollama"
ENV OPENAI_API_BASE_URL ""
ENV OPENAI_API_KEY ""

View file

@ -95,10 +95,10 @@ Don't forget to explore our sibling project, [Open WebUI Community](https://open
- **If Ollama is on a Different Server**, use this command:
- To connect to Ollama on another server, change the `OLLAMA_API_BASE_URL` to the server's URL:
- To connect to Ollama on another server, change the `OLLAMA_BASE_URL` to the server's URL:
```bash
docker run -d -p 3000:8080 -e OLLAMA_API_BASE_URL=https://example.com/api -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=https://example.com -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```
- After installation, you can access Open WebUI at [http://localhost:3000](http://localhost:3000). Enjoy! 😄
@ -110,7 +110,7 @@ If you're experiencing connection issues, its often due to the WebUI docker c
**Example Docker Command**:
```bash
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_API_BASE_URL=http://127.0.0.1:11434/api --name open-webui --restart always ghcr.io/open-webui/open-webui:main
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```
### Other Installation Methods

View file

@ -4,7 +4,7 @@
The Open WebUI system is designed to streamline interactions between the client (your browser) and the Ollama API. At the heart of this design is a backend reverse proxy, enhancing security and resolving CORS issues.
- **How it Works**: The Open WebUI is designed to interact with the Ollama API through a specific route. When a request is made from the WebUI to Ollama, it is not directly sent to the Ollama API. Initially, the request is sent to the Open WebUI backend via `/ollama/api` route. From there, the backend is responsible for forwarding the request to the Ollama API. This forwarding is accomplished by using the route specified in the `OLLAMA_API_BASE_URL` environment variable. Therefore, a request made to `/ollama/api` in the WebUI is effectively the same as making a request to `OLLAMA_API_BASE_URL` in the backend. For instance, a request to `/ollama/api/tags` in the WebUI is equivalent to `OLLAMA_API_BASE_URL/tags` in the backend.
- **How it Works**: The Open WebUI is designed to interact with the Ollama API through a specific route. When a request is made from the WebUI to Ollama, it is not directly sent to the Ollama API. Initially, the request is sent to the Open WebUI backend via `/ollama` route. From there, the backend is responsible for forwarding the request to the Ollama API. This forwarding is accomplished by using the route specified in the `OLLAMA_BASE_URL` environment variable. Therefore, a request made to `/ollama` in the WebUI is effectively the same as making a request to `OLLAMA_BASE_URL` in the backend. For instance, a request to `/ollama/api/tags` in the WebUI is equivalent to `OLLAMA_BASE_URL/api/tags` in the backend.
- **Security Benefits**: This design prevents direct exposure of the Ollama API to the frontend, safeguarding against potential CORS (Cross-Origin Resource Sharing) issues and unauthorized access. Requiring authentication to access the Ollama API further enhances this security layer.
@ -15,7 +15,7 @@ If you're experiencing connection issues, its often due to the WebUI docker c
**Example Docker Command**:
```bash
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_API_BASE_URL=http://127.0.0.1:11434/api --name open-webui --restart always ghcr.io/open-webui/open-webui:main
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```
### General Connection Errors
@ -25,8 +25,8 @@ docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_API_BASE_
**Troubleshooting Steps**:
1. **Verify Ollama URL Format**:
- When running the Web UI container, ensure the `OLLAMA_API_BASE_URL` is correctly set, including the `/api` suffix. (e.g., `http://192.168.1.1:11434/api` for different host setups).
- When running the Web UI container, ensure the `OLLAMA_BASE_URL` is correctly set. (e.g., `http://192.168.1.1:11434` for different host setups).
- In the Open WebUI, navigate to "Settings" > "General".
- Confirm that the Ollama Server URL is correctly set to `[OLLAMA URL]/api` (e.g., `http://localhost:11434/api`), including the `/api` suffix.
- Confirm that the Ollama Server URL is correctly set to `[OLLAMA URL]` (e.g., `http://localhost:11434`).
By following these enhanced troubleshooting steps, connection issues should be effectively resolved. For further assistance or queries, feel free to reach out to us on our community Discord.

View file

@ -3,15 +3,22 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
from fastapi.concurrency import run_in_threadpool
from pydantic import BaseModel, ConfigDict
import random
import requests
import json
import uuid
from pydantic import BaseModel
import aiohttp
import asyncio
from apps.web.models.users import Users
from constants import ERROR_MESSAGES
from utils.utils import decode_token, get_current_user, get_admin_user
from config import OLLAMA_BASE_URL, WEBUI_AUTH
from config import OLLAMA_BASE_URLS
from typing import Optional, List, Union
app = FastAPI()
app.add_middleware(
@ -22,27 +29,44 @@ app.add_middleware(
allow_headers=["*"],
)
app.state.OLLAMA_BASE_URL = OLLAMA_BASE_URL
# TARGET_SERVER_URL = OLLAMA_API_BASE_URL
app.state.OLLAMA_BASE_URLS = OLLAMA_BASE_URLS
app.state.MODELS = {}
REQUEST_POOL = []
@app.get("/url")
async def get_ollama_api_url(user=Depends(get_admin_user)):
return {"OLLAMA_BASE_URL": app.state.OLLAMA_BASE_URL}
# TODO: Implement a more intelligent load balancing mechanism for distributing requests among multiple backend instances.
# Current implementation uses a simple round-robin approach (random.choice). Consider incorporating algorithms like weighted round-robin,
# least connections, or least response time for better resource utilization and performance optimization.
@app.middleware("http")
async def check_url(request: Request, call_next):
if len(app.state.MODELS) == 0:
await get_all_models()
else:
pass
response = await call_next(request)
return response
@app.get("/urls")
async def get_ollama_api_urls(user=Depends(get_admin_user)):
return {"OLLAMA_BASE_URLS": app.state.OLLAMA_BASE_URLS}
class UrlUpdateForm(BaseModel):
url: str
urls: List[str]
@app.post("/url/update")
@app.post("/urls/update")
async def update_ollama_api_url(form_data: UrlUpdateForm, user=Depends(get_admin_user)):
app.state.OLLAMA_BASE_URL = form_data.url
return {"OLLAMA_BASE_URL": app.state.OLLAMA_BASE_URL}
app.state.OLLAMA_BASE_URLS = form_data.urls
print(app.state.OLLAMA_BASE_URLS)
return {"OLLAMA_BASE_URLS": app.state.OLLAMA_BASE_URLS}
@app.get("/cancel/{request_id}")
@ -55,9 +79,806 @@ async def cancel_ollama_request(request_id: str, user=Depends(get_current_user))
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
async def fetch_url(url):
try:
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.json()
except Exception as e:
# Handle connection error here
print(f"Connection error: {e}")
return None
def merge_models_lists(model_lists):
merged_models = {}
for idx, model_list in enumerate(model_lists):
for model in model_list:
digest = model["digest"]
if digest not in merged_models:
model["urls"] = [idx]
merged_models[digest] = model
else:
merged_models[digest]["urls"].append(idx)
return list(merged_models.values())
# user=Depends(get_current_user)
async def get_all_models():
print("get_all_models")
tasks = [fetch_url(f"{url}/api/tags") for url in app.state.OLLAMA_BASE_URLS]
responses = await asyncio.gather(*tasks)
responses = list(filter(lambda x: x is not None, responses))
models = {
"models": merge_models_lists(
map(lambda response: response["models"], responses)
)
}
app.state.MODELS = {model["model"]: model for model in models["models"]}
return models
@app.get("/api/tags")
@app.get("/api/tags/{url_idx}")
async def get_ollama_tags(
url_idx: Optional[int] = None, user=Depends(get_current_user)
):
if url_idx == None:
return await get_all_models()
else:
url = app.state.OLLAMA_BASE_URLS[url_idx]
try:
r = requests.request(method="GET", url=f"{url}/api/tags")
r.raise_for_status()
return r.json()
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
@app.get("/api/version")
@app.get("/api/version/{url_idx}")
async def get_ollama_versions(url_idx: Optional[int] = None):
if url_idx == None:
# returns lowest version
tasks = [fetch_url(f"{url}/api/version") for url in app.state.OLLAMA_BASE_URLS]
responses = await asyncio.gather(*tasks)
responses = list(filter(lambda x: x is not None, responses))
lowest_version = min(
responses, key=lambda x: tuple(map(int, x["version"].split(".")))
)
return {"version": lowest_version["version"]}
else:
url = app.state.OLLAMA_BASE_URLS[url_idx]
try:
r = requests.request(method="GET", url=f"{url}/api/version")
r.raise_for_status()
return r.json()
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class ModelNameForm(BaseModel):
name: str
@app.post("/api/pull")
@app.post("/api/pull/{url_idx}")
async def pull_model(
form_data: ModelNameForm, url_idx: int = 0, user=Depends(get_admin_user)
):
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
r = None
def get_request():
nonlocal url
nonlocal r
try:
def stream_content():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
r = requests.request(
method="POST",
url=f"{url}/api/pull",
data=form_data.model_dump_json(exclude_none=True),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class PushModelForm(BaseModel):
name: str
insecure: Optional[bool] = None
stream: Optional[bool] = None
@app.delete("/api/push")
@app.delete("/api/push/{url_idx}")
async def push_model(
form_data: PushModelForm,
url_idx: Optional[int] = None,
user=Depends(get_admin_user),
):
if url_idx == None:
if form_data.name in app.state.MODELS:
url_idx = app.state.MODELS[form_data.name]["urls"][0]
else:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.name),
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
r = None
def get_request():
nonlocal url
nonlocal r
try:
def stream_content():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
r = requests.request(
method="POST",
url=f"{url}/api/push",
data=form_data.model_dump_json(exclude_none=True),
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class CreateModelForm(BaseModel):
name: str
modelfile: Optional[str] = None
stream: Optional[bool] = None
path: Optional[str] = None
@app.post("/api/create")
@app.post("/api/create/{url_idx}")
async def create_model(
form_data: CreateModelForm, url_idx: int = 0, user=Depends(get_admin_user)
):
print(form_data)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
r = None
def get_request():
nonlocal url
nonlocal r
try:
def stream_content():
for chunk in r.iter_content(chunk_size=8192):
yield chunk
r = requests.request(
method="POST",
url=f"{url}/api/create",
data=form_data.model_dump_json(exclude_none=True),
stream=True,
)
r.raise_for_status()
print(r)
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class CopyModelForm(BaseModel):
source: str
destination: str
@app.post("/api/copy")
@app.post("/api/copy/{url_idx}")
async def copy_model(
form_data: CopyModelForm,
url_idx: Optional[int] = None,
user=Depends(get_admin_user),
):
if url_idx == None:
if form_data.source in app.state.MODELS:
url_idx = app.state.MODELS[form_data.source]["urls"][0]
else:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.source),
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
try:
r = requests.request(
method="POST",
url=f"{url}/api/copy",
data=form_data.model_dump_json(exclude_none=True),
)
r.raise_for_status()
print(r.text)
return True
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
@app.delete("/api/delete")
@app.delete("/api/delete/{url_idx}")
async def delete_model(
form_data: ModelNameForm,
url_idx: Optional[int] = None,
user=Depends(get_admin_user),
):
if url_idx == None:
if form_data.name in app.state.MODELS:
url_idx = app.state.MODELS[form_data.name]["urls"][0]
else:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.name),
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
try:
r = requests.request(
method="DELETE",
url=f"{url}/api/delete",
data=form_data.model_dump_json(exclude_none=True),
)
r.raise_for_status()
print(r.text)
return True
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
@app.post("/api/show")
async def show_model_info(form_data: ModelNameForm, user=Depends(get_current_user)):
if form_data.name not in app.state.MODELS:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.name),
)
url_idx = random.choice(app.state.MODELS[form_data.name]["urls"])
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
try:
r = requests.request(
method="POST",
url=f"{url}/api/show",
data=form_data.model_dump_json(exclude_none=True),
)
r.raise_for_status()
return r.json()
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class GenerateEmbeddingsForm(BaseModel):
model: str
prompt: str
options: Optional[dict] = None
keep_alive: Optional[Union[int, str]] = None
@app.post("/api/embeddings")
@app.post("/api/embeddings/{url_idx}")
async def generate_embeddings(
form_data: GenerateEmbeddingsForm,
url_idx: Optional[int] = None,
user=Depends(get_current_user),
):
if url_idx == None:
if form_data.model in app.state.MODELS:
url_idx = random.choice(app.state.MODELS[form_data.model]["urls"])
else:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.model),
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
try:
r = requests.request(
method="POST",
url=f"{url}/api/embeddings",
data=form_data.model_dump_json(exclude_none=True),
)
r.raise_for_status()
return r.json()
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class GenerateCompletionForm(BaseModel):
model: str
prompt: str
images: Optional[List[str]] = None
format: Optional[str] = None
options: Optional[dict] = None
system: Optional[str] = None
template: Optional[str] = None
context: Optional[str] = None
stream: Optional[bool] = True
raw: Optional[bool] = None
keep_alive: Optional[Union[int, str]] = None
@app.post("/api/generate")
@app.post("/api/generate/{url_idx}")
async def generate_completion(
form_data: GenerateCompletionForm,
url_idx: Optional[int] = None,
user=Depends(get_current_user),
):
if url_idx == None:
if form_data.model in app.state.MODELS:
url_idx = random.choice(app.state.MODELS[form_data.model]["urls"])
else:
raise HTTPException(
status_code=400,
detail="error_detail",
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
r = None
def get_request():
nonlocal form_data
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
if form_data.stream:
yield json.dumps({"id": request_id, "done": False}) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
print("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/api/generate",
data=form_data.model_dump_json(exclude_none=True),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
class ChatMessage(BaseModel):
role: str
content: str
images: Optional[List[str]] = None
class GenerateChatCompletionForm(BaseModel):
model: str
messages: List[ChatMessage]
format: Optional[str] = None
options: Optional[dict] = None
template: Optional[str] = None
stream: Optional[bool] = True
keep_alive: Optional[Union[int, str]] = None
@app.post("/api/chat")
@app.post("/api/chat/{url_idx}")
async def generate_chat_completion(
form_data: GenerateChatCompletionForm,
url_idx: Optional[int] = None,
user=Depends(get_current_user),
):
if url_idx == None:
if form_data.model in app.state.MODELS:
url_idx = random.choice(app.state.MODELS[form_data.model]["urls"])
else:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.model),
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
r = None
print(form_data.model_dump_json(exclude_none=True))
def get_request():
nonlocal form_data
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
if form_data.stream:
yield json.dumps({"id": request_id, "done": False}) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
print("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/api/chat",
data=form_data.model_dump_json(exclude_none=True),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
# TODO: we should update this part once Ollama supports other types
class OpenAIChatMessage(BaseModel):
role: str
content: str
model_config = ConfigDict(extra="allow")
class OpenAIChatCompletionForm(BaseModel):
model: str
messages: List[OpenAIChatMessage]
model_config = ConfigDict(extra="allow")
@app.post("/v1/chat/completions")
@app.post("/v1/chat/completions/{url_idx}")
async def generate_openai_chat_completion(
form_data: OpenAIChatCompletionForm,
url_idx: Optional[int] = None,
user=Depends(get_current_user),
):
if url_idx == None:
if form_data.model in app.state.MODELS:
url_idx = random.choice(app.state.MODELS[form_data.model]["urls"])
else:
raise HTTPException(
status_code=400,
detail=ERROR_MESSAGES.MODEL_NOT_FOUND(form_data.model),
)
url = app.state.OLLAMA_BASE_URLS[url_idx]
print(url)
r = None
def get_request():
nonlocal form_data
nonlocal r
request_id = str(uuid.uuid4())
try:
REQUEST_POOL.append(request_id)
def stream_content():
try:
if form_data.stream:
yield json.dumps(
{"request_id": request_id, "done": False}
) + "\n"
for chunk in r.iter_content(chunk_size=8192):
if request_id in REQUEST_POOL:
yield chunk
else:
print("User: canceled request")
break
finally:
if hasattr(r, "close"):
r.close()
if request_id in REQUEST_POOL:
REQUEST_POOL.remove(request_id)
r = requests.request(
method="POST",
url=f"{url}/v1/chat/completions",
data=form_data.model_dump_json(exclude_none=True),
stream=True,
)
r.raise_for_status()
return StreamingResponse(
stream_content(),
status_code=r.status_code,
headers=dict(r.headers),
)
except Exception as e:
raise e
try:
return await run_in_threadpool(get_request)
except Exception as e:
error_detail = "Open WebUI: Server Connection Error"
if r is not None:
try:
res = r.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
raise HTTPException(
status_code=r.status_code if r else 500,
detail=error_detail,
)
@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
async def proxy(path: str, request: Request, user=Depends(get_current_user)):
target_url = f"{app.state.OLLAMA_BASE_URL}/{path}"
async def deprecated_proxy(path: str, request: Request, user=Depends(get_current_user)):
url = app.state.OLLAMA_BASE_URLS[0]
target_url = f"{url}/{path}"
body = await request.body()
headers = dict(request.headers)

View file

@ -1,127 +0,0 @@
from fastapi import FastAPI, Request, Response, HTTPException, Depends
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
import requests
import json
from pydantic import BaseModel
from apps.web.models.users import Users
from constants import ERROR_MESSAGES
from utils.utils import decode_token, get_current_user
from config import OLLAMA_API_BASE_URL, WEBUI_AUTH
import aiohttp
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.state.OLLAMA_API_BASE_URL = OLLAMA_API_BASE_URL
# TARGET_SERVER_URL = OLLAMA_API_BASE_URL
@app.get("/url")
async def get_ollama_api_url(user=Depends(get_current_user)):
if user and user.role == "admin":
return {"OLLAMA_API_BASE_URL": app.state.OLLAMA_API_BASE_URL}
else:
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
class UrlUpdateForm(BaseModel):
url: str
@app.post("/url/update")
async def update_ollama_api_url(
form_data: UrlUpdateForm, user=Depends(get_current_user)
):
if user and user.role == "admin":
app.state.OLLAMA_API_BASE_URL = form_data.url
return {"OLLAMA_API_BASE_URL": app.state.OLLAMA_API_BASE_URL}
else:
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
# async def fetch_sse(method, target_url, body, headers):
# async with aiohttp.ClientSession() as session:
# try:
# async with session.request(
# method, target_url, data=body, headers=headers
# ) as response:
# print(response.status)
# async for line in response.content:
# yield line
# except Exception as e:
# print(e)
# error_detail = "Open WebUI: Server Connection Error"
# yield json.dumps({"error": error_detail, "message": str(e)}).encode()
@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
async def proxy(path: str, request: Request, user=Depends(get_current_user)):
target_url = f"{app.state.OLLAMA_API_BASE_URL}/{path}"
print(target_url)
body = await request.body()
headers = dict(request.headers)
if user.role in ["user", "admin"]:
if path in ["pull", "delete", "push", "copy", "create"]:
if user.role != "admin":
raise HTTPException(
status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED
)
else:
raise HTTPException(status_code=401, detail=ERROR_MESSAGES.ACCESS_PROHIBITED)
headers.pop("Host", None)
headers.pop("Authorization", None)
headers.pop("Origin", None)
headers.pop("Referer", None)
session = aiohttp.ClientSession()
response = None
try:
response = await session.request(
request.method, target_url, data=body, headers=headers
)
print(response)
if not response.ok:
data = await response.json()
print(data)
response.raise_for_status()
async def generate():
async for line in response.content:
print(line)
yield line
await session.close()
return StreamingResponse(generate(), response.status)
except Exception as e:
print(e)
error_detail = "Open WebUI: Server Connection Error"
if response is not None:
try:
res = await response.json()
if "error" in res:
error_detail = f"Ollama: {res['error']}"
except:
error_detail = f"Ollama: {e}"
await session.close()
raise HTTPException(
status_code=response.status if response else 500,
detail=error_detail,
)

View file

@ -108,7 +108,7 @@ class StoreWebForm(CollectionNameForm):
url: str
def store_data_in_vector_db(data, collection_name) -> bool:
def store_data_in_vector_db(data, collection_name, overwrite: bool = False) -> bool:
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=app.state.CHUNK_SIZE, chunk_overlap=app.state.CHUNK_OVERLAP
)
@ -118,6 +118,12 @@ def store_data_in_vector_db(data, collection_name) -> bool:
metadatas = [doc.metadata for doc in docs]
try:
if overwrite:
for collection in CHROMA_CLIENT.list_collections():
if collection_name == collection.name:
print(f"deleting existing collection {collection_name}")
CHROMA_CLIENT.delete_collection(name=collection_name)
collection = CHROMA_CLIENT.create_collection(
name=collection_name,
embedding_function=app.state.sentence_transformer_ef,
@ -355,7 +361,7 @@ def store_web(form_data: StoreWebForm, user=Depends(get_current_user)):
if collection_name == "":
collection_name = calculate_sha256_string(form_data.url)[:63]
store_data_in_vector_db(data, collection_name)
store_data_in_vector_db(data, collection_name, overwrite=True)
return {
"status": True,
"collection_name": collection_name,
@ -419,7 +425,7 @@ def get_loader(filename: str, file_content_type: str, file_path: str):
]
if file_ext == "pdf":
loader = PyPDFLoader(file_path)
loader = PyPDFLoader(file_path, extract_images=True)
elif file_ext == "csv":
loader = CSVLoader(file_path)
elif file_ext == "rst":

View file

@ -14,7 +14,7 @@ import json
from utils.utils import get_admin_user
from utils.misc import calculate_sha256, get_gravatar_url
from config import OLLAMA_API_BASE_URL, DATA_DIR, UPLOAD_DIR
from config import OLLAMA_BASE_URLS, DATA_DIR, UPLOAD_DIR
from constants import ERROR_MESSAGES
@ -75,7 +75,7 @@ async def download_file_stream(url, file_path, file_name, chunk_size=1024 * 1024
hashed = calculate_sha256(file)
file.seek(0)
url = f"{OLLAMA_API_BASE_URL}/blobs/sha256:{hashed}"
url = f"{OLLAMA_BASE_URLS[0]}/blobs/sha256:{hashed}"
response = requests.post(url, data=file)
if response.ok:
@ -147,7 +147,7 @@ def upload(file: UploadFile = File(...)):
hashed = calculate_sha256(f)
f.seek(0)
url = f"{OLLAMA_API_BASE_URL}/blobs/sha256:{hashed}"
url = f"{OLLAMA_BASE_URLS[0]}/blobs/sha256:{hashed}"
response = requests.post(url, data=f)
if response.ok:

View file

@ -200,27 +200,32 @@ if not os.path.exists(LITELLM_CONFIG_PATH):
####################################
# OLLAMA_API_BASE_URL
# OLLAMA_BASE_URL
####################################
OLLAMA_API_BASE_URL = os.environ.get(
"OLLAMA_API_BASE_URL", "http://localhost:11434/api"
)
if ENV == "prod":
if OLLAMA_API_BASE_URL == "/ollama/api":
OLLAMA_API_BASE_URL = "http://host.docker.internal:11434/api"
OLLAMA_BASE_URL = os.environ.get("OLLAMA_BASE_URL", "")
if OLLAMA_BASE_URL == "":
if ENV == "prod":
if OLLAMA_BASE_URL == "/ollama":
OLLAMA_BASE_URL = "http://host.docker.internal:11434"
if OLLAMA_BASE_URL == "" and OLLAMA_API_BASE_URL != "":
OLLAMA_BASE_URL = (
OLLAMA_API_BASE_URL[:-4]
if OLLAMA_API_BASE_URL.endswith("/api")
else OLLAMA_API_BASE_URL
)
OLLAMA_BASE_URLS = os.environ.get("OLLAMA_BASE_URLS", "")
OLLAMA_BASE_URLS = OLLAMA_BASE_URLS if OLLAMA_BASE_URLS != "" else OLLAMA_BASE_URL
OLLAMA_BASE_URLS = [url.strip() for url in OLLAMA_BASE_URLS.split(",")]
####################################
# OPENAI_API

View file

@ -48,3 +48,5 @@ class ERROR_MESSAGES(str, Enum):
lambda err="": f"Invalid format. Please use the correct format{err if err else ''}"
)
RATE_LIMIT_EXCEEDED = "API rate limit exceeded"
MODEL_NOT_FOUND = lambda name="": f"Model '{name}' was not found"

View file

@ -104,7 +104,7 @@ async def auth_middleware(request: Request, call_next):
app.mount("/api/v1", webui_app)
app.mount("/litellm/api", litellm_app)
app.mount("/ollama/api", ollama_app)
app.mount("/ollama", ollama_app)
app.mount("/openai/api", openai_app)
app.mount("/images/api/v1", images_app)
@ -125,6 +125,14 @@ async def get_app_config():
}
@app.get("/api/version")
async def get_app_config():
return {
"version": VERSION,
}
@app.get("/api/changelog")
async def get_app_changelog():
return CHANGELOG

View file

@ -22,6 +22,7 @@ google-generativeai
langchain
langchain-community
fake_useragent
chromadb
sentence_transformers
pypdf
@ -33,6 +34,7 @@ pandas
openpyxl
pyxlsb
xlrd
rapidocr-onnxruntime
faster-whisper

View file

@ -14,7 +14,7 @@ services:
build:
context: .
args:
OLLAMA_API_BASE_URL: '/ollama/api'
OLLAMA_BASE_URL: '/ollama'
dockerfile: Dockerfile
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
@ -25,7 +25,7 @@ services:
ports:
- ${OPEN_WEBUI_PORT-3000}:8080
environment:
- 'OLLAMA_API_BASE_URL=http://ollama:11434/api'
- 'OLLAMA_BASE_URL=http://ollama:11434'
- 'WEBUI_SECRET_KEY='
extra_hosts:
- host.docker.internal:host-gateway

View file

@ -40,7 +40,7 @@ spec:
- name: data
mountPath: /app/backend/data
env:
- name: OLLAMA_API_BASE_URL
- name: OLLAMA_BASE_URL
value: {{ include "ollama.url" . | quote }}
tty: true
{{- with .Values.webui.nodeSelector }}

View file

@ -26,8 +26,8 @@ spec:
cpu: "1000m"
memory: "1Gi"
env:
- name: OLLAMA_API_BASE_URL
value: "http://ollama-service.open-webui.svc.cluster.local:11434/api"
- name: OLLAMA_BASE_URL
value: "http://ollama-service.open-webui.svc.cluster.local:11434"
tty: true
volumeMounts:
- name: webui-volume

View file

@ -1,6 +1,6 @@
{
"name": "open-webui",
"version": "0.1.108",
"version": "0.1.109",
"private": true,
"scripts": {
"dev": "vite dev --host",

View file

@ -1,9 +1,9 @@
import { OLLAMA_API_BASE_URL } from '$lib/constants';
export const getOllamaAPIUrl = async (token: string = '') => {
export const getOllamaUrls = async (token: string = '') => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/url`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/urls`, {
method: 'GET',
headers: {
Accept: 'application/json',
@ -29,13 +29,13 @@ export const getOllamaAPIUrl = async (token: string = '') => {
throw error;
}
return res.OLLAMA_BASE_URL;
return res.OLLAMA_BASE_URLS;
};
export const updateOllamaAPIUrl = async (token: string = '', url: string) => {
export const updateOllamaUrls = async (token: string = '', urls: string[]) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/url/update`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/urls/update`, {
method: 'POST',
headers: {
Accept: 'application/json',
@ -43,7 +43,7 @@ export const updateOllamaAPIUrl = async (token: string = '', url: string) => {
...(token && { authorization: `Bearer ${token}` })
},
body: JSON.stringify({
url: url
urls: urls
})
})
.then(async (res) => {
@ -64,7 +64,7 @@ export const updateOllamaAPIUrl = async (token: string = '', url: string) => {
throw error;
}
return res.OLLAMA_BASE_URL;
return res.OLLAMA_BASE_URLS;
};
export const getOllamaVersion = async (token: string = '') => {
@ -151,7 +151,8 @@ export const generateTitle = async (
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/generate`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
@ -189,7 +190,8 @@ export const generatePrompt = async (token: string = '', model: string, conversa
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/generate`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
@ -223,7 +225,8 @@ export const generateTextCompletion = async (token: string = '', model: string,
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/generate`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
@ -251,7 +254,8 @@ export const generateChatCompletion = async (token: string = '', body: object) =
signal: controller.signal,
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify(body)
@ -294,7 +298,8 @@ export const createModel = async (token: string, tagName: string, content: strin
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/create`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
@ -313,19 +318,23 @@ export const createModel = async (token: string, tagName: string, content: strin
return res;
};
export const deleteModel = async (token: string, tagName: string) => {
export const deleteModel = async (token: string, tagName: string, urlIdx: string | null = null) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/delete`, {
method: 'DELETE',
headers: {
'Content-Type': 'text/event-stream',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
name: tagName
})
})
const res = await fetch(
`${OLLAMA_API_BASE_URL}/api/delete${urlIdx !== null ? `/${urlIdx}` : ''}`,
{
method: 'DELETE',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({
name: tagName
})
}
)
.then(async (res) => {
if (!res.ok) throw await res.json();
return res.json();
@ -336,7 +345,12 @@ export const deleteModel = async (token: string, tagName: string) => {
})
.catch((err) => {
console.log(err);
error = err.error;
error = err;
if ('detail' in err) {
error = err.detail;
}
return null;
});
@ -347,13 +361,14 @@ export const deleteModel = async (token: string, tagName: string) => {
return res;
};
export const pullModel = async (token: string, tagName: string) => {
export const pullModel = async (token: string, tagName: string, urlIdx: string | null = null) => {
let error = null;
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/pull`, {
const res = await fetch(`${OLLAMA_API_BASE_URL}/api/pull${urlIdx !== null ? `/${urlIdx}` : ''}`, {
method: 'POST',
headers: {
'Content-Type': 'text/event-stream',
Accept: 'application/json',
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify({

View file

@ -19,7 +19,7 @@
export let suggestionPrompts = [];
export let autoScroll = true;
let chatTextAreaElement:HTMLTextAreaElement
let filesInputElement;
let promptsElement;
@ -43,11 +43,9 @@
let speechRecognition;
$: if (prompt) {
const chatInput = document.getElementById('chat-textarea');
if (chatInput) {
chatInput.style.height = '';
chatInput.style.height = Math.min(chatInput.scrollHeight, 200) + 'px';
if (chatTextAreaElement) {
chatTextAreaElement.style.height = '';
chatTextAreaElement.style.height = Math.min(chatTextAreaElement.scrollHeight, 200) + 'px';
}
}
@ -86,9 +84,7 @@
if (res) {
prompt = res.text;
await tick();
const inputElement = document.getElementById('chat-textarea');
inputElement?.focus();
chatTextAreaElement?.focus();
if (prompt !== '' && $settings?.speechAutoSend === true) {
submitPrompt(prompt, user);
@ -191,8 +187,7 @@
prompt = `${prompt}${transcript}`;
await tick();
const inputElement = document.getElementById('chat-textarea');
inputElement?.focus();
chatTextAreaElement?.focus();
// Restart the inactivity timeout
timeoutId = setTimeout(() => {
@ -294,8 +289,7 @@
};
onMount(() => {
const chatInput = document.getElementById('chat-textarea');
window.setTimeout(() => chatInput?.focus(), 0);
window.setTimeout(() => chatTextAreaElement?.focus(), 0);
const dropZone = document.querySelector('body');
@ -663,6 +657,7 @@
<textarea
id="chat-textarea"
bind:this={chatTextAreaElement}
class=" dark:bg-gray-900 dark:text-gray-100 outline-none w-full py-3 px-3 {fileUploadEnabled
? ''
: ' pl-4'} rounded-xl resize-none h-[48px]"

View file

@ -223,33 +223,80 @@
}, 100);
};
// TODO: change delete behaviour
// const deleteMessageAndDescendants = async (messageId: string) => {
// if (history.messages[messageId]) {
// history.messages[messageId].deleted = true;
// for (const childId of history.messages[messageId].childrenIds) {
// await deleteMessageAndDescendants(childId);
// }
// }
// };
// const triggerDeleteMessageRecursive = async (messageId: string) => {
// await deleteMessageAndDescendants(messageId);
// await updateChatById(localStorage.token, chatId, { history });
// await chats.set(await getChatList(localStorage.token));
// };
const messageDeleteHandler = async (messageId) => {
if (history.messages[messageId]) {
history.messages[messageId].deleted = true;
for (const childId of history.messages[messageId].childrenIds) {
history.messages[childId].deleted = true;
const messageToDelete = history.messages[messageId];
const messageParentId = messageToDelete.parentId;
const messageChildrenIds = messageToDelete.childrenIds ?? [];
const hasSibling = messageChildrenIds.some(
(childId) => history.messages[childId]?.childrenIds?.length > 0
);
messageChildrenIds.forEach((childId) => {
const child = history.messages[childId];
if (child && child.childrenIds) {
if (child.childrenIds.length === 0 && !hasSibling) {
// if last prompt/response pair
history.messages[messageParentId].childrenIds = [];
history.currentId = messageParentId;
} else {
child.childrenIds.forEach((grandChildId) => {
if (history.messages[grandChildId]) {
history.messages[grandChildId].parentId = messageParentId;
history.messages[messageParentId].childrenIds.push(grandChildId);
}
});
}
}
}
await updateChatById(localStorage.token, chatId, { history });
// remove response
history.messages[messageParentId].childrenIds = history.messages[
messageParentId
].childrenIds.filter((id) => id !== childId);
});
// remove prompt
history.messages[messageParentId].childrenIds = history.messages[
messageParentId
].childrenIds.filter((id) => id !== messageId);
await updateChatById(localStorage.token, chatId, {
messages: messages,
history: history
});
};
// const messageDeleteHandler = async (messageId) => {
// const message = history.messages[messageId];
// const parentId = message.parentId;
// const childrenIds = message.childrenIds ?? [];
// const grandchildrenIds = [];
// // Iterate through childrenIds to find grandchildrenIds
// for (const childId of childrenIds) {
// const childMessage = history.messages[childId];
// const grandChildrenIds = childMessage.childrenIds ?? [];
// for (const grandchildId of grandchildrenIds) {
// const childMessage = history.messages[grandchildId];
// childMessage.parentId = parentId;
// }
// grandchildrenIds.push(...grandChildrenIds);
// }
// history.messages[parentId].childrenIds.push(...grandchildrenIds);
// history.messages[parentId].childrenIds = history.messages[parentId].childrenIds.filter(
// (id) => id !== messageId
// );
// // Select latest message
// let currentMessageId = grandchildrenIds.at(-1);
// if (currentMessageId) {
// let messageChildrenIds = history.messages[currentMessageId].childrenIds;
// while (messageChildrenIds.length !== 0) {
// currentMessageId = messageChildrenIds.at(-1);
// messageChildrenIds = history.messages[currentMessageId].childrenIds;
// }
// history.currentId = currentMessageId;
// }
// await updateChatById(localStorage.token, chatId, { messages, history });
// };
</script>
{#if messages.length == 0}
@ -258,57 +305,55 @@
<div class=" pb-10">
{#key chatId}
{#each messages as message, messageIdx}
{#if !message.deleted}
<div class=" w-full">
<div
class="flex flex-col justify-between px-5 mb-3 {$settings?.fullScreenMode ?? null
? 'max-w-full'
: 'max-w-3xl'} mx-auto rounded-lg group"
>
{#if message.role === 'user'}
<UserMessage
on:delete={() => messageDeleteHandler(message.id)}
user={$user}
{message}
isFirstMessage={messageIdx === 0}
siblings={message.parentId !== null
? history.messages[message.parentId]?.childrenIds ?? []
: Object.values(history.messages)
.filter((message) => message.parentId === null)
.map((message) => message.id) ?? []}
{confirmEditMessage}
{showPreviousMessage}
{showNextMessage}
{copyToClipboard}
/>
{:else}
<ResponseMessage
{message}
modelfiles={selectedModelfiles}
siblings={history.messages[message.parentId]?.childrenIds ?? []}
isLastMessage={messageIdx + 1 === messages.length}
{confirmEditResponseMessage}
{showPreviousMessage}
{showNextMessage}
{rateMessage}
{copyToClipboard}
{continueGeneration}
{regenerateResponse}
on:save={async (e) => {
console.log('save', e);
<div class=" w-full">
<div
class="flex flex-col justify-between px-5 mb-3 {$settings?.fullScreenMode ?? null
? 'max-w-full'
: 'max-w-3xl'} mx-auto rounded-lg group"
>
{#if message.role === 'user'}
<UserMessage
on:delete={() => messageDeleteHandler(message.id)}
user={$user}
{message}
isFirstMessage={messageIdx === 0}
siblings={message.parentId !== null
? history.messages[message.parentId]?.childrenIds ?? []
: Object.values(history.messages)
.filter((message) => message.parentId === null)
.map((message) => message.id) ?? []}
{confirmEditMessage}
{showPreviousMessage}
{showNextMessage}
{copyToClipboard}
/>
{:else}
<ResponseMessage
{message}
modelfiles={selectedModelfiles}
siblings={history.messages[message.parentId]?.childrenIds ?? []}
isLastMessage={messageIdx + 1 === messages.length}
{confirmEditResponseMessage}
{showPreviousMessage}
{showNextMessage}
{rateMessage}
{copyToClipboard}
{continueGeneration}
{regenerateResponse}
on:save={async (e) => {
console.log('save', e);
const message = e.detail;
history.messages[message.id] = message;
await updateChatById(localStorage.token, chatId, {
messages: messages,
history: history
});
}}
/>
{/if}
</div>
const message = e.detail;
history.messages[message.id] = message;
await updateChatById(localStorage.token, chatId, {
messages: messages,
history: history
});
}}
/>
{/if}
</div>
{/if}
</div>
{/each}
{#if bottomPadding}

View file

@ -22,6 +22,7 @@
import CodeBlock from './CodeBlock.svelte';
import Image from '$lib/components/common/Image.svelte';
import { WEBUI_BASE_URL } from '$lib/constants';
import Tooltip from '$lib/components/common/Tooltip.svelte';
export let modelfiles = [];
export let message;
@ -40,7 +41,7 @@
let edit = false;
let editedContent = '';
let editTextAreaElement: HTMLTextAreaElement;
let tooltipInstance = null;
let sentencesAudio = {};
@ -247,10 +248,9 @@
editedContent = message.content;
await tick();
const editElement = document.getElementById(`message-edit-${message.id}`);
editElement.style.height = '';
editElement.style.height = `${editElement.scrollHeight}px`;
editTextAreaElement.style.height = '';
editTextAreaElement.style.height = `${editTextAreaElement.scrollHeight}px`;
};
const editMessageConfirmHandler = async () => {
@ -341,9 +341,11 @@
<div class=" w-full">
<textarea
id="message-edit-{message.id}"
bind:this={editTextAreaElement}
class=" bg-transparent outline-none w-full resize-none"
bind:value={editedContent}
on:input={(e) => {
e.target.style.height = '';
e.target.style.height = `${e.target.scrollHeight}px`;
}}
/>
@ -462,189 +464,125 @@
</div>
{/if}
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
editMessageHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M16.862 4.487l1.687-1.688a1.875 1.875 0 112.652 2.652L6.832 19.82a4.5 4.5 0 01-1.897 1.13l-2.685.8.8-2.685a4.5 4.5 0 011.13-1.897L16.863 4.487zm0 0L19.5 7.125"
/>
</svg>
</button>
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition copy-response-button"
on:click={() => {
copyToClipboard(message.content);
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M15.666 3.888A2.25 2.25 0 0013.5 2.25h-3c-1.03 0-1.9.693-2.166 1.638m7.332 0c.055.194.084.4.084.612v0a.75.75 0 01-.75.75H9a.75.75 0 01-.75-.75v0c0-.212.03-.418.084-.612m7.332 0c.646.049 1.288.11 1.927.184 1.1.128 1.907 1.077 1.907 2.185V19.5a2.25 2.25 0 01-2.25 2.25H6.75A2.25 2.25 0 014.5 19.5V6.257c0-1.108.806-2.057 1.907-2.185a48.208 48.208 0 011.927-.184"
/>
</svg>
</button>
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded {message.rating === 1
? 'bg-gray-100 dark:bg-gray-800'
: ''} dark:hover:text-white hover:text-black transition"
on:click={() => {
rateMessage(message.id, 1);
}}
>
<svg
stroke="currentColor"
fill="none"
stroke-width="2"
viewBox="0 0 24 24"
stroke-linecap="round"
stroke-linejoin="round"
class="w-4 h-4"
xmlns="http://www.w3.org/2000/svg"
><path
d="M14 9V5a3 3 0 0 0-3-3l-4 9v11h11.28a2 2 0 0 0 2-1.7l1.38-9a2 2 0 0 0-2-2.3zM7 22H4a2 2 0 0 1-2-2v-7a2 2 0 0 1 2-2h3"
/></svg
>
</button>
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded {message.rating === -1
? 'bg-gray-100 dark:bg-gray-800'
: ''} dark:hover:text-white hover:text-black transition"
on:click={() => {
rateMessage(message.id, -1);
}}
>
<svg
stroke="currentColor"
fill="none"
stroke-width="2"
viewBox="0 0 24 24"
stroke-linecap="round"
stroke-linejoin="round"
class="w-4 h-4"
xmlns="http://www.w3.org/2000/svg"
><path
d="M10 15v4a3 3 0 0 0 3 3l4-9V2H5.72a2 2 0 0 0-2 1.7l-1.38 9a2 2 0 0 0 2 2.3zm7-13h2.67A2.31 2.31 0 0 1 22 4v7a2.31 2.31 0 0 1-2.33 2H17"
/></svg
>
</button>
<button
id="speak-button-{message.id}"
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
if (!loadingSpeech) {
toggleSpeakMessage(message);
}
}}
>
{#if loadingSpeech}
<svg
class=" w-4 h-4"
fill="currentColor"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_S1WN {
animation: spinner_MGfb 0.8s linear infinite;
animation-delay: -0.8s;
}
.spinner_Km9P {
animation-delay: -0.65s;
}
.spinner_JApP {
animation-delay: -0.5s;
}
@keyframes spinner_MGfb {
93.75%,
100% {
opacity: 0.2;
}
}
</style><circle class="spinner_S1WN" cx="4" cy="12" r="3" /><circle
class="spinner_S1WN spinner_Km9P"
cx="12"
cy="12"
r="3"
/><circle class="spinner_S1WN spinner_JApP" cx="20" cy="12" r="3" /></svg
>
{:else if speaking}
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M17.25 9.75 19.5 12m0 0 2.25 2.25M19.5 12l2.25-2.25M19.5 12l-2.25 2.25m-10.5-6 4.72-4.72a.75.75 0 0 1 1.28.53v15.88a.75.75 0 0 1-1.28.53l-4.72-4.72H4.51c-.88 0-1.704-.507-1.938-1.354A9.009 9.009 0 0 1 2.25 12c0-.83.112-1.633.322-2.396C2.806 8.756 3.63 8.25 4.51 8.25H6.75Z"
/>
</svg>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M19.114 5.636a9 9 0 010 12.728M16.463 8.288a5.25 5.25 0 010 7.424M6.75 8.25l4.72-4.72a.75.75 0 011.28.53v15.88a.75.75 0 01-1.28.53l-4.72-4.72H4.51c-.88 0-1.704-.507-1.938-1.354A9.01 9.01 0 012.25 12c0-.83.112-1.633.322-2.396C2.806 8.756 3.63 8.25 4.51 8.25H6.75z"
/>
</svg>
{/if}
</button>
{#if $config.images}
<Tooltip content="Edit" placement="bottom">
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
if (!generatingImage) {
generateImage(message);
editMessageHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M16.862 4.487l1.687-1.688a1.875 1.875 0 112.652 2.652L6.832 19.82a4.5 4.5 0 01-1.897 1.13l-2.685.8.8-2.685a4.5 4.5 0 011.13-1.897L16.863 4.487zm0 0L19.5 7.125"
/>
</svg>
</button>
</Tooltip>
<Tooltip content="Copy" placement="bottom">
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition copy-response-button"
on:click={() => {
copyToClipboard(message.content);
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M15.666 3.888A2.25 2.25 0 0013.5 2.25h-3c-1.03 0-1.9.693-2.166 1.638m7.332 0c.055.194.084.4.084.612v0a.75.75 0 01-.75.75H9a.75.75 0 01-.75-.75v0c0-.212.03-.418.084-.612m7.332 0c.646.049 1.288.11 1.927.184 1.1.128 1.907 1.077 1.907 2.185V19.5a2.25 2.25 0 01-2.25 2.25H6.75A2.25 2.25 0 014.5 19.5V6.257c0-1.108.806-2.057 1.907-2.185a48.208 48.208 0 011.927-.184"
/>
</svg>
</button>
</Tooltip>
<Tooltip content="Good Response" placement="bottom">
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded {message.rating === 1
? 'bg-gray-100 dark:bg-gray-800'
: ''} dark:hover:text-white hover:text-black transition"
on:click={() => {
rateMessage(message.id, 1);
}}
>
<svg
stroke="currentColor"
fill="none"
stroke-width="2"
viewBox="0 0 24 24"
stroke-linecap="round"
stroke-linejoin="round"
class="w-4 h-4"
xmlns="http://www.w3.org/2000/svg"
><path
d="M14 9V5a3 3 0 0 0-3-3l-4 9v11h11.28a2 2 0 0 0 2-1.7l1.38-9a2 2 0 0 0-2-2.3zM7 22H4a2 2 0 0 1-2-2v-7a2 2 0 0 1 2-2h3"
/></svg
>
</button>
</Tooltip>
<Tooltip content="Bad Response" placement="bottom">
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded {message.rating === -1
? 'bg-gray-100 dark:bg-gray-800'
: ''} dark:hover:text-white hover:text-black transition"
on:click={() => {
rateMessage(message.id, -1);
}}
>
<svg
stroke="currentColor"
fill="none"
stroke-width="2"
viewBox="0 0 24 24"
stroke-linecap="round"
stroke-linejoin="round"
class="w-4 h-4"
xmlns="http://www.w3.org/2000/svg"
><path
d="M10 15v4a3 3 0 0 0 3 3l4-9V2H5.72a2 2 0 0 0-2 1.7l-1.38 9a2 2 0 0 0 2 2.3zm7-13h2.67A2.31 2.31 0 0 1 22 4v7a2.31 2.31 0 0 1-2.33 2H17"
/></svg
>
</button>
</Tooltip>
<Tooltip content="Read Aloud" placement="bottom">
<button
id="speak-button-{message.id}"
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
if (!loadingSpeech) {
toggleSpeakMessage(message);
}
}}
>
{#if generatingImage}
{#if loadingSpeech}
<svg
class=" w-4 h-4"
fill="currentColor"
@ -679,6 +617,21 @@
r="3"
/></svg
>
{:else if speaking}
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M17.25 9.75 19.5 12m0 0 2.25 2.25M19.5 12l2.25-2.25M19.5 12l-2.25 2.25m-10.5-6 4.72-4.72a.75.75 0 0 1 1.28.53v15.88a.75.75 0 0 1-1.28.53l-4.72-4.72H4.51c-.88 0-1.704-.507-1.938-1.354A9.009 9.009 0 0 1 2.25 12c0-.83.112-1.633.322-2.396C2.806 8.756 3.63 8.25 4.51 8.25H6.75Z"
/>
</svg>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
@ -691,93 +644,166 @@
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m2.25 15.75 5.159-5.159a2.25 2.25 0 0 1 3.182 0l5.159 5.159m-1.5-1.5 1.409-1.409a2.25 2.25 0 0 1 3.182 0l2.909 2.909m-18 3.75h16.5a1.5 1.5 0 0 0 1.5-1.5V6a1.5 1.5 0 0 0-1.5-1.5H3.75A1.5 1.5 0 0 0 2.25 6v12a1.5 1.5 0 0 0 1.5 1.5Zm10.5-11.25h.008v.008h-.008V8.25Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0Z"
d="M19.114 5.636a9 9 0 010 12.728M16.463 8.288a5.25 5.25 0 010 7.424M6.75 8.25l4.72-4.72a.75.75 0 011.28.53v15.88a.75.75 0 01-1.28.53l-4.72-4.72H4.51c-.88 0-1.704-.507-1.938-1.354A9.01 9.01 0 012.25 12c0-.83.112-1.633.322-2.396C2.806 8.756 3.63 8.25 4.51 8.25H6.75z"
/>
</svg>
{/if}
</button>
</Tooltip>
{#if $config.images}
<Tooltip content="Generate Image" placement="bottom">
<button
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
if (!generatingImage) {
generateImage(message);
}
}}
>
{#if generatingImage}
<svg
class=" w-4 h-4"
fill="currentColor"
viewBox="0 0 24 24"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_S1WN {
animation: spinner_MGfb 0.8s linear infinite;
animation-delay: -0.8s;
}
.spinner_Km9P {
animation-delay: -0.65s;
}
.spinner_JApP {
animation-delay: -0.5s;
}
@keyframes spinner_MGfb {
93.75%,
100% {
opacity: 0.2;
}
}
</style><circle class="spinner_S1WN" cx="4" cy="12" r="3" /><circle
class="spinner_S1WN spinner_Km9P"
cx="12"
cy="12"
r="3"
/><circle
class="spinner_S1WN spinner_JApP"
cx="20"
cy="12"
r="3"
/></svg
>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m2.25 15.75 5.159-5.159a2.25 2.25 0 0 1 3.182 0l5.159 5.159m-1.5-1.5 1.409-1.409a2.25 2.25 0 0 1 3.182 0l2.909 2.909m-18 3.75h16.5a1.5 1.5 0 0 0 1.5-1.5V6a1.5 1.5 0 0 0-1.5-1.5H3.75A1.5 1.5 0 0 0 2.25 6v12a1.5 1.5 0 0 0 1.5 1.5Zm10.5-11.25h.008v.008h-.008V8.25Zm.375 0a.375.375 0 1 1-.75 0 .375.375 0 0 1 .75 0Z"
/>
</svg>
{/if}
</button>
</Tooltip>
{/if}
{#if message.info}
<button
class=" {isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition whitespace-pre-wrap"
on:click={() => {
console.log(message);
}}
id="info-{message.id}"
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
<Tooltip content="Generation Info" placement="bottom">
<button
class=" {isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition whitespace-pre-wrap"
on:click={() => {
console.log(message);
}}
id="info-{message.id}"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M11.25 11.25l.041-.02a.75.75 0 011.063.852l-.708 2.836a.75.75 0 001.063.853l.041-.021M21 12a9 9 0 11-18 0 9 9 0 0118 0zm-9-3.75h.008v.008H12V8.25z"
/>
</svg>
</button>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M11.25 11.25l.041-.02a.75.75 0 011.063.852l-.708 2.836a.75.75 0 001.063.853l.041-.021M21 12a9 9 0 11-18 0 9 9 0 0118 0zm-9-3.75h.008v.008H12V8.25z"
/>
</svg>
</button>
</Tooltip>
{/if}
{#if isLastMessage}
<button
type="button"
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition regenerate-response-button"
on:click={() => {
continueGeneration();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
<Tooltip content="Continue Response" placement="bottom">
<button
type="button"
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition regenerate-response-button"
on:click={() => {
continueGeneration();
}}
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Z"
/>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M15.91 11.672a.375.375 0 0 1 0 .656l-5.603 3.113a.375.375 0 0 1-.557-.328V8.887c0-.286.307-.466.557-.327l5.603 3.112Z"
/>
</svg>
</button>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Z"
/>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M15.91 11.672a.375.375 0 0 1 0 .656l-5.603 3.113a.375.375 0 0 1-.557-.328V8.887c0-.286.307-.466.557-.327l5.603 3.112Z"
/>
</svg>
</button>
</Tooltip>
<button
type="button"
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition regenerate-response-button"
on:click={regenerateResponse}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
<Tooltip content="Regenerate" placement="bottom">
<button
type="button"
class="{isLastMessage
? 'visible'
: 'invisible group-hover:visible'} p-1 rounded dark:hover:text-white hover:text-black transition regenerate-response-button"
on:click={regenerateResponse}
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0l3.181 3.183a8.25 8.25 0 0013.803-3.7M4.031 9.865a8.25 8.25 0 0113.803-3.7l3.181 3.182m0-4.991v4.99"
/>
</svg>
</button>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0l3.181 3.183a8.25 8.25 0 0013.803-3.7M4.031 9.865a8.25 8.25 0 0113.803-3.7l3.181 3.182m0-4.991v4.99"
/>
</svg>
</button>
</Tooltip>
{/if}
</div>
{/if}

View file

@ -5,6 +5,7 @@
import Name from './Name.svelte';
import ProfileImage from './ProfileImage.svelte';
import { modelfiles, settings } from '$lib/stores';
import Tooltip from '$lib/components/common/Tooltip.svelte';
const dispatch = createEventDispatcher();
@ -20,18 +21,17 @@
let edit = false;
let editedContent = '';
let messageEditTextAreaElement: HTMLTextAreaElement;
const editMessageHandler = async () => {
edit = true;
editedContent = message.content;
await tick();
const editElement = document.getElementById(`message-edit-${message.id}`);
editElement.style.height = '';
editElement.style.height = `${editElement.scrollHeight}px`;
messageEditTextAreaElement.style.height = '';
messageEditTextAreaElement.style.height = `${messageEditTextAreaElement.scrollHeight}px`;
editElement?.focus();
messageEditTextAreaElement?.focus();
};
const editMessageConfirmHandler = async () => {
@ -165,9 +165,11 @@
<div class=" w-full">
<textarea
id="message-edit-{message.id}"
bind:this={messageEditTextAreaElement}
class=" bg-transparent outline-none w-full resize-none"
bind:value={editedContent}
on:input={(e) => {
e.target.style.height = '';
e.target.style.height = `${e.target.scrollHeight}px`;
}}
/>
@ -245,55 +247,11 @@
</div>
{/if}
<button
class="invisible group-hover:visible p-1 rounded dark:hover:text-white hover:text-black transition edit-user-message-button"
on:click={() => {
editMessageHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M16.862 4.487l1.687-1.688a1.875 1.875 0 112.652 2.652L6.832 19.82a4.5 4.5 0 01-1.897 1.13l-2.685.8.8-2.685a4.5 4.5 0 011.13-1.897L16.863 4.487zm0 0L19.5 7.125"
/>
</svg>
</button>
<button
class="invisible group-hover:visible p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
copyToClipboard(message.content);
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M15.666 3.888A2.25 2.25 0 0013.5 2.25h-3c-1.03 0-1.9.693-2.166 1.638m7.332 0c.055.194.084.4.084.612v0a.75.75 0 01-.75.75H9a.75.75 0 01-.75-.75v0c0-.212.03-.418.084-.612m7.332 0c.646.049 1.288.11 1.927.184 1.1.128 1.907 1.077 1.907 2.185V19.5a2.25 2.25 0 01-2.25 2.25H6.75A2.25 2.25 0 014.5 19.5V6.257c0-1.108.806-2.057 1.907-2.185a48.208 48.208 0 011.927-.184"
/>
</svg>
</button>
{#if !isFirstMessage}
<Tooltip content="Edit" placement="bottom">
<button
class="invisible group-hover:visible p-1 rounded dark:hover:text-white hover:text-black transition"
class="invisible group-hover:visible p-1 rounded dark:hover:text-white hover:text-black transition edit-user-message-button"
on:click={() => {
deleteMessageHandler();
editMessageHandler();
}}
>
<svg
@ -307,10 +265,60 @@
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m14.74 9-.346 9m-4.788 0L9.26 9m9.968-3.21c.342.052.682.107 1.022.166m-1.022-.165L18.16 19.673a2.25 2.25 0 0 1-2.244 2.077H8.084a2.25 2.25 0 0 1-2.244-2.077L4.772 5.79m14.456 0a48.108 48.108 0 0 0-3.478-.397m-12 .562c.34-.059.68-.114 1.022-.165m0 0a48.11 48.11 0 0 1 3.478-.397m7.5 0v-.916c0-1.18-.91-2.164-2.09-2.201a51.964 51.964 0 0 0-3.32 0c-1.18.037-2.09 1.022-2.09 2.201v.916m7.5 0a48.667 48.667 0 0 0-7.5 0"
d="M16.862 4.487l1.687-1.688a1.875 1.875 0 112.652 2.652L6.832 19.82a4.5 4.5 0 01-1.897 1.13l-2.685.8.8-2.685a4.5 4.5 0 011.13-1.897L16.863 4.487zm0 0L19.5 7.125"
/>
</svg>
</button>
</Tooltip>
<Tooltip content="Copy" placement="bottom">
<button
class="invisible group-hover:visible p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
copyToClipboard(message.content);
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="M15.666 3.888A2.25 2.25 0 0013.5 2.25h-3c-1.03 0-1.9.693-2.166 1.638m7.332 0c.055.194.084.4.084.612v0a.75.75 0 01-.75.75H9a.75.75 0 01-.75-.75v0c0-.212.03-.418.084-.612m7.332 0c.646.049 1.288.11 1.927.184 1.1.128 1.907 1.077 1.907 2.185V19.5a2.25 2.25 0 01-2.25 2.25H6.75A2.25 2.25 0 014.5 19.5V6.257c0-1.108.806-2.057 1.907-2.185a48.208 48.208 0 011.927-.184"
/>
</svg>
</button>
</Tooltip>
{#if !isFirstMessage}
<Tooltip content="Delete" placement="bottom">
<button
class="invisible group-hover:visible p-1 rounded dark:hover:text-white hover:text-black transition"
on:click={() => {
deleteMessageHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
fill="none"
viewBox="0 0 24 24"
stroke-width="1.5"
stroke="currentColor"
class="w-4 h-4"
>
<path
stroke-linecap="round"
stroke-linejoin="round"
d="m14.74 9-.346 9m-4.788 0L9.26 9m9.968-3.21c.342.052.682.107 1.022.166m-1.022-.165L18.16 19.673a2.25 2.25 0 0 1-2.244 2.077H8.084a2.25 2.25 0 0 1-2.244-2.077L4.772 5.79m14.456 0a48.108 48.108 0 0 0-3.478-.397m-12 .562c.34-.059.68-.114 1.022-.165m0 0a48.11 48.11 0 0 1 3.478-.397m7.5 0v-.916c0-1.18-.91-2.164-2.09-2.201a51.964 51.964 0 0 0-3.32 0c-1.18.037-2.09 1.022-2.09 2.201v.916m7.5 0a48.667 48.667 0 0 0-7.5 0"
/>
</svg>
</button>
</Tooltip>
{/if}
</div>
</div>

View file

@ -15,6 +15,7 @@
let name = '';
let showJWTToken = false;
let JWTTokenCopied = false;
let profileImageInputElement: HTMLInputElement;
const submitHandler = async () => {
const updatedUser = await updateUserProfile(localStorage.token, name, profileImageUrl).catch(
@ -40,11 +41,12 @@
<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
<input
id="profile-image-input"
bind:this={profileImageInputElement}
type="file"
hidden
accept="image/*"
on:change={(e) => {
const files = e?.target?.files ?? [];
const files = profileImageInputElement.files ?? [];
let reader = new FileReader();
reader.onload = (event) => {
let originalImageUrl = `${event.target.result}`;
@ -86,7 +88,7 @@
// Display the compressed image
profileImageUrl = compressedSrc;
e.target.files = null;
profileImageInputElement.files = null;
};
};
@ -107,9 +109,7 @@
<button
class="relative rounded-full dark:bg-gray-700"
type="button"
on:click={() => {
document.getElementById('profile-image-input')?.click();
}}
on:click={profileImageInputElement.click}
>
<img
src={profileImageUrl !== '' ? profileImageUrl : '/user.png'}

View file

@ -22,6 +22,7 @@
let saveChatHistory = true;
let importFiles;
let showDeleteConfirm = false;
let chatImportInputElement: HTMLInputElement;
$: if (importFiles) {
console.log(importFiles);
@ -159,12 +160,17 @@
<hr class=" dark:border-gray-700" />
<div class="flex flex-col">
<input id="chat-import-input" bind:files={importFiles} type="file" accept=".json" hidden />
<input
id="chat-import-input"
bind:this={chatImportInputElement}
bind:files={importFiles}
type="file"
accept=".json"
hidden
/>
<button
class=" flex rounded-md py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
on:click={() => {
document.getElementById('chat-import-input').click();
}}
on:click={chatImportInputElement.click}
>
<div class=" self-center mr-3">
<svg

View file

@ -3,14 +3,15 @@
import { createEventDispatcher, onMount } from 'svelte';
const dispatch = createEventDispatcher();
import { getOllamaAPIUrl, getOllamaVersion, updateOllamaAPIUrl } from '$lib/apis/ollama';
import { getOllamaUrls, getOllamaVersion, updateOllamaUrls } from '$lib/apis/ollama';
import { getOpenAIKey, getOpenAIUrl, updateOpenAIKey, updateOpenAIUrl } from '$lib/apis/openai';
import { toast } from 'svelte-sonner';
export let getModels: Function;
// External
let API_BASE_URL = '';
let OLLAMA_BASE_URL = '';
let OLLAMA_BASE_URLS = [''];
let OPENAI_API_KEY = '';
let OPENAI_API_BASE_URL = '';
@ -25,8 +26,8 @@
await models.set(await getModels());
};
const updateOllamaAPIUrlHandler = async () => {
API_BASE_URL = await updateOllamaAPIUrl(localStorage.token, API_BASE_URL);
const updateOllamaUrlsHandler = async () => {
OLLAMA_BASE_URLS = await updateOllamaUrls(localStorage.token, OLLAMA_BASE_URLS);
const ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => {
toast.error(error);
@ -41,7 +42,7 @@
onMount(async () => {
if ($user.role === 'admin') {
API_BASE_URL = await getOllamaAPIUrl(localStorage.token);
OLLAMA_BASE_URLS = await getOllamaUrls(localStorage.token);
OPENAI_API_BASE_URL = await getOpenAIUrl(localStorage.token);
OPENAI_API_KEY = await getOpenAIKey(localStorage.token);
}
@ -53,11 +54,6 @@
on:submit|preventDefault={() => {
updateOpenAIHandler();
dispatch('save');
// saveSettings({
// OPENAI_API_KEY: OPENAI_API_KEY !== '' ? OPENAI_API_KEY : undefined,
// OPENAI_API_BASE_URL: OPENAI_API_BASE_URL !== '' ? OPENAI_API_BASE_URL : undefined
// });
}}
>
<div class=" pr-1.5 overflow-y-scroll max-h-[20.5rem] space-y-3">
@ -115,34 +111,81 @@
<div>
<div class=" mb-2.5 text-sm font-medium">Ollama Base URL</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none"
placeholder="Enter URL (e.g. http://localhost:11434)"
bind:value={API_BASE_URL}
/>
<div class="flex w-full gap-1.5">
<div class="flex-1 flex flex-col gap-2">
{#each OLLAMA_BASE_URLS as url, idx}
<div class="flex gap-1.5">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter URL (e.g. http://localhost:11434)"
bind:value={url}
/>
<div class="self-center flex items-center">
{#if idx === 0}
<button
class="px-1"
on:click={() => {
OLLAMA_BASE_URLS = [...OLLAMA_BASE_URLS, ''];
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M8.75 3.75a.75.75 0 0 0-1.5 0v3.5h-3.5a.75.75 0 0 0 0 1.5h3.5v3.5a.75.75 0 0 0 1.5 0v-3.5h3.5a.75.75 0 0 0 0-1.5h-3.5v-3.5Z"
/>
</svg>
</button>
{:else}
<button
class="px-1"
on:click={() => {
OLLAMA_BASE_URLS = OLLAMA_BASE_URLS.filter((url, urlIdx) => idx !== urlIdx);
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path d="M3.75 7.25a.75.75 0 0 0 0 1.5h8.5a.75.75 0 0 0 0-1.5h-8.5Z" />
</svg>
</button>
{/if}
</div>
</div>
{/each}
</div>
<button
class="px-3 bg-gray-200 hover:bg-gray-300 dark:bg-gray-600 dark:hover:bg-gray-700 rounded transition"
on:click={() => {
updateOllamaAPIUrlHandler();
}}
type="button"
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
<div class="">
<button
class="p-2.5 bg-gray-200 hover:bg-gray-300 dark:bg-gray-850 dark:hover:bg-gray-800 rounded-lg transition"
on:click={() => {
updateOllamaUrlsHandler();
}}
type="button"
>
<path
fill-rule="evenodd"
d="M15.312 11.424a5.5 5.5 0 01-9.201 2.466l-.312-.311h2.433a.75.75 0 000-1.5H3.989a.75.75 0 00-.75.75v4.242a.75.75 0 001.5 0v-2.43l.31.31a7 7 0 0011.712-3.138.75.75 0 00-1.449-.39zm1.23-3.723a.75.75 0 00.219-.53V2.929a.75.75 0 00-1.5 0V5.36l-.31-.31A7 7 0 003.239 8.188a.75.75 0 101.448.389A5.5 5.5 0 0113.89 6.11l.311.31h-2.432a.75.75 0 000 1.5h4.243a.75.75 0 00.53-.219z"
clip-rule="evenodd"
/>
</svg>
</button>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 20 20"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M15.312 11.424a5.5 5.5 0 01-9.201 2.466l-.312-.311h2.433a.75.75 0 000-1.5H3.989a.75.75 0 00-.75.75v4.242a.75.75 0 001.5 0v-2.43l.31.31a7 7 0 0011.712-3.138.75.75 0 00-1.449-.39zm1.23-3.723a.75.75 0 00.219-.53V2.929a.75.75 0 00-1.5 0V5.36l-.31-.31A7 7 0 003.239 8.188a.75.75 0 101.448.389A5.5 5.5 0 0113.89 6.11l.311.31h-2.432a.75.75 0 000 1.5h4.243a.75.75 0 00.53-.219z"
clip-rule="evenodd"
/>
</svg>
</button>
</div>
</div>
<div class="mt-2 text-xs text-gray-400 dark:text-gray-500">

View file

@ -2,7 +2,13 @@
import queue from 'async/queue';
import { toast } from 'svelte-sonner';
import { createModel, deleteModel, getOllamaVersion, pullModel } from '$lib/apis/ollama';
import {
createModel,
deleteModel,
getOllamaUrls,
getOllamaVersion,
pullModel
} from '$lib/apis/ollama';
import { WEBUI_API_BASE_URL, WEBUI_BASE_URL } from '$lib/constants';
import { WEBUI_NAME, models, user } from '$lib/stores';
import { splitStream } from '$lib/utils';
@ -13,7 +19,7 @@
let showLiteLLM = false;
let showLiteLLMParams = false;
let modelUploadInputElement: HTMLInputElement;
let liteLLMModelInfo = [];
let liteLLMModel = '';
@ -27,6 +33,9 @@
$: liteLLMModelName = liteLLMModel;
// Models
let OLLAMA_URLS = [];
let selectedOllamaUrlIdx: string | null = null;
let showExperimentalOllama = false;
let ollamaVersion = '';
const MAX_PARALLEL_DOWNLOADS = 3;
@ -45,7 +54,7 @@
let modelUploadMode = 'file';
let modelInputFile = '';
let modelFileUrl = '';
let modelFileContent = `TEMPLATE """{{ .System }}\nUSER: {{ .Prompt }}\nASSSISTANT: """\nPARAMETER num_ctx 4096\nPARAMETER stop "</s>"\nPARAMETER stop "USER:"\nPARAMETER stop "ASSSISTANT:"`;
let modelFileContent = `TEMPLATE """{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: """\nPARAMETER num_ctx 4096\nPARAMETER stop "</s>"\nPARAMETER stop "USER:"\nPARAMETER stop "ASSISTANT:"`;
let modelFileDigest = '';
let uploadProgress = null;
@ -236,9 +245,11 @@
};
const deleteModelHandler = async () => {
const res = await deleteModel(localStorage.token, deleteModelTag).catch((error) => {
toast.error(error);
});
const res = await deleteModel(localStorage.token, deleteModelTag, selectedOllamaUrlIdx).catch(
(error) => {
toast.error(error);
}
);
if (res) {
toast.success(`Deleted ${deleteModelTag}`);
@ -249,10 +260,12 @@
};
const pullModelHandlerProcessor = async (opts: { modelName: string; callback: Function }) => {
const res = await pullModel(localStorage.token, opts.modelName).catch((error) => {
opts.callback({ success: false, error, modelName: opts.modelName });
return null;
});
const res = await pullModel(localStorage.token, opts.modelName, selectedOllamaUrlIdx).catch(
(error) => {
opts.callback({ success: false, error, modelName: opts.modelName });
return null;
}
);
if (res) {
const reader = res.body
@ -358,6 +371,15 @@
};
onMount(async () => {
OLLAMA_URLS = await getOllamaUrls(localStorage.token).catch((error) => {
toast.error(error);
return [];
});
if (OLLAMA_URLS.length > 1) {
selectedOllamaUrlIdx = 0;
}
ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => false);
liteLLMModelInfo = await getLiteLLMModelInfo(localStorage.token);
});
@ -367,52 +389,137 @@
<div class=" space-y-3 pr-1.5 overflow-y-scroll h-[23rem]">
{#if ollamaVersion}
<div class="space-y-2 pr-1.5">
<div>
<div class=" mb-2 text-sm font-medium">Manage Ollama Models</div>
<div class="text-sm font-medium">Manage Ollama Models</div>
<div class=" mb-2 text-sm font-medium">Pull a model from Ollama.com</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter model tag (e.g. mistral:7b)"
bind:value={modelTag}
/>
</div>
<button
class="px-3 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded transition"
on:click={() => {
pullModelHandler();
}}
disabled={modelTransferring}
{#if OLLAMA_URLS.length > 1}
<div class="flex-1 pb-1">
<select
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={selectedOllamaUrlIdx}
placeholder="Select an Ollama instance"
>
{#if modelTransferring}
<div class="self-center">
<svg
class=" w-4 h-4"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_ajPY {
transform-origin: center;
animation: spinner_AtaB 0.75s infinite linear;
}
@keyframes spinner_AtaB {
100% {
transform: rotate(360deg);
{#each OLLAMA_URLS as url, idx}
<option value={idx} class="bg-gray-100 dark:bg-gray-700">{url}</option>
{/each}
</select>
</div>
{/if}
<div class="space-y-2">
<div>
<div class=" mb-2 text-sm font-medium">Pull a model from Ollama.com</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<input
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter model tag (e.g. mistral:7b)"
bind:value={modelTag}
/>
</div>
<button
class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
on:click={() => {
pullModelHandler();
}}
disabled={modelTransferring}
>
{#if modelTransferring}
<div class="self-center">
<svg
class=" w-4 h-4"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_ajPY {
transform-origin: center;
animation: spinner_AtaB 0.75s infinite linear;
}
}
</style><path
d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
opacity=".25"
/><path
d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
class="spinner_ajPY"
/></svg
@keyframes spinner_AtaB {
100% {
transform: rotate(360deg);
}
}
</style><path
d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
opacity=".25"
/><path
d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
class="spinner_ajPY"
/></svg
>
</div>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M8.75 2.75a.75.75 0 0 0-1.5 0v5.69L5.03 6.22a.75.75 0 0 0-1.06 1.06l3.5 3.5a.75.75 0 0 0 1.06 0l3.5-3.5a.75.75 0 0 0-1.06-1.06L8.75 8.44V2.75Z"
/>
<path
d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
/>
</svg>
{/if}
</button>
</div>
<div class="mt-2 mb-1 text-xs text-gray-400 dark:text-gray-500">
To access the available model names for downloading, <a
class=" text-gray-500 dark:text-gray-300 font-medium underline"
href="https://ollama.com/library"
target="_blank">click here.</a
>
</div>
{#if Object.keys(modelDownloadStatus).length > 0}
{#each Object.keys(modelDownloadStatus) as model}
<div class="flex flex-col">
<div class="font-medium mb-1">{model}</div>
<div class="">
<div
class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
style="width: {Math.max(15, modelDownloadStatus[model].pullProgress ?? 0)}%"
>
{modelDownloadStatus[model].pullProgress ?? 0}%
</div>
<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
{modelDownloadStatus[model].digest}
</div>
</div>
</div>
{:else}
{/each}
{/if}
</div>
<div>
<div class=" mb-2 text-sm font-medium">Delete a model</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<select
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={deleteModelTag}
placeholder="Select a model"
>
{#if !deleteModelTag}
<option value="" disabled selected>Select a model</option>
{/if}
{#each $models.filter((m) => m.size != null && (selectedOllamaUrlIdx === null ? true : (m?.urls ?? []).includes(selectedOllamaUrlIdx))) as model}
<option value={model.name} class="bg-gray-100 dark:bg-gray-700"
>{model.name + ' (' + (model.size / 1024 ** 3).toFixed(1) + ' GB)'}</option
>
{/each}
</select>
</div>
<button
class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
on:click={() => {
deleteModelHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
@ -420,262 +527,192 @@
class="w-4 h-4"
>
<path
d="M8.75 2.75a.75.75 0 0 0-1.5 0v5.69L5.03 6.22a.75.75 0 0 0-1.06 1.06l3.5 3.5a.75.75 0 0 0 1.06 0l3.5-3.5a.75.75 0 0 0-1.06-1.06L8.75 8.44V2.75Z"
/>
<path
d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
fill-rule="evenodd"
d="M5 3.25V4H2.75a.75.75 0 0 0 0 1.5h.3l.815 8.15A1.5 1.5 0 0 0 5.357 15h5.285a1.5 1.5 0 0 0 1.493-1.35l.815-8.15h.3a.75.75 0 0 0 0-1.5H11v-.75A2.25 2.25 0 0 0 8.75 1h-1.5A2.25 2.25 0 0 0 5 3.25Zm2.25-.75a.75.75 0 0 0-.75.75V4h3v-.75a.75.75 0 0 0-.75-.75h-1.5ZM6.05 6a.75.75 0 0 1 .787.713l.275 5.5a.75.75 0 0 1-1.498.075l-.275-5.5A.75.75 0 0 1 6.05 6Zm3.9 0a.75.75 0 0 1 .712.787l-.275 5.5a.75.75 0 0 1-1.498-.075l.275-5.5a.75.75 0 0 1 .786-.711Z"
clip-rule="evenodd"
/>
</svg>
{/if}
</button>
</div>
<div class="mt-2 mb-1 text-xs text-gray-400 dark:text-gray-500">
To access the available model names for downloading, <a
class=" text-gray-500 dark:text-gray-300 font-medium underline"
href="https://ollama.com/library"
target="_blank">click here.</a
>
</div>
{#if Object.keys(modelDownloadStatus).length > 0}
{#each Object.keys(modelDownloadStatus) as model}
<div class="flex flex-col">
<div class="font-medium mb-1">{model}</div>
<div class="">
<div
class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
style="width: {Math.max(15, modelDownloadStatus[model].pullProgress ?? 0)}%"
>
{modelDownloadStatus[model].pullProgress ?? 0}%
</div>
<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
{modelDownloadStatus[model].digest}
</div>
</div>
</div>
{/each}
{/if}
</div>
<div>
<div class=" mb-2 text-sm font-medium">Delete a model</div>
<div class="flex w-full">
<div class="flex-1 mr-2">
<select
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={deleteModelTag}
placeholder="Select a model"
>
{#if !deleteModelTag}
<option value="" disabled selected>Select a model</option>
{/if}
{#each $models.filter((m) => m.size != null) as model}
<option value={model.name} class="bg-gray-100 dark:bg-gray-700"
>{model.name + ' (' + (model.size / 1024 ** 3).toFixed(1) + ' GB)'}</option
>
{/each}
</select>
</div>
<button
class="px-3 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded transition"
on:click={() => {
deleteModelHandler();
}}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
fill-rule="evenodd"
d="M5 3.25V4H2.75a.75.75 0 0 0 0 1.5h.3l.815 8.15A1.5 1.5 0 0 0 5.357 15h5.285a1.5 1.5 0 0 0 1.493-1.35l.815-8.15h.3a.75.75 0 0 0 0-1.5H11v-.75A2.25 2.25 0 0 0 8.75 1h-1.5A2.25 2.25 0 0 0 5 3.25Zm2.25-.75a.75.75 0 0 0-.75.75V4h3v-.75a.75.75 0 0 0-.75-.75h-1.5ZM6.05 6a.75.75 0 0 1 .787.713l.275 5.5a.75.75 0 0 1-1.498.075l-.275-5.5A.75.75 0 0 1 6.05 6Zm3.9 0a.75.75 0 0 1 .712.787l-.275 5.5a.75.75 0 0 1-1.498-.075l.275-5.5a.75.75 0 0 1 .786-.711Z"
clip-rule="evenodd"
/>
</svg>
</button>
</div>
</div>
<div>
<div class="flex justify-between items-center text-xs">
<div class=" text-sm font-medium">Experimental</div>
<button
class=" text-xs font-medium text-gray-500"
type="button"
on:click={() => {
showExperimentalOllama = !showExperimentalOllama;
}}>{showExperimentalOllama ? 'Show' : 'Hide'}</button
>
</div>
</div>
{#if showExperimentalOllama}
<form
on:submit|preventDefault={() => {
uploadModelHandler();
}}
>
<div class=" mb-2 flex w-full justify-between">
<div class=" text-sm font-medium">Upload a GGUF model</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
if (modelUploadMode === 'file') {
modelUploadMode = 'url';
} else {
modelUploadMode = 'file';
}
}}
type="button"
>
{#if modelUploadMode === 'file'}
<span class="ml-2 self-center">File Mode</span>
{:else}
<span class="ml-2 self-center">URL Mode</span>
{/if}
</button>
</div>
</div>
<div class="flex w-full mb-1.5">
<div class="flex flex-col w-full">
{#if modelUploadMode === 'file'}
<div class="flex-1 {modelInputFile && modelInputFile.length > 0 ? 'mr-2' : ''}">
<input
id="model-upload-input"
type="file"
bind:files={modelInputFile}
on:change={() => {
console.log(modelInputFile);
}}
accept=".gguf"
required
hidden
/>
<div class="pt-1">
<div class="flex justify-between items-center text-xs">
<div class=" text-sm font-medium">Experimental</div>
<button
class=" text-xs font-medium text-gray-500"
type="button"
on:click={() => {
showExperimentalOllama = !showExperimentalOllama;
}}>{showExperimentalOllama ? 'Hide' : 'Show'}</button
>
</div>
</div>
<button
type="button"
class="w-full rounded text-left py-2 px-4 dark:text-gray-300 dark:bg-gray-850"
on:click={() => {
document.getElementById('model-upload-input').click();
}}
>
{#if modelInputFile && modelInputFile.length > 0}
{modelInputFile[0].name}
{:else}
Click here to select
{/if}
</button>
</div>
{:else}
<div class="flex-1 {modelFileUrl !== '' ? 'mr-2' : ''}">
<input
class="w-full rounded text-left py-2 px-4 dark:text-gray-300 dark:bg-gray-850 outline-none {modelFileUrl !==
''
? 'mr-2'
: ''}"
type="url"
required
bind:value={modelFileUrl}
placeholder="Type Hugging Face Resolve (Download) URL"
/>
</div>
{#if showExperimentalOllama}
<form
on:submit|preventDefault={() => {
uploadModelHandler();
}}
>
<div class=" mb-2 flex w-full justify-between">
<div class=" text-sm font-medium">Upload a GGUF model</div>
<button
class="p-1 px-3 text-xs flex rounded transition"
on:click={() => {
if (modelUploadMode === 'file') {
modelUploadMode = 'url';
} else {
modelUploadMode = 'file';
}
}}
type="button"
>
{#if modelUploadMode === 'file'}
<span class="ml-2 self-center">File Mode</span>
{:else}
<span class="ml-2 self-center">URL Mode</span>
{/if}
</button>
</div>
<div class="flex w-full mb-1.5">
<div class="flex flex-col w-full">
{#if modelUploadMode === 'file'}
<div class="flex-1 {modelInputFile && modelInputFile.length > 0 ? 'mr-2' : ''}">
<input
id="model-upload-input"
bind:this={modelUploadInputElement}
type="file"
bind:files={modelInputFile}
on:change={() => {
console.log(modelInputFile);
}}
accept=".gguf"
required
hidden
/>
<button
type="button"
class="w-full rounded-lg text-left py-2 px-4 dark:text-gray-300 dark:bg-gray-850"
on:click={modelUploadInputElement.click}
>
{#if modelInputFile && modelInputFile.length > 0}
{modelInputFile[0].name}
{:else}
Click here to select
{/if}
</button>
</div>
{:else}
<div class="flex-1 {modelFileUrl !== '' ? 'mr-2' : ''}">
<input
class="w-full rounded-lg text-left py-2 px-4 dark:text-gray-300 dark:bg-gray-850 outline-none {modelFileUrl !==
''
? 'mr-2'
: ''}"
type="url"
required
bind:value={modelFileUrl}
placeholder="Type Hugging Face Resolve (Download) URL"
/>
</div>
{/if}
</div>
{#if (modelUploadMode === 'file' && modelInputFile && modelInputFile.length > 0) || (modelUploadMode === 'url' && modelFileUrl !== '')}
<button
class="px-3 text-gray-100 bg-emerald-600 hover:bg-emerald-700 disabled:bg-gray-700 disabled:cursor-not-allowed rounded transition"
type="submit"
disabled={modelTransferring}
>
{#if modelTransferring}
<div class="self-center">
<svg
class=" w-4 h-4"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_ajPY {
transform-origin: center;
animation: spinner_AtaB 0.75s infinite linear;
}
@keyframes spinner_AtaB {
100% {
transform: rotate(360deg);
}
}
</style><path
d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
opacity=".25"
/><path
d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
class="spinner_ajPY"
/></svg
>
</div>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M7.25 10.25a.75.75 0 0 0 1.5 0V4.56l2.22 2.22a.75.75 0 1 0 1.06-1.06l-3.5-3.5a.75.75 0 0 0-1.06 0l-3.5 3.5a.75.75 0 0 0 1.06 1.06l2.22-2.22v5.69Z"
/>
<path
d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
/>
</svg>
{/if}
</button>
{/if}
</div>
{#if (modelUploadMode === 'file' && modelInputFile && modelInputFile.length > 0) || (modelUploadMode === 'url' && modelFileUrl !== '')}
<button
class="px-3 text-gray-100 bg-emerald-600 hover:bg-emerald-700 disabled:bg-gray-700 disabled:cursor-not-allowed rounded transition"
type="submit"
disabled={modelTransferring}
>
{#if modelTransferring}
<div class="self-center">
<svg
class=" w-4 h-4"
viewBox="0 0 24 24"
fill="currentColor"
xmlns="http://www.w3.org/2000/svg"
><style>
.spinner_ajPY {
transform-origin: center;
animation: spinner_AtaB 0.75s infinite linear;
}
@keyframes spinner_AtaB {
100% {
transform: rotate(360deg);
}
}
</style><path
d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
opacity=".25"
/><path
d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
class="spinner_ajPY"
/></svg
>
</div>
{:else}
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 16 16"
fill="currentColor"
class="w-4 h-4"
>
<path
d="M7.25 10.25a.75.75 0 0 0 1.5 0V4.56l2.22 2.22a.75.75 0 1 0 1.06-1.06l-3.5-3.5a.75.75 0 0 0-1.06 0l-3.5 3.5a.75.75 0 0 0 1.06 1.06l2.22-2.22v5.69Z"
/>
<path
d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
/>
</svg>
{/if}
</button>
{/if}
</div>
{#if (modelUploadMode === 'file' && modelInputFile && modelInputFile.length > 0) || (modelUploadMode === 'url' && modelFileUrl !== '')}
<div>
<div>
<div class=" my-2.5 text-sm font-medium">Modelfile Content</div>
<textarea
bind:value={modelFileContent}
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none resize-none"
rows="6"
/>
</div>
</div>
{/if}
<div class=" mt-1 text-xs text-gray-400 dark:text-gray-500">
To access the GGUF models available for downloading, <a
class=" text-gray-500 dark:text-gray-300 font-medium underline"
href="https://huggingface.co/models?search=gguf"
target="_blank">click here.</a
>
</div>
{#if uploadProgress !== null}
<div class="mt-2">
<div class=" mb-2 text-xs">Upload Progress</div>
<div class="w-full rounded-full dark:bg-gray-800">
<div
class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
style="width: {Math.max(15, uploadProgress ?? 0)}%"
>
{uploadProgress ?? 0}%
<div>
<div class=" my-2.5 text-sm font-medium">Modelfile Content</div>
<textarea
bind:value={modelFileContent}
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-800 outline-none resize-none"
rows="6"
/>
</div>
</div>
<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
{modelFileDigest}
</div>
{/if}
<div class=" mt-1 text-xs text-gray-400 dark:text-gray-500">
To access the GGUF models available for downloading, <a
class=" text-gray-500 dark:text-gray-300 font-medium underline"
href="https://huggingface.co/models?search=gguf"
target="_blank">click here.</a
>
</div>
{/if}
</form>
{/if}
{#if uploadProgress !== null}
<div class="mt-2">
<div class=" mb-2 text-xs">Upload Progress</div>
<div class="w-full rounded-full dark:bg-gray-800">
<div
class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
style="width: {Math.max(15, uploadProgress ?? 0)}%"
>
{uploadProgress ?? 0}%
</div>
</div>
<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
{modelFileDigest}
</div>
</div>
{/if}
</form>
{/if}
</div>
</div>
<hr class=" dark:border-gray-700 my-2" />
{/if}
@ -693,7 +730,7 @@
type="button"
on:click={() => {
showLiteLLMParams = !showLiteLLMParams;
}}>{showLiteLLMParams ? 'Advanced' : 'Default'}</button
}}>{showLiteLLMParams ? 'Hide Additional Params' : 'Show Additional Params'}</button
>
</div>
</div>
@ -702,7 +739,7 @@
<div class="flex w-full mb-1.5">
<div class="flex-1 mr-2">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter LiteLLM Model (litellm_params.model)"
bind:value={liteLLMModel}
autocomplete="off"
@ -710,7 +747,7 @@
</div>
<button
class="px-3 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded transition"
class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
on:click={() => {
addLiteLLMModelHandler();
}}
@ -734,7 +771,7 @@
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter Model Name (model_name)"
bind:value={liteLLMModelName}
autocomplete="off"
@ -748,7 +785,7 @@
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter LiteLLM API Base URL (litellm_params.api_base)"
bind:value={liteLLMAPIBase}
autocomplete="off"
@ -762,7 +799,7 @@
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter LiteLLM API Key (litellm_params.api_key)"
bind:value={liteLLMAPIKey}
autocomplete="off"
@ -776,7 +813,7 @@
<div class="flex w-full">
<div class="flex-1">
<input
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
placeholder="Enter LiteLLM API RPM (litellm_params.rpm)"
bind:value={liteLLMRPM}
autocomplete="off"
@ -803,7 +840,7 @@
<div class="flex w-full">
<div class="flex-1 mr-2">
<select
class="w-full rounded py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
bind:value={deleteLiteLLMModelId}
placeholder="Select a model"
>
@ -818,7 +855,7 @@
</select>
</div>
<button
class="px-3 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded transition"
class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
on:click={() => {
deleteLiteLLMModelHandler();
}}

View file

@ -29,6 +29,6 @@
});
</script>
<div bind:this={tooltipElement}>
<div bind:this={tooltipElement} aria-label={content}>
<slot />
</div>

View file

@ -15,7 +15,7 @@
export let show = false;
export let selectedDoc;
let uploadDocInputElement: HTMLInputElement;
let inputFiles;
let tags = [];
@ -69,7 +69,7 @@
}
inputFiles = null;
document.getElementById('upload-doc-input').value = '';
uploadDocInputElement.value = '';
} else {
toast.error(`File not found.`);
}
@ -126,14 +126,19 @@
}}
>
<div class="mb-3 w-full">
<input id="upload-doc-input" hidden bind:files={inputFiles} type="file" multiple />
<input
id="upload-doc-input"
bind:this={uploadDocInputElement}
hidden
bind:files={inputFiles}
type="file"
multiple
/>
<button
class="w-full text-sm font-medium py-3 bg-gray-850 hover:bg-gray-800 text-center rounded-xl"
type="button"
on:click={() => {
document.getElementById('upload-doc-input')?.click();
}}
on:click={uploadDocInputElement.click}
>
{#if inputFiles}
{inputFiles.length > 0 ? `${inputFiles.length}` : ''} document(s) selected.

View file

@ -2,12 +2,11 @@
import { onMount } from 'svelte';
export let messages = [];
let textAreaElement: HTMLTextAreaElement;
onMount(() => {
messages.forEach((message, idx) => {
let textareaElement = document.getElementById(`${message.role}-${idx}-textarea`);
textareaElement.style.height = '';
textareaElement.style.height = textareaElement.scrollHeight + 'px';
textAreaElement.style.height = '';
textAreaElement.style.height = textAreaElement.scrollHeight + 'px';
});
});
</script>
@ -27,16 +26,17 @@
<div class="flex-1">
<textarea
id="{message.role}-{idx}-textarea"
bind:this={textAreaElement}
class="w-full bg-transparent outline-none rounded-lg p-2 text-sm resize-none overflow-hidden"
placeholder="Enter {message.role === 'user' ? 'a user' : 'an assistant'} message here"
rows="1"
on:input={(e) => {
e.target.style.height = '';
e.target.style.height = e.target.scrollHeight + 'px';
textAreaElement.style.height = '';
textAreaElement.style.height = textAreaElement.scrollHeight + 'px';
}}
on:focus={(e) => {
e.target.style.height = '';
e.target.style.height = e.target.scrollHeight + 'px';
textAreaElement.style.height = '';
textAreaElement.style.height = textAreaElement.scrollHeight + 'px';
// e.target.style.height = Math.min(e.target.scrollHeight, 200) + 'px';
}}

View file

@ -7,7 +7,7 @@ export const WEBUI_BASE_URL = dev ? `http://${location.hostname}:8080` : ``;
export const WEBUI_API_BASE_URL = `${WEBUI_BASE_URL}/api/v1`;
export const LITELLM_API_BASE_URL = `${WEBUI_BASE_URL}/litellm/api`;
export const OLLAMA_API_BASE_URL = `${WEBUI_BASE_URL}/ollama/api`;
export const OLLAMA_API_BASE_URL = `${WEBUI_BASE_URL}/ollama`;
export const OPENAI_API_BASE_URL = `${WEBUI_BASE_URL}/openai/api`;
export const AUDIO_API_BASE_URL = `${WEBUI_BASE_URL}/audio/api/v1`;
export const IMAGES_API_BASE_URL = `${WEBUI_BASE_URL}/images/api/v1`;
@ -90,8 +90,3 @@ export const SUPPORTED_FILE_EXTENSIONS = [
// This feature, akin to $env/static/private, exclusively incorporates environment variables
// that are prefixed with config.kit.env.publicPrefix (usually set to PUBLIC_).
// Consequently, these variables can be securely exposed to client-side code.
// Example of the .env configuration:
// OLLAMA_API_BASE_URL="http://localhost:11434/api"
// # Public
// PUBLIC_API_BASE_URL=$OLLAMA_API_BASE_URL

View file

@ -34,10 +34,11 @@
import Sidebar from '$lib/components/layout/Sidebar.svelte';
import ShortcutsModal from '$lib/components/chat/ShortcutsModal.svelte';
import ChangelogModal from '$lib/components/ChangelogModal.svelte';
import Tooltip from '$lib/components/common/Tooltip.svelte';
let ollamaVersion = '';
let loaded = false;
let showShortcutsButtonElement: HTMLButtonElement;
let DB = null;
let localDBChats = [];
@ -184,7 +185,7 @@
if (isCtrlPressed && event.key === '/') {
event.preventDefault();
console.log('showShortcuts');
document.getElementById('show-shortcuts-button')?.click();
showShortcutsButtonElement.click();
}
});
@ -201,15 +202,18 @@
{#if loaded}
<div class=" hidden lg:flex fixed bottom-0 right-0 px-3 py-3 z-10">
<button
id="show-shortcuts-button"
class="text-gray-600 dark:text-gray-300 bg-gray-300/20 w-6 h-6 flex items-center justify-center text-xs rounded-full"
on:click={() => {
showShortcuts = !showShortcuts;
}}
>
?
</button>
<Tooltip content="help" placement="left">
<button
id="show-shortcuts-button"
bind:this={showShortcutsButtonElement}
class="text-gray-600 dark:text-gray-300 bg-gray-300/20 w-6 h-6 flex items-center justify-center text-xs rounded-full"
on:click={() => {
showShortcuts = !showShortcuts;
}}
>
?
</button>
</Tooltip>
</div>
<ShortcutsModal bind:show={showShortcuts} />

View file

@ -42,7 +42,7 @@
let stopResponseFlag = false;
let autoScroll = true;
let processing = '';
let messagesContainerElement: HTMLDivElement;
let currentRequestId = null;
let selectedModels = [''];
@ -140,8 +140,7 @@
};
const scrollToBottom = () => {
const element = document.getElementById('messages-container');
element.scrollTop = element.scrollHeight;
messagesContainerElement.scrollTop = messagesContainerElement.scrollHeight;
};
//////////////////////////
@ -340,7 +339,7 @@
content: $settings.system
}
: undefined,
...messages.filter((message) => !message.deleted)
...messages
]
.filter((message) => message)
.map((message, idx, arr) => ({
@ -548,7 +547,7 @@
content: $settings.system
}
: undefined,
...messages.filter((message) => !message.deleted)
...messages
]
.filter((message) => message)
.map((message, idx, arr) => ({
@ -821,8 +820,11 @@
<div
class=" pb-2.5 flex flex-col justify-between w-full flex-auto overflow-auto h-0"
id="messages-container"
bind:this={messagesContainerElement}
on:scroll={(e) => {
autoScroll = e.target.scrollHeight - e.target.scrollTop <= e.target.clientHeight + 50;
autoScroll =
messagesContainerElement.scrollHeight - messagesContainerElement.scrollTop <=
messagesContainerElement.clientHeight + 50;
}}
>
<div
@ -830,10 +832,7 @@
? 'max-w-full'
: 'max-w-2xl md:px-0'} mx-auto w-full px-4"
>
<ModelSelector
bind:selectedModels
disabled={messages.length > 0 && !selectedModels.includes('')}
/>
<ModelSelector bind:selectedModels />
</div>
<div class=" h-full w-full flex flex-col py-8">

View file

@ -45,7 +45,7 @@
let stopResponseFlag = false;
let autoScroll = true;
let processing = '';
let messagesContainerElement: HTMLDivElement;
let currentRequestId = null;
// let chatId = $page.params.id;
@ -160,8 +160,7 @@
};
const scrollToBottom = () => {
const element = document.getElementById('messages-container');
element.scrollTop = element.scrollHeight;
messagesContainerElement.scrollTop = messagesContainerElement.scrollHeight;
};
//////////////////////////
@ -353,7 +352,7 @@
content: $settings.system
}
: undefined,
...messages.filter((message) => !message.deleted)
...messages
]
.filter((message) => message)
.map((message, idx, arr) => ({
@ -561,7 +560,7 @@
content: $settings.system
}
: undefined,
...messages.filter((message) => !message.deleted)
...messages
]
.filter((message) => message)
.map((message, idx, arr) => ({
@ -852,8 +851,11 @@
<div
class=" pb-2.5 flex flex-col justify-between w-full flex-auto overflow-auto h-0"
id="messages-container"
bind:this={messagesContainerElement}
on:scroll={(e) => {
autoScroll = e.target.scrollHeight - e.target.scrollTop <= e.target.clientHeight + 50;
autoScroll =
messagesContainerElement.scrollHeight - messagesContainerElement.scrollTop <=
messagesContainerElement.clientHeight + 50;
}}
>
<div
@ -861,10 +863,7 @@
? 'max-w-full'
: 'max-w-2xl md:px-0'} mx-auto w-full px-4"
>
<ModelSelector
bind:selectedModels
disabled={messages.length > 0 && !selectedModels.includes('')}
/>
<ModelSelector bind:selectedModels />
</div>
<div class=" h-full w-full flex flex-col py-8">

View file

@ -21,7 +21,7 @@
let inputFiles = '';
let query = '';
let documentsImportInputElement: HTMLInputElement;
let tags = [];
let showSettingsModal = false;
@ -524,6 +524,7 @@
<div class="flex space-x-2">
<input
id="documents-import-input"
bind:this={documentsImportInputElement}
bind:files={importFiles}
type="file"
accept=".json"
@ -558,9 +559,7 @@
<button
class="flex text-xs items-center space-x-1 px-3 py-1.5 rounded-xl bg-gray-50 hover:bg-gray-100 dark:bg-gray-800 dark:hover:bg-gray-700 dark:text-gray-200 transition"
on:click={async () => {
document.getElementById('documents-import-input')?.click();
}}
on:click={documentsImportInputElement.click}
>
<div class=" self-center mr-2 font-medium">Import Documents Mapping</div>

View file

@ -16,11 +16,14 @@
let localModelfiles = [];
let importFiles;
let modelfilesImportInputElement: HTMLInputElement;
const deleteModelHandler = async (tagName) => {
let success = null;
success = await deleteModel(localStorage.token, tagName);
success = await deleteModel(localStorage.token, tagName).catch((err) => {
toast.error(err);
return null;
});
if (success) {
toast.success(`Deleted ${tagName}`);
@ -235,6 +238,7 @@
<div class="flex space-x-1">
<input
id="modelfiles-import-input"
bind:this={modelfilesImportInputElement}
bind:files={importFiles}
type="file"
accept=".json"
@ -262,9 +266,7 @@
<button
class="flex text-xs items-center space-x-1 px-3 py-1.5 rounded-xl bg-gray-50 hover:bg-gray-100 dark:bg-gray-800 dark:hover:bg-gray-700 dark:text-gray-200 transition"
on:click={async () => {
document.getElementById('modelfiles-import-input')?.click();
}}
on:click={modelfilesImportInputElement.click}
>
<div class=" self-center mr-2 font-medium">Import Modelfiles</div>

View file

@ -1,4 +1,4 @@
<script>
<script lang="ts">
import { goto } from '$app/navigation';
import { onMount, tick } from 'svelte';
@ -21,15 +21,17 @@
let mode = 'chat';
let loaded = false;
let text = '';
let selectedModelId = '';
let loading = false;
let currentRequestId;
let currentRequestId = null;
let stopResponseFlag = false;
let messagesContainerElement: HTMLDivElement;
let textCompletionAreaElement: HTMLTextAreaElement;
let system = '';
let messages = [
{
@ -39,13 +41,7 @@
];
const scrollToBottom = () => {
let element;
if (mode === 'chat') {
element = document.getElementById('messages-container');
} else {
element = document.getElementById('text-completion-textarea');
}
const element = mode === 'chat' ? messagesContainerElement : textCompletionAreaElement;
if (element) {
element.scrollTop = element?.scrollHeight;
@ -96,6 +92,10 @@
while (true) {
const { value, done } = await reader.read();
if (done || stopResponseFlag) {
if (stopResponseFlag) {
await cancelChatCompletion(localStorage.token, currentRequestId);
}
currentRequestId = null;
break;
}
@ -112,7 +112,11 @@
let data = JSON.parse(line.replace(/^data: /, ''));
console.log(data);
text += data.choices[0].delta.content ?? '';
if ('request_id' in data) {
currentRequestId = data.request_id;
} else {
text += data.choices[0].delta.content ?? '';
}
}
}
}
@ -150,16 +154,6 @@
: `${OLLAMA_API_BASE_URL}/v1`
);
// const [res, controller] = await generateChatCompletion(localStorage.token, {
// model: selectedModelId,
// messages: [
// {
// role: 'assistant',
// content: text
// }
// ]
// });
let responseMessage;
if (messages.at(-1)?.role === 'assistant') {
responseMessage = messages.at(-1);
@ -184,6 +178,11 @@
while (true) {
const { value, done } = await reader.read();
if (done || stopResponseFlag) {
if (stopResponseFlag) {
await cancelChatCompletion(localStorage.token, currentRequestId);
}
currentRequestId = null;
break;
}
@ -200,17 +199,21 @@
let data = JSON.parse(line.replace(/^data: /, ''));
console.log(data);
if (responseMessage.content == '' && data.choices[0].delta.content == '\n') {
continue;
if ('request_id' in data) {
currentRequestId = data.request_id;
} else {
textareaElement.style.height = textareaElement.scrollHeight + 'px';
if (responseMessage.content == '' && data.choices[0].delta.content == '\n') {
continue;
} else {
textareaElement.style.height = textareaElement.scrollHeight + 'px';
responseMessage.content += data.choices[0].delta.content ?? '';
messages = messages;
responseMessage.content += data.choices[0].delta.content ?? '';
messages = messages;
textareaElement.style.height = textareaElement.scrollHeight + 'px';
textareaElement.style.height = textareaElement.scrollHeight + 'px';
await tick();
await tick();
}
}
}
}
@ -221,48 +224,6 @@
scrollToBottom();
}
// while (true) {
// const { value, done } = await reader.read();
// if (done || stopResponseFlag) {
// if (stopResponseFlag) {
// await cancelChatCompletion(localStorage.token, currentRequestId);
// }
// currentRequestId = null;
// break;
// }
// try {
// let lines = value.split('\n');
// for (const line of lines) {
// if (line !== '') {
// console.log(line);
// let data = JSON.parse(line);
// if ('detail' in data) {
// throw data;
// }
// if ('id' in data) {
// console.log(data);
// currentRequestId = data.id;
// } else {
// if (data.done == false) {
// text += data.message.content;
// } else {
// console.log('done');
// }
// }
// }
// }
// } catch (error) {
// console.log(error);
// }
// scrollToBottom();
// }
}
};
@ -417,12 +378,14 @@
<div
class=" pb-2.5 flex flex-col justify-between w-full flex-auto overflow-auto h-0"
id="messages-container"
bind:this={messagesContainerElement}
>
<div class=" h-full w-full flex flex-col">
<div class="flex-1 p-1">
{#if mode === 'complete'}
<textarea
id="text-completion-textarea"
bind:this={textCompletionAreaElement}
class="w-full h-full p-3 bg-transparent outline outline-1 outline-gray-200 dark:outline-gray-800 resize-none rounded-lg text-sm"
bind:value={text}
placeholder="You're a helpful assistant."

View file

@ -11,7 +11,7 @@
let importFiles = '';
let query = '';
let promptsImportInputElement: HTMLInputElement;
const sharePrompt = async (prompt) => {
toast.success('Redirecting you to OpenWebUI Community');
@ -208,6 +208,7 @@
<div class="flex space-x-2">
<input
id="prompts-import-input"
bind:this={promptsImportInputElement}
bind:files={importFiles}
type="file"
accept=".json"
@ -241,9 +242,7 @@
<button
class="flex text-xs items-center space-x-1 px-3 py-1.5 rounded-xl bg-gray-50 hover:bg-gray-100 dark:bg-gray-800 dark:hover:bg-gray-700 dark:text-gray-200 transition"
on:click={async () => {
document.getElementById('prompts-import-input')?.click();
}}
on:click={promptsImportInputElement.click}
>
<div class=" self-center mr-2 font-medium">Import Prompts</div>
@ -266,7 +265,7 @@
<button
class="flex text-xs items-center space-x-1 px-3 py-1.5 rounded-xl bg-gray-50 hover:bg-gray-100 dark:bg-gray-800 dark:hover:bg-gray-700 dark:text-gray-200 transition"
on:click={async () => {
// document.getElementById('modelfiles-import-input')?.click();
// promptsImportInputElement.click();
let blob = new Blob([JSON.stringify($prompts)], {
type: 'application/json'
});