tdpeuter/open-webui
tdpeuter/open-webui
1
Fork 0
forked from open-webui/open-webui
Code Issues Pull requests 1 Releases 2 Packages Activity Actions

Merge pull request #784 from dannyl1u/feat/prompt-token/s

Browse source 
feat: show prompt token/s in perf info
This commit is contained in:
Timothy Jaeryang Baek 2024-02-18 14:11:03 -05:00 committed by GitHub
commit 564970d33e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 6 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
src/lib/components/chat/Messages/ResponseMessage.svelte
View file

@ -72,13 +72,18 @@
if (message.info) { if (message.info) {
tooltipInstance = tippy(`#info-${message.id}`, { tooltipInstance = tippy(`#info-${message.id}`, {
content: `<span class="text-xs" id="tooltip-${message.id}">token/s: ${ content: `<span class="text-xs" id="tooltip-${message.id}">response_token/s: ${
`${ `${
Math.round( Math.round(
((message.info.eval_count ?? 0) / (message.info.eval_duration / 1000000000)) * 100 ((message.info.eval_count ?? 0) / (message.info.eval_duration / 1000000000)) * 100
) / 100 ) / 100
} tokens` ?? 'N/A' } tokens` ?? 'N/A'
}<br/> }<br/>
prompt_token/s: ${
Math.round(
((message.info.prompt_eval_count ?? 0) / (message.info.prompt_eval_duration / 1000000000)) * 100
) / 100 ?? 'N/A'
} tokens<br/>
total_duration: ${ total_duration: ${
Math.round(((message.info.total_duration ?? 0) / 1000000) * 100) / 100 ?? Math.round(((message.info.total_duration ?? 0) / 1000000) * 100) / 100 ??
'N/A' 'N/A'