Self Denial
075fbedb02
More format fixes
2024-04-04 12:07:42 -06:00
Self Denial
3b66aa55c0
Improve embedding model update & resolve network dependency
...
* Add config variable RAG_EMBEDDING_MODEL_AUTO_UPDATE to control update behavior
* Add RAG utils embedding_model_get_path() function to output the filesystem path in addition to update of the model using huggingface_hub
* Update and utilize existing RAG functions in main: get_embedding_model() & update_embedding_model()
* Add GUI setting to execute manual update process
2024-04-04 11:01:23 -06:00
Mmx233
947c392f72
fix: manually check the docs' filename
2024-04-03 23:37:13 +08:00
Timothy J. Baek
5558514ff1
fix
2024-04-01 15:23:12 -07:00
KoreLogic Disclosures
6c96361402
Suggested mitigation for KL-CAN-2024-002.
2024-04-01 15:55:14 -05:00
Self Denial
144c9059a3
Improve logging. Move print()
statements to appropiate log()
.
...
Add COMFYUI and WEBHOOK logging and associated environment variable
control. Add WEBHOOK payload & request debug logs.
2024-03-31 13:17:29 -06:00
Timothy J. Baek
3688955c77
fix: encoding issue
2024-03-25 23:50:52 -07:00
Timothy J. Baek
6307adfba1
feat: better error handling
2024-03-25 23:47:08 -07:00
Doug Danat
c91a5d8b1f
switch to using BeautifulSoup HTML loader so title is also captured
2024-03-25 11:26:18 +01:00
Doug Danat
784a6ec85e
include html langchain loader for RAG
2024-03-25 09:50:53 +01:00
Timothy Jaeryang Baek
371dfc1143
Merge branch 'dev' into debug_print
2024-03-24 18:04:03 -05:00
Timothy J. Baek
ff8a55a861
refac: rag api
2024-03-24 00:41:41 -07:00
Timothy J. Baek
7e0ea8f77d
feat: RAG text ingestion(store) api
2024-03-24 00:40:27 -07:00
Self Denial
e6dd0bfbe0
Migrate to python logging module with env var control.
2024-03-20 17:11:36 -06:00
Timothy J. Baek
91efd6cb63
fix: file upload encoding issue
2024-03-15 23:52:37 -07:00
Timothy J. Baek
98948814fd
feat: toggle pdf ocr
2024-03-10 13:32:34 -07:00
Timothy J. Baek
7e5e2c42c9
refac: rag routes
2024-03-08 19:26:39 -08:00
Timothy J. Baek
b88c64f80e
fix: ocr issue
2024-03-06 17:54:42 -08:00
Timothy J. Baek
bb98c10abb
revert: ocr feature
2024-03-06 17:04:40 -08:00
Timothy Jaeryang Baek
8fb5f54751
Merge pull request #1050 from jannikstdl/rag-pdf-ocr
...
feat: added ocr functionality to the pdf loader
2024-03-06 00:45:33 -05:00
Jannik Streidl
089a63e0c6
feat: added ocr functionality to the pdf loader
2024-03-05 22:25:25 +01:00
Firat Birlik
6782e95c75
recreate rag collection is now optional and only used for web requests
2024-03-04 10:00:06 -06:00
Firat Birlik
5d4ff85228
recreate rag collection instead of falling back to stale version
2024-03-03 21:25:00 -06:00
Timothy J. Baek
47a05a47b4
feat: add rag top k value setting
2024-03-02 18:56:57 -08:00
Ased Mammad
b473ad574f
fix: RAG scan unsupported mimetype
...
This fixes an issue with RAG that stops loading documents as soon
as it reaches a file with unsupported mimetype.
2024-02-23 14:27:31 +03:30
Timothy J. Baek
7c127c35fc
feat: dynamic embedding model load
2024-02-19 11:05:45 -08:00
Jannik Streidl
acf999013b
storing vectordb in project cache folder + device types
2024-02-19 07:51:17 +01:00
Timothy J. Baek
0cb0358485
refac: more descriptive var names
2024-02-18 11:16:10 -08:00
Jannik S
4b88e7e44f
Merge branch 'main' into choose-embedding-model
2024-02-18 09:20:54 +01:00
Jannik Streidl
bc3dd34d8b
collection query fix
2024-02-18 09:17:43 +01:00
Timothy J. Baek
07b451995e
feat: reset rag template
2024-02-17 22:49:18 -08:00
Timothy J. Baek
5270efa9e5
feat: editable rag template
2024-02-17 22:41:03 -08:00
Timothy J. Baek
ccf08fb91e
feat: editable chunk params
2024-02-17 22:29:52 -08:00
Timothy J. Baek
a94e4161f7
fix: file content type issue
2024-02-17 21:31:46 -08:00
Timothy J. Baek
e07001e5f6
feat: rag folder scan support
2024-02-17 21:06:08 -08:00
Jannik Streidl
1846c1e80d
choose embedding model when using docker
2024-02-17 19:38:29 +01:00
Tim Farrell
08e8e922fd
Endpoint role-checking was redundantly applied but FastAPI provides a nice abstraction mechanic...so I applied it. There should be no logical changes in this code; only simpler, cleaner ways for doing the same thing.
2024-02-08 18:05:01 -06:00
Timothy J. Baek
683650ec00
feat: collection rag integration
2024-02-03 15:57:06 -08:00
Timothy J. Baek
00803c92f2
feat: doc tagging
2024-02-03 14:44:49 -08:00
Timothy J. Baek
50f7b20ac2
refac
2024-02-01 13:35:41 -08:00
Timothy J. Baek
28226a6f97
feat: web rag support
2024-01-26 22:17:28 -08:00
Timothy J. Baek
4e468dc58c
refac
2024-01-25 00:24:49 -08:00
Timothy Jaeryang Baek
fa5918ad13
Merge branch 'main' into main
2024-01-25 00:13:12 -08:00
Marclass
8bfda730d9
add excel document support
2024-01-23 14:03:22 -07:00
Timothy Jaeryang Baek
ca943d0795
Merge pull request #549 from Marclass/main
...
Bugfix: Fix toast error popup when front end can't figure out file type.
2024-01-22 23:13:53 -08:00
Timothy Jaeryang Baek
7054f02891
Merge pull request #466 from baumandm/feat/epub-support
...
feat: Add epub support
2024-01-22 23:12:46 -08:00
Marclass
7eea3ef313
copy list of file ext from backend to front end
2024-01-23 00:00:07 -07:00
Marclass
35ace57784
add rst document for RAG
2024-01-19 10:48:04 -07:00
Dave Bauman
f559068186
feat: Add epub support
2024-01-19 12:23:59 -05:00
Marclass
aa1d386042
Allow any file to be used for RAG.
...
Changed RAG parser to prefer file extensions over MIME content types. If the type of file is not recognized assume it's a text file.
2024-01-18 20:41:14 -07:00