Self Denial
bcf79c8366
Format fixes
2024-04-04 12:02:48 -06:00
Self Denial
3b66aa55c0
Improve embedding model update & resolve network dependency
...
* Add config variable RAG_EMBEDDING_MODEL_AUTO_UPDATE to control update behavior
* Add RAG utils embedding_model_get_path() function to output the filesystem path in addition to update of the model using huggingface_hub
* Update and utilize existing RAG functions in main: get_embedding_model() & update_embedding_model()
* Add GUI setting to execute manual update process
2024-04-04 11:01:23 -06:00
Mmx233
947c392f72
fix: manually check the docs' filename
2024-04-03 23:37:13 +08:00
Timothy J. Baek
5558514ff1
fix
2024-04-01 15:23:12 -07:00
KoreLogic Disclosures
6c96361402
Suggested mitigation for KL-CAN-2024-002.
2024-04-01 15:55:14 -05:00
Timothy J. Baek
a6c154d839
feat: rag context logging
2024-03-31 14:02:31 -07:00
Self Denial
144c9059a3
Improve logging. Move print()
statements to appropiate log()
.
...
Add COMFYUI and WEBHOOK logging and associated environment variable
control. Add WEBHOOK payload & request debug logs.
2024-03-31 13:17:29 -06:00
Timothy J. Baek
3688955c77
fix: encoding issue
2024-03-25 23:50:52 -07:00
Timothy J. Baek
6307adfba1
feat: better error handling
2024-03-25 23:47:08 -07:00
Doug Danat
c91a5d8b1f
switch to using BeautifulSoup HTML loader so title is also captured
2024-03-25 11:26:18 +01:00
Doug Danat
784a6ec85e
include html langchain loader for RAG
2024-03-25 09:50:53 +01:00
Timothy Jaeryang Baek
371dfc1143
Merge branch 'dev' into debug_print
2024-03-24 18:04:03 -05:00
Timothy J. Baek
ff8a55a861
refac: rag api
2024-03-24 00:41:41 -07:00
Timothy J. Baek
7e0ea8f77d
feat: RAG text ingestion(store) api
2024-03-24 00:40:27 -07:00
Self Denial
e6dd0bfbe0
Migrate to python logging module with env var control.
2024-03-20 17:11:36 -06:00
Timothy J. Baek
91efd6cb63
fix: file upload encoding issue
2024-03-15 23:52:37 -07:00
Timothy J. Baek
072b499a50
fix: backslash rag content issue
2024-03-15 13:34:52 -07:00
Timothy J. Baek
8df6b137cb
fix: rag
2024-03-10 18:40:50 -07:00
Timothy J. Baek
98948814fd
feat: toggle pdf ocr
2024-03-10 13:32:34 -07:00
Timothy J. Baek
c49491e516
refac: rag to backend
2024-03-08 22:34:47 -08:00
Timothy J. Baek
7e5e2c42c9
refac: rag routes
2024-03-08 19:26:39 -08:00
Timothy J. Baek
b88c64f80e
fix: ocr issue
2024-03-06 17:54:42 -08:00
Timothy J. Baek
bb98c10abb
revert: ocr feature
2024-03-06 17:04:40 -08:00
Timothy Jaeryang Baek
8fb5f54751
Merge pull request #1050 from jannikstdl/rag-pdf-ocr
...
feat: added ocr functionality to the pdf loader
2024-03-06 00:45:33 -05:00
Jannik Streidl
089a63e0c6
feat: added ocr functionality to the pdf loader
2024-03-05 22:25:25 +01:00
Firat Birlik
6782e95c75
recreate rag collection is now optional and only used for web requests
2024-03-04 10:00:06 -06:00
Firat Birlik
5d4ff85228
recreate rag collection instead of falling back to stale version
2024-03-03 21:25:00 -06:00
Timothy J. Baek
47a05a47b4
feat: add rag top k value setting
2024-03-02 18:56:57 -08:00
Ased Mammad
b473ad574f
fix: RAG scan unsupported mimetype
...
This fixes an issue with RAG that stops loading documents as soon
as it reaches a file with unsupported mimetype.
2024-02-23 14:27:31 +03:30
Timothy J. Baek
7c127c35fc
feat: dynamic embedding model load
2024-02-19 11:05:45 -08:00
Jannik Streidl
acf999013b
storing vectordb in project cache folder + device types
2024-02-19 07:51:17 +01:00
Timothy J. Baek
0cb0358485
refac: more descriptive var names
2024-02-18 11:16:10 -08:00
Jannik S
4b88e7e44f
Merge branch 'main' into choose-embedding-model
2024-02-18 09:20:54 +01:00
Jannik Streidl
bc3dd34d8b
collection query fix
2024-02-18 09:17:43 +01:00
Timothy J. Baek
07b451995e
feat: reset rag template
2024-02-17 22:49:18 -08:00
Timothy J. Baek
5270efa9e5
feat: editable rag template
2024-02-17 22:41:03 -08:00
Timothy J. Baek
ccf08fb91e
feat: editable chunk params
2024-02-17 22:29:52 -08:00
Timothy J. Baek
a94e4161f7
fix: file content type issue
2024-02-17 21:31:46 -08:00
Timothy J. Baek
e07001e5f6
feat: rag folder scan support
2024-02-17 21:06:08 -08:00
Jannik Streidl
1846c1e80d
choose embedding model when using docker
2024-02-17 19:38:29 +01:00
Tim Farrell
08e8e922fd
Endpoint role-checking was redundantly applied but FastAPI provides a nice abstraction mechanic...so I applied it. There should be no logical changes in this code; only simpler, cleaner ways for doing the same thing.
2024-02-08 18:05:01 -06:00
Timothy J. Baek
683650ec00
feat: collection rag integration
2024-02-03 15:57:06 -08:00
Timothy J. Baek
00803c92f2
feat: doc tagging
2024-02-03 14:44:49 -08:00
Timothy J. Baek
50f7b20ac2
refac
2024-02-01 13:35:41 -08:00
Timothy J. Baek
28226a6f97
feat: web rag support
2024-01-26 22:17:28 -08:00
Timothy J. Baek
4e468dc58c
refac
2024-01-25 00:24:49 -08:00
Timothy Jaeryang Baek
fa5918ad13
Merge branch 'main' into main
2024-01-25 00:13:12 -08:00
Marclass
8bfda730d9
add excel document support
2024-01-23 14:03:22 -07:00
Timothy Jaeryang Baek
ca943d0795
Merge pull request #549 from Marclass/main
...
Bugfix: Fix toast error popup when front end can't figure out file type.
2024-01-22 23:13:53 -08:00
Timothy Jaeryang Baek
7054f02891
Merge pull request #466 from baumandm/feat/epub-support
...
feat: Add epub support
2024-01-22 23:12:46 -08:00