Integrating pre-trained LLMS with RAG for efficient content retrieval
DOI:
https://doi.org/10.61591/jslhu.20.633Từ khóa:
Large Language Models; Retrieval-Augmented Generation; Optimizing chunking. Data retrieval.Tóm tắt
Large Language Models (LLMs) are highly effective at replicating human tasks and boosting productivity but face challenges in accurate data extraction due to prioritizing fluency over factual precision. Researchers are addressing these limitations by combining LLMs with Retrieval-Augmented Generation (RAG) models. This approach utilizes chunking, searching, and ranking algorithms to streamline data retrieval from unstructured text, improving LLMs’ precision and processing. The findings provide key insights into optimizing chunking strategies and set the stage for the advancement and broader application of RAG-enhanced systems.
Tài liệu tham khảo
Mei Kobayashi and Koichi Takeda, "Information retrieval on the web", ACM Comput. Surv, 32(2), pp.144–173. 2010.
Yujuan D. et al., "Discriminative dual-stream deep hashing for large-scale image retrieval", Information Processing & Management, 57(6), 2020.
Jiatong L. et al, "Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective", IEEE Transactions on Knowledge and Data Engineering, pp. 1-13, 2024.
Junda Wu, et al., "CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendation", arXiv preprint arXiv:2403.06447, 2024.
Lewis P. et al., "Retrieval-augmented generation for knowledge-intensive NLP tasks", In Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS '20). Curran Associates Inc., Red Hook, NY, USA, Article 793, 9459–9474.
Hongjin S. et al., "Selective Annotation Makes Language Models Better Few-Shot Learners", arXiv preprint: 2209.01975, 2022.
Liu, S. et al, "Multi-modal molecule structure–text model for text-based retrieval and editing", Nature Machine Intelligence, 5(12), pp. 1447–1457, 2023.
Vlad Krotov, Leigh Johnson, "Big web data: Challenges related to data, technology, legality, and ethics", Business Horizons, 66(4), pp. 481-491, 2023.
Han S. et al, "Web Scraping for Hospitality Research: Overview, Opportunities, and Implications", Cornell Hospitality Quarterly, 62(1), pp. 89-10.4, 2021.
Andrews, J. T. et al, "Ethical considerations for collecting human-centric image datasets". arXiv preprintarXiv:2302.03629, 2023.
Brown, T. et al, "Language Models are Few-Shot Learners", arXiv, 2020.
Lewis, P. et al. "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", arXiv, 2020.
Zhang Y., et al, "A Survey on Retrieval-Augmented Language Models", Journal of Machine Learning Research, 1, pp. 1-35, 2020.
Radford A., et al, "Language Models are Unsupervised Multitask Learners", OpenAI, 2019.
Yin Z. et al, “Furthest Reasoning with Plan Assessment: Stable Reasoning Path with Retrieval-Augmented Large Language Models”, arXiv preprint arXiv:2309.12767, 2023.
Fangyuan X. et al, “RECOMP: Improving retrieval-augmented LMs with context compression and selective augmentation”, arXiv:2310.04408, 2023.
Sebastian H. et al, “FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation”. In SIGIR. ACM, pp. 1437–1447, 2023.
Devlin J. et al, "BERT: Pre-training of deep bidirectional transformers for language understanding", arXiv, 2019.
Johnson J. et al. "Billion-Scale Similarity Search with GPUs", IEEE Transactions on Big Data, 7(3), pp. 535-547, 2021.
Lewis P. et al. "MLQA: Evaluating cross-lingual extractive question answering", in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7315-7330, 2020.
Reimers N. et al, "Sentence-BERT: Sentence embeddings using Siamese BERT-networks", in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pp. 3980-3990, 2019.
Manning C. D. et al, "Introduction to Information
Retrieval", Cambridge University Press, 2008.
Jacob D. et al, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2019.
Boyu Z. “Enhancing financial sentiment analysis via retrieval augmented large language model”, in Proceedings of the Fourth ACM International Conference on AI in Finance, pp.349–35, 2023.
Demiao L, et al, “Revolutionizing Retrieval-Augmented Generation with Enhanced PDF Structure Recognition”, arXiv preprint arXiv:2401.12599, 2024.
Antonio J. Y. et al, “Financial Report Chunking for Effective Retrieval Augmented Generation”, arXiv preprint arXiv:2402.05131, 2024.
Yinghao Z. et al, “REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models”, arXiv preprint arXiv:2402.07016, 2024.
